[sthlmb]

I guess I'll reply here and cover both of the comments :D There are currently 80 SBCs in the database, and they're all boards that I've obtained over the last few years. It's a side project/hobby and buying many more boards would be a very expensive affair :( New boards will be a mixture of those that interest me, and those that vendors (very kindly) send to be included on the website. Sadly, as much as I'd love to, I can't do a Pokémon and catch them all, so there will be some missing!

[aspenmayer]

Do you accept donations of hardware to be tested and added to the site?

[sthlmb]

I do accept them from vendors directly (and it has no bearing on how I run the tests) but I've not taken any from individuals before. I'm not quite sure how I'd do that logistically, but if there's something interesting then I'm open to discussing it!

I'm currently in talks with a few of the vendors to start filling the database with missing boards from their ranges so we should have more data available in the coming weeks. There should be 2 BeagleBoard (Eco Green and PocketBeagle 2) arriving tomorrow/Tuesday to expand their range on the site!

[aspenmayer]

As for donations from individuals, I have seen folks who do this kind of thing get a P.O. Box or use a mailbox service provider/remailer[0] to maintain their opsec, though that is an added expense. It would likely be easier to accept donations earmarked for equipment purchases, but that requires a bit of trust and reputation, but you have to begin where you are, which is pretty good so far, as you are doing your own tests on your own equipment in your possession. I’m sure many would be willing and able to donate funds and/or hardware, if the option were presented to them, but I don’t know if that is worth the added efforts involved in engaging with the community.

Directly engaging with vendors for donations is somewhat fraught with concerns of its own, as ideally you are doing blind purchases so that vendors can’t give you known-good units, but rather are giving you units that the vendor doesn’t already know are going to be subject to enhanced scrutiny.

Maybe get in touch with the OpenWRT folks to see if you can test the OpenWRT One? I believe they’re using a modified BananaPi design.

https://openwrt.org/toh/openwrt/one

> OpenWrt One is based on the MediaTek Filogic 820 SoC and has WiFi 6, dual-band, 3×3/2×2, 1x 2.5Gbit WAN, 1x 1Gbit LAN, 1GB DDR4 RAM, 256 MiB NAND, 16 MiB NOR (for recovery), M.2 SSD, USB-C Serial console and USB 2.0. Power Over Ethernet (POE): an IEEE 802.3af/at compliant device can power the device via the RJ-45 2.5 Gbps WAN connector.

https://docs.banana-pi.org/en/OpenWRT-One/BananaPi_OpenWRT-O...

> OpenWrt One uses Mediatek’s MT7981B SoC with a dual-arm Cortex-a53 core at 1.3 GHz, 1 GB DDR4 RAM and 256 MiB SPI NAND storage. It also integrates 16 MiB of additional protected storage as a system backup, dual storage hardware to ensure that the onboard system is unbrickable, and finally integrated M.2. 2230/2242 NVMe PCIe 2 X1 ports can be used to add external storage. And with a battery holder for an internal clock, OpenWrt One offers a USB 2.0 Type-A and a mikroBUS expansion port to provide more interfaces to a host of technical possibilities

> OpenWrt One is the first board design with OpenWrt opensource communtity.and designed in collaboration with Banana Pi that will also handle manufacturing and distribution of the router board. The OpenWrt One/AP-24.XY router should provide a source of income for the project, for example, to cover the cost of hosting and OpenWrt conferences, with Banana Pi selling the board through their distribution network, and for every device sold, donating to the Software Freedom Conservancy (SDC) with the funds earmarked for OpenWrt.

[0] Techlore on YouTube interviewed the operator of such a service in the video linked below. I won’t name the company as this is not a review or endorsement (pun intended), as I have not used their service, nor can I speak to their operations, but the interview is a decent explainer of the service itself from the point of view of an operator of such a service, and other competitors in that field are also named.

https://youtube.com/watch?v=idSBvjaaFSk

[sthlmb]

I do have a ko-fi available in the footer of the website, and if someone wants their donation to go towards a specific piece of hardware then I'll most certainly do what I can to honour that, or return it if I can't.

On the samples from vendors front, yeah, that's definitely a concern, though in the years I've been working with manufacturers and had samples from them, I'm fairly confident that no "golden sample" binning is going on as I've had quite a few shockers, hah! Not to say that it couldn't/wouldn't happen, I wouldn't want to introduce that doubt. Perhaps, like my review website, I should add a quick note/indicator of whether a board was obtained directly from a vendor, or if it was purchased.

[aspenmayer]

> I do have a ko-fi available in the footer of the website, and if someone wants their donation to go towards a specific piece of hardware then I'll most certainly do what I can to honour that, or return it if I can't.

That’s pretty good of you to mention, as I didn’t notice the footer when I visited to see if you already had tested the OpenWRT One before posting my suggestion to test it. Perhaps you could mention that you would make a best effort to test hardware suggested by donors, if you haven’t already done so on the site? I approve of you asking for donations openly, as your time and efforts are worth something to you and to visitors, and if you communicate that it’s a two-way street, with some incentives to donors in that they can help suggest what you spend their donations on, I think that your transparency and good faith are more likely to be rewarded with more and larger donations that a simple “buy me a coffee” pitch might otherwise attract.

> Perhaps, like my review website, I should add a quick note/indicator of whether a board was obtained directly from a vendor, or if it was purchased.

I think such a disclaimer/disclosure would assuage (m)any concerns most folks would have on that front!

You seem to have a good grasp on what you’re doing here. Do you share/open source your testing methodology and/or code run to derive your data?

Edit: I see now that you have some of your testing methodology posted. I would suggest gathering as much room temp data during testing as possible, and maybe even re-running the test suite at variable temperatures as much as possible if you can, so you can plot performance over time with different cooling setups in different ambient temperature conditions, as many folks use SBCs in (semi) permanent indoor and outdoor installs in less-than-ideal environments and enclosures, usually with little to no cooling.

If you were to share your complete test scripts, temp probe/jig setup, etc., other users could reality-test their own hardware against other devices in an apples-to-apples way.

[sthlmb]

I'm discussing some other bits with Banana Pi so maybe I can ask about that, I'd need to look into the device to see how viable it is, let's see! I can definitely add the "How did you get this board" disclaimer there somewhere

I don't know if I'd release everything as is, there's a lot of hard coded nonsense that I'd need to filter out and after testing 80 boards, I have a bunch of improvements I want to make. Opening up the test scripts to others isn't/wasn't a high priority item, as I didn't even plan on making this site initially :D

Room temperature data is also nice, but I'll need to draw a line somewhere, on average these tests take around 3-4 hours to run, and with 80 boards right now, best case scenario is 10 straight days of testing (in reality, more with setup and.. life). I could use data on the most popular boards and only do a handful, but I probably have to limit the scope at this point and clearly state the testing conditions and how it may differ in your real world setup.

I can definitely do more to expand on the testing process though, and I can look at sharing the exact commands that are run with my automation so that users can at least mimic the benchmark runs if not the full automation. I really appreciate the responses and feedback, though, so thank you!