[GordonS]

Not sure this actually happens. In practice.

A friend works at an oil company, and they have dozens of rpis deployed on offshore rigs, and in land rigs in the middle east. They are working in pretty harsh conditions, with high salinity, sand in the air, high vibrations, very low temps, very high temps... and yet over the 6 years they have been running, not 1 has had a fault.

All use SD cards, and they haven't been glued in or secured in any way.

At this point, the RPi SD-card fault thing is becoming a meme.

[kavalg]

Not my experience with commodity SD card, power supply and stock settings of Raspbian. They must be using some "secret sauce" to make them run 6 years without outages. I would suspect some industrial grade SD card (SLC or at least AMLC), high quality power supply (e.g. UPS with surge protection) and also some way to limit writes on the SD-card.

[airbreather]

Commodity SD cards - there's your problem. It's not a camera you need to buy industrial grade to get 0.5-1M write cycles.

Just bought industrial grade SD cards for Siemens PLCs the other day, almost $2k for 32 gig and due to certification for safety only good for ten years regardless of condition/use.

Equivelents spec cards without the magic S word on them are still around $800.

They are actually a different kind of memory used, so I guess not manufactured in same consumer number quantities.

I always liked the look of something like puppy Linux that loaded to ram, pretty sure most Linux variants allow this as an option, that would potentially make use of consumer grade palletable.

[kn0where]

I have an original RPi in my parents garage in Arizona that checks if the garage door is open and sends a notification if it’s left open too long. It’s been there a few years now, survived the hot summers just fine. Using an ordinary SanDisk SD card and a decent quality phone charger for power. Occasionally it locks up and my parents unplug it and plug it back in and it boots fine.

[Ductapemaster]

Any disk-IO heavy application will brick a card in record time. I worked on a lot of Pi-based systems at my last job (long term time-lapse photography) and we learned quickly to:

1. Buy high quality SD cards

2. Remove as much disk-IO as possible, using an external USB stick for raw data storage, logs, swap, etc

3. If you really need to make it stable, make the OS read-only

[GordonS]

You mentioned USB sticks, but I just assumed they used the same type of flash memory that SD cards use?

Aside from USB sticks, any ideas on alternative storage suitable for RPis (something small)? I know you can get eMMC modules that fit into the SD card slot, but I would have guessed they'd have the same issue with writes.

[GordonS]

They are running stock Raspbian. For SD cards, they are using a mix of SanDisk, Kingston and others - consumer versions.

They are indeed using UPS', and the workloads naturally have a low write load.

[soylentcola]

That's nuts. I had to replace some old Cisco media players that we used as digital schedule boards outside of ~20 classrooms. The Cisco players were quite old and only needed to load a webpage that pulled schedule info from elsewhere and displayed it in the proper format.

I decided to do this with RPi and went through several revisions as I learned/relearned a lot about Linux. The first version was built on full Raspbian and added a basic server listening on some port or another so I could remotely reboot the things by clicking links on a simple website I put together. That build would usually crap out after a few months and the Pis would stop booting fully from the Sandisk MicroSD cards.

Most recent rev was built from the "Lite" version of Raspbian as I was looking to stay slim and have less stuff to update/run in the background/cause issues with the basic operation of a device that only needed to boot, load Chromium, and open a webpage defined in the startup script.

Those have been a lot more stable (no more reimaging a card here and there every month or two) but they still occasionally crap out and a reimage is the only option (maybe once every 6-8 months). This is in an air conditioned building with stable power, wired LAN connection, and fast SD cards. I can't imagine they would be even this reliable running all the time in any sort of "hostile" environment or even using wireless networking.

I looked into network boot but didn't really have the time or resources to set it all up on the school's network.

[ClumsyPilot]

Is the OS perhaps booting in readonly mode? My experience with the first and second gen pi was of frequent data corruption. I have the later models, but i have hardly used them, so cant tell.

[theamk]

The problem ia not vibration or heat, it is power off while data is being written.

The default rpi os writes to disk all the time, there is even syslog with periodic messages. I bet your oil company uses read-only root, or at least a customized os which does not log things unless it has to.

[GordonS]

Nope, it's stock Raspbian.

I mentioned in another comment though, that the particular workload doesn't result in many writes.

[kube-system]

I've have first hand experience with a failure or two. I'd bet the SD card failure rates depend mostly on write load.