Hacker News new | ask | show | jobs
by slaymaker1907 1837 days ago
Just to add to the explanation, SSDs are able to do this because they have a layer of indirection akin to virtual memory. This means that what your OS thinks is byte 800000 of the SSD may change it's actual physical location on the SSD over time even in the absence of writes or reads to said location.

This is a very important property of SSDs and is a large reason why log structured storage is so popular in recent times. The SSD is very fast at appends, but changing data is much slower.

3 comments

> The SSD is very fast at appends, but changing data is much slower.

No, it's worse than that. The fact that it's an overly subtle distinction is the problem.

SSDs are fast while write traffic is light. From an operational standpoint, the drive is lying to you about its performance. Unless you are routinely stress testing your system to failure, you may have a very inaccurate picture of how your system performs under load, meaning you have done your capacity planning incorrectly, and you will be caught out with a production issue.

Ultimately it's the same sentiment as people who don't like the worst-case VACUUM behavior of Postgres - best-effort algorithms in your system of record make some people very cranky. They'd rather have higher latency with a smaller error range, because at least they can see the problem.

Are there write-once SSDs? They would have a tremendous capacity. Probably good for long term backups or archiving. Also possibly with a log structured filesystem only.
Making them write-once doesn't increase the capacity; that's mostly limited by how many analog levels you can distinguish on the stored charge, and how many cells you can fit. The management overhead and spare capacity to make SSDs rewritable is –to my knowledge– in the single digit percentages.

(Also you need the translation layer even for write-once since flash generally doesn't come 100% defect free. Not sure if manufacturers could try to get it there, but that'd probably drive the cost up massively. And the translation layer is there for rewritable flash anyway... the cost/benefit tradeoff is in favor of just living with a few bugged cells.)

I suspect that hawki was assuming that a WORM SSD would be based on a different non-flash storage medium. I don't know any write once media that has similar read/write access times to an SSD.

FWIW, there are WORM microsd cards available but it looks like they still use flash under the hood.

I don't know enough specifics, so I didn't assume anything :) In fact I was not aware of non-flash SSDs.

Because of the Internet age there probably is not much place for write once media anyway, even it would be somewhat cheaper. But maybe for specialized applications or if it would be much much cheaper per GB.

The only write once media I'm aware of that is in significant use are WORM tapes. They don't offer significant advantages over regular tapes, but for compliance reasons it can be useful to just make it impossible to modify the backups.
What about EPROMs? I mean could those be scaled down with 7nm lithography to be energy efficient incorruptible fast storage?
> Making them write-once doesn't increase the capacity

It could theoretically make them cheaper. But I guess that there wouldn't be enough demand, so you'd be better off having some kind of OS enforced limitation on it.

I find this a super interesting question. I always assumed that long term stability of electronic non-volatile memory is worse than that of magnetic memory. When I think about it, I can't think of any compelling reason why that should be the case. Trapped electrons vs magnetic regions; I have no intuition which one of them is likely to be more stable.

There is a question on stackoverflow about this topic with many answers but no definitive conclusion. There seem to be some papers touching the subject but at a glance I couldn't find anything useful in them.

[1] https://superuser.com/questions/4307/what-lasts-longer-data-...

According to https://www.ni.com/en-no/support/documentation/supplemental/... (Seems kinda reputable at least)

"The level of charge in each cell must be kept within certain thresholds to maintain data integrity. Unfortunately, charge leaks from flash cells over time, and if too much charge is lost then the data stored will also be lost.

During normal operation, the flash drive firmware routinely refreshes the cells to restore lost charge. However, when the flash is not powered the state of charge will naturally degrade with time. The rate of charge loss, and sensitivity of the flash to that loss, is impacted by the flash structure, amount of flash wear (number of P/E cycles performed on the cell), and the storage temperature. Flash Cell Endurance specifications usually assume a minimum data retention duration of 12 months at the end of drive life."

> During normal operation, the flash drive firmware routinely refreshes the cells to restore lost charge. However, when the flash is not powered the state of charge will naturally degrade with time.

You have to be careful how you interpret this bit. "Normal operation" here assumes not just that the SSD is powered, but that it is actively used to perform IO. Writes to the SSD will eventually cause data to be refreshed as a consequence of wear leveling; if you write 1TB per month to a 1TB drive then every (in-use) cell will be refreshed approximately monthly, and data degradation won't be a problem.

If you have an extremely low-write workload, the natural turnover due to wear leveling won't keep the data particularly fresh and you'll be dependent on the SSD re-writing data when it notices (correctable) read errors, which means data that is never accessed could degrade without being caught. But in this scenario, you're writing so little to the drive that the flash stays more or less new, and should have quite long data retention even without refreshing stored data.

> When I think about it, I can't think of any compelling reason why that should be the case. Trapped electrons vs magnetic regions; I have no intuition which one of them is likely to be more stable.

My layman intuition (which could be totally wrong) is that trapped electrons have a natural tendency to escape due to pure thermal jitter. Whereas magnetic materials tend to stick together, so there's at least that. Don't how much of this matches the actual electron physics/technology though...

Hmm I don't think this is conclusive. Thermal jitter makes magnetic boundaries change too, and of course you have to add to it that it is more susceptible to magnetic interference.

I don't have intuition either, but I don't think this explanation is sufficient

> Trapped electrons vs magnetic regions;

From the physics point of view, aren't both cases the same thing?.

Isn't magnetic regions a state of the electric field? so if I move electrons in and out, the electric field should be changing as well

No. A region of a piece of material is magnetized in a certain direction when its (ionized) atoms are mostly oriented in that direction, the presence of a constant magnetic field is (roughly speaking) only a consequence of that.

So flash memory is about the electrons, while magnetic memory is about the ions.

Aren't permanent magnetics a direct result of oriented spins? (So due to quantum effects?)
Modern multi-bit-per-cell flash has quite terrible data retention. It is especially low if it is stored in a warm place. You'd be lucky to see ten years without an occasional re-read + error-correct + re-write operation going on
Any SSD you go through the trouble of building a max capacity disk image for, then dd'ing onto the disk before removing?

I mean... This is general purpose HW here. Write once SSD is a workflow more than an economically tenable use-case in terms of making massive size write once then burn the write circuit devices.

I don't think anyone would make literally write-once drives with flash memory; that's more optical disk territory. But zoned SSDs and host-managed SMR hard drives make explicit the distinction between writes and larger-scale erase operations, while still allowing random-access reads.
That would be magnetic tapes.
Append-only garbage-collected storage was used in data center even when hard disks were (and are) popular because it's more reliable and scalable.