![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
miriam_eWell, this is disturbing:
http://lwn.net/Articles/349970/
I've always disliked flash memory, or rather the common implementation of it as thumbdrives and little flash cards, but I thought the journalling in Linux's Ext3 and Ext4 made data pretty secure. It seems it ain't necessarily so. It can keep your filesystem hierarchy clean, but still lose data from the files in that filesystem without you knowing about it... which makes it almost pointless.
RAID is more secure, but even it can face corruption if power is lost suddenly. I have to admit I don't entirely understand all the details of the different RAID arrangements though, and it worries me less because I'm frankly unlikely to have enough money to be able use it in the near future.
Why do I particularly dislike flash drives?
They seem like a case of three steps forward and two steps back. Granted they give us nice, small, rewriteable memory that doesn't need power to retain its data, but it is also a system that requires large blocks of many thousands of bytes to be written at once, which means writes must be buffered. If the flash drive is to be unplugged, any buffers that remain incompletely filled, waiting to be written, must be emptied onto the flash drive first. So the current, unsatisfactory solution is that buffers must be flushed by manually unmounting before unplugging the flash drive. But we humans have better things to do than micro-manage our computers -- they're are supposed to do that, so we don't have to. It would have been a simple to avoid this problem. If the operating system let us know that the buffer wasn't empty, and warned, on unplugging a flash drive that either a write had been interrupted or that a buffer was still waiting to be emptied then we would have a much more human-friendly and safer system. The old Amiga did this with floppy drives. Now 30 years later it seems we are moving backwards in the human-friendliness of our machines. If we fail to remember to unmount a flash drive before unplugging it our modern computers quietly allow the data to be corrupted.
Power failure is a slightly more difficult problem to solve, but even that can be avoided with a little forethought. Capacitors can hold power for the milliseconds required to save to either CMOS battery-backed RAM or flash memory the current state of the machine. This would hold info about, for example, any write operation to either hard drive or flash drive that had been in progress at the time of power failure. When power resumed this could be read to find out if the system was in the middle of a write operation and what addresses had been involved. This is not rocket science.
Beyond the problems of buffered writes coupled with half-thought-out driver design, I have this awful nagging knowledge that we are just waiting for all these flash drives to begin corrupting data as they reach the end of their lives (they support only a limited number of write/delete cycles). While flash drives keep getting bigger as fast as they currently are, this doesn't feel like a problem, but in actual fact we are just walking deeper into shark-infested waters. As the drives get bigger then we put more data at risk on them. They will fail eventually and begin to corrupt our data. It is guaranteed. You can't even put the drive away to keep it safe, because the data on a flash drive will also simply evaporate after some years. "Years" may seem like a long time, but it isn't if you have a lot of essential data that you need to refer to once a decade. Putting it on flash is just asking to lose it.
http://lwn.net/Articles/349970/
I've always disliked flash memory, or rather the common implementation of it as thumbdrives and little flash cards, but I thought the journalling in Linux's Ext3 and Ext4 made data pretty secure. It seems it ain't necessarily so. It can keep your filesystem hierarchy clean, but still lose data from the files in that filesystem without you knowing about it... which makes it almost pointless.
RAID is more secure, but even it can face corruption if power is lost suddenly. I have to admit I don't entirely understand all the details of the different RAID arrangements though, and it worries me less because I'm frankly unlikely to have enough money to be able use it in the near future.
Why do I particularly dislike flash drives?
They seem like a case of three steps forward and two steps back. Granted they give us nice, small, rewriteable memory that doesn't need power to retain its data, but it is also a system that requires large blocks of many thousands of bytes to be written at once, which means writes must be buffered. If the flash drive is to be unplugged, any buffers that remain incompletely filled, waiting to be written, must be emptied onto the flash drive first. So the current, unsatisfactory solution is that buffers must be flushed by manually unmounting before unplugging the flash drive. But we humans have better things to do than micro-manage our computers -- they're are supposed to do that, so we don't have to. It would have been a simple to avoid this problem. If the operating system let us know that the buffer wasn't empty, and warned, on unplugging a flash drive that either a write had been interrupted or that a buffer was still waiting to be emptied then we would have a much more human-friendly and safer system. The old Amiga did this with floppy drives. Now 30 years later it seems we are moving backwards in the human-friendliness of our machines. If we fail to remember to unmount a flash drive before unplugging it our modern computers quietly allow the data to be corrupted.
Power failure is a slightly more difficult problem to solve, but even that can be avoided with a little forethought. Capacitors can hold power for the milliseconds required to save to either CMOS battery-backed RAM or flash memory the current state of the machine. This would hold info about, for example, any write operation to either hard drive or flash drive that had been in progress at the time of power failure. When power resumed this could be read to find out if the system was in the middle of a write operation and what addresses had been involved. This is not rocket science.
Beyond the problems of buffered writes coupled with half-thought-out driver design, I have this awful nagging knowledge that we are just waiting for all these flash drives to begin corrupting data as they reach the end of their lives (they support only a limited number of write/delete cycles). While flash drives keep getting bigger as fast as they currently are, this doesn't feel like a problem, but in actual fact we are just walking deeper into shark-infested waters. As the drives get bigger then we put more data at risk on them. They will fail eventually and begin to corrupt our data. It is guaranteed. You can't even put the drive away to keep it safe, because the data on a flash drive will also simply evaporate after some years. "Years" may seem like a long time, but it isn't if you have a lot of essential data that you need to refer to once a decade. Putting it on flash is just asking to lose it.