Tuesday, June 28, 2011

More on de-duplicating flash controllers

My ACM Queue piece describing the problems caused by storage devices invisibly doing de-duplication attracted the attention of Robin Harris who actually asked SandForce and other manufacturers to comment. The details of SandForce's response are in Robin's StorageMojo article but the key claims are:
There is no more likelihood of DuraWrite loosing data than if it was not present.
and:
That is why SandForce created RAISE (Redundant Array of Independent Silicon Elements) and includes it on every SSD that uses a SandForce SSD Processor. ... if the ECC engine is unable to correct the bit error RAISE will step in to correct a complete failure of an entire sector, page, or block. ...This combination of ECC and RAISE protection provides a resulting UBER of 10-29 virtually eliminates the probabilities of data corruption.
I would regard both claims with considerable skepticism:

1 comment:

David. said...

According to Robin Harris SandForce claims a UBER of 10e-29. That is a remarkable number. To see how remarkable, consider a black box containing an exabyte, approximately 10e19 bits, that has the claimed UBER of 10e-29. On average, you would get an error about once in every 10e(29-19=10) times you read the box's contents.

Suppose the box's read bandwidth was 10Gb or 10e10bit/s. It would take 10e(19-10=9)s to read the contents. So on average you would wait about 10e(9+10=19)s before seeing an error. This is a long time. The current estimate of the age of the universe is about 4.4*10e17s, so it is roughly 23 times the age of the universe.

Remarkable claims need remarkable evidence. It would be interesting to see the basis for SandForce's claim.