tag:blogger.com,1999:blog-4503292949532760618.post2559957615024466363..comments2024-03-28T13:39:27.601-07:00Comments on DSHR's Blog: Compression vs. PreservationDavid.http://www.blogger.com/profile/14498131502038331594noreply@blogger.comBlogger1125tag:blogger.com,1999:blog-4503292949532760618.post-52625884709101754732019-04-07T15:05:33.984-07:002019-04-07T15:05:33.984-07:00Thanks David, for another excellent post. It'...Thanks David, for another excellent post. It's certainly an interesting point that the selection process is essentially a form of lossy compression.<br /><br />There are one or two things I don't necessarily agree with, eg "File system: this layer typically neither reduces nor increases redundancy, although file systems that compress content exist, as do file systems that deduplicate (see Caveat below). Neither is appropriate for preservation system use."<br /><br />I think you're saying that this cancels out redundancy. This is true, however in practice it is normally used to cancel out unintended redundancy and then use other specific functions for intentional redundancy (e.g. cancel out multiple copies of a file saved in numerous network locations, but then replicate these copies to multiple sites for redundancy). The advantage of this is that you are using known, specific techniques for redundancy and saving space where it is not needed rather than depending on incidental or ad-hoc methods for recovery in the event of failures (which are only every likely to allow for recovery of some data, rather than all).<br /><br />On file system compression - it would be good to hear definitively why it is not appropriate for preservation system use. Are there really compelling reasons to exclude such systems? Of course, a risk assessment would need to be carried out based on the specific system and software in use, but that might determine the risk of turning off the compression is greater, eg a storage system that uses compression by default (turning off the compression would on the face of it be a non-standard implementation).James Doighttps://www.blogger.com/profile/00122814918462610034noreply@blogger.com