Jan 182011
 

The risk of computer hard disk failure is fairly well known: the disk crashes and your computer stops working.

Corrupt 6, by gusset, on Flickr
Corrupt 6, by gusset, on Flickr

Less well-known is a phenomenon known as silent data corruption, where an undetected error occurs in content stored on a drive.  Errors creep in from bugs in both software and hardware (firmware).  “Silent” means the drive does not report it, even in situations where special precautions such as multiple redundant disks (RAID) are used.  The problem remains unknown until you attempt to retrieve the data.

An Analysis of Data Corruption in the Storage Stack, A 2008 study of 1.53 million disk drives over a period of 41 months, found 400,000 silent errors.   That is a disturbingly high number, even though the overall percentage of bad to good data was very small.  The bad news for personal users is that the type of hard disk they are most likely to have–a SATA drive–has a failure rate that is an order of magnitude large than more expensive “enterprise class” drives.

If you have important digital data that you need to keep for a long time the best thing to do is to keep multiple copies in different places and stored on different kinds of media, even different kinds of hard disks.  As I wrote earlier, all varies of digital storage media will eventually fail, so it is essential to have replicated copies.

The more complicated issue is how to detect and fix silent errors.  The most common method is to use a checksum–an unique numerical code for each file–to find bad data.  Once an error is found the next step is to “scrub” it, a process where bad data is replaced by good data from a trusted source.

Individual users have limited choices in this regard, unfortunately.  Checksum comparisons and scrubbing require advanced knowledge and can take a long time.   Commercial data recovery services might be able to help, but they charge a hefty premium.  A cloud storage provider may–or may  not–provide the service; if it does, and if the service runs automatically in the background, this is a fact very much in its favor.

Jan 022011
 

All digital storage media–hard drives, flash disks, CD-ROMs, and the like–have a short life.  This is why digital preservation requires active management, including regular migration of content from older storage devices to newer devices.

Do you have a back-up plan?

Do you have a back-up plan? by Images by John 'K', on Flickr

Individuals face an especially serious challenge.  Unlike many organizations, people at home typically do not have special services to guard their digital data from loss or corruption.

Another way to put it is that everyone is now their own digital archivist.  If you don’t attend to preserving your own digital photographs, videos, email, social media and so on, there is an excellent chance they will be lost.

And, unlike what some vendors imply, relying solely on the cloud is not foolproof. A commercial service can choose to pull the plug–literally–on a cloud service at any time.  If you want to keep it, you need to take responsibility for it.

Individual users need to know that the life of storage media are cut short by at least three factors:

  1. media durability
  2. media usage, storage and handling
  3. media obsolescence

Media Durability

Computer storage media devices vary in how long they last. The quality and construction of individual media items differ widely. The following estimates for media life are approximate; a specific item can easily last longer–or fail much sooner.

  • Floppy disk: 3-5 years.  Though no longer made, many still exist; examples include 8”, 5.25” and 3.5” disks, along with items such as Zip and Jaz disks.
  • Flash media: 1-10 years.  This category includes USB flash drives (also known as jump drives or thumb drives), SD/SDHC cards and solid-state drives; all generally are less reliable than traditional spinning-disk hard drives.
  • Hard drive: 2-8 years.  The health of a spinning disk hard drive often depends on the environment; excessive heat, for example, can lead to quick failure.
  • CD/DVD/Blu-ray optical disk: 2-10 years.  There is large variation in the quality of optical media; note that “burnable” discs typically have a shorter life than “factory pressed” discs).
  • Magnetic tape: 10-30 years.  Tape is a more expensive storage option for most users–it depends on specialty equipment–but it is the most reliable media available.

Media use handling and storage

People have a direct impact on the lives of storage media:

  • The more often media are handled and used, the greater the chance they will fail; careful handling can extend media life, rough handling has the opposite effect.
  • Stable and moderate temperature and humidity, along with protection from harmful elements (such as sun and salt) helps keep media alive.
  • Good-quality readers and other hardware media connections are beneficial; poor connections can kill media quickly.
  • Media that are not labeled or safely stored can be lost or accidentally thrown away.
  • Fires, floods and other disasters are very bad for media!

Media obsolescence

Computer technology changes very quickly.  Commonly used storage media can become obsolete within a few years.  Current and future computers may not:

  • Have drives that can read older media.
  • Have hardware connections that can attach to older media (or media drives).
  • Have device drivers that can recognize older media hardware.
  • Have software that can read older files on media.

What you need to do

Actively manage your important digital content!  Steps to consider:

  • Have at least two separate copies of your content on separate media—more copies are better.
  • Use different kinds of media (DVDs, CDs, portable hard drives, thumb drives or Internet storage);  use reputable vendors and products.
  • Store media copies in different locations that are as physically far apart as practical.
  • Label media properly and keep in secure locations (such as with important papers).
  • Create new archival media copies at least every five years to avoid data loss.

For more information

  1. Care and Handling of CDs and DVDs —A Guide for Librarians and Archivists
  2. Digital Media Life Expectancy and Care
  3. Do Burned CDs Have a Short Life Span?
  4. Mag Tape Life Expectancy 10-30 years
  5. Personal Archiving: Preserving Your Digital Memories (Library of Congress)
  6. Retro Media: Memory (and Memories) Lost; Which of these media will be readable in 10 years?  50 years?  150 years?
  7. Care, Handling and Storage of Removable media (UK National Archives)
  8. Do You Have a Back-up Plan?

Note: This is adopted from information developed for digitalpreservation.gov at the Library of Congress