Washington, Apr 22 : Long term data storage is the biggest shortcoming linked with digital tapes and hard disk drives, but now researchers have found a solution by introducing a new approach, called Pergamum, to store vast amounts of digital data for recovery by future generations.
Pergamum, a distributed network of intelligent, disk-based storage devices, developed by scientists from University of California, Santa Cruz, may make today's archival data storage techniques, using tapes and hard disks turn obsolete.
Developed by a team led by UCSC graduate students Mark Storer and Kevin Greenan, Pergamum will provide reliable, energy-efficient data storage using off-the-shelf components. It also has the ability to evolve over time as storage technologies change.
Today, businesses need to store archival data due to legal requirements for the preservation of financial and business records, and as data mining strategies can turn stored data into a valuable resource. In fact, long-term storage is also a problem for individuals who are filling their personal computers with digital photos, movies, and documents.
"There is a risk that an entire generation's cultural history could be lost if people aren't able to retrieve that data. Everyone is switching to digital cameras, but we've never demonstrated that digital data can be reliably preserved for a long time," Storer said.
"You want to avoid 'forklift upgrades,' where you have to get rid of the old system and transfer all your data to a whole new system," said Miller.
Storer claimed that businesses are beginning to recognize that archival storage is very different from simply backing up their data, saying: "A backup is a safety net--you hope you won't need it. Archival data you do want to use--it's a valuable resource and you want to be able to mine it for information."
Ideally, archival data should be easy to read, query, browse, and search, and that's where tape lags behind. Existing disk-based systems offer excellent performance, but rely on power-hungry central controllers.
"Energy usage is a big issue, so a lot of our effort in designing Pergamum focused on dramatically reducing power use," said Miller.
Pergamum is made by individual building blocks consisting of a hard drive; a small, low-power processor (like the chip in an iPhone); a flash memory card; and an ethernet port. These units, called "tomes," are connected using reasonably priced ethernet switches.
"Each tome is like a minicomputer, but with very low power demands. When not in use, it can shut down almost completely," said Miller.
Despite being active, the devices use very little power (less than 13 watts), which can be delivered over the network using Power over Ethernet technology. Thus, it uses two levels of redundancy--within and between disks--to protect from both disk failures and errors in writing data to a disk (so-called "latent sector errors"). Tomes can be easily added to expand the system or to replace failed disks.
In case, hard disk drives become obsolete in 10 years, Pergamum won't suffer the same fate. The system doesn't care what the actual storage medium is, as long as the device can implement the simple protocol that will allow it to function as part of the network.
"In 50 years, the devices might use holographic storage. As long as you can wrap the new storage medium in this intelligent layer that speaks the protocol, it can participate in the network," said Storer.
Pergamum was presented at the USENIX Conference on File and Storage Technologies (FAST '08), held in San Jose in February.