by Angela Guess
A recent article makes the case for more companies to turn to tape rather than or in addition to the internet for their Big Data storage needs. The article notes, “Conventional wisdom, or at least the wisdom of the disk manufacturers, is that the entire Big Data store needs to be online and available (how convenient). The reality is that putting all data online is not a viable solution for most businesses from a cost perspective. An example of a Big Data use case that I heard recently was of a farming conglomerate that installed soil sensors on every tractor. These sensors took samples every time they rotated through the soil on every field that was worked. For the conglomerate, this resulted in tractors taking hundreds of samples of soil every minute across thousands of acres of farmland throughout the United States.”
It continues, “The intent is to store all of these data for decades so that seasonal soil and weather predictions can be made. As you can imagine it will take petabytes of storage per year to maintain all this information. When extended to decades, the prospect of storing this data on even the cheapest of disk systems is totally impractical. Also consider that this is not a backup disk area, high performance analytics require high performance disk systems. Purchasing PBs of this kind of tier one disk would render almost any Big Data project too costly to undertake.”
The article goes on, “The answer is to leverage tape as part of the access tier through the use of an Active Archive. Active Archiving is the ability to marry high performance primary disk storage with secondary disk and then tape to create a single, fully integrated access point. The data that needs to be analyzed at a given moment in time would be loaded onto the high performance tier which, in the above example, could be a comparison of 2011 soil samples to 2009 soil samples. The secondary disk tier could store inbound data from the tractor collection devices, which would come over slower broadband connections. Finally, tape could be used to store all the other years’ worth of data. The Active Archive software would automatically move the data between the various tiers based on access or the movement could be pre-programmed into the application.”
photo credit: Arild Andersen
