Before starting this column, what exactly is a petabyte? Technically, it is a measure of storage capacity equal to 2 to the 50th power bytes, or 1,024 to the 5th power bytes. Practically, it is about a million gigabytes or a thousand terabytes. Pretty big, isn't?
Petabyte levels of storage are not that usual today. They can be found at some large universities, and research or government labs. But the petabyte age is coming, and companies will need petabyte CIOs. Who are they? Here is Fred Hapgood's definition.
[This is] the person responsible for developing and maintaining the science-fiction-like applications that will be running on tomorrow's immense storage capacities. Currently, petabyte responsibilities are mostly restricted to the IT departments of universities, research organizations, and microbiology and genetics labs. But the law of technological adoption ("If you build it, they will use it") says that sooner or later most CIOs will be crossing that line, discovering a new world of applications, responsibilities, costs and problems.
As usual with CIO Magazine, this article provides us with an Executive Summary.
As companies' storage needs increase exponentially, many will eventually discover the world of petabyte storage. Such massive storage levels create applications that can retain and process vast amounts of visual data (especially data from video cameras), applications that provide simulations and predictive models, and apps that access networked devices such as counters, meters and switches. Management headaches come when searches conducted on larger volumes of data generate more errors. The costs of the core overhead tasks (like buffer management) will grow, and data quality will deteriorate as more space opens up. While some observers say petabyte storage is just not worth the hassle, optimists believe new technologies -- such as systems that will automatically detect and resolve data conflicts -- will keep petabyte problems of cost, error and time under control.
After developing all these ideas, here are some of Fred Hapgood's conclusions.
Many observers think the transition to petabyte levels is going to introduce changes even more sweeping than those associated with previous leaps in storage. "Traditionally vendors have built standalone data mining engines and moved the data into them," says Richard Winter, president of the Winter Corp., a Waltham, Mass-based consultancy specializing in the architecture of very large databases. "But are you going to be able to move a petabyte around like that?" Winter foresees radical changes in engine architecture, probably involving breakthroughs in the engineering of parallelization.
I'm pretty sure he's right on this.
[And] Scot Klimke, vice president and CIO for Network Appliance, a storage services vendor in Sunnyvale, Calif., argues that as the petabyte revolution picks up steam, the struggle to measure and manage data quality will increasingly define the CIO's job. While he might or might not be right about this specific point, it's clear that anyone exploring the petabyte world should bring a good map, watch out for booby traps and carry a rabbit's foot for luck.
For more details, please read the full article.
Source: Fred Hapgood, CIO Magazine, October 15, 2002 Issue
6:07:46 PM Permalink
|
|
