Petabyte Disk Drives in Seven Years--What Does That Mean for You?

"So just how big is a petabyte drive and what could you put on it?

One certainty is that you will not fill the space with personal jottings or reading matter. In round numbers, a book is a megabyte. If you read one book a day for every day of your life for 80 years, your personal library will amount to less than 30 gigabytes. Remember a petabyte is 1 million gigabytes so you will still have 999,970 gigabytes left over.

To fill any appreciable fraction of the drive with text you’ll need to acquire a major research library. The Library of Congress would be a good candidate; it is said to hold 24 million volumes, which would take up one-fiftieth of your disk. So you could fit 50 Library of Congresses on your petabyte drive.

Other kinds of information are bulkier than text. A picture, for example, is worth much more than a thousand words; for high-resolution images a round-number allocation might be 10 megabytes each.

And this is being generous. Most images from a digital camera are one to four megabytes, not 10. How many such pictures can a person look at in a lifetime? I can only guess, but 100 images a day certainly ought to be enough for a family album. After 80 years, that collection of snapshots would add up to 30 terabytes. So your petabyte disk will have 970,000 gigabytes left after a lifetime of high quality photos.

What about music? MP3 audio files run a megabyte a minute, more or less. At that rate, a lifetime of listening--24 hours a day, 7 days a week for 80 years--would consume 42 terabytes of disk space. So with all your music and pictures for a lifetime you will have 928,000 gigabytes free on your disk.

The one kind of content that might possibly overflow a petabyte disk is video. In the format used on DVDs, the data rate is about two gigabytes per hour. Thus the petabyte disk will hold some 500,000 hours worth of movies; if you want to watch them all day and all night without a break for popcorn, they will actually fill up your petabyte drive if you have a lifetime of video on it as it will give you 57 years of video....

Still another nagging question is how anyone will be able to organize and make sense of a personal archive amounting to 1 million gigabytes. Computer file systems and the human interface to them are already creaking under the strain of managing a few gigabytes; using the same tools to index the Library of Congress is unthinkable.

Perhaps this is the other side of the economic equation: information itself becomes free (or do I mean worthless?), but metadata--the means of organizing information--is priceless.

The notion that we may soon have a surplus of disk capacity is profoundly counterintuitive. A well-known corollary of Parkinson’s Law says that data, like everything else, always expands to fill the volume allotted to it. Shortage of storage space has been a constant of human history; I have never met anyone who had a hard time filling up closets or bookshelves or file cabinets.

But closets and bookshelves and file cabinets don’t double in size every year. Now it seems we face a curious Malthusian catastrophe of the information economy: the products of human creativity grow only arithmetically, whereas the capacity to store and distribute them increases geometrically. The human imagination can’t keep up." [Mercola.com, via LibraryPlanet.com]

Picking up where Joi leaves off.....

So Joi and Marc are sitting at Inakaya in Roppongi, ordering robota-yaki in a theater-like atmosphere and discussing memes, happenings, white papers, public servers, RSS feeds, XML-RPC protocols and other techniques to implement the mesh. "But Joi-san, we CAN do it!  I know it sounds ambitious, but I'm tired of just linking - can't we go beyond that?  Can't knowledge, communication and personal expression be personified in other ways than just linking?"

"Yes - Marc-san I see what you're saying (only don't take all the asparagus - it's my favorite!)  Let's start with 'the Matrix: Reloaded' - since we just saw it, and it rocked my world - and I REALLY wanna tell everyone about it NOW - and not wait till I get home to blog it."  Marc then adds "Yes - Alice can post a photo or audio snippet, but that goes onto her private Aliceblog server - but the world has to know about AliceBlog or some blog related to AliceBlog - to find out about her feelings on 'theMatrix'.  Wouldn't it be cool if there was a public server - who's only job in life was to keep track of such 'topics'?"

"Oh, you mean the InternetTopicExchange" exclaimed Joi.  "Yes, but since we're entrepreneurs and inventors - let's imagine going beyond what Phil Pearson and Matt Mower have done so far."  "So what you're saying is that blogs are just ONE kind of micro-content - that a review can be another type..." says Joi.

Marc lunges for the grilled giant crab leg........"Yes" (he mumbles as he gobbles down the crab leg) "by establishing new forms of micro-content, and setting up registries to track those posts of these new micro-content types, we can create entirely NEW WAYS of connecting to each other.  So your review of the movie 'TheMatrix: Reloaded" or a review of this restaurant or of your new Hasselblad camera - can all reside in some place - that forms a central respository for reviews.  These reviews are placed into shared databases - forming a new kind of contributory community of interests." 

"Yes" exclaimed Joi, as he orders more Sake, "that respository doesn't have to house the actual content, but just the pointers to the reviews.  And we can have all sorts of these registries, shared databases, etc. - where tools, services and a myriad of systems can all ping, contribute to - and in general - implement the semantic web!"

"Bingo" Marc crys, as he also bellows along with the wait staff at Inakaya "He wants more kinki-fish!"

"So first we establish new micro-content types for reviews and conversations, then we create a topic registry to ricochet all querries and links to those topic names of these reviews, conversations and memes, and THEN we enable a level playing field with the notion of publicly available media and people profiles - all of which can be thought of as 'web services'."

"Boy is Idei-san gonna get off on this" Joi says.  "Yes and the powers that be, like Macromedia, Adobe, Ofoto, Microsoft, Pressplay, Yahoo, etc. are not really gonna like it." "But I bet Rob Glaser at Real - will.........."

"So a review could be posted from a cell phone, which is accessed via a public shared database of reviews and flowed through the blogosphere just like today's blog micro-content." "And someone's persistent digital ID can also be thought of as 'micro-content' as well, only with rules attached."  "As will 'legal content', such as music or movies, have 'rules' attached too."  "Or a conversation on spiratuality or an actual Passover Hadagah could become a shared public conversaion - bringing Palestinian and Israeli youth together."

"This is starting to get interesting....."  "We'll take some more Sake here, please......"

"So all sorts of new communities will form - neh?"

"Yes - this all doesn't have to be limited by the current blogosphere.  My buddy BigDave doesn't want to blog, but he has a digital lifestyle with every kind of device there is, a Home LAN, Broadband connections and plenty of people and memes he wants to help propogate.  He just doesn't want to blog."

"So he might be a member of a Grateful Dead community where they share taped performances of Dead concerts in a shared living room, and view art posters in shared gallery, and the same techniques we talked about for the blogoshere get utilized to enable this private Grateful Dead 'club'."

"And he might have a BMW M5 - which has another kind of community associated with it, where folks discuss new struts and engine blocks, and sell each other custom decals and videos."  "Or he might want to help his kid's school out - with fund raising efforts."  These new kinds of communities would all utilize public servers, private clouds, XML-RPC protocols, RSS feeds, new forms of micro-content, etc. - they're just not part of the blogosphere proper. They have their own "spheres".

"So let me get this straight - digital lifestyles will have private networks, community networks and public networks - all weaved together with a mesh of memes, micro-content and RSS feeds - connecting one's Home LAN and devices to the Web - via their always on broadband connections." recaps Joi. "That's certainly going beyond what Mr. Berners-Lee gave us!"

"Yes" says Marc. "Now you're starting to sound like a marketing guy......."

"OK - Marc-san" - now who's gonna pick up this $800 tab we just ran up for the two of us?