Updated: 10/18/2005; 9:42:41 AM

  Thursday, December 11, 2003

Brain dump on discovery infrastructure for personal content

Holy Sh*t!

This means that we need a subscription signalling infrastructure for the three fundamental methods of communication:

  • One to one.
  • One to many.
  • Many to one.

Content delivery uses:

One to one is email. One to many is RSS. Many to one BitTorrent.

So now we need a global infrastructure like we have for DNS, right? Signaling in a standard format what's out there for me, where it is, and how to get it. My aggregator knows how to get it because the infrastructure tells it where it is.

So why the expletive? Well, we've got all of the delivery methods in place, so let's stop trying to reinvent them and instead reinvent the way we dicover our own content. Home servers are coming closer to a reality, so we can store the content we have for others locally. We can backup content for us locally or leave it on the place where it came from, getting it when we want to, not when we have to.

(ever been so excited about an idea that you can't type fast enough?)

TiVo does this to some extent.

Think TiVo meets email meets BitTorrent meets iTunes Music Store meets personal publishing. That's the client.

Think DNS as the infrastructure. I need "yahoo.com", where is it? My local DNS server sends me to the root and that sends me to Yahoo!'s DNS server. It returns the IP for the web/ftp/email/IM server. DNS is our content infrastructure now, we're just not using it for that now.

Enter your DNS servers in a Network dialog box to make your Internet access make sense. No more IPs for you! Just those simple to remember names. So now, open your aggregator and enter the content directory servers. Oops! Can't do that!

Apple/next used to do this with NetInfo, but for local resources. Now they are slowly redoing it with Rendesvouz.

Wow... brain empty for now. Man, this is exciting...

My comment on Dave's Rant

I'm reposting this here from comments on this post:

All:

Great comments so far. Very interesting discussion. Let's explore some other real-life situations where things needed to change but the situation didn't.

Stop signals:

Two roads cross creating an intersection. There is a shared goal for all intersection users: allow free flow of traffic but minimize the potential for accidents. First, stop signals were whistles, but later semaphores and then finally colored lights. What hasn't changed? Why, it's the color of the lights, folks. Red means stop, weather it's on a static sign or a colored light. The mechanism has changed over the last 100 years, but the concept is the same.

Most importantly, the shared goal of traffic control hasn't changed. The world of RSS/Atom/content-syndication does not have a shared goal. Therefore, there cannot be a universally accepted method of "signalling" a way to subscribe to a content feed.

Read my traffic signalling source site: Dave's Traffic Signal Page

The last paragraph contains a lesson for us all: "However, these early electric traffic signals provided little flexibility in traffic coordination as compared to the observations of a police officer. This led to the development of traffic signal coordination. The first coordination development was the simultaneous traffic signal system which was installed in Houston, Texas in 1922. In this system, all the traffic signals on the main roadway would change to green at the same time. After that, the next coordination development was the alternate traffic signal system which was installed in the District of Columbia in 1926. The alternate system is the type of coordination system that is used today."

Mulitple systems that do the same thing don't offer the user the flexibility to coordinate traffic (subscribed content). We need a coordinated development with a shared goal. Not a goal that everyone likes, a shared goal. Focus on the shared goal and the traffic lights will turn green.

Comments are golden

There's a debate in comments about a Dave Winer 'rant' on a contest to rename RSS. This nugget was in the comments:

Saying that the only way to get content is to have aggregators pulling it is missing a whole other way of doing it, that's proven itself as scalable and resilient over the years: NNTP

I'm not talking about using NNTP directly but taking ideas from it. Having a network of servers that talk to each other, where items propagate between servers, and where users talk to a single server that's close to them instead of reaching all over the world to poll, makes a lot more sense.

Rather than having a million aggregators hitting 20 or 30 sites an hour each, those million aggregators would talk to server, and the servers would talk to each other, so that your aggregator client would simply talk to an "RSS Server" near you to pick up all the things you subscribed to.

No polling involved - at least, not polling of the sites themselves.

If email was implemented the way RSS is implemented, then my mail client would be polling all my friends every half hour to see if they've sent me an email. Nobody would implement email that way - but that's what RSS is.

SMTP moves mail from the author to a server close to the user, in the background, maybe involving multiple servers in between. Long term, I think this is going to have to happen with RSS as well. Steve Tibbett [apple] 12/9/03; 5:07:54 PM

My favorite part is the sentence about email working like RSS. Classic.