Sunday, September 12, 2004

Popularity doesn't scale I could have quoted from any of a numbers of posters about the RSS non-scaling non-controversy, but Dare offers a round-up of the bleating masses The RSS Sky is Falling...Again.This is becoming a broken record. Every couple of months some web site that hasn't properly prepared for the amount of bandwidth consumed by having a popular RSS feed loudly complains and the usual suspects complain that RSS is broken. [Dare Obasanjo aka Carnage4Life] Sure, RSS doesn't scale. It doesn't not scale either. All of the problems with scaling have thus far been shown to depend on nothing particular to RSS. Scoble attemps to show RSS not scaling a) generating a gedankenexperiment where hits on the RSS are a fixed factor greater than HTML hits, and b) by quoting figures that the bandwidth consumed by RSS requests (for what site I'm not sure) per month is growing at a much faster rate that bandwidth for HTML. The former shows exact parity in scaling between RSS and HTML, while the latter fails to factor in the relative popularity of RSS versus HTML readers. If new traffic for RSS is N times new traffic for HTML, but new requestors of RSS are 10 times new requestors for HTML , this is also parity in scaling between RSS and HTML. There are attempts to blame the automated nature of  RSS requests, but HTML can be and is requested by machine as well. Either way, requests at regular intervals may affect the absolute value of the traffic but bear no relation to the scaling of the traffic growth. Certainly Amdahl's law would tell you that if RSS is a significatly larger portion of  your traffic that it may be worth while applying some RSS specific fixes. However, the discussion is not really about a scaling problem in the sense that the proposed solutions do not affect scaling. If requests are still growing exponentially, reducing response size may delay the point at which you hit any particular traffic level but it cannot prevent it. This is independant of  what the response is, and also independant of the absolute size of the response. 10:42:15 AM     Categories: Pushing rectangles... Radio

Sunday, August 08, 2004

Radio Userland aggregator eating it's own entrails I just saw someone posting this problem on the radio discussion boards that I was also having. I responded thre, but I'm copying the information here for dispersal. The situation was that the aggregator in Radio Userland was failing to pick up any new items out of any subscribed RSS. No error message was thrown up in any obvious location, other than the "0 new stories" in the log. I found that in aggregatorData.root every subscribed feed had an error message that flChanges was undefined. Digging in, I discovered a strange change in xml.rss.readService. The "local (flChanged=true)" that sets the default value for flChanged had moved to the bottom of the script. flChanged is used in the code to prevent processing a feed if it has not changes since the last download. The checks for Etags and such clear it to false, so the default value must be set before these checks are done. If not, the conditional that surrounds teh compileService call sees an undefined value and trows an error. If you are comfortable with editing the scripts in the Radio database, you can open xml.rss.readService and drag the "local (flChanged=true)" at the bottom up to the top of the code. 10:44:36 AM     Categories: Radio

Sunday, August 01, 2004

Compiling a service in Radio after a 301 I was having an error for the longest time with the viewRssBox on the right of my Radio site, but until now couldn't be bothered to look at it. I was actually taking out the blogroll but commented the wrong thing by mistake. In the debugging process I discovered a weird interaction with caching and 301 URLs. An HTTP asside: 301 is the HTTP numeric response for "Permanently  Moved". It is an indication that you should never use this URL in the future but instead always go directly to the refered URL. I had been refering to my LiveJournal RSS as "htttp://username.livejournal.com", which will 301 to "http://www.livejournal.com/users/username/data/rss". In Radio Userland, xml.rss.readService caches the RSS that it reads as a table at @user.xml.rss.cache.[url]. Somewhere along the line there's confusion about that URL as the 301 changes it from one to the other. What I think is happening is that in xml.rss.readService where redirectInfo is checked the table.rename to the new URL fails because the table from the last time in still there in the cache. readService probably worked the first time and only failed from the second on. I have correctly chaned the URL to that refered to by the 301, like a good little automaton, but that table.rename ought to be dealt with as well. 8:24:52 PM     Categories: Radio

Saturday, March 27, 2004

Now you know Be happy!. Matthew Ernst: content:encoded makes me sad: People that include the full text in their RSS 2.0 feed instead of just an automatically truncated "excerptoid" make me happy. However, including the full text in a while still keeping the exceptoid in the makes me sad. I'm not even sure what the motivation is to having , since as far as I know you can put that sort of thing in an RSS 2.0 just fine. You certainly can put anything you want in an RSS 2.0 . That doesn't mean you should. I think it's a great thing that so many people are reading blogs in RSS aggregators these days, and as a result agitating to have more people do full-content feeds. However, we need to not lose sight of the fact that reading in an aggregator isn't the only use for RSS. My blog is syndicated (in the original sense of Really Simple Syndication, publishing on another website) in a couple of places that don't really make use of a hand-crafted excerpt, but if you look at Henrik Gemal's Mozilla-related blogs page, and hover over the links, you'll see that some people have short ugly chopped off posts from Movable Type's auto-excerpts, and some people have slightly longer ugly chopped off posts from Gemal's own auto-excerpting, but a few people have a perfectly sensible, readable and understandable excerpt, because they put one in for things that syndicate, along with for people who are using RSS for reading. It's a marvelous combination, having two elements for each of two different uses (three, if you count people who have intentionally built their aggregator to only show them a short description because they would rather read posts in the original web page), and I only hope that Radio includes Matthew's code or something like it (just like it did with his code that now allows him to talk about those tags with HTML entities without double-decoding them and interpreting them) so that Radio users can enjoy full content in their aggregator without forcing people who are also providing their feed to syndicators to not provide them what they want. [phil ringnalda dot com] 2:00:55 PM     Categories: Fifteen Radio

Wednesday, March 17, 2004

Which way does it point? So Tim Bray gets a little careless with with is programming, and commits an uneven number of encodes() which screws up his RSS. Then he blames RSS. Huh? <’s Pointy End. Dave Walker over at freeform goodness catches me with my XML pants, figuratively speaking, down. I wrote a piece about leaving the W3C TAG entitled (cleverly I thought) . Unfortunately that < in the title caused all sorts of grief and breakage, both here at ongoing and downstream in the world of syndication and aggregation. I can fix my own problems, but it’s deeper downstream; long term, the answer is Atom. Herewith some thoughts on good programming practices and the larger problem. [Update: A couple of notes on the “href problem.â€�]... [ongoing] If you read the full article, you'll note that: a) he freely admits that it was his own programming error that produced the problem in his RSS; and b) he never actually explains how using Atom prevents you from making a programming error like that. I'll be the first to admit that I have no idea whether Atom really prevents that problem, however I'm also not the one claiming that it does. I'll also freely admit that I do have a Unicode problem in here somewhere...8-( 10:01:09 PM     Categories: LiveJournal Radio

Friday, March 05, 2004

Content:encoded makes me sad People that include the full text in their RSS 2.0 feed instead of just an automatically truncated "excerptoid" make me happy. However, including the full text in a while still keeping the exceptoid in the makes me sad. I'm not even sure what the motivation is to having , since as far as I know you can put that sort of thing in an RSS 2.0 just fine. So, I finally got around to doing something about it. In Radio, I've gone into xml.rss.compileService and added this bundle: bundle { //use content:encoded if possible try{contentEncoded=xml.getAddress(item,"encoded")}; if defined(contentEncoded) { cdata=decode(contentEncoded^.["/cdata"]); if sizeOf(cdata)>sizeOf(description) { description=cdata}}} I'm sure this is a completely unsafe way of grabbing a out of the xmlstruct, but hey... it works so far! 8:56:36 PM     Categories: Radio

Thursday, August 28, 2003

Catching up Paolo notes a problem with Yahoo's feeds, announced last night. They flow a huge amount of stuff, even if you just subscribe to one feed, there might be dozens of new stories each hour. Maybe this is just something to get used to. I remember the NY Times felt strange that way too at the beginning, now it's very normal to get several dozen stories in one update. There's another concern, linkrot. Links into Yahoo News rot relatively quickly, as compared to News.Com, for example, which is near perfect. So, until I hear something has changed, I'm going to use Yahoo to read news but try hard not to point to stories on Yahoo from my weblog. [Scripting News] The number one feature I've wanted to handle this situation is "Catch up on this feed". You can set Radio to have all the delte boxes checked by default or unchecked by default. However, whether I am most likely to want them checked or unchecked really depends on the type of feed. For these straight news kinds of feeds, I would typically want scan the headlines an select a few to save while discarding most. Other feeds I want to keep until I've read through. In addition, sometimes a feed just goes wonky, and I get a large quantity of old articlesshowing up from teh feed again. Sure, you could spend a lot of time trying to debug why they didn't match agains the record of articles already deleted, but no one is really that interested and it's going to be different every time. There are even good reasons why the articles would show up again in a way that couldn't be matched. "Catch up on this feed" is less elegant, but less effort too. An even better feature, which I think might actualy be easier considering the way that Radio does things, would be "Catch up on this contiguous block from this feed" since the aggregator keeps things organized by reverse time order. A batch of new articles that were dowloaded together will show up together, but the batch from the last download will be further down the page (or even a different page). And if I were half the programmer that people think I am, I would just write it myself. But I'm not. 7:55:45 AM     Categories: Radio

© Copyright 2004 Matthew Ernest. Last update: 9/12/2004; 10:42:56 AM. hT This theme is based on the SoundWaves (blue) Manila theme.

radaR's LiveJournal   8/30/2004 Geek gamers rejoice. Finally, a hot chick they can score with. Video game vampire to go topless in October Playboy Ernest Miller sez, "Videogame character BloodRayne (a red-headed 'Dhampir' who hunts supernatural baddies for the 'Brimstone Society') will be topless in October's Playboy. According to her creators, 'This is a first in videogame history and trust us when we say that Rayne does not disappoint.'" Link (Thanks, Ernest!) [Boing Boing] By radar@poboxes.com. Keeping America Safe from wireless internet. AKMA gets hassled by the man: So Weirdly Wrong: And I walked back to the studio, dumbfounded that someone just rousted me for picking an open wireless signal in public — indeed (as it turns out) for using a laptop within a wireless signal’s range of the library. Weird. We should all be glad that the local contstabulary are able to invent federal laws at a moments notice to save us from the scourge of freely available internet access. However, those godless commie librarians seem to have gotten off scott free. By radar@poboxes.com. Keeping America Safe from Ted Kennedy. Ted Kennedy's name is similar to an alias of some "evil doer". Proof that this list only catches the innocent is left as an exercise for the reader. If Senators are allowed to roam freely about the country, then the terrorists have already won! Reuters. Kennedy -- one of the most recognizable figures in American politics -- told a Senate committee hearing on Thursday he had been blocked several times from boarding commercial airline flights because his name was on a "no-fly" list intended to exclude potential terrorists. [John Robb's Weblog] By radar@poboxes.com. Any sufficiently nice person is indistinguishable from someone who likes you. Yet again, I find I have underestimated just how deep into pathetic geekdom I am when confronted by others of the species. We're pointed that this damning refelction by Joey de Villa, an actual example of the cool geek that the rest of us pretend that we could possibly be but really can't. I don't know how I ended up looking at a page in Everything2 (imagine a less academic Wikipedia written by LiveJournalers), but someone has come up with a geek lament treatment of the Clarke Axiom: Any sufficiently nice person is indistinguishable from someone who likes you. [The Adventures of Accordion Guy in the 21st Century :: Joey deVilla's Weblog] By radar@poboxes.com. fun Fun FUN. Whee... isn't working late fun? Someone bring be some dinner, okay? By radar@poboxes.com.