rssCloud Blog

VizWorld asks: “We’ve been using the rssCloud plugin for a while. How do we know if it’s working?”

As with all questions asked on Twitter, it can be hard to know what someone means if all they had was 140 chars to express the question.

So I’m assuming they have WordPress and have installed the rssCloud plug-in.

The first thing to check is the feed. Here’s the feed for my WordPress site, unberkeley.com. To see what’s in the XML you may have to View Source (in Firefox it’s the Page Source command in the View menu). Look for a <cloud> element there. If you see it, then it’s probably working. Here’s a screen shot of what it looks like.

However, the real test of whether it’s working, is to subscribe to it in a realtime-aware RSS reader. There are a few of them, including my own River2. I could and probably should put up an app that tests a feed. I just put it on my to-do list.

Update: I checked out VizWorld’s feed, and it does have a cloud element.

I know programmers aren’t into marketing that much, but someday someone will ask why you’re so darned excited about Realtime RSS and related technologies. For that day you might want to keep a link to this piece around:

12/29/09: Why it’s smart to publish Realtime RSS now.

Dave Winer
Berkeley CA

Over the holidays I’ve been busily putting together the XML protocol for CloudPipe and developing a client and server running in the River2 aggregator.

It’s all been deployed, with examples and a public server for developers to test against.

I documented my work on Scripting News.

There has also been a proposal from Chuck Shotton that the fatPing element have an optional type attribute. I’m listening to hear what other developers think.

The project is nearing completion of its initial phase. You should now have all the pieces to evaluate and to develop compatible software.

I’m especially interested in seeing large-scale implementations of this protocol.

I think the Tornado environment, the core software for FriendFeed, would be especially interesting. If someone wanted to create a CloudPipe server in Tornado, I’d contribute a Linux server on EC2 or Rackspace to run it. It’s supposed to scale really well.

I also included source code listings for my client and server. You can see that if you have a good XML library there isn’t much code to write.

This is the last major piece of the rssCloud architecture, the only piece not specified in 2001.

With Cloud Pipe, it will be possible to flow realtime updates from the cloud to desktop and mobile devices, even if they are not “net-accessible,” that is, even if they are behind a firewall or NAT.

The technology here is not new, in fact the design is modeled after the realtime part of the FriendFeed API. I chose this approach because: 1. It’s known to work. 2. It’s easy to implement. 3. The server-side to scale. 4. Many other developers are familiar with it. 5. There aren’t many ways to solve this problem.

See rsscloud.org for a technical narrative and discussion.

I’m a big fan of Amazon’s web services, especially EC2. So many cool things about it, but perhaps the coolest is that I get to create my own version of Windows and offer it to others. How they got Microsoft to go along with this, I don’t know, but it’s great.

Last night Amazon announced a new feature for EC2 that’s hard to explain, largely because it makes EC2 work the way you always thought it should. Even so, it’s an important development, so it’s worth trying to explain, by analogy…

Suppose you went to Fry’s or Best Buy and bought a new laptop. You brought it home, set it up, installed all the software, Firefox, WinZip, Dropbox, etc, tweaked up the settings so it works the way you like it. This could take a couple of hours, but you do it once and that’s it. However, imagine if you had to do this every time you booted the system.

If it worked this way, you’d probably put all the custom stuff on an external drive, to keep the amount of work you have to do each time you reboot to a minimum. And you’d avoid rebooting at all costs.

What Amazon did this morning was eliminate the difference between the external drive and the boot drive. So now you can get your (virtual) computer set up the way you want it, stop it, and only restart it when you need it running. When it’s not running you don’t pay the CPU rates.

They also made starting much faster than launching the old style instances.

I put all this new stuff through its paces this morning and it worked as advertised. As with all EC2 services, the docs make it sound stranger that it really is. They invent all these weird terms that don’t help you understand what’s going on. But now that we know this, they’re not so intimidating, and the tutorials they provide work well, as does the service.

Coming soon: EC2-based rivers!

The timing of this release is very good for my work. I’ve been preparing a new version of River2 that allows users to create EC2-based rivers.

As with EC2 for Poets, it’s something a technical end-user can set up in a few minutes. And because of the way the new instances work, you won’t have to leave your river running all the time, you’ll be able to start and stop it whenever you like.

What does this mean? Well, previously the benefits of realtime RSS were only available to the few people who could setup and run servers. By offering an easy-to-launch EC2-based river, the number of people who can do this will shoot up by a huge factor. As I said, Amazon’s timing is very good.

After getting this release done, the next step is to make the EC2-based river multi-user so one instance can serve dozens of users. I think we’ll reach this milestone before the end of the year. Knock wood!

Eventually, when all this is done, we’ll have a loosely-coupled realtime network running completely outside the confines of any single company, all based on an open format and protocol, RSS and rssCloud.

Bing!

It would do more or less exactly what the twitter.com website does. Same prefs, same commands, same user experience. Think Apache for the Twitter user interface.

In my explorations of a hypothetical decentralized Twitter, at first I thought the clients would be where decentralization would happen. But, lately I’ve come to realize that it probably won’t happen there because as the market has evolved they’ve become too dependent on Twitter Corp, and are unlikely to do anything that might threaten a friendly relationship with the company.

I saw this first-hand in the Mac software market in the early 90s. Even when it would have been in the interests of developers to work with each other, each of them tried to do special deals with Apple. Of course no one really got those deals, so we all went down. But it’s human nature to think you’re special and if you play nice with the platform vendor, they’ll play nice with you. And the platform vendor may totally mean to be nice, but they can’t help acting in their own self-interest and that almost always is at the expense of the developers.

So if the commercial developers can’t or won’t break free of the platform vendor, let’s create an open source client that can be repurposed in as many different ways as we, as individuals want. Some of us may want to do deals with Twitter Corp, and that’s fine — but others may wish to embark on paths that are independent of Twitter. They wouldn’t try to guess what would make the platform vendor happy, and instead follow the grain of the Internet, or go where the users want to go, or some users, or to scratch their own itch. Some may want to be part of the Cathedral and others part of the Bazaar.

I’m not even 100 percent sure what I’m asking for, but I’ll know it when I see it. Probably a JavaScript framework that comes with a Twitter timeline object. So displaying a timeline is automatic as are the user interactions. So any kind of client, one written in any language — Python, Perl, Java, JavaScript, PHP, C, etc — could store data in it. It wouldn’t know anything about the Twitter API. It would be up to the applications to put data in the structure.

It would do more or less exactly what the twitter.com website does. Same prefs, same commands, same user experience. Think Apache for the Twitter user interface. It would, of course, be programmable through a user scripting language.

Having this one component would let a thousand flowers bloom in exactly the place where we need them to bloom. The key thing is to find out what would happen if we could take a path that was not designed to please the platform vendor. Note I carefully did not say “to piss off the platform vendor.” I really do mean to chart courses that are independent of the vendor.

What would be even cooler is if one of the client vendors decided to release their code under the GPL. Or, even better would be if Twitter Corp did it. That would be hugely disruptive and would likely lead to some very serious innovation.

Much discussion about the programmable Twitter client concept thanks to a link from Gruber. He may be a Yankees fan (boo) but he has a good eye for interesting tech stufff. And the man is a flow machine. :-)

Loic LeMeur of Seesmic says he had the idea first, but perhaps there was a misunderstanding. I want programming language built into Twitter clients to enable end-user scripting. I think Loic is talking about support for plugins. A different idea. Scripting enables a much wider group of people to customize their use of Twitter. Plugins are more heavyweight and writing and debugging a plugin is a much more serious undertaking that writing a script or macro.

Meanwhile Chuck Shotton has a great idea which works really well with the idea of the programmable client — a proxy server that talks to Twitter on your behalf. I love this idea. Many interesting possibilities.

Unix had a shell language. DOS had a batch language. Lotus 1-2-3 had its macro language. Emacs is a programming tool as much as it is a text editor. We have gotten out of the habit of making programmable end-user products, but they are still just as important today as they were a couple of decades ago.

Every few weeks Scoble and I have an hour-plus conversation about what’s on each of our minds. The last few years it’s been all about Twitter of course, just like everyone else (or so it seems).

Lately our conversations have been ending up in the same dead-end.

Seems all good ideas begin and end with a phrase like: Of course they’ll never do it.

The “they” in the conversation is Twitter Corp, of course.

But we kept talking, and Scoble re-stated an idea that he’s been promoting publicly called SuperTweets, which was more or less exactly the idea for RSS enclosures, which led to podcasting. I didn’t need to pitch anyone to make that one happen, so those were happier times.

In our dead-end brainstorming I think we’ve been making an incorrect assumption, that all would be good if Twitter would just become an extensible metadata platform, allowing any developer to latch any data they wanted on any tweet, with Twitter storing at least a pointer to the data with the tweet, if not the actual data.

Now I think this was incorrect, because it assumed there would be client developers who are creative or brave enough to work with one another without waiting for Twitter to tell them where to go. I actually think the developer space around Twitter has been so scared of Twitter Corp for so long that even if they knew exactly how to turn the market upside-down they’d never risk pissing off the mother ship. So I think there’s virtually zero percent chance of any disruptive stuff coming from the developers. And without disruption, TwitterSpace will remain moribund and stuck.

Unless…

Of course you’ve read the title of this post so you know what I’m going to say. :-)

What if there were a relatively simple and low-power programming language built into a Twitter client that allowed power users to build their own little apps on top of Twitter. User interfaces for grouping tweets, or flowing groups of ideas to two places, Twitter and somewhere else. So that the bits that end up on Twitter are coherent and useful to people who don’t use the client, but somehow more useful to those who do.

And on the reading side, I want to add features that not everyone will want, but perhaps more people than just myself. And I don’t want to have write a whole client just to have those features.

For example, it’s been about two years since I first asked for an “unfollow-with-timeout.”

Use-case: Someone is live-tweeting a conference I don’t care about and hogging up all my bandwidth. I want to unfollow them for just a day. Now I don’t think the client guys are going to implement this anytime soon, but it’s the kind of feature a few hundred people would kill for.

I’d also like a “block-with-timeout” feature.

Shouldn’t have to block someone to remove a single tweet from view. Can’t tell you how many times I’ve blocked people just to get a turd they sent to me out of my @replies tab.

So I suspect the answer to Scoble’s void is we need a programmable Twitter client.

PS: I’m sure some or most of the comments will be from people who say they don’t need one. Okay maybe they can skip the comment because I know that most people don’t want this, at least until someone ships a script that they can’t live without. :-)

Before you study the pie chart below, please take a careful look at the first word in the title of this post.

As you can see polling remains the most popular update method, but rssCloud is catching up fast. SUP has a respectable following, as does PubSubHubBub. But the real contest is between polling and rssCloud.

Please note that I made these numbers up, based on how popular I feel each of the protocols should be. I normally wouldn’t do this, but I saw a pie chart on Superfeedr that was created this way. So I figured if they could have fun with made-up numbers, so could I!

It’s kind of liberating for science to be so separated from fact. I just hope they’re a little more rigorous in debugging their software. :-)

Dave Winer

Before you study the pie chart below, please take a careful look at the first word in the title of this post.

As you can see polling remains the most popular update method, but rssCloud is catching up fast. SUP has a respectable following, as does PubSubHubBub. But the real contest is between polling and rssCloud.

Please note that I made these numbers up, based on how popular I feel each of the protocols should be. I normally wouldn’t do this, but I saw a pie chart on Superfeedr that was created this way. So I figured if they could have fun with made-up numbers, so could I!

It’s kind of liberating for science to be so separated from fact. I just hope they’re a little more rigorous in debugging their software. :-)

Dave Winer