{
"attachments": [],
"comments_archived": true,
"date": "2004-08-05T13:04:56.000Z",
"excerpt": "It's a new feed aggregator, my third attempt at such. Everything is clunky and command-line driven at present--but I've got further plans, like a REST API for feed queries and manipulation of various things such as feed subscriptions and the read/unread state of items. Pair this with an XSLT-driven browser UI, and the possibility of other clients (not the least of include other Atom-consuming aggregators).\n\nThe goal is to make a Client/Server Aggregator.",
"layout": "post",
"tags": [
"syndication",
"python"
],
"title": "Introducing dbagg3, an Atom-powered client/server aggregator",
"wordpress_id": 536,
"wordpress_slug": "introducing-dbagg3-an-atom-powered-clientserver-aggregator",
"wordpress_url": "http://www.decafbad.com/blog/?p=536",
"year": "2004",
"month": "08",
"day": "05",
"isDir": true,
"slug": "introducing-dbagg3-an-atom-powered-clientserver-aggregator",
"type": "entry",
"postName": "2004-08-05-introducing-dbagg3-an-atom-powered-clientserver-aggregator",
"html": "Update: I've just dumped what code I have into my CVS repository. So, go ahead and poke fun at it:
\n\nOr, fetch it from CVS:
\n$ cvs -d:pserver:anoncvs@www.decafbad.com:/cvsroot login\n(Logging in to anoncvs@www.decafbad.com)\nCVS password: anoncvs\n$ cvs -d:pserver:anoncvs@www.decafbad.com:/cvsroot co dbagg3\n
\n
\nSo, besides the funk, there's a little project in which I've gotten immersed. Here's a teaser diagram:
\n![](\"http://www.decafbad.com/2004/08/dbagg3-demo/dbagg3-phase1.jpg\")
\n\nIt's a feed aggregator, my third attempt at such. At present, things are roughly close to the diagram above. Everything is clunky and command-line driven at present--but I've got further plans, like a REST API for feed queries and manipulation of various things such as feed subscriptions and the read/unread state of items. Pair this with an XSLT-driven browser UI, and the possibility of other clients (not the least of include other Atom-consuming aggregators).
\nThe goal is to make a Client/Server Aggregator. Somewhat serendipitously, I just caught Bob DuCharme's xml.com article on Amazon.com's web services, which I really like and have drawn inspiration from in thinking about this new aggregator. Eventually, I want to offer the same sort of XML+XSL style of service that they've put together, along with some futher inspiration from the Atom API.
\nAnyway, as another teaser, check out this early demo involving the XML this thing has been producing, coupled with some experimental XSL:
\n\nYou'll notice that there are three URLs involved here:
\n\nThe XSLT expects Atom (plus a few extensions of my own), so you can also do things like this:
\n\nThese are the sorts of tricks I was looking forward to when I started thinking about things like a universal blog transfer format and rose-colored glasses. XSLT used like this could just as easily produce a blog or RSS 2.0 content.
\nAnyway, hoping to get some code into CVS by this weekend that's not entirely embarassing. So if you're interested in this stuff, stay tuned. I'm hoping that this thing can provide a base for others interested in feed aggregation--if you just want to play with UI, use the scanning and storage as-is and tinker with XSLT; if you want to play with storage and query, leave the scanning and UI alone; if you want to tinker with parsing... er, talk to Mark Pilgrim.
\nMore soon!
\n(Oh yeah, and I will be working on coming up with a better name than dbagg3. Unfortunately, I probably won't be coming up with a more visually appealing design for the UI, since what you see is the best I can do. Heh, heh. Don't let your programmers do visual design...)
\n\n\n\n",
"body": "**Update**: I've just dumped what code I have into my CVS repository. So, go ahead and poke fun at it:\r\n\r\n * \r\n\r\nOr, fetch it from CVS:\r\n\r\n $ cvs -d:pserver:anoncvs@www.decafbad.com:/cvsroot login\r\n (Logging in to anoncvs@www.decafbad.com)\r\n CVS password: anoncvs\r\n $ cvs -d:pserver:anoncvs@www.decafbad.com:/cvsroot co dbagg3\r\n\r\n* * *\r\n\r\nSo, besides the funk, there's a little project in which I've gotten immersed. Here's a teaser diagram:\r\n\r\n\r\n\r\nIt's a feed aggregator, my third attempt at such. At present, things are roughly close to the diagram above. Everything is clunky and command-line driven at present--but I've got further plans, like a REST API for feed queries and manipulation of various things such as feed subscriptions and the read/unread state of items. Pair this with an XSLT-driven browser UI, and the possibility of other clients (not the least of include other Atom-consuming aggregators).\r\n\r\nThe goal is to make a [Client/Server Aggregator][clientserveragg]. Somewhat serendipitously, I just caught Bob DuCharme's xml.com [article on Amazon.com's web services][amazonrest], which [I really like][wishes] and have drawn inspiration from in thinking about this new aggregator. Eventually, I want to offer the same sort of XML+XSL style of service that they've put together, along with some futher inspiration from the [Atom API][atomapi]. \r\n\r\nAnyway, as another teaser, check out this early demo involving the XML this thing has been producing, coupled with some experimental XSL:\r\n\r\n* [dbagg3 demo page][dbagg3demo]\r\n\r\nYou'll notice that there are three URLs involved here:\r\n\r\n* [An XSLT processor service][dbagg3xsltproc]\r\n* [An XSLT stylesheet][dbagg3xsl]\r\n* [Some aggregated Atom feeds][dbagg3xml]\r\n\r\nThe XSLT expects Atom (plus a few extensions of my own), so you can also do things like this:\r\n\r\n* [Dive Into Mark][mark]\r\n* [Intertwingly][sam]\r\n* [0xDECAFBAD][me]\r\n\r\nThese are the sorts of tricks I was looking forward to when I started thinking about things like a [universal blog transfer format][blogxfer] and [rose-colored glasses][roseglasses]. XSLT used like this could just as easily produce a blog or RSS 2.0 content.\r\n\r\nAnyway, hoping to get some code into CVS by this weekend that's not entirely embarassing. So if you're interested in this stuff, stay tuned. I'm hoping that this thing can provide a base for others interested in feed aggregation--if you just want to play with UI, use the scanning and storage as-is and tinker with XSLT; if you want to play with storage and query, leave the scanning and UI alone; if you want to tinker with parsing... er, [talk to Mark Pilgrim][ufp].\r\n\r\nMore soon! \r\n\r\n(Oh yeah, and I *will* be working on coming up with a better name than `dbagg3`. Unfortunately, I probably won't be coming up with a more visually appealing design for the UI, since what you see is the best I can do. Heh, heh. Don't let your programmers do visual design...)\r\n\r\n[atomapi]: http://www.atomenabled.org/developers/api/atom-api-spec.php\r\n[urchin]: http://urchin.sourceforge.net/\r\n[ufp]: http://www.feedparser.org/\r\n[roseglasses]: http://www.decafbad.com/blog/2004/05/03/put_on_your_rsscolored_glasses_and_forget_about_atom\r\n[blogxfer]: http://www.decafbad.com/blog/2004/05/17/use_atom_for_a_universal_blog_transfer_protocol\r\n[me]: http://www.decafbad.com/2004/08/dbagg3-demo/xsltproc.cgi?xsl=http://www.decafbad.com/2004/08/dbagg3-demo/new.xsl&xml=http://www.decafbad.com/blog/atom.xml\r\n[sam]: http://www.decafbad.com/2004/08/dbagg3-demo/xsltproc.cgi?xsl=http://www.decafbad.com/2004/08/dbagg3-demo/new.xsl&xml=http://www.intertwingly.net/blog/index.atom\r\n[mark]: http://www.decafbad.com/2004/08/dbagg3-demo/xsltproc.cgi?xsl=http://www.decafbad.com/2004/08/dbagg3-demo/new.xsl&xml=http://www.diveintomark.org/xml/atom.xml\r\n[dbagg3xsltproc]: http://www.decafbad.com/2004/08/dbagg3-demo/xsltproc.cgi\r\n[dbagg3xsl]: http://www.decafbad.com/2004/08/dbagg3-demo/new.xsl\r\n[dbagg3xml]: http://www.decafbad.com/2004/08/dbagg3-demo/demo.xml\r\n[awsrss]: http://www.decafbad.com/2004/03/xml-rss091.xsl\r\n[dbagg3demo]: http://www.decafbad.com/2004/08/dbagg3-demo/xsltproc.cgi?xsl=http://www.decafbad.com/2004/08/dbagg3-demo/new.xsl&xml=http://www.decafbad.com/2004/08/dbagg3-demo/demo.xml\r\n[clientserveragg]: http://www.intertwingly.net/wiki/pie/ClientServerAggregator\r\n[amazonrest]: http://www.xml.com/pub/a/2004/08/04/tr-xml.html\r\n[wishes]: http://www.decafbad.com/blog/2004/06/16/wishofthemonthclub1\r\n\r\n\r\n ",
"parentPath": "./content/posts/archives/2004/2004-08-05-introducing-dbagg3-an-atom-powered-clientserver-aggregator",
"path": "2004/08/05/introducing-dbagg3-an-atom-powered-clientserver-aggregator",
"thumbnail": "http://www.decafbad.com/2004/08/dbagg3-demo/dbagg3-phase1.jpg",
"needsBuild": true,
"prevPostPath": "2004/07/29/kibo-kibo-kibo",
"prevPostTitle": "Kibo Kibo Kibo",
"nextPostPath": "2004/08/05/dbagg3cvs",
"nextPostTitle": "dbagg3 code in CVS"
} ![](\"http://www.gravatar.com/avatar.php?gravatar_id=e7368d797917d0df7942c26d4ed2e580&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\"/)
\r\n Benjamin\r\n This is great. It seems that this could be the base, for what I've been dreaming/thinking about the last half year or so. In contrast to you, I haven't come up with any code yet (mostly due to the fact, that I don't python yet, and I found no other reasonably good feed parser, than the Universal Feed Parser), but with a name: Roosster \r\n\r\nFeatures I'd really like to have in such a ClientServerAggregator are:\r\n(a) search over entries the aggregator has seen to date\r\n(b) add and arbitrary number of notes to any entry\r\n(c) add multiple category-tags to entries\r\n(d) interlink entries (I know this could be solved by some 'Related Entries' stuff, but somtimes I don't trust these \"automatic classifiers\")\r\n\r\nCan't wait, until I can take a look at the code and see if I can extend it. This would finally be a real reason to learn python.\r\n\r\nBy the way, I really like the interface. Might be, that's because I'm a programmer too?!\r\n \r\n \r\n \r\n ![](\"http://www.gravatar.com/avatar.php?gravatar_id=e1cc8e7103bb0f77a7d7abf91b1a961b&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\"/)
\r\n Christopher Drake\r\n Very nice. The demo has me salivating a little.\r\n\r\nOddly enough, I like the UI. If you can slim it down, side-to-side-wise a bit, it would also be a great aggregator to rip whole chunks from and feed to a Palm Pilot or PocketPC, as well. They're not as big as they were in the past, but with decent strides every couple of weeks, they still have a lot of following. Your page-width is dynamic, and that means a lot to me, a Palm user.\r\n\r\nA feature I'd like to see is kind of like Bloglines, where you receive a small box of info from the RSS feed, or what-have-you to give you a taste of an article. If going 1 link deep from any link on the aggregator page linked to a small page with such a tidbit, it would make surfing the articles a lot faster. This would allow a PDA or cellphone to drag an aggregator page 1 link deep with specialised software and get all of the highlights from say, 20 blogs.\r\n\r\nThat could be a major boon, especially for people that connect while at wireless access points on business trips, etc.\r\n\r\nKeep it up, this looks like it'll rock when you are done.\r\n \r\n \r\n \r\n ![](\"http://www.gravatar.com/avatar.php?gravatar_id=0038a1e9f4492e2f964f0f39d2d0dc7f&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\"/)
\r\n Christoph Görn\r\n Hi,\r\n and I will try to plug an jabber/XMPP bot into it, so it will deliver the aggragated content rigth to my beloved IM app...\r\n \r\n \r\n \r\n
Archived Comments
\n\n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=e7368d797917d0df7942c26d4ed2e580&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nBenjamin\n \n\n\n \n\n\nThis is great. It seems that this could be the base, for what I've been dreaming/thinking about the last half year or so. In contrast to you, I haven't come up with any code yet (mostly due to the fact, that I don't python yet, and I found no other reasonably good feed parser, than the Universal Feed Parser), but with a name: Roosster \nFeatures I'd really like to have in such a ClientServerAggregator are:\n(a) search over entries the aggregator has seen to date\n(b) add and arbitrary number of notes to any entry\n(c) add multiple category-tags to entries\n(d) interlink entries (I know this could be solved by some 'Related Entries' stuff, but somtimes I don't trust these \"automatic classifiers\")\nCan't wait, until I can take a look at the code and see if I can extend it. This would finally be a real reason to learn python.\nBy the way, I really like the interface. Might be, that's because I'm a programmer too?! \n\n\n \n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=e1cc8e7103bb0f77a7d7abf91b1a961b&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nChristopher Drake\n \n\n\n \n\n\nVery nice. The demo has me salivating a little.\nOddly enough, I like the UI. If you can slim it down, side-to-side-wise a bit, it would also be a great aggregator to rip whole chunks from and feed to a Palm Pilot or PocketPC, as well. They're not as big as they were in the past, but with decent strides every couple of weeks, they still have a lot of following. Your page-width is dynamic, and that means a lot to me, a Palm user.\nA feature I'd like to see is kind of like Bloglines, where you receive a small box of info from the RSS feed, or what-have-you to give you a taste of an article. If going 1 link deep from any link on the aggregator page linked to a small page with such a tidbit, it would make surfing the articles a lot faster. This would allow a PDA or cellphone to drag an aggregator page 1 link deep with specialised software and get all of the highlights from say, 20 blogs.\nThat could be a major boon, especially for people that connect while at wireless access points on business trips, etc.\nKeep it up, this looks like it'll rock when you are done. \n\n\n \n- \n
\n\n![](\"http://www.gravatar.com/avatar.php?gravatar_id=0038a1e9f4492e2f964f0f39d2d0dc7f&size=32&default=http://mediacdn.disqus.com/1320279820/images/noavatar32.png\")
\nChristoph Görn\n \n\n\n \n\n\nHi,\nand I will try to plug an jabber/XMPP bot into it, so it will deliver the aggragated content rigth to my beloved IM app... \n\n\n \n
\n\n\n\n
\n
\n