Problems with the Man

Adam Curry mentioned me again in his Daily Source Code program, and included some excerpts of my audioblog entry on RSS and the iPod platform. He included the part where I was sort of frothing at the mouth about how much I hope the RIAA machinations in getting the internet broadcasting pay-per-stream licensing pushed through bites them in the ass. My dream is that having shot their wad on that front, then a different mode of interaction comes along that is different and in most ways superior to the streaming they tried to squash, and they are helpless and forced to give in because they put all their eggs in the wrong basket. That’s the world I want. To clarify, I’m not against record labels in general, I’m against the heavy-handed RIAA tactics, suing of the fans who are their customers, trying end runs to get the government to protect their future-proof their business models from the advance of technology so they can sit on their piles of money without being bothered to change with the times, etc. He did correctly tag me, though, as “having problems with the Man in general.” I’ll admit to that.

The really interesting part is at the end when he suggest that Prince might be a good candidate for getting into this world. He is a massive generator of the “bundles of passion”, being both prolific and inspiring fiercely loyal fans. Adam suggests that he’s very smart and very wired, and points out that he is one of the innovators in the world of enabling downloads for music. That’s the other thing about all this, is that when we have the “killer app” then that will help the brush fire catch. All it takes is one feed that is the candy everyone wants, and this will really get rolling.

Changing the Caching Mechanism

I’m going to change the way the caching for the files works in get_enclosures. The way it works now is that when a file is downloaded, the current timestamp is saved. Before a file is downloaded, it is checked to see if there exists a timestamp for it. If so, it is not downloaded. I realize that this case is too simplistic, and I thought of a use case that would make this break while I was thinking of something else that I thought would be cool. But first, a digression.

In this talk of the “iPod platform”, for over two years now I’ve been saving the MP3 files from the WREK streaming archives off for specific shows. I would then burn them to CD and listen to them offline. I did this with custom scripts and Windows scheduled tasks. It occurred to me that this could easily be something that reused all this infrastructure. I realized that it would be quite simple to create a cron task that would write out an RSS feed with enclosures for the various programs on that station. Then, the get_enclosure script could just download them when it was doing its thing anyway.

Here’s where the mechanism described in paragraph 1 falls apart: every week, the URL to get the MP3 archive for that same half-hour of programming is the same. With the existing mechanism, that URL would be downloaded once and only once, the first time the script ran. All subsequent runs would find that URL as one that has already been downloaded. Damn, so close yet so far.

Here’s how that can be fixed, and how perhaps it makes things more robust in all cases. The RSS 2.0 spec defines (requires?) an element for the item, pubDate. I’ve altered the caching mechanism to use this value rather than the current timestamp. Then, when examining whether to get the file it checks the value contained in the pubDate of the item in the current feed versus the one in the cache. If the feed is newer than the cache, get it again. This allows for getting a file down like the WREK situation, where the file name and URL will be reused every week as the contents of the file are rewritten with the new week’s stream. When assembling the RSS 2.0 feed with the enclosure, the pubDate is set to the correct value for that week and everything will work out. Conceivably, this could also allow for redownloading of a file that was edited and republished with everything else the same but the pubDate updated to the new publish time. Because these are textual times, I wrote a simple function that compares two RFC 822 dates and finds out which is the earliest, so for the individual download URLs everything will be used, compared and stored with those dates from the item tag. There are better, more robust ways such as using Date::Manip, but I don’t want to require people to install any more modules than they already do. In fact, I might think about getting rid of the dependence on XML::Simple.

This updated mechanism will be part of the 0.2 release. As well, I will pick a WREK show or three to prepare these experimental feeds for. If they like it and want to do it, I’ll let them have it and they can put it on their own site.

The iPod Platform

This is a list of some of the points from yesterday’s audioblog entry. I’m not including a lot of the discussion, just the bulleted points. You can hear the full thing here.

Aspects of the “iPod Platform”:

  1. The units of submission are “bundles of passion” (my term rather than the vague and dry “content”). These are discrete things with a beginning and an end – a thing, where a stream is indefinite – a place. They are also by definition something you care about, either directly (choosing an individual entry to listen to from a web page) or indirectly (contained in a feed you subscribe to.)
  2. Data transimission is asynchronous, like using TiVo or buying a DVD of a TV show. You are not required to view or listen during a fixed transmission time.
  3. Automated (or at least automatable) and passive, so you aren’t waiting on it (no “Wait for rebuffering”) and that you don’t have to think about getting it. I do this in RadioLover where I set up my recordings, you already do this in TiVo or your VCR.
  4. With interests definable at levels of arbitrary granularity. I want this specific episode of a program, I subscribe to everything from this feed, I want any audioblog that discusses a given topic or any program with involvement of a given person. In other words, there needs to be a robust and flexible way of defining these interest.

On the issues of monetizing audioblogs/programs delivered passively via RSS:
I think the best place to start is where radio and TV began – the single sponsored program, with an advertiser/underwriter that attaches themselves to a program and funds it completely. When I listen to MP3s of old episodes of You Bet Your Life most of them were sponsored by Elgin America. You could have a sponsor or microsponsor that was willing to pay a given dollar value per unique download per sponsored show. In my example, I suggested that because my blog is heavily Apple specific, maybe a company like Ambrosia Software would be willing to sponsor me at something like a dime a download. With specified threshholds, maybe they pay $100 to start and every time the download numbers cross another 1000 episode downloads they are billed another $100. This could be even more complex, such as paying more for downloads from a given geographic area or network or something. The limits are only the creativity and the quality of data collection and analysis of your audience.

There is nothing revolutionary here, much the opposite. This is how I paid for Reality Break, back when Dueling Modems or Del Rey Books were underwriting production costs. This is tried and true, and the best part from the end of the producer or blogger is that you take one larger check from one party, rather than trying to collect micropayments from all your audience. The sponsor gets to attach themselves to whatever cachet you might have, and you get to rent out a piece of your attention, monetizing to some degree the attention economy. It’s the oldest possible model in broadcasting which is how we know it works.

iPodder for Windows

Via a comment, Pieter Overbeeke informs me that he has a script for downloading files and controlling iTunes for Windows XP available! This does the same stuff as get_enclosures or iPodder on the Mac, by getting the files and also adding them to the iTunes library.

In the shower this morning, I was wondering if there were COM libraries for Perl that I could use to control Windows iTunes from get_enclosurest. Now that Pieter has invented this wheel, there is no need. If you are on Windows you should definitely give his script a try. More infrastructure for the “iPod platform!”

Multiplatform it is!

I let the updated get_enclosures script run overnight on a Windows box with Active Perl installed, and it worked just fine. Right on! There were a few minor issues, but the files all came down, so that’s good.

I tried to test it against Cygwin’s version of Perl but couldn’t get the modules installed. I recently had to wipe and reinstall my Windows 2000 OS because Windows is such a fragile piece of shit that it eats itself over time, and when I did I had to start from scratch. This Cygwin is newly installed, and I get all kinds of make errors trying to install the LWP. It’s really weird because if I go into the build directories and manually run make it works. I’ve never seen this before on any Cygwin install. If some kind soul out there could test this script with Cygwin Perl and let me know how it goes, I’d highly highly appreciate it. I’m not planning on spending any time fighting with Cygwin.

Update: I did get it to work on Cygwin after all, and it seemed to work fine. If anyone has success with this on any other platforms, let me know.

Audioblog for August 25, 2004

The new audioblog entry is online. In this one, I talk solely about RSS, the “iPod platform” and monetizing the whole thing with the oldest damn idea in broadcasting but with better technology and data. Ignore the part at the beginning where I say that it will be short. I thought it would be when I started, but it wasn’t. No personal stuff in this one, just my shot at trying to define some terms and lay out some ideas.

get_enclosures version 0.1

Here is the updated version of get_enclosures, version 0.1. The zip now includes a changes.txt which covers the differences from the previous version. It now is no longer dependent on Mac::AppleScript, which means it will run without alteration on Linux or Cygwin, etc (it still depends on XML::Simple and LWP). It caches the RSS time so that feeds are not redownloaded unless something has changed since last time. Thanks to Brian Tol you can now get nicely formatted documentation via POD (run “perldoc get_enclosures.pl” to see it.)

Thanks to all who have given suggestions and used this. I highly recommend everyone upgrade to this if you downloaded the previous one, particularly the person who had this on a cron job to download my RSS feed every minute. Thanks, anonymous friend, you reminded me that any reasonable RSS consumer should be using Last-Modified out of etiquette.

Update: That enhancement of not fetching the RSS every time introduced a bug, because I was clearing out URLs from the cache if there weren’t in the RSS feeds. Well, when you don’t fetch the RSS feed at all, there are no URLs in it at all so it was clearing out the cache when nothing was new. For the time being, I have just turned off the cache cleanup altogether. This cache is not going to be getting large relative to an audiofile in any reasonable timescale anyway.

get_enclosures Category

Since this is taking off, and a little faster than I expected, I am creating a category for this on the blog. From here on forward, I’m posting everything about it in this category. I do ask everyone that uses this to, if you don’t subscribe to the whole blog RSS here, to at least subscribe to this category. If there is some sort of bug fix or new release, I’ll post it here and then you’ll know about it. Through the miracle of blosxom, you can automatically subscribe to the RSS for any subcategory, and the RSS feed for just this category is here. There will be a release of a 0.1 version (what is out there now I am retroactively calling 0.0) before I go to bed tonight. It will have enough new stuff, including a serious performance tweak, that all current users should upgrade. In addition, it includes what people like Gordon Smith suggested here and make this so that it will work on non Apple platforms. My original conception was that this would be specific to Mac and iTunes, but there is no reason to be that specific. Now, it will work as a downloader for anyone that can have the right Perl stuff installed, on Linux, on Windows (straight or Cygwin), etc. Cool stuff.

Top Ten Subversion Tips

Via Coop comes this cool article on top ten Subversion tips for CVS users. I’ve pretty much switched from CVS to Subversion for everything I do at the house. It’s still a bear to set up from scratch on a clean box that lacks the prerequisite libraries, but on something like Fedora core where it comes preinstalled, there is no reason not to use it. It is highly spiffy.

get_enclosures

Oddly enough, people are actually using this damn thing! I’ve seen a number of people getting the audio from this weblog with a user agent of “LWP::Simple”, so I’m assuming most or all of them are users of the script. Now the bad part is that I see all these weaknesses of the script. One kind soul has already emailed it back to me with my explanatory comments from the top in perldoc format. I also realized from looking at my logs when someone had set this up to run every minute (!) on their cron job that I really need to be checking the Last-Modified when I go to get the RSS. As it is, I give people the power to hammer the living shit out of web servers by fetching that RSS over and over. I found a great example of setting up the headers using the LWP::UserAgent to not fetch it unless new. I’ll be doing that as I can. The day job is stepping up a little, so I might be doing less evening coding for the next little bit, until I get a handle on it. There should be another release of this in the next day or two, though.

Politics in a Small Town

On September 9th, Michael Waldman and William Kristol will be in this small town, debating at Coastal Carolina. My wife asked if I wanted to attend this event, and my response was “Only if I think there is a reasonable chance of pumping a poisoned blowdart into William Kristol’s neck.” It’s worth noting that I have a paper copy of a Coastal publication, and it has both guys’ photos in it. On the web page, only Kristol’s grimacing face is up there. I wonder why that would be?

Audioblog for August 24, 2004

Here is today’s entry in the audioblog, in which we discuss life in a small coastal southern town, being mentioned by Adam Curry, organizing one’s personal space and fighting the packrat within, and how to fight social anxiety with apathy rather than medication. For once, it is an audioblog entry not about audioblogging!

RSS – It’s All 2.0 Here, Baby!

I couldn’t think of a reason not to anymore, so I converted it such that all RSS feeds on this blog are now 2.0 unless you specifically ask for the 0.91. If your url ends in index.rss20, that’s the RSS 2.0 feed. If it is index.rss, that’s also the 2.0 feed. If for some reason, you explicitly need the 0.91 feed, access that via index.rss091. In fact, if you do please leave me a writeback. I’m really curious about what that situation that would be. I’m thinking that for anyone, this should Just Work. The only difference might be that if you were subscribed to the default feed with an aggregator that supports enclosures, all of the sudden all of the audioblog posts will show up. Otherwise, it shouldn’t impact anyone but please let me know if it does impact you.

Streaming Zydeco

From the most recent KRVS email newsletter:

On Saturday, September 4th, KRVS starts an exciting season of live broadcasts of festivals and concerts! At noon on that Saturday, KRVS will broadcast the Zydeco Festival live from Plaisance, Louisiana until 6pm. From 6-7:30pm (CT) we’ll go live from Eunice for Rendevous des Cajuns, and then over to Acadian Village in Lafayette for a live Bluegrass concert featuring Ronnie Reno! Check the Local Events site on the KRVS website for more information on upcoming live broadcasts.

I capture a number of their shows with RadioLover and I’ll capture the entirety of this event. The Rendez-Vouz des Cajuns is one I get every week anyway (a Prairie Home Companion style weekly variety show but mostly in French). If you have any interest in these types of music, I strongly recommend checking out this station. You might even could kick them a few bucks. I’m trying to get organized to send a small contribution to all the various public radio stations that I capture streams from.

Audioblog for August 23, 2004

The latest episode of the audioblog is now available. As my pal Jonny X requested, I am not using the Sharp Zaurus and its hissy audio for this entry (and the MP3 link will open a new window). I discuss the notion of the “iPod Platform” and how I hope that leads into a world of delicious irony where the RIAA efforts to kill internet radio turn around and bite them in the ass. I discuss a little more about bands I like, and how I have informed Gentle Readers that I use them for a theme song that that “I’m asking for your permission, or at least your forgiveness.” I also discuss how I might one day finally discuss on the blog why I don’t believe in God, and how the audioblog entries make it more likely that I will finally go through with that.

Some links mentioned in this episode:
RadioLover – MP3 stream capturing for Mac OS X
WREK 91.1 FM – Ga Tech Student Radio
KRVS 88.7 FM – Radio for the University of Lousiana at Lafayette (geaux Cajuns!)

Old Crow Medicine Show

Band of the Day! They’ve been playing the hell out of this band on World Cafe, a five piece from Nashville TN called Old Crow Medicine Show. The song they’ve been playing is their cover of Bob Dylan’s “Wagon Wheel” – a song I had never heard before but love their version of. They have a few MP3s available on their download page and all of these tunes rock out. They are to bluegrass what the “cow punk” bands are to traditional country. What would you call that? “Fiddle punk?” Whatever it is, they put out high energy ass kicking bluegrass music that rocks. Check it out.

get_enclosures.pl

Normally I’m pretty spastic about releasing unpolished code. I tend to want to hang onto it until I’m proud enough of it to let it out in the wild. I’m making an exception for this thing, my Perl equivalent of Adam Curry’s iPodder AppleScript. You should only download this if you are comfortable running Perl from the command line. If you don’t know what that means, this is not for you I am afraid. For those who can handle that, and who don’t have a problem installing CPAN modules, I present my script get_enclosures.pl.

Read the beginning comments inside that script to get the basics of how to use this. I include a paste of the crontab entry I use to automate this thing firing off twice a day. Anyone can take this and do anything with this. I only ask that if you improve it, please send me a copy of your changes so that I too can benefit from your expertise. As I say, I’m not being a control freak perfectionist and waiting until it is perfect to let it ride. We’re all sharing first drafts here, so keep that perspective in mind as you look at it (ie, don’t judge me harshly by this – this is a unique situation and not indicative of my standard work product.)

Update: As Adam pointed out, the download was boned. I had to zip it up, otherwise my web server was trying to execute the file rather than just serve it out as text. Whoops. Since I’m zipping it up now anyway, I went ahead and included my feeds.txt in there as well.

Update #2: For all of you coming from Adam’s site or anyone that has downloaded this previously, see this post about how to subscribe to the get_enclosure specific feed to keep up with developments (no pun intended.) 0.1 is now available.

Office Before and After

So I’ve been talking about getting organized and my self-loathing at my own worst shitmongering tendencies. This morning I took an hour and tried to rectify some of that. We start here with a before picture. It’s not the most terrible situation I’ve ever seen, but I really have no excuse considering I set this up from scratch in a new office barely a week ago! As you can see, cords run everywhere, including many from the PC on the right on the floor to the big monitor on the left. As you can also see, the light from the built-in fluorescents gets eclipsed by the huge ass CRT. Even though this is a fresh office, because I needed to get things up and running ASAP you can see that things were wired up the most expedient way possible. Whatever network cable I first found, that went on the machine that most needed hooked up. It quickly became an explosion of tangled wires. Since I’m spending 8-16 hours a day working in this office, I need something better.

Today I decided to get serious on this. I powered down all computers and unplugged every single cable from everything. I laid it all out on the floor and put everything back one single item at a time. I had bought this velcro cable wrapping stuff that you can use like removable zip ties. I started by laying out the power strips where I wanted them, one at each end of the desk and a third off to the side. My guiding precepts were “place everything as close as it needs to be to other things, use the shortest posssible cable to connect it, and bundle up all possible slack.” I plugged back in the Holy Trinity of cable modem, Netgear SOHO router and Linksys WAP. For bonus points, this time I ran the cable coax through the surge protector which has two connectors for that. I happened to have two very short network cable and used those for the three adjacent connections. I then put back the two PCs and the monitor on the right side. As I connected everything up, I bundled the slack in the cable. If anything got tangled, I stopped and backed up and rearranged it so there were no tangle and no knots. I moved the KVM box from the desk, where it just took up space, to the top of the monitor. This had the added advantage of letting me rest all the bundles of slack cable on top there.

By the time I hooked up the Mac on the left and the networked Laser Printer off to the left of the desk, I had everything firmly in control. I chose power cords of appropriate length, plugged everything into the nearest power source, made the things that are frequently accessed the easiest to get to. Peter Walsh would be proud of my desk (he’d beat my ass over the rest of the office, though.) I’m not generally anal about organization like this (some of the readers of this blog are, and you know who you are.) Once I got started, it just seemed to take root. You might not be able to make it out in the photo, but with one exception, beige boxes have beige cords running to them and black boxes have black cords. I had two rolls of the velcro wrap, black and white. You guessed it – black cords got black wrap and same with white.

Now, the exact same stuff fits on the desk with no rat’s nest of cable and no tangles that pull on inappropriate things. Even though it’s all there, I have more free space on the desk and most importantly, I feel calmer and more centered and organized as I sit there. Subtly, I think this will help out my days and make me a more productive person. Dear lord, I hope so. To the sides and behind of the camera, it’s still shitville. I want to take this thinking and spread it throughout the whole room as I am able to. I really am turning over a new leaf. It’s just such a huge leaf that it can’t all be turned at one time. It is happening, just in shifts.

Shrook Meltdown

Shrook is having serious problems. All day long, I was getting nothing in my feeds. At first I thought it was network related, since I had taken down every part of the network earlier in the day. However, long after my normal connectivity was restored I was still seeing nothing. My client would try to connect and then immediately go to the “Disconnected” state. I went to the web page and this is what you see:

Warning: Too many connections in /www/shrook/functions.php on line 11

Warning: MySQL Connection Failed: Too many connections
in /www/shrook/functions.php on line 11
Unable to Connect to Database

This either is an isolated problem on that server with a runaway process or something boned up, or they have just exceeded what their resources can handle. This kept it from doing anything, because all my channels are set to use distributed checking. I could and just did do a manual check, which worked, but until I noticed that it was just sitting there like a stone.