How TweeVo survived the OAuthcalypse

Last month, I spent a few evenings adding OAuth support to Brian Peek's TweeVo application.  TweeVo is a little WPF based application that runs in the background and logs what your TiVo has recorded to a specified Twitter account.  I’ve been running it on and off as @AnotherTiVo. Brian keeps a Twitter list of known TweeVo accounts as tweevousers.

It's a good learning tool for showing how to query a web server and how to post to Twitter using the Twitter API, all wrapped up as a WPF application.  What it does is very clever and Brian did a nice article about it on the Coding4Fun site.

The web server is the built in HTTP server that runs on the TiVo box.  You can use that web server to get a list of everything that has been recorded by your TiVo.  Brian wrote a nice, clean application that would query the selected TiVo units on your home network.  TweeVo polls each unit and checks the "Now Playing" list to see what shows were recorded since the last check by TweeVo.  It then posts the name of the show, plus a http://www.zap2it.com/ link to the specified Twitter account.  The Zap2It link will list some information about the show, plus a link to tell your TiVo to record that show.

The original version of TweeVo posted to Twitter using the username and password for the account.  This was called Basic Authentication or just Basic Auth. The user’s credentials were stored in a config file by TweeVo and they were encrypted so nothing else could read it.  Brian released it a while back and and it was a lot of fun for the people who used it.  Then came the OAuthcalypse.

Twitter supported two forms of authentication, Basic Auth and OAuth.  Twitter announced in the Spring that support for Basic Auth was being phased out and everyone using the Twitter API needed to implement OAuth.

With OAuth, the application requesting Twitter access with the application key.  To get the application key, you would request one for your application from Twitter.  If they approves your request, you would get a consumerKey and a consumerSecret.The user would be presented with a web dialog asking if they wanted to allow access to their account to the application and they would be prompted for their user name and password. 

If they allowed it, Twitter would send back an access token and application would use that token and their own api key to access the Twitter API.  The web dialog would redirect back to the calling web application and life was good for the user.

That’s an over simplification of the process, but it describes the basic mechanism for allowing an web application to post to your Twitter timeline.  There are a few advantages to using OAuth.  Since the application uses an access token, you could change your password without having to update the application.  Plus you could revoke the access token at any time from your Twitter web page.

For desktop application, it was a little trickier.  You still needed to present the web dialog from Twitter to request access.  Since you couldn’t get back to desktop application from a web page, the user would be presented with a PIN from the web dialog.  He would then manually type the PIN into an entry field provided by the desktop application.  The app would then request the access token from Twitter by providing the application key and the PIN.

While this mechanism keeps the user’s credentials away from the application, it’s annoying to use.  Plus you have now introduced a point of failure where the user types in the PIN.  A more streamlined approach called XAuth was made available by Twitter for desktop applications.

XAuth works by consolidating a few of the steps.  The user provides the user name and password to the application.  The application then requests the access token by passing the credentials and the application key.  This skips over the access request dialog and sends back the access token.  For the end user, this is a much simpler. 

The original shutoff date for Basic Auth was June 30th, 2010.  This date became commonly known as the OAuthcalypse.  Due to heavy Twitter usage around World Cup activity, the Twitter team pushed the OAuthcalypse date to August.  On August 16th, Basic Auth usage would start get rate limited and a final shutoff occurred on August 30th.

The OAuthcalypse basically prevented TweeVo from posting to Twitter.  Brian was a little busy in September and I offered to help add OAuth/XAuth support to TweeVo.  I did some reading and played around with some .NET implementations of XAuth.

We implemented XAuth with TweeVo and tested it in late September.  Much of the XAuth code was based on code that been posted in a set of blog posts by Shannon Whitely.  I made a few changes to Shannon’s code, but the his implementation was sound and it saved me quite a bit of time.  That allowed me to spend more time reworking the TweeVo code to use XAuth and do more testing.  If you were using TweeVo 1.0, now is the time to get version 1.1.

Twitter? Time Warner Cable gets it

I have a love/hate relationship with Twitter.  Some days, I think it’s greatest time waste since they invented the meeting.  Other days, it’s an endless source of amusement.  And on some days, it’s actually useful.  Yesterday was one of the useful days.

At about 9m last night, my 10 year daughter came running into my home office.  Kathryn was convinced that she had broken the TV.  Since the destruction of electronic devices is her sister’s department, I went in to the bedroom to take a look.  Kathryn was watching a show on TLC HD and it was not coming in well.  The audio was distorted and the image had tiling artifacts.  I assured my daughter that it wasn’t her fault and sent her off to read a book.

I checked a few channels and sure enough, it wasn’t just TLC HD that was having problems.  Most of the channels had the same problem.  And yet, a small number of channels were just fine.  My decidedly non-scientific search had determined that the SDV channels were having the problem and the non-SDV channels were just fine. 

SDV or Switched Digital Video is a technical hack that allows cable companies to provide more channels than they actually have the physical bandwidth to carry them to every home.  What they do is to only provide a channel when a view switches to that channel.  A cable node will supply somewhere between 1000 to 2000 homes in a neighborhood.  If the cable company provides 200 channels, only a fraction of that number is actually being watched at any time. 

The “Switched” part of SDV means that the frequency that the channels goes out on is not fixed, it can changed based on usage.  When you select a SDV channel from your set top box, the channel request goes back to the cable office.  They check to see if any other box on your node is watching that channel.  If it’s in use, the cable office sends back the frequency that the channel is on, and you get to watch “Matlock”. 

It’s a clever hack, but it’s still a hack.  If you get enough people on your node to watch a different channel, it will fall apart like a cheap suit.  While I doubt that would actually happen, it’s technically feasible.  They can spend the money running Fiber optic to each node, and making the last run to house over copper.  Running fiber to each house is very expensive.  Just ask Verizon how much it costs to wire a neighborhood for FiOS.

If no one else is currently watching that channel, the cable company assigns the channel to an open frequency for your node and sends that frequency back down to your set top box.  If you have a TiVo, the process works the same way.  Your cable company provides a device called a Tuning Adapter, that manages the frequency negotiation between the TiVo and the cable office.

I have both a set top box and a TiVo HD with a Tuning Adapter.  The set top box was having the audio distortion and video artifacts.  The TiVo wasn’t even displaying the SDV channels.  Since the non SDV channels were crystal clear, I guessed that it wasn’t a signal problem and the problem was more likely than not an issue at the cable office.  Since I could reproduce the problem with just the cable company’s equipment, I decided to call their tech support line.

My cable provider is Time Warner Cable of Albany, NY.  I went to their web site to look up their support number.  I called it and heard a recorded message explaining that there would be a long wait time for technical support.  TWC Albany is pretty well staffed, that only happens when something bad has happened. 

While I was in the hold queue, I saw an option for chat support on their web site.  I tried that and the best thing that I can say about that experience was that it occupied 45 minutes of my time without any resolution.   The sum of the support rep’s skill set was to have me reboot the set top box and when that did not resolve the issue, he then scheduled a service call for a time slot that was 9 hours in the past.  Since my cable package did not include any time travel options, I wrapped up that chat session and tried calling the Albany support number again.  This was about 10pm.

While I was in a never ending on hold queue, I decided to see who from Time Warner Cable was on Twitter at that hour. As it turns out, the corporate office of Time Warner Cable has a bunch of people on Twitter.  Their main support account is @TWCableHelp and I could see on Tweetdeck that they were active at this late hour.  I tweeted the following message:

Can @TWCableHelp help with local issues? My SDV channels are unwatchable and I made the mistake of trying TWC online chat

The message was a little snarky, but 45 minutes of chatting with a CSR in India will do that to you.  Within two minutes, I got the following response:

@anotherlab Can certainly take a look into your issue. Had eChat already rebooted the box? ^BP

We then traded a few tweets and Bryan (the ^BP in the message) ended up calling me.  He was very professional and had a pretty good sense of humor.  We immediately ruled out time travel as an option and he was able to remotely check the signal strength coming into my set top box.  Everything looked normal.  While we were discussing what the root cause could be, the problem cleared up.  What ever had been going on at the local office, the problem appeared to have been resolved. 

We checked the channels on the set top box and on the TiVo.  From his end, he could see that about 50 channels where being sent over SDV on my node.  So we picked a few channels were not being sent at that moment. Everything was normal. 

We chatted a bit about my TiVo and I mentioned that a few people had been complaining about the latest software update that went out to the DVRs that TWC supplies.  He wanted to know more about those complaints.  I told him that I didn’t know too much about those problems, but I had read a few complaints on some local sites.  I gave him a few links (Kristi Gustafson at the Albany Times Union and Albany HDTV) and he checked them out while we were talking.

Bryan then asked if I had any other questions.  So I asked him about the CCI Flag.  Almost all of the digital channels (except for local stations) carried by Time Warner Albany have the CCI Flag set to 0x02, which is usually referred to as “Copy Once”.  “Copy Once” means that the show can be recorded to a DVR, but it can not be copied from the DVR.  The DVR must respect the CCI Flag, otherwise it would not be allowed to use a CableCARD, which is needed to access the digital content.

This is not an issue with the DVR that the cable company supplies, but TiVo can do quite a bit more than a cable company DVR.  If you have multiple TiVo DVRs in your house, you can copy shows from one to another.  If you have young children like I do, that is a very handy feature.  We can also copy the shows to a PC for storage and transfer them back at a later date.  The video files are copy protected, it’s not like you can just upload them to the great unwashed masses on the Internet.

I have a couple of TiVo boxes and it’s been handy to watch a show recorded on one TiVo from another.  When the CCI flag is set to “Copy Once”, the only copy allowed is the copy on the first TiVo.  It just wont copy it to another TiVo in the house.  Many cable providers set most of the channels (excluding Pay Per View and premium channels like HBO) to 0x00, which is usually called “Copy Freely”.

Bryan told me that it was a local cable office decision. I told him that the local cable office had made it clear to me that the decision was mandated by Time Warner Corporate.  He said that he would look into it and then would get back to me on that.  i think that he will find that it’s a dead end, but it was worth a shot.

All and all, I was very satisfied with the support provided by Time Warner Cable through their Twitter account.  But please change the CCI flag.

Just watched a movie in HD using Amazon Video On Demand through my Tivo

I had some credits with my Amazon account so I decided to “rent” a movie through their Amazon Video On Demand program.  TiVo had offered the Amazon Video on Demand for while, I decided to see what the experience would be with a HD movie.

From what I read on the TiVo site, HD movies from Amazon are only available through a select hardware devices, you can’t order them on through the Amazon site for a PC.  I have a TiVo HD, which is one of the approved hardware devices. 

I’m not sure why you can’t get HD movies on your PC.  It’s not a hardware limitation, you can download TV shows in HD through Amazon.com.  It must be a studio thing.  It’s not an issue for me, if I’m going to download a movie in HD, I’m going to want to watch it on my HDTV while sitting on my couch.

With Amazon Video on Demand, you can buy a movie outright or you rent it.  If you buy it, it’s downloaded to your device and you can watch it as often as you like.  You can download it to 2 locations (like a TiVo and a Windows PC), plus two portable devices.  The portable devices must be “Play for Sure” compliant, which rules out anything designed at One Infinite Loop.

With a rental, you have up to 30 days to watch the movie once it starts downloading.  Once you start watching, you have 48 hours to watch it.  Within that 48 hours, you can watch it as many times as you want, but then it’s gone.  Which is just as well, a full length HD movie is going to take a chunk out of the available space on a TiVo.

I decided to rent a movie.  I wanted to see Terminator Salvation and the rental price was $2.99.  The purchase price for download was $11.99, but if I wanted to own it, I think I would just get the DVD for $15.99.  I don’t own a Blu-Ray player yet, so the Amazon download would be at a higher resolution than DVD.

So I selected Terminator Salvation in HD from the Amazon Video on Demand list on my TiVo.  I already synced up my Amazon account with my TiVo so I only needed to enter my 5 digit pass code to complete the purchase.  It then displayed a screen that said that download would start within 30 minutes. It was a pretty easy process.

Sure enough with 15 minutes, the little blue light on my TiVo came on.  That was the signal that the TiVo was downloading content from the Internet.  I gave it a 20 minute head start and then started watching the movie.

I don’t know how it works on other devices, but with Amazon Video on Demand on the TiVo, it’s streaming at a fixed bit rate.  It doesn’t scale the quality based on the bandwidth of the Internet connection.  I have a decent download pipe, 20 Mb/s through Verizon FiOS.  My TiVo has a wired connection to my router, so I don’t have to deal with the general vagueness of 802.11g wireless.  So that means I can start watching while it was still downloading.  If my TiVo was going through a wireless connection, I think I would have waited until the movie had been completely downloaded before starting to watch it.

The quality was very good.  The bitrate was 2800 kbps and the resolution was 1080i.  That beats what I would get from the DVD version.  Amazon Video on Demand to the TiVo only supports stereo sounds, but I don’t have a true 5:1 surround sound setup yet and that loss didn’t really affect me,  The movie was downloading at roughly the same rate I was watching it, with the 20 minute head start, I was able to watch from start to finish with out any hiccups.

The experience was very good.  $2.99 seems like the going rate for movie rentals.  I’m not sure what my cable company charges, but since they have a “$2.99 value section”, I would assume that new HD releases would cost more.  Plus, my TiVo can’t access the cable company’s VOD, so it’s a moot point.

As for the movie itself?  Terminator Salvation was fun.  I think it was better than the reviews that it got.  I thought the ending was a hokey, but I got my money’s worth out of it.  I would do this again with another movie.

Some odd TiVo issues

I have  couple a couple of Series2 TiVo DVRs and they just received the TiVo Fall 2008 Service Update for 9.3.2.  It’s basically a few tweaks to the UI, but the odd thing was that I needed to manually restart the TiVo.  Usually the TiVo restarts itself after getting a new system update.  Very odd, but hopefully just a one time glitch.

My in-laws have a TiVO HD and sometime last week it decided to stop recording the shows on the season passes.  I went over and took a look.  The season passes were in order and when you selected view upcoming episodes, the shows would be listed, but not listed as being recorded.  Very odd.  The fix was simple, but annoying.  I selected each season pass and saved it without making any changes.  After saving the pass, the correct shows were marked as going to be recorded.  Fortunately they only had a handful of season passes, it took a minute or two to update all of them. 

After fixing the season passes, I tried to bring up YouTube on the TiVo HD.  When I selected YouTube from the menu, the TiVo froze and stopped responding.  I gave it a few minutes to figure it on it’s own and I restarted their TiVo the hard way.  Thanks to TiVo not having a reset button, I had to unplug and plug it back in again.  After waiting a few minutes to boot abck up, it was back to normal.

That was the first time I have ever had to reboot a TiVo, their software has been very stable.  As with my Series2 units, I’m hoping that this was a one time glitch caused by system updates being pushed down the to TiVo.

It’s not a TiVo, it’s just a DVR

There’s a post on the TV Board blog that really illustrates the differences between a TiVo and a cable company DVR.  A friend of the blog’s author (Mitch Oscar) was contacted by his cable operator, Comcast, and was offered a DVR.  We now join that conversation already in progress:

“I don’t need your DVR, I already have a TiVo,” she said.

“Our DVR is the same as TiVo but it comes free with our service,” the sales rep adamantly replied.

“Oh, really, well I have a lifetime membership with TiVo. Can you log on from any computer and schedule something to record?” she retorted.

“No.”

“Can the DVR you offer be networked to my computer?”

“No.”

“Can I transfer shows between TVs?”

“No.”

“Can you store whatever you record on the DVR to your computer?”

“No.”

“Can you convert anything that you have recorded to play on your iPod video?”

“No.”

“Can you burn anything you have recorded on your DVR onto a DVD?”

“No.”

“Then I would have to say your DVR is nothing like a TiVo. All yours does is record.”

”I guess you’re right, it’s not the same.”

 

That really sums up the advantage that TiVo has over CableCo DVRs.  On the flip side, you do have to pay for TiVo and that’s a $200 or more that you don’t pay with the CableCo DVR.

I came across that post via the TiVo Lovers blog.

Why I like owning a TiVo

This tip just popped in my email as part of the TiVo Newsletter:

TiVo Tip: Only the Best, Thank You

So you want to see the 100 greatest American movies of all time, do you? Easy! Get the AFI, 100 Years… Guru Guide and TiVo will automatically record every movie from the list that comes into your living room, from Tootsie to Toy Story, Swing Time to Star Wars, and 96 more. Here's all you need to do:

• From TiVo Central, select Find Programs & Downloads.
• Select KidZone Recommendations & Guru Guides.
• Scroll down to find AFI, 100 Years…
• Movies will magically appear on your Now Playing List, just like regular TiVo recordings.

Note: You must be broadband-connected to get a Guru Guide using your TiVo remote. Alternatively, try browsing the guide and scheduling recordings online.

Being able to do stuff like that is one of the reasons I love having TiVo in the house.

Displaying what your TiVo recorded as a blog badge

There have been some people blogging about having what was recorded on TiVo being displayed on their blog.  Something like how you have display what you have been listening to with Last.FM.  Personally, I wouldn't want the world to know what was just recorded on my TiVos.  One TiVo gets filled up with cooking shows and NY Mets baseball, the other one gets an eclectic collection of kid shows and the SciFi channel, with some Animal Planet for extra seasoning.

The programmer in me keeps thinking about from a how to build it viewpoint.  There are some scripts out there that will pull the recorded show information from the TiVo mini web server.  That part is pretty easy.  Getting it to your blog is another story.  Unless you host your own blog, you need some place to store the data so a blog widget can render it in Web 2.0 approved colors (plus the bottom reflection with alpha channel blending goodness).  Since your TiVo is behind a firewall (well, it should be behind one), your blog can't pull the data directly form the TiVo.  And you really don't want to pull the data, that's too much traffic.

What you want to do is to push the data up once you find a place to host the data.  Since we don't want to hack the TiVo, the TiVo wont be pushing the data.  You'll need a home PC to query the recorded show history and push the data up.  A pretty simple task, but if you step back and think a little more about the whole TiVo experience, there is a better way.

Your TiVo box is connecting back to the TiVo mothership on a regular basis.  It gets program updates, verifies that it's subscription is still good, sends usage data back to TiVo.  Other than creating the infrastructure, it would be pretty easy for the TiVo company to supply to you what your TiVo has recorded.  That's where Louis Gray was heading with his article about TiVo adding social network related functionality.

I wonder if anyone at TiVo has thought of doing this.  They already have the hardware and networking stuff in place.  It's basically just taking the data that TiVo is already sending back and allowing the users to display and share that data.  It adds a coolness factor that you wont get with a cable company DVR.  I mean TiVo spent resources enabling you to program your TiVo from a mobile phone, why not social networking?

Not a fan of Vibrant Media's intellitxt popups (and how to get rid of them)

I was reading a blog about pulling an XML feed from my TiVo and I kept being distracted by popup ads over key words in the article.  You've probably seen them on other sites, the words have a double underline and they are not actually links. 

In the middle of a sentence, you see a word like this Internet and it looks almost like a regular link.  Except the color is wrong and the underline is not the same a regular link.  When your mouse hovers over them, a popup ad that has some contextual relationship to the text appears.  In the fake link above, nothing will happen when you mouse over it.  I just used an inline CSS style to do the double underlining.

Vibrant Media calls it "in-text advertising".  I call it a popup and I want it to go away.  As in go away right now.

I can understand the desire to have ads on your blog and I have them here myself.  I'm using Google to supply the ads on this blog and they are pretty unobtrusive.  I don't have any beef with web based advertising as long as it doesn't get in the way of the reading the web page.  I don't like popups and if I can avoid them, I will. 

In this case it was pretty easy.  I am using Firefox 2, so I installed the Adblock Plus addin.  That little gem will give you fine control over what ads you will see and experience.  For Vibrant based popups, it's pretty easy.  For a web site to use Vibrant's popups, they need to register an account with Vibrant and add some code to their web page that scans the text and creates something funky with Javascript when the page is rendered.  The web site author gets a url that includes their blog name on the intellitxt.com domain.  With AdBlock Plus, I just added a filter for the text http://*.intellitxt.* and after I refreshed the web page the popups were gone.  For Opera users, this link will show you how to block Intellitxt popups.

I would prefer not using AdBlock Pro. When I installed it, it prompted me to pick a subscription for automatically picking up ad filters.  When I did this, it took out most of the ads on every page.  And I don't have a problem with Google Ads or other ads that don't interfere with the consuming of the content.  So I yanked the subscription list out.  From here on out, I'm just going to kill the popups and leave the other ads in place.