oldmatebrendo.com

After making the sickipedia twitter bot, I decided to make another that made use of a little bit more of the Twitter API this time. I call him lyricbot. Basically, you send a reply to lyricbot (after you have followed) with a line from a song and it will have a red hot go at telling you what the song is…

I decided to post some of the code for anyone interested in how I did it.

The bot is written in Python, and uses the python-twitter library for the Twitter API calls. I will apologize upfront, the code is probably disgusting, but this is only the second thing I’ve ever written in python, so I’m still an L plater.

The first thing we want to do is import the 3 libraries we need; re, twitter and urllib2.

import twitter
import re
import urllib2

Next we need to authenticate with the twitter API, then we’ll get the replies in the form of a Status object.

api = twitter.Api(username='lyricbot', password='nottelling:)')
status = api.GetReplies()

Now that we have all the replies sent to that user, we can iterate through them to decipher what people have said:

for stat in status: #iterate through replies
lyric = stat.GetText()[10:] #strip the '@lyricbot ' from the front and grab the text
userN = str(stat.GetUser().GetScreenName()) #grab the name of the user that replied
idno = stat.GetId() #the unique id no of the message. This is for ensuring people don't get multiple replies for one request

Next I parse the string for use in the URL and get the html:

lyric = urllib2.quote(lyric)
f = urllib2.urlopen(url) #open the URL into the stream
html = f.read() #read the url into a string


Once I have the html of the search results, I need to use regular expressions to find the song names from that site.

#compile the regex that finds the first 3 song names returned
re_lyr = re.compile('[123].([A-Za-z0-9].{10,80}).Lyrics', re.DOTALL)
#find the matches for the expression in the html
newRes = re_lyr.findall(html,1)
songs = ''


Once I have the (1, 2 or 3) song results stored in the newRes tuple, I need to check how many (if any) it found, or deal with none being found.

reslen = len(newRes)
if reslen > 0 :
songs = songs+newRes[0] #append to the string that gets sent
if tuplen > 1 :
songs = songs+' '+newRes[1]
if tuplen > 2 :
songs = songs+' '+newRes[2]
else : #if there were no songs found
songs = 'Sorry I couldn\'t find that one!


Now that we have a string to post (either the song names or the fail message), it’s time to send:

api.PostDirectMessage(userN, songs)

That’s about it really.
There will be another post after I launch it properly, probably with some updates.

When playing with SSL certificates and Java I came across a very weird error that I could not find a solution to anywhere.

I had submitted my CSR and got the certificate back in an email, so I copied it and pasted it into a .cer file ready to SSL my applet, but when trying to import the certificate, I got the following error:

keytool java.security.cert.CertificateParsingException: signed overrun, bytes = 139

After much searching of forums and Google, I could not find any reference to this specific error message other than something about copy and pasting from the wrong format. I thought that maybe it was something to do with Outlook, so logged into Gmail and copy and pasted from there, voila!

I decided to make sure Outlook was my problem before slaying it all over the intarwebz and sure enough, copy and pasting from Outlook again threw the error, but doing it from Gmail did not. I no longer copy and paste anything important out of Outlook.

Security is painful. My attempts to make our applet that is using RMI use SSL as well have absolutely done my head in.

First there was the fact that a self signed certificate wouldn’t do. This wasn’t a big surprise, but it wouldn’t even allow me to test my app to ensure it was working before I forked out the big bucks for the proper SSL certificate.

I had an idea, I have a code signing certificate from Thawte that is worth a pretty penny, surely it will at least allow me to test my app, if not be my solution. Strike 2. A Code-Signing certificate can only be used to sign code (JARs in my case), not for authentication – again, no real surprise.

So an SSL certificate was purchased from a company we have a good relationship with, so we got it for wholesale price and away we went… almost. Java’s Virtual Machine looks in a certain spot for trusted certificates, but unlike the major browsers, only has about 15 in there. Of course the certificate we bought was not one of them. Strike 3 – back to the drawing board. For anyone reading this because they are experiencing similar problems, use:

keystore –list –v –keystore %JAVA_HOME%/lib/security/cacerts

Password: changeit

to view the certificates that are in that cacerts file. This is the default file java will look in if you do not specify a trustStore property when calling your applet/application.

We found a certificate in here, were able to obtain a refund on the previous and now it was time to invoke methods remotely over secured sockets and love life… nearly.

The first test of this saw love. The problem was that test didn’t bring browsers into play. I ran the RMI Registry and the Server Implementation, downloaded the jar and ran it from the command line on my PC. Voila!

Hello World!

I was ecstatic. Until I ran the applet from the same jar and got an error telling me “bad certificate�. This didn’t make sense as the root CA of the certificate we were using was certainly in the browser (both Firefox and IE7). A bit of looking around the forums provided very few answers so I decided to post myself (I very rarely post on java forums as the answers to my questions are there if I look hard enough).

It turns out that the Root CA certificates in the browsers have a property that specifies what actions a certificate using that Root CA can perform. The certificate we were using was marked for Code Signing, Server Authentication, Email Authentication. Notice Client Authentication missing. *sigh*.

This was getting thoroughly annoying as I have now spent 3 weeks trying to test code that only took me a couple of days to fully merge with the older code.

I tried checking the Client Authentication box in the browser for out certificate provider, but this didn’t fool the browser for a second, and I was still denied. After a long chat with an RMI expert, it was concluded that it is not currently possible to enable client authentication in RMI using SSL in an applet. If the application is not running from a browser however, this works as it should. The reason is that when you use the browser, it looks for the certificate there rather than in the parameters specified by the applet.

The work around is buy one of the (I think) 2 certificates that support client authentication, though this is untested as we added security other ways and as such, didn’t purchase one of these certificates.

The applet is now running with SSL enabled using Remote Method Invocation. Hurrah!

I had a dabble at modifying a Firefox extension this morning.

http://netusage.iau5.com/ – The broadband usage extension is open source, but doesn’t yet support my new ISP. I got the information that the developer would need to add it and thought I’d have a stab myself.

Patience wasn’t my friend this morning and I ended up giving up after about 2 hours of poking and editing, but not really knowing enough about it to be able to make it work. If you aren’t with Australis for your internet, it is a great extension! Hopefully the developer has more success than I did.