Stelios found a cool use for my Twitter API into a hardware-hacking project, detailed here and here. Glad to see someone found it useful!
—★—
Getting pymunk to work under OS X
The Python binding for the brilliant Chimpunk physics library, pymunk is a bit of a pain to get set up on OS X. You need the source for Chipmunk to spit out a dylib, or dynamic library, but the setup.py bundled with pymunk bungles the compilation. With a lot of trial and error, I managed to compile the source using gcc. The recommended cmake didn’t work either. For future reference, here is how you compile Chipmunk on OS X:
gcc -O3 -std=gnu99 -fno-common -ffast-math -c *.c
gcc -dynamiclib -o libChipmunk.dylib *.o
Enjoy!
—★—
Sometimes I think people go out of their way to poorly brand something. Case in point: phpass. It’s a PHP password hashing library that’s called…PHP Ass. Sure, it’s probably meant to be read pee-eych-pass rather than php-ass, but PHP is PHP, not PH. It’s like that running joke that if you don’t take care of your branding you’ll end up with a domain name like PenIsland.com.
—★—
Python’s range and xrange
range and xrange are very different in Python; do you use them correctly? I see a lot of code where the author seemingly didn’t know the difference. At first glance, there doesn’t seem to be one at all—in fact, when I started hacking in Python, I used them interchangeably because I couldn’t figure out what the difference was, and in all the code I was writing at the time, swapping one for the other made no conceivable difference.
It’s a subtle difference. For the sake of simplicity, let’s omit the the optional start and step parameters in both functions. range returns exactly what you think: a list of consecutive integers, of a defined length beginning with 0. xrange, however, returns an “xrange object”, which acts a great deal like an iterator. If you’ve spent anytime with iterators, this difference should make sense. Here’s an example:
range(1000000)
xrange(1000000)
range(1000000) will return a list of one million integers, whereas xrange(1000000) will return (what is essentially) an iterator sequence. Indeed, xrange supports iteration, whereas range does not. The overall benefit is minimal, because xrange (in the words of the Python manual) “still has to create the values when asked for them,” but at each call, it consumes the same amount of memory regardless of the size of the requested list. At extremely large values, this is a major benefit over range. Another benefit is also apparent: if your code is going to break out while traversing over a generated list, then xrange is the better choice as you are going to consume less memory overall if you break.
I’d love to see any cool xrange iterator tricks if you’d like to share them.
—★—
The Caps Lock button is in a horrible place. On most Unix workstations, you’ll actually find Ctrl button living there. Most hackers should be able to recognize the problem with having Ctrl where is now lives by default, under the Shift key. It used to be a pain to fix, but since I picked up Emacs/Vim on my Mac I immediately got frustrated at the claw my left hand became when I needed to perform any trivial task. Programmers have traditionally remapped their Ctrl keys and Caps Lock keys backwards, putting Ctrl on the home row. It’s trivial in Tiger and beyond: System Preferences > Keyboard And Mouse > Keyboard > Modifier Keys. Swap them, and as an added bonus, the little light on my Caps Lock key goes green correctly when I hit Ctrl now.
—★—
John Resig, Javascript guru, ported Processing to Javascript. The magnitude of the project gave me one of those jaw drop moments. I’m a huge fan of Processing, and to be able to write “applets” without Java…well, let’s just say I’m excited.
—★—
Everytime you linkjack a YouTube video on any number of social news websites, God kills a baby seal. Or kitten—whatever. It’s a disgusting habit and pathetic attempt at driving traffic to your website. Don’t do it.
To restore the balance to the Force, I will be vehemently down-voting/burying any submissions that follow this vile practice. Help me out, and do it too.
—★—
Quicksilver vs. Spotlight
I’m no stranger to Macs—both my parents are long-time users. My mother still has a candy green G3 tower sitting around somewhere. I, however, bounced between various versions of Windows and Linux until last December, when I picked up a shiny new Macbook.
Quicksilver was one of “those” apps I was told to get—the list that included wonders like TextMate and Adium X—and I had it installed literally minutes after I first saw Leopard’s desktop.
I learned pretty quickly that Spotlight is a pretty decent application launcher itself. This is a change from the Tiger days, when there were little nags that prevented it from being a truly functional launcher. Now, I can type, see my result and tap Enter. I can type a word into it and wait for an in-line Dictionary.app definition to show up.
Quicksilver isn’t afraid of functionality; it’s known for it’s vast library of plugins and “power-user” functionality. Spotlight definitely pales in comparison to Quicksilver’s functionality, despite the overhauls that Leopard brought. But I use Quicksilver exclusively as an application launcher. When I want to Google/Wikipedia/Internet something, I do it in Firefox—it’s almost always open anyway. When I want to control iTunes, I use the dedicated buttons the newer Macbooks (mine included) have to play/pause and skip songs. Other crazy stuff like compose new emails inline, or send text to various applications is far beyond what I want my application launcher to do.
Logic would dictate that since the core functionality that I want is inherent in both and is performed exactly the same in both, I should be using Spotlight—after all, there isn’t an extra application running in the background eating RAM by indexing on top of what the operating system is already doing. So I did: I ditched Quicksilver for a week. I re-mapped Spotlight back to its default Command-Space, and Quicksilver back to it’s default of Control-Space. I then took Quicksilver off my startup items and rebooted.
The week went well—in fact, too well. I was almost sold. I didn’t really notice any improvements from the fact that Quicksilver wasn’t running, and so I decided to, as control, go back to Quicksilver for a week.
The first moment I hit Command-Space after getting Quicksilver running again had me sold. I find Quicksilver more responsive; I love how it is in my face instead of far in the corner; I like how it indexes new applications faster; and most of all, I find I prefer how it guesses what application I want better.
Believe me, if you’ve been sold on Spotlight for application launching already, give Quicksilver a shot.
—★—
Python Decorators: Syntactic Sugar
Python decorators are syntactic sugar—you’ll hear that a lot. Decorators are an extremely powerful tool that, at first, don’t seem to offer any real use. Take, for example:
def decorator_test(function):
print function.__name__
def foobar():
print "Hello, world!"
decorator_test(foobar)
That will output foobar—quite clear, and a simple example. To convert this into decorator syntax:
def decorator_test(function):
print function.__name__
@decorator_test
def foobar():
print "Hello, world!"
Again, you’ll see the output foobar. So what happened? You’ll notice the @decorator_test line above the function definition of foobar(). This is the syntax for applying a decorator to a function. Comparing our two examples, you’ll see applying a decorator to an arbitrary function and then calling function, is the equivalent of calling decorator(function).
Why bother doing this at all? There is plenty of real-world examples for decorators, and I would consider it to be “modern” Python—it applies well to object-oriented programming, and properly, in a Pythonic way, exposes Python’s functions as first-class objects. Here’s a more practical application:
class safe:
def __init__(self, function):
self.function = function
def __call__(self, *args):
try:
return self.function(*args)
except Exception, e:
print "Error: %s" % (e)
There’s some major changes over the last example function. Firstly, I’ve encapsulated the functionality into the class; notice how it doesn’t affect the decorator. Rather than decorating a function with another function, I’d do it with a class here. To make things work, I’m using the class’ __call__ method, which is going to be how I pass the functionality to my target function. I also need to __init__ my class so that I can take the target function as a first-class object into my class. The functionality is very simple: I’m going to receive a function (self.function, created at __init__), and test it’s execution safely. I use *args to receive all arguments from the target function so that functionality is preserved and completely generalized. Here’s a sample on how to use this:
@safe
def unsafe(x):
return 1 / x
print "unsafe(1): ", unsafe(1)
print "unsafe(0): ", unsafe(0)
This outputs:
unsafe(1): 1
unsafe(0): Error: integer division or modulo by zero
Python doesn’t like when you divide by zero1, and so safe catches that and cleanly lets us know without killing the application.
This class can be used almost as a template for handling a large proportion of decorator functions; the combination of __init__ and __call__ is a lot more powerful and Pythonic—at least in my opinion—than declaring a wrapper function with another one inside it to achieve the same functionality.
Outside of Django, I haven’t really used decorators a whole lot, but spending a lot of time on Project Euler meant I needed to speed up a lot of my recursive algorithms. Decorators really came to the rescue in the form of memoization.
Let’s take a very simple Fibonacci number generator:
def fibonacci(n):
if n in (0, 1): return n
return fibonacci(n - 1) + fibonacci(n - 2)
It’s clear this is a very inefficient algorithm: the amount of function calls increases exponentially for increasing values of n—this is because the function calls values that it has already calculated again and again. The easy way to optimize this would be to cache the values in a dictionary and check to see if that value of n has been called previously. If it has, return it’s value in the dictionary, if not, proceed to call the function. This is memoization. Let’s look at our memoize class:
class memoize:
def __init__(self, function):
self.function = function
self.memoized = {}
def __call__(self, *args):
try:
return self.memoized[args]
except KeyError:
self.memoized[args] = self.function(*args)
return self.memoized[args]
This is very similar to the safe class structurally. There is now a dictionary, self.memoized, that acts as our cache, and a change in the exception handling that looks for KeyError, which throws an error if a key doesn’t exist in a dictionary. Again, this class is generalized, and will work for any recursive function that could benefit from memoization.
Let’s run a few comparisons. First, the setup:
def fibonacci(n):
if n in (0, 1): return n
return fibonacci(n - 1) + fibonacci(n - 2)
@memoize
def fibonacci_memoized(n):
if n in (0, 1): return n
return fibonacci_memoized(n - 1) + fibonacci_memoized(n - 2)
Notice how fibonacci_memoized is extremely clean—it’s the exact same function. We don’t have any extraneous cache = {} calls outside the function, and there is nothing in the algorithm that detracts from the natural flow of the process. That is what I think is the biggest benefit of decorators: it abstracts away functionality that isn’t relevant to the core of the function.
Using a simple home-brewed timer function:
Beginning trial for fibonacci_memoized(30).
fibonacci_memoized(30) = 832040 in 0.000516s.
Beginning trial for fibonacci(30).
fibonacci(30) = 832040 in 1.147118s.
The memoized function is over 2223 times faster. Even better, in this case, it scales very well.
Beginning trial for fibonacci_memoized(40).
fibonacci_memoized(40) = 102334155 in 0.000699s.
Beginning trial for fibonacci(40).
fibonacci(40) = 102334155 in 145.366141s.
The memoized function went up about 35% (an increase of 0.000183s) whereas the vanilla version went up almost 126% (an increase of 144.219023s). While the percentage values might not show a great deal of improvement, take a look at the actual values: this is effective. In fact, you can easily reach the maximum value Python will accept before you hit maximum recursion depth:
>>> fibonacci_memoized(332)
1082459262056433063877940200966638133809015267665311237542082678938909
0.009884s
I’m not going to give you a comparison—I guess I’m just not too big on leaving my laptop on for a few hours ever with the CPU working on overload. While this essentially just became a post on the glory and wonders of memoization, note how easy it was to get was to get speed improvements of several orders of magnitude by using decorator functions. I’ve already created memoize, you just have to use it. No hassle.
Here’s a bit of homework to practice your decorator-fu: write a decorator function that rounds off the output of another function down to an arbitrary precision. Email it to me (I have a contact form). Let me know if you have some cool uses for decorators, again, email me.
- Then, who does? ↩
—★—
Python and Twitter
I’m a user of Twitter, and my favorite part of the Twitter experience is the API. Twitter has allowed users access to a simple, yet expansive API that has spawned a whole community of users that create apps, bots, mash-ups etc. using Twitter.
I figured I’d write a little Python wrapper to work with the Twitter API. For some reason, I didn’t do any research, and only very recently found python-twitter. I’m not upset I spent the time—I actually like my implementation because it’s so light. Here it is:
import base64, httplib, urllib
class TwitterAPI:
def __init__(self, username, password):
# generate authentication header string
self.authentication = { "Authorization": "Basic %s"
% base64.encodestring("%s:%s" % (username, password)).strip() }
# create connection
self.connection = httplib.HTTPConnection("twitter.com", 80)
def update_status(self, status):
# send post response with authenticated status
self.connection.request("POST", "/statuses/update.xml",
urllib.urlencode({ "status": status }), self.authentication)
response = self.connection.getresponse()
return response.status
It’s pretty simple—I’ve written out update_status which, surprisingly, updates the status of the authenticated user. To use the other API functions, just adjust self.connection.request accordingly. For example, to fetch the public timeline:
def get_public_timeline(self):
# send get response--no authentication needed
self.connection.request("GET", "/statuses/public_timeline.xml")
response = self.connection.getresponse()
return response.read()
response.read() just spits out the request (which we have asked for in XML form) in it’s entirety. Use your favorite method of parsing XML in Python—alternatively, you may opt to return any of the other supported formats. A few things to point out. Notice how self.connection.request is now passed a GET parameter rather than POST—this is defined in the API and is function-specific. I didn’t need to send any further information because Twitter doesn’t require it. update_statushowever, did: status. To send a direct message, for example, you need to provide user and text for the recipient.
Simple and easy. Feel free to use and abuse this. Credit would be appreciated—or, buy me something nice.
—★—