Here’s another installment in what seems to be turning into a series of compilation instructions for Windows of libraries that were born and raised on Linux.

Python has only the most basic support for secure SSL and HTTPS and if you know anything about how SSL works, you’ll know that support doesn’t provide enough security. I’ll leave the discussion of SSL, TLS, HTTPS and other related protocols and technologies to people who actually know something about it (any good links I should put here?), but the following quote from the Python documentation should put even the uninitiated on their toes:

class HTTPSConnection(host[, port, key_file, cert_file])

A subclass of HTTPConnection that uses SSL for communication with secure servers. Default port is 443. key_file is the name of a PEM formatted file that contains your private key. cert_file is a PEM formatted certificate chain file.

Warning: This does not do any certificate verification!

The red color is mine, but the warning is there (at least in Python 2.4.4 – I’ve been a bit slow to adopt 2.5 yet, but I don’t think it has changed).

What that means is that although you might think you’re using a secure connection when you’re using HTTPSConnection you really aren’t. At least not as secure as you thought. Although all the data transferred between you and the server will be encrypted, you won’t actually know you’re talking to the right server and wil be vulnerable to the man-in-the-middle attack.

But fear not, because M2Crypto comes to the rescue. M2Crypto is a Python library based on the well known OpenSSL library which does all the right cryptographic magic in all the right ways. M2Crypto has a compatible HTTPSConnection class that should work as a drop-in replacement of the one in httplib and actually authenticate the server correctly.

Now that we’ve got all this unimportant stuff out of the way, lets get our hands a dirty with building the library on Windows.

Tools you’ll need

Here are the programs you’ll need installed before you dig in:

1. Python 2.4 or later – might work with earlier versions, but I haven’t tested it with anything but Python 2.4.4.
2. Microsoft Visual Studio 2003 – this is the version that Python 2.4/2.5 is built with and this is the version you need to build M2Crypto. I don’t think any other (including 2005) will work.
3. ActivePerl 5.8.7 – that’s the version I used, but I guess any reasonable Perl will do.
4. Command prompt – you don’t need to install it, but you’re going to be using it a lot so you’d best be familiar with it.

Building OpenSSL for Windows

The first thing we’ll need to do is build us a fresh OpenSSL DLL.

1. Download the latest OpenSSL source package from http://www.openssl.org/source/.
2. Unzip and untar the package somewhere and open a command prompt there.
3. > perl Configure VC-WIN32 –prefix=c:/openssl
4. > ms\do_masm
5. > nmake -f ms\ntdll.mak
6. > nmake -f ms\ntdll.mak install

If something doesn’t work, refer to the INSTALL.W32 file in  the OpenSSL source package. I followed the intructions there to the letter and they worked.

Building M2Crypto for Windows

M2Crypto uses a tool called SWIG to help write the Python code that wraps the OpenSSL library that is written in C, so we’ll have to download and install it.

Let’s go.

1. Download the latest SWIG Windows binaries from http://www.swig.org/download.html .
2. Unzip and untar the SWIG package to some directory and add that directory to your PATH.
3. Download the latest M2Crypto sources from http://chandlerproject.org/bin/view/Projects/MeTooCrypto.
4. Unzip and untar the M2Crypto source somewhere and open a command prompt there.
5. > python setup.py build_ext –openssl c:/openssl
6. > python setup.py bdist_wininst

That last command will create a nice M2Crypto-0.18.win32-py2.4.exe file in the dist subdirectory which you can run to install M2Crypto in the Python site-packages directory.

To test your build, run python and do import M2Crypto. If you get an error that says ‘ImportError: DLL load failed with error code 182′, it’s because the M2Crypto library can’t find the OpenSSL DLLs. You’ll need to place the libeay32.dll and ssleay32.dll files somewhere python can find them. The directory in which your script resides is a good bet.

A couple of weeks ago Jeff Atwood of Coding Horror wrote about the paradox of large displays. He mentioned two tools that helped him manage the overlapped windows on his giant monitors, WinSplit Revolution and GridMove. I tried both and neither had the feature I wanted most – splitters I could drag to resize the windows relative to each other.

I too use a couple of 19″ monitors and although they are plain old 4×3, I still often have the feeling that I’m underutilizing my screen space. This issue has been on my mind for the longest time. There’s some code on my HD called TidyDesktop that I wrote about 4 years ago. The idea was to turn the desktop into something akin to modern IDE’s, with floating windows, dock points and most importantly splitters – those narrow bars you drag to change the relative size of windows. I was doing it in C at the time, playing around with windows hooks and message loops and it was going very slowly. I did get to a point where a single window could be dragged to the edge of the screen and it would dock there, but didn’t have the time to see it through.

I was thinking about this yesterday, and it suddenly dawned on me that I was going about this the wrong way. Instead of trying to bring the splitters to the desktop, I could try to put the windows within a large container window, a general purpose IDE if you’d like.

Turns out you can change the parent of an application’s window and attach it to another window. Not only that, but when you maximize the inner window, it stays within the borders of the container window.

Based on this idea I’ve build a wxPython application that can host other applications’ windows and it looks like this:

As you can see in the screenshot, I’ve arranged Firefox, Notepad and a command prompt in a nice way that allows me to use all of them at once. You can also use this technique when you watch online movies and want to jot down some notes or when you’re playing the guitar and need both the chords and the lyrics alongside on the screen.

This should really boost productivity, but don’t take my word for it,
try it out and let me know how it goes.

This thing looks interesting. It’s a mouse replacement shaped like a small ballpoint pen. Well, no, that’s not exactly right – it is a pen. Imagine drawing on a piece of paper and having the result display on the computer screen or project to your audience..

I want one.

(via AskDaveTaylor)

I love Python, I really do. It has clear syntax, a nice library support and I feel very productive using it.

But Python is an interpreted language and it’s slow. You don’t feel it until you do something stupid like going through all the pixels in a bitmap and converting them into gray scale. I tried doing that once to create grayed out versions of my button images in a wxPython app I was building. I couldn’t belive how slow that was.  Then again, the slowness might have been due to the calls to the wxWidgets C++ layer and back.

Anyway..

I was completely unaware of the fact that the guys at YouTube, the mega site that serves millions of users daily, use Python. And they don’t just use it, their whole damn server side runs on it.

Here’s a lecture by Cuong Do Cuong, the engineering manager at YouTube. Apparently, he was there from the very start and he tells an interesting story of dealing with exponential growth and handling the scalability issues as they arised.

[Update: The lecture is a bit boring, but the real interesting stuff is during the Q&A at the last 10 minutes or so. You should watch them.]

The amazing thing is that the guys that did all kinds of insanely amazing things to acommodate their growth rate, including hacking the lighttpd source code to improve the way it does multithreading, still use Python for their server side code. According to Coung, it never was the bottleneck and the speed problems they did have were easily solved by throwing in several more servers. The speed of development in Python on the other hand, helped them respond quickly to the changes and implement new ideas almost on the spot.

When I chose Python as the main language for Tuzig about two years ago, I knew very little about it. I knew only that everything I tried thus far (C++, Java and .NET) wasn’t going get us quickly enough to where we were going and that Python looked very promising. I haven’t looked back since. I usually try to choose the best tool for the job, but Python seems to excel in many areas : from small one-off scripts to complex GUI applications (thanks to wxPython) to web applications. I can even write Python stored procedures for PostgreSQL.

And speedwise, if it’s good enough for YouTube, it should be good enough for me.

If you don’t know Python, you should take a look at DiveIntoPython. It’s a great book by Mark Pilgrim that is written for experienced programmers and focuses on the things unique to Python instead of explaining, like many other books, what a for loop is. It’s also freely available on the net so can dive right in.

This is absolutely amazing. I had no idea that robots have advanced so far. A bit scary though.

About a month ago I saw a post on the JoS Business of Software about a piece of software called Kirix Strata.

… Strata lets you access and manipulate data from pretty much anywhere on the web.  It’ll let you work with HTML tables, RSS Feeds and CSV files natively in tables.  And then once you have the data, you can do all sorts of ad hoc analysis, such as creating calculations, sorting, filtering, creating queries and reports — similar to the kinds of things you might do with a desktop database or a spreadsheet. 

That sounded interesting enough, although I had no idea what it might be useful for, so I went to the site to see. It’s a nicely designed site, easy on the eyes and with very obvious access to two of the most important pieces : the screencast and the download. I watched the screencast, which was also very professionaly done and downloaded the software. The software itself has some usability quirks, but it’s a first public beta so we’ll forgive the folks.

But after going through all those steps I couldn’t figure out what am I supposed to do with it. I mean, the site is nice, the movie is well done, the program seems to be thought out, but why haven’t anybody tell me why whould I need a specialty data browser.

The list of benefits on the site didn’t help me either:

• Access Data From Anywhere
• Integrate Data From Across the Web
• Manipulate Data Quickly
• Create Reports Easily
• Customize and Extend

Alright, so I can do all those cool things with data. What data? Why data? Why would I ever need to do something with data I find on the web? It’s not like the proposition doesn’t sound reasonable. We all like data, at least the geeks among us. We probably manipulate data everyday, and most of it comes from the web. But for the life of me, while reviewing the site and the application, I couldn’t come up with a single example.

Fast forward about three weeks and I wanted to do some research about the development outsource market. Specifically, I needed to know the number of active developers listed on www.rentacoder.com. They have a nice statistics page, but alas, it’s a table. Hmm.. how am I going to compute the sum of the “Coders with completed projects” column?

Finally I had a use for Kirix. I downloaded it again, installed it, browsed with it to the stats page and got the data I needed. And then a thought came to me. Wouldn’t it be much nicer if I could do that right from Firefox? All I needed was to copy the data from the table to Excel and I could do all the computing there in a familiar way.

A short search came up with Table2Clipboard, a nifty Firefox extension that lets you copy an entire table or just a bunch of cells in a format that can be pasted into Excel (should work for OpenOffice Calc as well, though I haven’t tried).

Just hold Ctrl and select the cells you want to copy dragging the left mouse button. You can also use Shift+Click to select a range of cells without dragging.

When pasting to Excel, use the Edit > Paste Special > Paste as text command, otherwise Excel might not parse the data correctly.

I finally found out what I might need Kirix Strata for, but I also came to understand that doing analisys from web data is a rare activity for me and that there is a much simpler solution to my needs. I used to think that market research is something you don’t need to really do when creating a small mISV. But it seems that it’s important after all. I imagine that if the folks at Kirix were to do their homework, they would determine whether there is a need for their product, and if so, what specific niche they will be filling.

Obviously, I’m not the kind of user they had in mind.