YouTube runs on Python

I love Python, I really do. It has clear syntax, a nice library support and I feel very productive using it.

But Python is an interpreted language and it’s slow. You don’t feel it until you do something stupid like going through all the pixels in a bitmap and converting them into gray scale. I tried doing that once to create grayed out versions of my button images in a wxPython app I was building. I couldn’t belive how slow that was.  Then again, the slowness might have been due to the calls to the wxWidgets C++ layer and back.

Anyway..

I was completely unaware of the fact that the guys at YouTube, the mega site that serves millions of users daily, use Python. And they don’t just use it, their whole damn server side runs on it.

Here’s a lecture by Cuong Do Cuong, the engineering manager at YouTube. Apparently, he was there from the very start and he tells an interesting story of dealing with exponential growth and handling the scalability issues as they arised.

[Update: The lecture is a bit boring, but the real interesting stuff is during the Q&A at the last 10 minutes or so. You should watch them.]

The amazing thing is that the guys that did all kinds of insanely amazing things to acommodate their growth rate, including hacking the lighttpd source code to improve the way it does multithreading, still use Python for their server side code. According to Coung, it never was the bottleneck and the speed problems they did have were easily solved by throwing in several more servers. The speed of development in Python on the other hand, helped them respond quickly to the changes and implement new ideas almost on the spot.

When I chose Python as the main language for Tuzig about two years ago, I knew very little about it. I knew only that everything I tried thus far (C++, Java and .NET) wasn’t going get us quickly enough to where we were going and that Python looked very promising. I haven’t looked back since. I usually try to choose the best tool for the job, but Python seems to excel in many areas : from small one-off scripts to complex GUI applications (thanks to wxPython) to web applications. I can even write Python stored procedures for PostgreSQL.

And speedwise, if it’s good enough for YouTube, it should be good enough for me.

If you don’t know Python, you should take a look at DiveIntoPython. It’s a great book by Mark Pilgrim that is written for experienced programmers and focuses on the things unique to Python instead of explaining, like many other books, what a for loop is. It’s also freely available on the net so can dive right in.

Technorati tags: , ,

11 Comments on “YouTube runs on Python”


By Ido Schacham. August 23rd, 2007 at 21:46

Watched some of the presentation (the dude is a pretty boring lecturer despite his attempts to joke). It’s interesting how many optimizations these guys have to make so that everything would work smoothly. Some of their ‘conclusions’ though are basic stuff that I learned in my Computer Sciences Bachelors degree. Still, much respect, handling exponential growth like that must be crazy.

By import this. » Blog Archive » YouTube (still) runs on Python. August 24th, 2007 at 02:40

[...] GVR mentioned it back in 2006, I think a lot of people are still completely unaware that YouTube is almost all Python: I was completely unaware of the fact that the guys at YouTube, the mega site that serves millions [...]

By James. August 24th, 2007 at 02:42

Like you I am happy with python, and try to use it in many areas, but I can’t find anything that could match Delphi for developing database desktop application, but still searching for it.

By Lars. August 24th, 2007 at 16:37

There’s also a MMORPG out there which runs on Python: EvE-Online. Though they use a special fork of python (Stackless Python). If I am not mistaken their server farm is capable of handling 30.000+ concurrent players.

By Just another WordPress weblog. August 25th, 2007 at 08:13

[...] gooli.org – YouTube runs on Python [...]

By Python Bytes - Today’s Top Blog Posts on Python - Powered by SocialRank. September 11th, 2007 at 15:31

[...] gooli.org – YouTube runs on Python [...]

By python web development - DesignersTalk. December 24th, 2009 at 01:56

[...] gooli.org – YouTube runs on Python Article of YouTube on Python, with attached video. The beauty of python is templating so it's not tooo messy. And you can embed templates in .php pages iirc. __________________ dA | twitter The people who cast the votes don't decide an election, the people who count the votes do. ~ Joseph Stalin. [...]

By YouTube « Acordo Coletivo (Petroleiros, Bancários, Prof de Saúde). November 7th, 2012 at 18:11

[...] Lextrait, Vincent (July 2010). “YouTube runs on Python”. Archived from the original on 2012-05-30. Retrieved 5 September [...]

By You Tube « comzoo27work. January 8th, 2013 at 08:30

[...] Vincent (July 2010). “YouTube runs on Python”. Archived from the original on 2012-05-30. Retrieved 5 September [...]

By You Tube Traffic - Why Men Lie. January 28th, 2013 at 18:32

[...] Lextrait, Vincent (July 2010). “YouTube runs on Python”. Archived from the original on 2012-05-30. http://archive.is/20120530/http://www.lextrait.com/Vincent/implementations.html. [...]

By Viacom-0 Google-1 | elcidharth. April 19th, 2013 at 06:22

[...] Lextrait, Vincent (July 2010). “YouTube runs on Python”. Archived from the original on 2012-05-30. Retrieved 5 September [...]