Python at Google (Greg Stein - SDForum) [panela.blog-city.com]

Python at Google (Greg Stein - SDForum) [panela.blog-city.com]

Swig is your friend
Google makes extensive use of Swig. Greg indicated that Swig has improved much in recent years. All C++ projects in Google have swig generators created during build time, so python programmers can take advantage of this work. Greg said that neither Boost nor ctypes were as direct or clean as using SWIG.

RPC
RPC is the method Google uses to scale horizontally so well. They have their own internal binary wire format that speaks over http. Programmers can easy speak this format using Java, C++ or Python. Using RPC allows Google to divide computing problems up across large numbers of servers.

Python at Google
Python programmers at Google must follow a strict style guideline (based on PEP8 with 2 spaced indenting). When engineers are first granted commit access to their SCM system, they must pass a style test. All code must pass through two sets of eyes before being checked in. That combined with liberal doses of unittest, pychecker and code coverage eliminates most non-algorithmic issues that might appear in python code.

Where is Python used?
* The Google build system is written in python. All of Google's corporate code is checked into a repository and the dependency and building of this code is managed by python. Greg mentioned that to create code.google.com took about 100 lines of python code. But since it has so many dependencies, the build system generated a 3 megabyte makefile for it!
* Packaging. Google has an internal packaging format like RPM. These packages are created using python.
* Binary Data Pusher. This is the area where Alex Martelli is working, on optimizing pushing bits between thousands of servers
* Production servers. All monitoring, restarting and data collection functionality is done with python
* Reporting. Logs are analyzed and reports are generated using Python.
* A few services including code.google.com and google groups. Most other front ends are in C++ (google.com) and Java (gmail). All web services are built on top of a highly optimizing http server wrapped with SWIG.