George V. Reilly

Recursive Generators in Python 2 and 3

Generators decouple iteration from the code that uses the results of the iteration.

—David Beazley, Generators

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

Python generators have a variety of uses. One such is to lazily evaluate sequences. Another is for coroutines. Yet another is to re­cur­sive­ly traverse a tree or a graph, yielding an iterable sequence.

Consider this simple tree of nodes:

node_tree = Node(
    'a', [
        Node('b', [
            Node('e', [
                Node('g')
…continue.

Obfuscating Passwords in URLs in Python

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

RFC 1738 allows passwords in URLs, in the form ://:@:/. Although passwords are deprecated by RFC 3986 and other newer RFCs, it's oc­ca­sion­al­ly useful. Several important packages in the Python world allow such URLs, including SQLAlchemy ('post­gresql://scott:tiger@lo­cal­host:5432/my­data­base') and Celery ('amqp://guest:guest@lo­cal­host:5672//'). It's also useful to be able to log such URLs without exposing the password.

Python 2 has urlparse.urlparse (known as urllib.parse.urlparse in Python 3 and six.moves.url­lib_­parse.urlparse in the Six com­pat­i­bil­i­ty library) to split a URL into six components, scheme, netloc, path, parameters, query, and fragment. The netloc cor­re­sponds to :@:.

Un­for­tu­nate­ly, neither Python 2 nor 3's urlparse properly handle the userinfo (username + optional password in the netloc), as …continue.

Including Data Files in Python Packages

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

I spent some time today struggling with setuptools, trying to make a Python source package not only include a data file, but also install that file.

Building the installer

Consider the following source tree layout:

├── MANIFEST.in
├── README.md
├── my_stuff/
│   ├── bar.py
│   ├── foo.py
│   ├── __init__.py
│   └── quux.py
├── models/
│   └── long_ugly_name_20151221.json
└── setup.py*

I wanted to create a Python source dis­tri­b­u­tion, some_­pack­age-N.N.N.tar.gz, which contains the code in the my_stuff directory, as well as models/long_ug­ly_­name_20151221.json, using python setup.py sdist.

It's not that hard to get models/long_ug­ly_­name_20151221.json included in the tarball. Add an entry in MANIFEST.in:

include models/*.json

Then be sure to set in­clude_­pack­age_­da­ta=True in the call to setup():

from setuptools
…continue.

Setting Up a Pairing Workstation for Chrome and Git

[Pre­vi­ous­ly published at the now defunct MetaBrite Dev Blog.]

At MetaBrite, we believe in the power of pair pro­gram­ming. We find that pairing helps for col­lab­o­ra­tion on difficult tasks, for exploring new areas, for knowledge transfer, and for spurring each other to better solutions. I find it to be fun, though it can also be exhausting. It's not ideal for all our work—there's no value in tying up two developers on some rote task that both know well.

Last week, I rebuilt our primary pairing work­sta­tion. In its previous in­car­na­tion, we had an account for each developer. Each of us had set up some personal pref­er­ences in our own …continue.

« Next