C++ Boost

Boost.Python

June 2002 Progress Report

Contents

Introduction
handle<T>
object
object operators
object conversions
list
Numerics
Community
What's Next

Introduction

July was mostly focused on allowing expressive manipulation of individual Python objects, or what Ralf Grosse-Kunstleve calls "Writing Python in C++". The work began with this posting, which outlines the issues and intention.

handle<T>

The most basic element needed was a replacement for the reference<> class template and the ref typedef from Boost.Python v1, a simple smart pointer to a Python object. The old v1 typedef "ref" (for reference<PyObject>) had to be retired because I thought it would be too confusing given the importance of boost::ref() to this library. I began a discussionof possible names, and it was eventually decided to rename reference to handle and supply a default argument so that ref could be spelled handle<> without an additional typedef. There were also some interface changes to make it safer and more-efficient to interface with the raw PyObject*s forced on us by Python's 'C' API. A discussion of those protocols can be found here.

object

It is intended that users will seldom need or want to work with handle<>; its major distinguishing features are that it gives direct access to the underlying object representation through operator* and operator->, and that can be NULL, both sources of danger. Instead the library provides a class called object, which encapsulates a valid Python object and provides a similar interface to Python's.

object operators

The first challenge was to provide support for object manipulations using a Python-like syntax, mostly in the form of operator overloads:
Python C++
y = x.foo y = x.attr("foo");
x.foo = 1 x.attr("foo") = 1;
y = x[z] y = x[z];
x[z] = 1 x[z] = 1;
y = x[3:-1] y = x.slice(3,-1);
y = x[3:] y = x.slice(3,_);
y = x[:-2] y = x.slice(_,-2);
z = x(1, y) z = x(1, y);
z = x.f(1, y) z = x.attr("f")(1, y);
not x !x
x and y x and y
I'm still a unsatisfied with the interface for attribute access. There original proposal used a syntax like this one: 
y = x._("foo"); 
x._("foo") = 1;
which was only marginally better than what we've got. Niki Spahiev then pointed out a potential conflict with the macro which GNU Gettext suggests people define. This unfortunate state of affairs forced us into using attr instead. I'd still like to find a better interface, but the lack of overloadable C++ operators which aren't already used in Python is an obstacle. The comma operator is still a possibility, but it has the wrong precedence: 
y = x,"foo"    // error
x,"foo" = 1;   // error

y = (x,"foo"); // ok
(x,"foo") = 1; // ok
Well, I guess we could consider adding that to the interface without removing attr(), to see how it plays out...

object conversions

The object class also provided an opportunity to replace Boost.Python v1's to_python() as a user-level interface. Instead, object has a templated constructor which can be used to convert any C++ object to Python using the same underlying mechanisms used for the arguments to call<>.

Incidentally, the implementation of operator and conversion support for object uncovered an inordinate number of compiler bugs in our targeted platforms. It was a lot more "interesting" than it should have been.

list

With object implemented, it was time to begin replacing the ad-hoc implementations of list, string, and dictionary supplied by Boost.Python v1 with something more robust. I started with list as an example. Because object already provides all of the requisite operators, publicly deriving list from object seemed like a good choice. The remaining issues were what do do about the one-argument list constructor (which in Python attempts to convert its argument to a list), and how to deal converting with list arguments to wrapped functions. Some of the issues are laid out in this thread. Ultimately, it was decided that list(x) should do the same thing in C++ as in Python (conversion), while list arguments should only match Python lists (and list subclasses). The implementation worked well, and provided a roadmap for the protocol to be used for implementation of the other built-in types.

Numerics

Support for C++ long long and unsigned long long (and __int64 on MSVC) to/from python conversions was added this month. We also improved handling of numeric overflows when converting, e.g., a Python int to a type with a more limited range of representation.

Community

Deep thanks to all the Boost.Python contributors! This project wouldn't be possible without your participation.

What's Next

As I write this we are already well into the month of July, so I suggest you consult the Mailing List Archive if you want to know what's been happening. Otherwise you'll just have to wait till next month (hopefully the beginning).

Revised 13 November, 2002

© Copyright Dave Abrahams 2002.