|
March 2002 Progress Report
|
Contents
- Accomplishments
- Calling Python from C++
- Virtual Functions
- Abstract Classes
- C++ Implicit Conversions
- C++ Data Members
- Miscellaneous
- The Near future
- Notes
March was mostly devoted to the reimplementation of features from
Boost.Python v1, and some new features. Re-examination of the features
from Boost.Python v1 allowed me to make significant improvements.
The ability to call Python from C++ is crucial for virtual function
support. Implementing this feature well for V2 proved to be more
interesting than I expected. You can review most of the relevant
design decisions
here.
One point which isn't emphasized in that document is that there
are subtle differences in the way from_python conversions
work when used for C++ function arguments and Python function return
values. In particular, while T const& arguments may
invoke rvalue converters, a reference-to-const return value requires
an lvalue converter, since a temporary conversion result would leave
the returned reference dangling.
I'm not particularly pleased with the current callback interface,
since it usually results in constructs like:
return returning<X&>::call(f, obj);
However, I think the following may be possible and I plan to investigate:
return apply<X&>(f, obj);
I'm open to suggestion for better names (and syntaxes)!
Once Python callbacks were implemented, it was just a short step to
implementing virtual functions. Python extension class exposing a C++
class whose virtual functions are overridable in Python must actually
hold a C++ instance of a class derived from the one exposed to
Python. Needing some way for users to specify that class, I added an
optional template argument to value_holder_generator and
pointer_holder_generator<> to specify the class
actually held. This move began to put pressure on the
class_<> interface, since the need for the user to
produce complicated instantations of
class_<> was increased:
class<Foo, bases<>, value_holder_generator<Foo_callback> >("Foo")
.def("hello", &Foo::hello)
...
Normally when a C++ class is exposed to Python, the library registers
a conversion function which allows users to wrap functions returning
values of that type. Naturally, these return values are temporaries,
so the conversion function must make a copy in some
dynamically-allocated storage (a "holder") which is managed
by the corresponding Python object.
Unfortunately, in the case of abstract classes (and other types
without a publicly-accessible copy constructor), instantiating this
conversion function causes a compilation error. In order to support
non-copyable classes, there had to be some way to prevent the library
from trying to instantiate the conversion function. The only practical
approach I could think of was to add an additional template parameter
to the class_<> interface. When the number of
template parameters with useful defaults begins to grow, it is often
hard to choose an order which allows users to take advantage of the
defaults.
This was the straw that broke the
class_<> interface's back and caused the redesign
whose outcome is detailed here.
The approach allows the user to supply the optional parameters in an
arbitrary order. It was inspired by the use of named
template parameters in the Boost Iterator Adaptor
Library, though in this case it is possible to deduce the meaning
of the template parameters entirely from their type properties,
resulting in a simpler interface. Although the move from a
policy-based design to what resembles a configuration DSL usually
implies a loss of flexibility, in this case I think any costs are far
outweighed by the advantages.
Note: working around the limitations of the various compilers I'm
supporting was non-trivial, and resulted in a few messy implementation
details. It might be a good idea to switch to a more-straightforward
approach once Metrowerks CodeWarrior Pro8 is released.
Support for C++ implicit conversion involves creating
from_python converters for a type U which in
turn use from_python converters registered for a type
T where there exists a implicit conversion from
T to U. The current implementation is
subject to two inefficiencies:
- Because an rvalue
from_python converter produces two
pieces of data (a function and a void*) from its
convertible() function, we end up calling the function
for T twice: once when the converter is looked up in the
registry, and again when the conversion is actually performed.
- A vector is used to mark the "visited" converters, preventing
infinite recursion as
T to
U and U to T converters
continually search through one-another.
I consider the former to be a minor issue. The second may or may not
prove to be computationally significant, but I believe that
architecturally, it points toward a need for more sophisticated
overload resolution. It may be that we want CLOS-style multimethod
dispatching along with C++ style rules that prevent more than one
implicit conversion per argument.
To supply the ability to directly access data members, I was able to
hijack the new Python property
type. I had hoped that I would also be able to re-use the work of make_function to create callable python
objects from C++ functions which access a data member of a given
class. C++ facilities for specifying data member pointer non-type
template arguments require the user to explicitly specify the type of
the data member and this under-utilized feature is also not
well-implemented on all compilers, so passing the member pointer as a
runtime value is the only practical approach. The upshot is that any
such entity would actually have to be a function object, and I
haven't implemented automatic wrapping of C++ callable function
objects yet, so there is less re-use in the implementation than I'd
like. I hope to implement callable object wrapping and refactor this
code one day. I also hope to implement static data member support,
for which Python's property will not be an appropriate descriptor.
- Moved
args<> and bases<> from unnamed namespace to boost::python in their own header files.
- Convert
NULL pointers returned from wrapped C++ functions to None.
- Improved some compile-time error checks.
- Eliminated
boost/python/detail/eval.hpp in favor of
more-general boost/mpl/apply.hpp.
- General code cleanup and refactoring.
- Works with Microsoft Visual C++ 7.0
- Warning suppression for many compilers
- Elegant interface design for exporting
enum types.
Before April 15th I plan to
- Document all implemented features
- Implement a
CallPolicy interface for constructors of wrapped
classes
- Implement conversions for
char types.
- Implement automated code generation for all headers containing
families of overloaded functions to handle arbitrary arity.
I also hope to implement a mechanism for generating conversions
between arbitrary Python sequences and C++ containers, if time permits
(and others haven't already done it)!
The older version of KCC used by Kull is generating lots of warnings
about a construct I use to instantiate static members of various class
templates. I'm thinking of moving to an idiom which uses a function
template to suppress it, but worry about bloating the size of debug
builds. Since KCC users may be moving to GCC, I'm not sure that it's
worth doing anything about it.
Revised
13 November, 2002
© Copyright Dave Abrahams
2002.