A Practical Guide to Python C++ Extensions and Bindings

This guide is born from my recent experiences of adding Python bindings to a large C++ library. After stumbling around, I was finally able to make things work. I don’t think it’s difficult to write Python bindings or C++ extensions; it’s just hard to know how you should approach doing so for your particular case. I hope to do that in a series of blog posts. This will be just an introduction.

What are Python bindings for C++?

Writing C++ that is used by Python has a lot of names: “Python bindings”, “C++ extensions”, “wrappers”, etc. All of them entail the same thing: you’re writing code in C++ to be used in Python.

Where they differ is in mostly context:

  • “Python bindings” typically refers to adding a Python API to a C++ library.
  • “C++ extensions” more generally means extending Python with functionality written in C++.
  • “wrappers” more or less the same as “Python bindings”

And by doing so, you can:

  1. implement new built-in types
  2. call C++ functions

How is C++ code interfaced with Python?

The Python implementation you’re currently using is most likely written in C. This is known as CPython. Python can actually expose the C code that it’s using so that you can access native types and objects in C. And you can write them in C++ too.

And there’s lots of pros and cons that come with doing this…

When should I use Python bindings?

  • you want to take advantage of Python’s expressiveness to quickly develop
  • your C++ component is taking on the more CPU intensive tasks of your app but the rest of your app can be in Python.
  • you like developing in Python
  • any task where there’s a lot of “pipelining” and “infrastructure” code that needs to be written.

Python bindings allow you to leverage the strengths and weaknesses of both languages. Mainly, you can leverage the ease and speed of developing in Python coupled with the performance of C++.

Where do I wrap/attach binding/extend?

You usually have a set of functions that serve as the interface into your C++ library. You wrap around or attach bindings to these functions.

How will it be used?

Suppose you had a C++ library called myLib.a or myLib.so. Now you can use it like this in Python: import myLib. Now your CPython is extended with myLib.

How to use your large C++ library in Python…

There’s only one thing you need to do to use your C++ library in Python: You need to write adapters that convert Python objects to C++ objects and then call your C++ functions. Vice versa, you need to convert C++ objects returned from your C++ function into Python objects.

That’s it.

The task is more time-consuming than it is difficult. And it varies by how many functions you want to expose and how many objects you need to convert. There are many tools out there to help you. They help by reducing the amount of conversion code you need to write. In exchange, you make it more difficult for other users to install your package because they’ll need to obtain the same tools.

For example, consider this C++ method. It’s the entry point to a much large library with complex functionality. CustomClass, A, and B are all C++ classes.

// megaFoo.h

#include "scary.h"
#include "complex.h"

CustomClass megaFoo(vector<A>& vectA, B b);

Your C++ binding will work like this.

// megaFooAdapter.h

#include "path/to/megaFoo.h"
#include "Python.h"

class pyCustomClass(PyObject)
{
  ...
};

class pyA(PyObject)
{
  ...
};

class pyB(PyObject)
{
  ...
};

pyCustomClass pyMegaFoo(list listA, pyB b)
{
  vector<A> vA = convertListAToVectorA(listA);
  B = convertpyBToB(b);

  CustomClass cc = megaFoo(vA, B);
  return covertCustomClassTopyCustomClass(cc);
}

As you can see, megaFooAdapter does all the work. The function pyMegaFoo() accepts all Python arguments and converts them to classes that the original megaFoo() understands. But megaFoo() returns a C++ class. So we convert it back to a Python class that we’ll use.

Check out the next post for writing these functions.