Python Extension Programming with C





Any code that you write using any compiled language like C, C++, or Java can be integrated or imported into another Python script. This code is considered as an "extension."

A Python extension module is nothing more than a normal C library. On Unix machines, these libraries usually end in .so (for shared object). On Windows machines, you typically see .dll (for dynamically linked library).

Pre-Requisites for Writing Extensions

To start writing your extension, you are going to need the Python header files.

Additionally, it is assumed that you have good knowledge of C or C++ to write any Python Extension using C programming.

First look at a Python Extension

For your first look at a Python extension module, you need to group your code into four part −

The Header File Python.h

You need include Python.h header file in your C source file, which gives you access to the internal Python API used to hook your module into the interpreter.

Topics You May Be Interested In
Python Overview Python - Http Server
Python - Ds Home Python - Imap
Python - Dictionary Python - Ftp
Python - 2d Array Python - Remote Procedure Call
Python - Stack Python Overview

Make sure to include Python.h before any other headers you might need. You need to follow the includes with the functions you want to call from Python.

The C Functions

The signatures of the C implementation of your functions always takes one of the following three forms −

static PyObject *MyFunction( PyObject *self, PyObject *args );

static PyObject *MyFunctionWithKeywords(PyObject *self,
                                 PyObject *args,
                                 PyObject *kw);

static PyObject *MyFunctionWithNoArgs( PyObject *self );

Each one of the preceding declarations returns a Python object. There is no such thing as a void function in Python as there is in C. If you do not want your functions to return a value, return the C equivalent of Python's None value. The Python headers define a macro, Py_RETURN_NONE, that does this for us.

The names of your C functions can be whatever you like as they are never seen outside of the extension module. They are defined as static function.

Topics You May Be Interested In
Python Lists Python - Word Replacement
Python Multithreaded Programming Python - Chunks And Chinks
Python Tools/utilities Python Decision Making
Python - Ip Address Python Functions
Python - Http Headers Python Regular Expressions

Your C functions usually are named by combining the Python module and function names together, as shown here −

static PyObject *module_func(PyObject *self, PyObject *args) {
   /* Do your stuff here. */
   Py_RETURN_NONE;
}

This is a Python function called func inside of the module module. You will be putting pointers to your C functions into the method table for the module that usually comes next in your source code.

The Method Mapping Table

This method table is a simple array of PyMethodDef structures. That structure looks something like this −

struct PyMethodDef {
   char *ml_name;
   PyCFunction ml_meth;
   int ml_flags;
   char *ml_doc;
};

Here is the description of the members of this structure −

Topics You May Be Interested In
Python - Ds Home Python - Uploading Data
Python - Sets Python - Remote Procedure Call
Python - Divide And Conquer Python - Text Processing Environment
Python - Recursion Python - Bigrams
Python - Telnet Python Basic Syntax

This table needs to be terminated with a sentinel that consists of NULL and 0 values for the appropriate members.

Topics You May Be Interested In
Python Networking Programming Python - Telnet
Python Gui Programming (tkinter) Python - Process Pdf
Python - Algorithm Design Python Lists
Python - Http Client Python Modules
Python - Building Urls Python Regular Expressions

Example

For the above-defined function, we have following method mapping table −

static PyMethodDef module_methods[] = {
   { "func", (PyCFunction)module_func, METH_NOARGS, NULL },
   { NULL, NULL, 0, NULL }
};

The Initialization Function

The last part of your extension module is the initialization function. This function is called by the Python interpreter when the module is loaded. It is required that the function be named initModule, where Module is the name of the module.

The initialization function needs to be exported from the library you will be building. The Python headers define PyMODINIT_FUNC to include the appropriate incantations for that to happen for the particular environment in which we're compiling. All you have to do is use it when defining the function.

Your C initialization function generally has the following overall structure −

Topics You May Be Interested In
Python Modules Python - Text Processing
Python Sending Email Using Smtp Python - Reformatting Paragraphs
Python - Matrix Python - Corpora Access
Python - Backtracking Python - Chunk Classification
Python - Web Servers Python - Process Word Document
PyMODINIT_FUNC initModule() {
   Py_InitModule3(func, module_methods, "docstring...");
}

Here is the description of Py_InitModule3 function −

Putting this all together looks like the following −

#include <Python.h>

static PyObject *module_func(PyObject *self, PyObject *args) {
   /* Do your stuff here. */
   Py_RETURN_NONE;
}

static PyMethodDef module_methods[] = {
   { "func", (PyCFunction)module_func, METH_NOARGS, NULL },
   { NULL, NULL, 0, NULL }
};

PyMODINIT_FUNC initModule() {
   Py_InitModule3(func, module_methods, "docstring...");
}

Example

A simple example that makes use of all the above concepts −

#include <Python.h>

static PyObject* helloworld(PyObject* self) {
   return Py_BuildValue("s", "Hello, Python extensions!!");
}

static char helloworld_docs[] =
   "helloworld( ): Any message you want to put here!!\n";

static PyMethodDef helloworld_funcs[] = {
   {"helloworld", (PyCFunction)helloworld, 
      METH_NOARGS, helloworld_docs},
      {NULL}
};

void inithelloworld(void) {
   Py_InitModule3("helloworld", helloworld_funcs,
                  "Extension module example!");
}

Here the Py_BuildValue function is used to build a Python value. Save above code in hello.c file. We would see how to compile and install this module to be called from Python script.

Building and Installing Extensions

The distutils package makes it very easy to distribute Python modules, both pure Python and extension modules, in a standard way. Modules are distributed in source form and built and installed via a setup script usually called setup.py as follows.

Topics You May Be Interested In
Python Sending Email Using Smtp Python - Http Data Download
Python Extension Programming With C Python - Smtp
Python - Ds Home Python - Rpc Json Server
Python - Binary Tree Python - Wordnet Interface
Python - Algorithm Justification Python Basic Syntax

For the above module, you need to prepare following setup.py script −

from distutils.core import setup, Extension
setup(name='helloworld', version='1.0',  \
      ext_modules=[Extension('helloworld', ['hello.c'])])

Now, use the following command, which would perform all needed compilation and linking steps, with the right compiler and linker commands and flags, and copies the resulting dynamic library into an appropriate directory −

$ python setup.py install

On Unix-based systems, you'll most likely need to run this command as root in order to have permissions to write to the site-packages directory. This usually is not a problem on Windows.

Importing Extensions

Once you installed your extension, you would be able to import and call that extension in your Python script as follows −

Topics You May Be Interested In
Python Mysql Database Access Python - Sockets Programming
Python - Dictionary Python - Text Processing Environment
Python - Heaps Python - Reformatting Paragraphs
Python - Algorithm Classes Python - Process Word Document
Python - Network Environment Python Strings
#!/usr/bin/python
import helloworld

print helloworld.helloworld()

This would produce the following result −

Hello, Python extensions!!

Passing Function Parameters

As you will most likely want to define functions that accept arguments, you can use one of the other signatures for your C functions. For example, following function, that accepts some number of parameters, would be defined like this −

static PyObject *module_func(PyObject *self, PyObject *args) {
   /* Parse args and do something interesting here. */
   Py_RETURN_NONE;
}

The method table containing an entry for the new function would look like this −

static PyMethodDef module_methods[] = {
   { "func", (PyCFunction)module_func, METH_NOARGS, NULL },
   { "func", module_func, METH_VARARGS, NULL },
   { NULL, NULL, 0, NULL }
};

You can use API PyArg_ParseTuple function to extract the arguments from the one PyObject pointer passed into your C function.

Topics You May Be Interested In
Python Tutorial Python - Sockets Programming
Python Variable Types Python - Http Client
Python Xml Processing Python - Convert Binary To Ascii
Python - Dequeue Python - Capitalize And Translate
Python - Network Programming Python Basic Syntax

The first argument to PyArg_ParseTuple is the args argument. This is the object you will be parsing. The second argument is a format string describing the arguments as you expect them to appear. Each argument is represented by one or more characters in the format string as follows.

static PyObject *module_func(PyObject *self, PyObject *args) {
   int i;
   double d;
   char *s;

   if (!PyArg_ParseTuple(args, "ids", &i, &d, &s)) {
      return NULL;
   }
   
   /* Do something interesting here. */
   Py_RETURN_NONE;
}

Compiling the new version of your module and importing it enables you to invoke the new function with any number of arguments of any type −

module.func(1, s="three", d=2.0)
module.func(i=1, d=2.0, s="three")
module.func(s="three", d=2.0, i=1)

You can probably come up with even more variations.

The PyArg_ParseTuple Function

Here is the standard signature for PyArg_ParseTuple function −

int PyArg_ParseTuple(PyObject* tuple,char* format,...)

This function returns 0 for errors, and a value not equal to 0 for success. tuple is the PyObject* that was the C function's second argument. Here format is a C string that describes mandatory and optional arguments.

Here is a list of format codes for PyArg_ParseTuple function −

Code C type Meaning
c char A Python string of length 1 becomes a C char.
d double A Python float becomes a C double.
f float A Python float becomes a C float.
i int A Python int becomes a C int.
l long A Python int becomes a C long.
L long long A Python int becomes a C long long
O PyObject* Gets non-NULL borrowed reference to Python argument.
s char* Python string without embedded nulls to C char*.
s# char*+int Any Python string to C address and length.
t# char*+int Read-only single-segment buffer to C address and length.
u Py_UNICODE* Python Unicode without embedded nulls to C.
u# Py_UNICODE*+int Any Python Unicode C address and length.
w# char*+int Read/write single-segment buffer to C address and length.
z char* Like s, also accepts None (sets C char* to NULL).
z# char*+int Like s#, also accepts None (sets C char* to NULL).
(...) as per ... A Python sequence is treated as one argument per item.
|   The following arguments are optional.
:   Format end, followed by function name for error messages.
;   Format end, followed by entire error message text.

Returning Values

Py_BuildValue takes in a format string much like PyArg_ParseTuple does. Instead of passing in the addresses of the values you are building, you pass in the actual values. Here's an example showing how to implement an add function −

static PyObject *foo_add(PyObject *self, PyObject *args) {
   int a;
   int b;

   if (!PyArg_ParseTuple(args, "ii", &a, &b)) {
      return NULL;
   }
   return Py_BuildValue("i", a + b);
}

This is what it would look like if implemented in Python −

def add(a, b):
   return (a + b)

You can return two values from your function as follows, this would be cauptured using a list in Python.

static PyObject *foo_add_subtract(PyObject *self, PyObject *args) {
   int a;
   int b;

   if (!PyArg_ParseTuple(args, "ii", &a, &b)) {
      return NULL;
   }
   return Py_BuildValue("ii", a + b, a - b);
}

This is what it would look like if implemented in Python −

def add_subtract(a, b):
   return (a + b, a - b)

The Py_BuildValue Function

Here is the standard signature for Py_BuildValue function −

PyObject* Py_BuildValue(char* format,...)

Here format is a C string that describes the Python object to build. The following arguments of Py_BuildValue are C values from which the result is built. The PyObject* result is a new reference.

Following table lists the commonly used code strings, of which zero or more are joined into string format.

Code C type Meaning
c char A C char becomes a Python string of length 1.
d double A C double becomes a Python float.
f float A C float becomes a Python float.
i int A C int becomes a Python int.
l long A C long becomes a Python int.
N PyObject* Passes a Python object and steals a reference.
O PyObject* Passes a Python object and INCREFs it as normal.
O& convert+void* Arbitrary conversion
s char* C 0-terminated char* to Python string, or NULL to None.
s# char*+int C char* and length to Python string, or NULL to None.
u Py_UNICODE* C-wide, null-terminated string to Python Unicode, or NULL to None.
u# Py_UNICODE*+int C-wide string and length to Python Unicode, or NULL to None.
w# char*+int Read/write single-segment buffer to C address and length.
z char* Like s, also accepts None (sets C char* to NULL).
z# char*+int Like s#, also accepts None (sets C char* to NULL).
(...) as per ... Builds Python tuple from C values.
[...] as per ... Builds Python list from C values.
{...} as per ... Builds Python dictionary from C values, alternating keys and values.

Code {...} builds dictionaries from an even number of C values, alternately keys and values. For example, Py_BuildValue("{issi}",23,"zig","zag",42) returns a dictionary like Python's {23:'zig','zag':42}.



Frequently Asked Questions

+
Ans: Python GUI Programming (Tkinter) - Learn Python in simple and easy steps starting from basic to advanced concepts with examples including Python Syntax Object Oriented Language, Methods, Tuples, Tools/Utilities, Exceptions Handling, Sockets, GUI, Extentions, XML Programming. view more..
+
Ans: Python XML Processing - Learn Python in simple and easy steps starting from basic to advanced concepts with examples including Python Syntax Object Oriented Language, Methods, Tuples, Tools/Utilities, Exceptions Handling, Sockets, GUI, Extentions, XML Programming. view more..
+
Ans: Python Multithreaded Programming - Learn Python in simple and easy steps starting from basic to advanced concepts with examples including Python Syntax Object Oriented Language, Methods, Tuples, Tools/Utilities, Exceptions Handling, Sockets, GUI, Extentions, XML Programming. view more..
+
Ans: Python Extension Programming with C - Learn Python in simple and easy steps starting from basic to advanced concepts with examples including Python Syntax Object Oriented Language, Methods, Tuples, Tools/Utilities, Exceptions Handling, Sockets, GUI, Extentions, XML Programming. view more..
+
Ans: Python Quick Guide - Learn Python in simple and easy steps starting from basic to advanced concepts with examples including Python Syntax Object Oriented Language, Methods, Tuples, Tools/Utilities, Exceptions Handling, Sockets, GUI, Extentions, XML Programming. view more..
+
Ans: Python Tools/Utilities - Learn Python in simple and easy steps starting from basic to advanced concepts with examples including Python Syntax Object Oriented Language, Methods, Tuples, Tools/Utilities, Exceptions Handling, Sockets, GUI, Extentions, XML Programming. view more..
+
Ans: Python Useful Resources - Learn Python in simple and easy steps starting from basic to advanced concepts with examples including Python Syntax Object Oriented Language, Methods, Tuples, Tools/Utilities, Exceptions Handling, Sockets, GUI, Extentions, XML Programming. view more..
+
Ans: Discuss Python - LLearn Python in simple and easy steps starting from basic to advanced concepts with examples including Python Syntax Object Oriented Language, Methods, Tuples, Tools/Utilities, Exceptions Handling, Sockets, GUI, Extentions, XML Programming. view more..
+
Ans: Python Data Structure Tutorial for Beginners - Learn Python Data Structure in simple and easy steps starting from basic to advanced concepts with examples including Introduction, Environment, Arrays, Lists, Tuples, Dictionary, 2-D Array, Matrix, Sets, Maps, Linked Lists, Stack, Queue, Dequeue, Advanced Linked list, Hash Table, Binary Tree, Search Tree, Heaps, Graphs, Algorithm Design, Divide and conquer, Recursion, backtracking, Tree Traversal, Sorting, Searching, Graph Algorithms, Algorithm Analysis, Big-O Notation, Algorithim classes, Amortized analysis, Algorithm Justifications. view more..
+
Ans: Python Data Structure Introduction - Learn Python Data Structure in simple and easy steps starting from basic to advanced concepts with examples including Introduction, Environment, Arrays, Lists, Tuples, Dictionary, 2-D Array, Matrix, Sets, Maps, Linked Lists, Stack, Queue, Dequeue, Advanced Linked list, Hash Table, Binary Tree, Search Tree, Heaps, Graphs, Algorithm Design, Divide and conquer, Recursion, backtracking, Tree Traversal, Sorting, Searching, Graph Algorithms, Algorithm Analysis, Big-O Notation, Algorithim classes, Amortized analysis, Algorithm Justifications. view more..
+
Ans: Python Data Structure Introduction - Learn Python Data Structure in simple and easy steps starting from basic to advanced concepts with examples including Introduction, Environment, Arrays, Lists, Tuples, Dictionary, 2-D Array, Matrix, Sets, Maps, Linked Lists, Stack, Queue, Dequeue, Advanced Linked list, Hash Table, Binary Tree, Search Tree, Heaps, Graphs, Algorithm Design, Divide and conquer, Recursion, backtracking, Tree Traversal, Sorting, Searching, Graph Algorithms, Algorithm Analysis, Big-O Notation, Algorithim classes, Amortized analysis, Algorithm Justifications. view more..
+
Ans: Python Data Structure Introduction - Learn Python Data Structure in simple and easy steps starting from basic to advanced concepts with examples including Introduction, Environment, Arrays, Lists, Tuples, Dictionary, 2-D Array, Matrix, Sets, Maps, Linked Lists, Stack, Queue, Dequeue, Advanced Linked list, Hash Table, Binary Tree, Search Tree, Heaps, Graphs, Algorithm Design, Divide and conquer, Recursion, backtracking, Tree Traversal, Sorting, Searching, Graph Algorithms, Algorithm Analysis, Big-O Notation, Algorithim classes, Amortized analysis, Algorithm Justifications. view more..
+
Ans: Python Data Structure Introduction - Learn Python Data Structure in simple and easy steps starting from basic to advanced concepts with examples including Introduction, Environment, Arrays, Lists, Tuples, Dictionary, 2-D Array, Matrix, Sets, Maps, Linked Lists, Stack, Queue, Dequeue, Advanced Linked list, Hash Table, Binary Tree, Search Tree, Heaps, Graphs, Algorithm Design, Divide and conquer, Recursion, backtracking, Tree Traversal, Sorting, Searching, Graph Algorithms, Algorithm Analysis, Big-O Notation, Algorithim classes, Amortized analysis, Algorithm Justifications. view more..
+
Ans: Python Data Structure Introduction - Learn Python Data Structure in simple and easy steps starting from basic to advanced concepts with examples including Introduction, Environment, Arrays, Lists, Tuples, Dictionary, 2-D Array, Matrix, Sets, Maps, Linked Lists, Stack, Queue, Dequeue, Advanced Linked list, Hash Table, Binary Tree, Search Tree, Heaps, Graphs, Algorithm Design, Divide and conquer, Recursion, backtracking, Tree Traversal, Sorting, Searching, Graph Algorithms, Algorithm Analysis, Big-O Notation, Algorithim classes, Amortized analysis, Algorithm Justifications. view more..
+
Ans: Python Data Structure Introduction - Learn Python Data Structure in simple and easy steps starting from basic to advanced concepts with examples including Introduction, Environment, Arrays, Lists, Tuples, Dictionary, 2-D Array, Matrix, Sets, Maps, Linked Lists, Stack, Queue, Dequeue, Advanced Linked list, Hash Table, Binary Tree, Search Tree, Heaps, Graphs, Algorithm Design, Divide and conquer, Recursion, backtracking, Tree Traversal, Sorting, Searching, Graph Algorithms, Algorithm Analysis, Big-O Notation, Algorithim classes, Amortized analysis, Algorithm Justifications. view more..
+
Ans: Python Data Structure Introduction - Learn Python Data Structure in simple and easy steps starting from basic to advanced concepts with examples including Introduction, Environment, Arrays, Lists, Tuples, Dictionary, 2-D Array, Matrix, Sets, Maps, Linked Lists, Stack, Queue, Dequeue, Advanced Linked list, Hash Table, Binary Tree, Search Tree, Heaps, Graphs, Algorithm Design, Divide and conquer, Recursion, backtracking, Tree Traversal, Sorting, Searching, Graph Algorithms, Algorithm Analysis, Big-O Notation, Algorithim classes, Amortized analysis, Algorithm Justifications. view more..
+
Ans: Python Data Structure Introduction - Learn Python Data Structure in simple and easy steps starting from basic to advanced concepts with examples including Introduction, Environment, Arrays, Lists, Tuples, Dictionary, 2-D Array, Matrix, Sets, Maps, Linked Lists, Stack, Queue, Dequeue, Advanced Linked list, Hash Table, Binary Tree, Search Tree, Heaps, Graphs, Algorithm Design, Divide and conquer, Recursion, backtracking, Tree Traversal, Sorting, Searching, Graph Algorithms, Algorithm Analysis, Big-O Notation, Algorithim classes, Amortized analysis, Algorithm Justifications. view more..
+
Ans: Python Data Structure Introduction - Learn Python Data Structure in simple and easy steps starting from basic to advanced concepts with examples including Introduction, Environment, Arrays, Lists, Tuples, Dictionary, 2-D Array, Matrix, Sets, Maps, Linked Lists, Stack, Queue, Dequeue, Advanced Linked list, Hash Table, Binary Tree, Search Tree, Heaps, Graphs, Algorithm Design, Divide and conquer, Recursion, backtracking, Tree Traversal, Sorting, Searching, Graph Algorithms, Algorithm Analysis, Big-O Notation, Algorithim classes, Amortized analysis, Algorithm Justifications. view more..




Rating - NAN/5
466 views

Advertisements