When it comes to languages, every programmer has a preference. I've been using Python a lot, because its very-high-level nature fits my way of thinking. What Python sacrifices, namely speed and robustness, is usually not a problem. It's still hard not to want for more, especially after working in languages like Nim.
But Python can't go there. It very much wants to be interpreted... or does it?
I've been aware of Cython for a while, thanks to a friend (hi, Avaxar!) but wasn't interested until discovering three things:
- It's included in Linux distributions like Debian and Alpine.
- It can make executables, not just (shared) libraries.
- It doesn't require modifying Python sources.
Yes, seriously. Cython is a compiler you can use to turn your unchanged Python scripts into executables, then make them faster. They won't be any easier to distribute, since they'll still depend on the Python runtime and libraries you compiled against, but there are other advantages:
- More upfront type-checking; in Python it's otherwise all too easy to return different types from a function on a whim, or forget to remove old code paths that refer to now-missing variables.
All your source code (and any non-native libraries) get bundled together and compiled in, so you can just drop the resulting executable in
/usr/local/bin and call it a day.
- Faster startup and shutdown times (especially noticeable when using Pygame).
So how to get started? Once Cython is installed, it can be as simple as
cython3 -3 --embed wireview.py -o wireview.c
gcc -I /usr/include/python3.7 wireview.c -lpython3.7m -O2 -o wireview
Obviously the exact invocation depends on the local environment. Tip: check the size of the intermediate C file before enabling optimization, they grow large.
Most Python code is supported out of the box, and just like that, it's now an executable. There's just one problem: it's probably a little bit slower than in the interpreter. Time to declare types.
Now, there are three different ways to do that in Cython:
- By applying decorators from the
- By using Cython's extended syntax (in a file with the .pyx extension).
- By adding just the declarations in a separate .pxd file.
Perhaps surprisingly, Python's own type annotations aren't supported at all. (Edit: apparently they are, but I didn't see it documented until recently.)
But what types can be declared? It's a mix of Python and C. While
tuple are what you expect,
int will be understood as the native C type. Same with
float... which isn't the same as Python's; that would be a
double. You can declare types for variables, functions and classes, though in the latter case I had trouble with inheritance. If that happens, simply don't declare them and Cython will treat them as pure Python code that should work, if slowly.
(Note, Python types are still checked at runtime. It's a mix of static and dynamic. On the plus side, you retain access to all libraries you were using.)
The only problem with declaration files is, you need to write one for each Python module you want to compile, then make each of them into a shared library that the main executable will load? Something like that, I'm not sure yet.
(In fact, documentation needs more work; vital information is buried in unlikely places, at other details I had to guess, and nobody gets
Luckily, games like Tomb of the Snake are already all in one file, and so are my interpreters. Simply declaring argument and return types for most functions can speed up one of the latter 8.5 times. And some kinds of interpreters are a lot easier to code in Python than in less flexible languages.
Of course, I've been using Cython for a few days now, in limited ways, and frankly it's designed to speed up scientific software, not so much other code. But it's also hard to notice any improvements from compilation when the machine I have now is dozens of times faster than back when I wrote some of my projects.
It's still a different way of looking at the same code, a new reason to write it more cleanly, and a new lease on life for once-abandoned experiments.
Be right back, got a lot more of those to revive and make interesting again.
P.S. Turns out Cython doesn't follow imports after all, and since multiple files can't be used with
--embed, a different approach is needed for programs made of several modules.