Shed Skin restricted-Python-to-C++ compiler 0.9.9

I have just released version 0.9.9 of Shed Skin, a restricted-Python-to-C++ compiler. It comes with many changes under the hood to improve the code base. For example, Shakeeb has started adding type annotations to Shed Skin itself, whereas I have fixed many C++ compiler warnings. We have also replaced the old CPython-based dict and set implementations with STL unordered_map and unordered_set. While this may cost some performance for especially small toy programs, it really helps with maintainability.

There was also a type inference improvement, which hasn't happened in a while since it wasn't needed. It was however needed to add a cool new example called "pycsg", which shows how one can perform CSG (constructive solid geometry) using binary space partitioning (BSP). After some investigation I decided to make the type inference slightly less "optimistic", which should make it scale a lot better, at the cost of being slower overall. In the end this new example runs about 15 times faster after compilation on my system (not 15%.. 15 times!).

Most notably on the outside, Shed Skin now comes with --floatXX/--intXXX options, which allow you to specify the desired float/int precision. Unfortunately under windows, --int128 does not yet work, since C++ has not yet standardized 128-bit integers.. The relatively new othello2 and collatz examples now require --int64 and --int128, respectively, to function properly.

For the full list of changes, please see the release notes.

I would like to take this opportunity to (once again) invite others to help out in Shed Skin development. There is always enough to do, also low-hanging fruit, on both the Python and the C++ side of things. I at least think it is rather fun to work on.. :) Let me know if you'd like to contribute but aren't sure what you could bring to the table.

Of course just sending in feedback or new example programs (especially if they fail!) can be very motivating as well to keep improving things.

Fast DOOM WAD renderer in 999 lines of Python

For the longest time, I've wanted to re-implement the original DOOM engine in Python, and compile it with Shedskin to get reasonable performance. So when I finally ran across a pretty small engine written in Java, by Leonardo Ono, I decided to convert his version to Python.

I left out some optimizations (most notably "visplanes"), for readability and in order to keep everything under 1000 lines of code (while following PEP8). So it could run quite a bit faster still. This is including a WAD loader and interactive Pygame wrapper, that makes it almost feel like you are playing DOOM.

It's interesting how using Python (using just CPython) seems to be about as fast as using assembler back in the day.

This video shows the day-and-night difference in FPS after compilation using Shedskin.

The source code can be found in the Shedskin examples directory.

I may revisit the code now and then, to try and improve readability by adding comments and/or refactoring parts. Patches in this direction very much welcome of course.

In general, we are also always looking to receive more feedback and contributions from the community! While not the 'final' solution to Python performance, Shed Skin is great fun to hack on. See some potential issues for the next release to start looking into.

Shed Skin restricted-Python-to-C++ compiler 0.9.6

I recently decided to sit down and finally port Shed Skin, an experimental restricted-Python-to-C++ compiler in the works since 2005 or so, to Python3. Three painful months and a total diff of 50k lines later, everything now works with Python3 (Shed Skin itself, and all tests and examples..)

This does not mean that every Python3 feature is supported, but what was there now at least works fine with Python3.. For example, unicode is still restricted to 1-byte characters, and there is no support (yet) for nice new features such as f-strings. Python2 support has been dropped with the new release, and subsequent releases should add support for various new Python3 features.

The following people have contributed along the way:

• Johan Kristensen (large patch for moving from compiler.ast to ast, still on Python2)
• Shakeeb Alireza (extension module support for Python3, ported many examples, improved OSX support, and various code cleanups)
• Folkert van Heusden (some fixes on the C++ side, move to c++17)
• Jeremie Roquet, Thomas Spura, Paul Boddie and others who kept doing maintenance on the project in my absence

I started work on the port after realizing that Shed Skin was being removed from distributions, especially Debian, as it was still tied to Python2, and I really want to keep Shed Skin in a working state if possible.

So what about the future of the project? Not really sure, but I'm happy to sit down again in a few months to prepare a new release. Any feedback on what to support or improve would be very welcome!

For fun, here are screenshots of some of the Shed Skin example programs (in total 75 working example programs can be found on the github site):

Shed Skin 0.9.4

I have just released Shed Skin 0.9.4, a (restricted-)Python-(2.x)-to-C++ compiler. The full release notes can be found here, as usual.

Major thanks go to Ernesto Ferro, who has been making many large refactorings in the code to improve maintainability. He also found a nice new example, called Gh0stenstein (see picture below). Paul Haeberli has also triggered several very useful improvements.

Besides refactoring, most changes are minor fixes or improved tests. But there are quite a few of them. There are also 3 new examples, meaning there are now 75 examples in total, which I think is a milestone in itself.

Shed Skin 0.9.3

I have just released version 0.9.3 of Shed Skin, a restricted-Python (2.4-2.6) to C++ compiler. This is the third maintenance release since 0.9, and as such there were (almost) no major changes.

For the full list of changes since 0.9.2, please see the release notes.

Five interesting new examples were added for this release, bringing the total to 72. Most interesting to me are Pygasus, a 1,500 line NES emulator that is able to play mario bros (Maciek "Mistrall" Żuk), as well as a pure Python implementation of timsort, the famous Python sorting algorithm (ported from java to python by Dan Stromberg).

In the meantime, the C64 emulator (see my earlier posts) has grown to about 6,000 lines of code (sloccount) but is still compiling fine (though it takes about 10 minutes here, still not that bad). I'm tempted to update the scalability graph I posted earlier.

In any case, I would like to thank the authors of the new example programs for sharing their code (maciek, dan, thomas, piotr, keegan) and those who provided improvements (danny milosavljevic) or feedback since 0.9.2.

Shed Skin 0.9.2

I have just released version 0.9.2 of Shed Skin, a restricted-Python (2.4-2.6) to C++ compiler. This is the second maintenance release since 0.9, so no new major features were added.

The biggest improvement is probably some optimizations for the array module. Other than this, the changes are mostly unrelated minor optimizations and bug fixes. Please see the release notes for more detail.

Three nice new examples were also added for this release, bringing the number of examples to 67, with a total linecount of about 17,000 lines (sloccount): a "stereo vision" program, and two Rubik's cube solvers.

Shed Skin 0.9.1

I have just released version 0.9.1 of Shed Skin, a restricted-Python (2.4-2.6) to C++ compiler. This is a maintenance release, so no new major features were added. Most interesting perhaps are optimizations for itertools.product and str.join. Other than that, it's mostly bugfixes and a new (hq2x image scaling) example. Please see the release notes for more details.

In the meantime, Paul Boddie has succeeded in getting Shed Skin 0.9 accepted into the Debian repositories, which is awesome for me personally.