A (biased) Pure Python Performance Comparison

The following is a performance comparison of several (pure) Python implementations, for a large part of the Shed Skin example set. I left out some of the examples, that would result in an unfair comparison (mostly because of randomization), or that were too interactive to easily measure. Obviously this comparison is very biased, and probably unfair in some way to the other projects (though I've tried to be fair, for example by letting PyPy stabilize before measuring).

This first plot shows the speedups versus CPython 3.10, for CPython 3.14, Nuitka, Pypy and Shed Skin.

Shed Skin is able to speed up the examples by an average factor of about 29 times (not percent, times :)), while PyPy is able to speed up the examples by an average factor of about 16 times. Given that PyPy has its hands tied behind its back trying to support unrestricted Python code, and was not optimized specifically for these examples (that I am aware of), that is actually still quite an impressive result.

As for the few cases where PyPy performs better than Shed Skin, this appears to be mainly because of PyPy being able to optimize away heap allocations for short-lived objects (in many cases, custom Vector(x,y,z) instances). In a few cases also, the STL unordered_map that Shed Skin uses to implement dictionaries appears to perform poorly compared to more modern implementations. Of course it is possible for Shed Skin to improve in these areas with future releases.

Note that some of the examples can run even faster with Shed Skin by providing --nowrap/--nobounds options, which disable wrap-around/bounds-checking respectively. I'm not sure if PyPy has any options to make it run faster, at the cost of certain features (in the distant past there was talk of RPython - does that still exist?).

As the CPython 3.14 and Nuitka results are a bit hard to see in the above plot, here is the same plot but with a logarithmic y-scale:

CPython 3.14 is about 60% faster on average for these examples than CPython 3.10, which to me is actually very promising for the future. While Nuitka outperforms CPython 3.10 by about 30% on average, unfortunately it cannot match the improvements in CPython since.

If there are any CS students out there who would like to help improve Shed Skin, please let me know. I think especially memory optimizations (where PyPy still seems to have an edge) would be a great topic for a Master's Thesis!