The following is a performance comparison of several (pure) Python implementations, for a large part of the Shed Skin example set. I left out some of the examples, that would result in an unfair comparison (mostly because of randomization), or that were too interactive to easily measure. Obviously this comparison is very biased, and probably unfair in some way to the other projects (though I've tried to be fair, for example by letting PyPy stabilize before measuring).
This first plot shows the speedups versus CPython 3.10, for CPython 3.14, Nuitka, Pypy and Shed Skin.
Shed Skin is able to speed up the examples by an average factor of about 29 times (not percent, times :)), while PyPy is able to speed up the examples by an average factor of about 16 times. Given that PyPy has its hands tied behind its back trying to support unrestricted Python code, and was not optimized specifically for these examples (that I am aware of), that is actually still quite an impressive result.
As for the few cases where PyPy performs better than Shed Skin, this appears to be mainly because of PyPy being able to optimize away heap allocations for short-lived objects (in many cases, custom Vector(x,y,z) instances). In a few cases also, the STL unordered_map that Shed Skin uses to implement dictionaries appears to perform poorly compared to more modern implementations. Of course it is possible for Shed Skin to improve in these areas with future releases.
Note that some of the examples can run even faster with Shed Skin by providing --nowrap/--nobounds options, which disable wrap-around/bounds-checking respectively. I'm not sure if PyPy has any options to make it run faster, at the cost of certain features (in the distant past there was talk of RPython - does that still exist?).
As the CPython 3.14 and Nuitka results are a bit hard to see in the above plot, here is the same plot but with a logarithmic y-scale:
CPython 3.14 is about 60% faster on average for these examples than CPython 3.10, which to me is actually very promising for the future. While Nuitka outperforms CPython 3.10 by about 30% on average, unfortunately it cannot match the improvements in CPython since.
If there are any CS students out there who would like to help improve Shed Skin, please let me know. I think especially memory optimizations (where PyPy still seems to have an edge) would be a great topic for a Master's Thesis!
4 comments:
I believe RPython still exists, but it's just an implicitly static version of the Python language that Pypy knows how to translate to an executable. Pypy itself is written in RPython, but RPython isn't really something you'd want to write a typical Python project in.
BTW, I Love Shedskin, but I wish I could write a generator callable from CPython with it. Last time I looked, there wasn't a way to do that. I have a Python project that would benefit greatly from that.
hi dan, thanks!! I will have a look at exporting generators/iterators, hopefully next week. it might help if you could send me a small example of what you are trying to achieve. cheers, mark.
Hi Mark, this is amazing work. I note that shedskin's performance for your Othello example is literally off the charts!
haha, indeed, thanks! :) yeah, I think GCC is able to find a SIMD optimization there.. this is usually also used (manually I don't know?) for fast othello bitboarding.
Post a Comment