Some nice additions to /examples

(Shed Skin is an experimental (restricted-)Python-to-C++ compiler.)

Because the set of example programs available at the Shed Skin homepage (ssprogs.tgz) hasn't changed in a while, I decided to add some nice programs to it. With some relatively simple (but as usual, important) fixes, I was able to add support for a Jpeg decompressor (!), at 1200 lines, an iPod shuffle programmer, at 600 lines, and a multi-module mastermind program, at 300 lines.

I will be adding these programs to ssprogs.tgz with 0.0.31 (or 0.1), but for now they can be found here. They should compile using Shed Skin SVN (see the note in the top of rdb.py; TonyJpegDecoder.py should be run with two arguments, e.g. "python TonyJpegDecoder.py tiger1.jpg tiger1.bmp").

Thanks to ********** for sending me the jpeg decompressor, and the authors of these programs for the nice test cases!

I would have probably added more programs (and fixed more things) if I could find more interesting test cases, but these seem hard to come by. If you happen to know of any nice programs between 500 and, say, 2,000 lines that might be statically compiled, possibly after some minor changes, please let me know. Some more bug reports would also be very welcome.

Shed Skin 0.0.30

I have just released version 0.0.30 of Shed Skin, an experimental (restricted) Python-to-C++ compiler.

Most importantly, this release adds (efficient) support for user-defined classes in generated extension modules, which should make it much easier to integrate compiled code within larger projects. More specifically, compiled classes can now be instantiated on the CPython side, and instances can be passed freely between CPython and Shed Skin without any conversion taking place. (Instances of builtin classes are still (recursively) copied, though, at the moment..)

Another major improvement was contributed by FFAO: a new 'set' implementation, directly based on the CPython code. While I haven't tested it on many benchmarks, it is clear that is now much faster, and on one benchmark it even outperforms CPython on my system by about 35%.

Other notable changes include complex number support, mapping None to NULL instead of 0 and printing it as 'None', as well as an important type inference fix.

With support for user-defined classes in extension modules, it looks like all the major pieces are now there to do a 0.1 release. The only thing I'd really like to do before that, is to improve support for the 'os' module. Please let me know if you'd like to help out here! Hopefully, with many details out of the way, I can have another good look at type inference for 0.2..

Shed Skin in the News

(Shed Skin is an experimental (restricted-)Python-to-C++ compiler).

Because a new version (0.0.30) is still some weeks away, I decided to post, in the meantime, some links to other people's blog entries about Shed Skin.

making-python-math-196-faster-with-shedskin
taking-on-shed-skin
more performance python
compiled kamaelia

Please let me know if I have missed an interesting entry.

And if you are really bored, here are also some links to (old) discussions on slashdot and osnews:

slashdot article
osnews article

Shed Skin 0.0.29

I have just released version 0.0.29 of Shed Skin, an experimental (restricted) Python-to-C++ compiler. It's been a while since the last release (well, just under 4 months), because of work and vacation, but mostly because there have been so many changes.

Thanks to the work of Karel Heyse, Pavel Vinogradov, FFAO and David Marek, there is now a pretty much complete implementation of the datetime module. Thanks to a suggestion by Albert Hofkamp, I added support for most of the ConfigParser module (by compiling it with Shed Skin of course), which led me to fix several outstanding but important problems. For example, support for inheritance hierarchies was greatly improved, and mapping keys should now work, too ('%(key)x..' % some_dict).

Since the previous release, I've also gone through the pains of installing FreeBSD, OpenSolaris and 64-bit Ubuntu on a spare PC and testing Shed Skin on each of them, with the result that these platforms are now 'officially' supported. Please see the updated 'installation' section of the tutorial. I also received some GCC 4.3 patches (Seo Sanghyeon and Winterknight), which should work fine now, too. So I guess most platforms should be covered now.

Some other improvements:

-improved support for importing from nested modules such as os.path
-__init__ methods are much less a special-case
-improved support for calling ancestor methods (e.g. Parent.__init__)
-all example programs (ss-progs) now compile as extension modules
-staticmethod and property decorator support (Seo Sanghyeon)
-Shed Skin doesn't crash on highly dynamic/recursive types anymore, making it easier to debug programs to get them to compile
-several fixes in the re module, e.g. re.sub now accepts a replacement function
-tuple hash caching was disabled, as CPython doesn't do this either

For the full changelog, see the release notes wiki at the Googlecode site. Please try out the new release, and let me know about any problems. Note that I probably won't be very responsive for at least a week or so - it has been a somewhat difficult release, and I could use some rest.. :)

Shed Skin 0.0.28

I have just released Shed Skin 0.0.28, with the following changes. Thanks to those mentioned for helping out!

- basic 'socket' support (Michael Elkins)
- support for os.{popen3, popen4} under UNIX (Jaroslaw Tworek)
- support for time.strptime under Windows (David Marek)
- options for changing output dir, disabling annotation (Dave Tweed)
- support for 'cmp' and 'reverse' arguments of 'sorted' and 'list.sort'
- fixes for cross-module default arguments
- important fixes for type inference and inheritance
- restore compatibility with Python 2.3
- many minor bugfixes

I would really like to receive more bug reports. Please try out the new version, and let me know about any problems.

With the socket support, 15 common modules are now largely supported. For a 0.1 release, I'd really like to have support for one more module: datetime. Thanks to the GHOP, there is a type model already (lib/datetime.py), so we only still need a C++ implementation..

Shed Skin 0.0.27

Thanks to the GHOP students, this new release contains support for 're' (via libpcre!), 'time' (except strptime under windows) and 'fnmatch' and 'glob', as well as much improved support for 'os' (POSIX), and also a typemodel for 'datetime'. This means Shed Skin now supports (most of) the following standard library modules:

bisect, collections, copy, fnmatch, getopt, glob, math, os, os.path, random, re, string, sys and time.

If anyone is interested in implementing/bootstrapping 'datetime', 'socket' or in improving 'os' support (esp. under Windows) or other modules, please let me know!

I got more help for this release. Brian Blais helped fix OSX support again, and added support for building extension modules under OSX. And Mark Dewing (in attempts to compile parts of Quameon, a monte carlo quantum atom simulator) sent in several bug reports. Several other people also sent in bug reports.

Of course I have added some things myself, too ;) For one, I added basic support for the 'staticmethod' and 'property' builtins. I also (finally, might I say) split up the compiler core (ss.py) into several files, added many fixes for multi-dir/multi-file projects, further optimized a few builtins, and fixed many minor issues.

In all, I'm very happy with this release, and hope to get as much help for the next release!

Shed Skin 0.0.26

After my 'rant' about not getting much help with Shed Skin, I received a lot of feedback on how to improve packaging and usability in general. Several people even started helping out with this :-) Most notably, Paul helped me to create a Debian package, and James wrote a tutorial, which has by now replaced the README. Thanks guys!!

Shortly after my rant, somehow I also got involved in the Google GHOP project, in which high school students are paid to help out with open source projects. Some of these students turned out to be a real help, and one of them even managed to add complete 're' support to Shed Skin, using PCRE (perl-compatible-regular-expression library, thanks Cyril for the suggestion; this will be available in the next release!).

All of this, in turn, inspired me to add (almost) complete support for os.path and collections.defaultdict (deque was already supported). The combined result of all this should be a much more 'usable' project, although many things can still be improved of course. For the next release, I am planning on splitting up the compiler core (ss.py), and adding support for time, datetime, socket and as mentioned re.

When all this works well, I think the time has finally come to have another look at type inference scalability. Two ideas that stand out in my mind are iterative deepening (restart after an increasing number of iterations, each time combining everything that was learned) and selector-based filtering (basically, the idea is that for a method call x.bleh(), obviously x can only be of a type that has a bleh method, though it can be generalized further, e.g. for x=y).

Anyway, getting back to the subject, I have just released Shed Skin 0.0.26. It took me a while to get there, but this is mostly because it has many improvements:

-support for:
-most os.path methods (bootstrapped using Shed Skin)
-many os methods (ghop; many still remaining)
-collections.defaultdict (completing collections)
-getopt.gnu_getopt (bootstrapped, completing getopt)
-5 of the last 7 missing str methods (ghop)

-optimized string addition (a+b+c..)
-improved locale support (ghop)
-removed many leading underscores from generated code
-new documentation/tutorial
-added a Debian package
-many minor bugfixes

Additionally, I moved the project from SourceForge (sorry, guys) to Google code hosting. This is the new address:

http://shedskin.googlecode.com