Friday, December 29, 2006

Approaching 0.1

After 0.0.15 I have quickly released 0.0.16. It essentially adds frozenset and fixes some bugs, as reported by users of 0.0.15. The next release is coming along nicely too, and should be ready within about a week. Here's a list of changes:

-added frozenset
-time.sleep now works on WIN32
-constant-string expressions and __doc__ attributes are made into nice C++ comments
-added --nowrap optimization option to ss.py (disables checking for negative indices)
-several minor bug-fixes reported by users of 0.0.15

Because I have a huge amount of spare time coming up to work on Shed Skin, I have started to think about what is necessary to do a 0.1 release. These are currently the most important shortcomings:

-support for iterators, generators and generator expressions

The first two seem mostly a case of how to do this nicely in C++, and the latter one seems easy.

-support for tuples t of len > 2 and with different types of elements

Again this seem mostly a case of how to do this nicely in C++. I'm looking forward to finding out how to do this.

-scalability of the type analysis

Because of all the details involved in doing a python compiler, I still haven't focused on this too much. The basic techniques seem to work very well, but I have many ideas yet to work out to improve scalability. Based on my work so far, I'm convinced it is possible to quickly analyze most programs up to a few thousands lines, even without any form of profiling.

-two-way integration with CPython

There may be some progress at this front, as another developer has decided to look into using boost.python for this purpose. It's rather exciting to see small fragments of this work already. If anything, it has made me optimistic about having something like this work in the near future.

-some optimizations: strings (replace std::string), indexing (remove wrap-around checks by proving an index expression is always positive), heap allocation (transform heap allocation into stack- and static preallocation, when possible)

While not essential for a 0.1 release, an important goal of Shed Skin is to generate really fast code. Especially the latter optimization would be a great master's thesis subject.. :-)

7 comments:

Baczek said...

check out boost::tuple.

srepmub said...

hi again baczek,

yes I'm aware of this, and it looks pretty nice. but it won't fit in the hierarchy of my C++ implementation of the python builtins. I will probably have a look at the code though, to see what (tricks) I can use.

thanks!
mark.

Unknown said...

Hi Mark,

Do you think the two way integration with CPython will be ready for 0.1?
thanks,
Flavio

srepmub said...

hello flavio,

yes, for me that is the most important goal for a 0.1 release :-) if you cannot build extension modules or import 're', for example, shed skin is not very useful in general.


mark.

Unknown said...

Hi MarK,

I have a wish for 0.1:

support for the threading module.

Is it too hard?

I hope it can make it...

It would free python programs from the GIL!!! Will it?

I have a question also:
If I want to add support to a pure-python package to SS, is there a way to automate the convertion of all modules within a package, or do I have to convert one module at a time?

thanks,

Flávio

Unknown said...

As a follow up, I actually tried to convert Threadin but got this error message from SS:

*ERROR* threading:5: no such class: 'ImportError'

Does SS support all built-in exception classes?

Flávio

p.s. This error occurs in the line where threadin tries to import thread (a built-in module).

srepmub said...

hello flavio,

thanks again for the comments/questions. it may be more useful to use the mailing list for extensive discussions though ^^ see the sourceforge site on how to subscribe.

>support for the threading module. Is it too hard?

I don't think it's too hard, but I'm not planning on looking at it anytime soon.. unfortunately I don't get much help, so other things have higher priority right now. you might have a go at it - see the README on how to implement libraries.

>If I want to add support to a pure-python package to SS, is there a way to automate the convertion of all modules within a package, or do I have to convert one module at a time?

if you have a single module, you can try to convert it in one go (this is what I did with getopt). support for hierarchies/packages of modules is still absent though in shed skin, so you'd have to hack your way around some limitations. the threading module/package is probably very nasty though..

>It would free python programs from the GIL!!! Will it?

I think you still need some kind of locks..

>Does SS support all built-in exception classes?

no, not yet. I'm adding them when I see them in a program I'm trying to support. you can see the ones are supported in lib/builtin.py. note that some are only there but never thrown. e.g., indexerror doesn't work yet (I will add a command-line option to ss.py to enable these). let me know if you really need an exception type hat is not supported.

as you can see from the version number also, shed skin is still in quite an early stage of development. I counted 22 largish programs though, and the stability is improving, but there's a lot more work to be done. please let me know if you encounter any other problems, have any wishes or would like to help out.


thanks ^^
mark.