py2cr.py

A code translator using AST from Python to Crystal. This is basically a NodeVisitor with Crystal output. See AST documentation (https://docs.python.org/3/library/ast.html) for more information.

Status

Currently more than 80% of the relevant tests are passing. See more information below.

Installation

Execute the following:

pip install py2cr

or

git clone git://github.com/nanobowers/py2cr.git
cd py2cr
python setup.py

Versions

• Python 3.6 .. 3.9
• Crystal 1.1+

Dependencies

Crystal

• num.cr - Preliminary support for translating numpy python code into crystal relies on num.cr. This requires the latest (master) branch.

Methodology

In addition to walking and writing the AST tree and writing a Crystal syntax output, this tool either:

• Monkey-patches some common Crystal stdlib Structs/Classes in order to emulate the Python equivalent functionality.
• Calls equivalent Crystal methods to the Python equivalent
• Calls wrapped Crystal methods that provide Python equivalent functionality

Usage

Generally, py2cr.py somefile.py > somefile.cr

There is a Crystal shim/wrapper library in src/py2cr (and linked into lib/py2cr) that is also referenced in the generated script. You may need to copy that as needed, though eventually it may be appropriate to convert it to a shard if that is more appropriate.

Tests

$ ./run_tests.py

Will run all tests that are supposed to work. If any test fails, its a bug. (Currently there are a lot of failing tests!!)

$ ./run_tests.py -a

Will run all tests including those that are known to fail (currently). It should be understandable from the output.

$ ./run_tests.py basic

Will run all tests matching basic. Useful because running the entire test-suite can take a while.

$ ./run_tests.py -x or $ ./run_tests.py --no-error

Will run tests but ignore if an error is raised by the test. This is not affecting the error generated by the test files in the tests directory.

For additional information on flags, run:

./run_tests.py -h

Writing new tests

Adding tests for most new or existing functionality involves adding additional python files at tests/<subdirectory/<testname>.py.

The test-runner scripts will automatically run py2cr to produce a Crystal script, then run both the Python and Crystal scripts, then compare stdout/stderr and check return codes.

For special test-cases, it is possible to provide a configuration YAML file on a per test basis named tests/<subdirectory>/<testname>.config.yaml which overrides defaults for testing. The following keys/values are supported:

min_python_version: [int, int] # minimum major/minor version
max_python_version: [int, int] # maximum major/minor version
expected_exit_status: int      # exit status for py/cr test script
argument_list: [str, ... str]  # list of strings as extra args for argv

Typing

Some amount of typing support in Python is translated to Crystal. Completely untyped Python code in many cases will not be translatable to compilable Crystal. Rudimentary for python Optional and Union should convert appropriately to Crystal typing.

Some inference of bare list/dict types can now convert to [] of X and {} of X, however set and tuple may not work properly.

Status

This is project is and will continue to be incomplete because

• transpiling is hard
• the surface area of Python is extremely large
• not all Python things have a simple/good mapping into Crystal

The goal is to cover common cases and reduce the additional work to create a minimum-viable-program.

Many of the tests brought forward from py2rb do not pass. Some of them may never pass as-is due to significant language / compilation differences (even moreso than Python vs. Ruby)

Some numpy tests have recently been re-enabled with recent support for transpiliing to num.cr. Numpy related tests can be run with ./run_tests.py numpy

Additional tests have been imported from py2many. Many of these do not operate (known to fail), but some have been used to enhance coverage of py2cr. 🎉 Run these with ./run_tests.py py2many It is possible some of the py2many tests cover pre-existing tests, so some many be pruned out later on.

Limitations

• Many Python run-time exceptions are not translatable into Crystal as these issues manifest in Crystal as compile-time errors.
• A significant portion of python code is untyped and may not translate properly in places where Crystal demands type information.
  • e.g. Crystal Lambda function parameters require typing and this is very uncommon in Python, though may be possible with Callable[] on the python side.
• Python importing is significantly different than Crystal and thus may not ever map well.
• Python Unittest does not have an equivalent in Crystal. With some significant additional work, converting tests into Spec format may be possible via https://github.com/jaredbeck/minitest_to_rspec as a guide.

To-do

• Remove python2/six dependencies to reduce clutter. Py2 has been end-of-lifed for a while now.
• Remove numpy dependencies unless/until a suitable target for Crystal can be identified (targeting num.cr now)
• Add additional Crystal shim methods to translate common python3 stdlib methods. Consider a mode that just maps to a close Crystal method rather than using a shim-method to reduce the python-ness.
• Refactor the code-base. Most of it is in the __init__.py
• Add additional unit-tests
• Multi-thread the test-suite so it can run faster.

Contribute

Free to submit an issue. This is very much a work in progress, contributions or constructive feedback is welcome.

If you'd like to hack on py2cr, start by forking the repo on GitHub:

https://github.com/nanobowers/py2cr

Contributing

The best way to get your changes merged back into core is as follows:

1. Fork it (https://github.com/nanobowers/py2cr/fork)
2. Create a thoughtfully named topic branch to contain your change (git checkout -b my-new-feature)
3. Hack away
4. Add tests and make sure everything still passes by running crystal spec
5. If you are adding new functionality, document it in the README
6. If necessary, rebase your commits into logical chunks, without errors
7. Commit your changes (git commit -am 'Add some feature')
8. Push to the branch (git push origin my-new-feature)
9. Create a new Pull Request

License

MIT, see the LICENSE file for exact details.