Python 2 EOL Roadmap?

[fake-name]

LinuxCNC includes a lot of python components, and as everyone is probably aware, Python 2 is EOL in a little more then 2 years.

What's the roadmap for the python 3 migration?

It might be a decent idea to start requiring that people who touch python files make them support either platform. It's completely possible to write python code that works in both versions.

[SebKuzminsky]

I would love an effort to make our python code 2/3 compatible.

Are there tools in the python world that can identify 2/3 compatibility problems in python programs and modules? Maybe through static analysis? "Flake" maybe?

If so we should add build-time tests to run them; for now the test will just warn on failures, then once we have full 2/3 compatibility we'll switch it to fail the build on failures.

[SebKuzminsky]

pylint --py3k seems like an option. Here's the output for axis:
pylint-axis.txt

[gmoccapy]

I agree, that we should go the way to write code for both python versions. A good way to start is also the page:

Python future

I will check also on pylint

Norbert

[KurtJacobson]

The python code will be easy to convert to work with both Python 2 and 3, the part I am not so sure about is the C/C++ python bindings. I think boost python is used for those. We probably will have to add a switch to compile for one or the other version of python. Ideally the bindings would work with both, but I am not sure how feasible that is.

[fake-name]

I would love an effort to make our python code 2/3 compatible.

Note that you don't have to have the same code work for both. There are many packages written for Py2k that support py3k via 2to3, which is invoked during package installation. Basically, they're written with a little care so that the automatic translation allows them to be converted to python3 without issue.

[c-morley]

@fake-name as Kurt mentioned, the key problem is the c/c++ binding code.
It seems that we really need to make a choice at one point and change everything all at once.
And python touches almost everything in linuxcnc! Huge job.
Nice job there Python :(

[SebKuzminsky]

Here's somebody else's experience converting an old, big python2 codebase to python3: https://medium.com/@boxed/moving-a-large-and-old-codebase-to-python3-33a5a13f8c99

There might be something helpful for us to glean from this....

[fake-name]

FWIW, I'm rewriting axis.py for work things. Primarily, I'm doing cleanup (holy cow it's the worst python I've ever worked on), but it should be basically 2/3 compatible when I'm done.

It actually resulted in https://github.com/fake-name/autoflake/blob/master/autostar.py, (pr here), which is a kind of cool thing that tries very hard to fix all the from x import * mess in a python file, by doing some crazy ast hijinks to figure out which packages items not defined on the local scope come from, and rename them to their actual full names.

Actually, I'm unclear why axis.py involves tcl at all, but that's a different discussion.

[c-morley]

SebKuzminsky: An interesting read there. I see nothing of python extensions or python embedding unfortunately, which of course is the hardest part. It's a good reference though.

[jepler]

@fake-name It's great that you are taking an interest in working on axis and on the externally imposed python2 time bomb that our project faces. Thank you.

But even as you have the urge to rewrite axis.py wholesale (who doesn't? It's more a question of having the energy to do so), please keep in mind how you'll be able to ultimately create that as a pull request where the individual commits are reviewable. This is particularly important since we don't have any tests of GUI components of LinuxCNC, like AXIS. (Unless you're going to fix that too. 😉) A single commit in which the whole structure and design of axis.py is upended is probably unreviewable and is likely to languish indefinitely as a pull request.

The way lots of direct access to Tcl/Tk is mixed with Python/Tkinter does have an historical motivation: Long ago at my $DAY_JOB, I created a GUI builder for Tcl, which I got permission to release as an open source product. (it didn't gain any users and the open source version is unmaintained) A never-realized goal was for programmers to be able to reconfigure the AXIS UI by modifying it in the GUI builder. AXIS never achieved this goal, and eventually the GUI builder files were renamed to ".tcl" and when it felt expedient, developers (mostly me) wrote code in Tcl instead of in Python. Coming from my particular $DAY_JOB's background, mixing Python and Tcl like this doesn't even feel weird; but I know it's not common practice in widely published or read open source Python projects.

[fake-name]

But even as you have the urge to rewrite axis.py wholesale (who doesn't? It's more a question of having the energy to do so), please keep in mind how you'll be able to ultimately create that as a pull request where the individual commits are reviewable.

My primary focus here is to get axis into a state where I can extend it as an internal testing tool for work-related things. The rework is basically only because the code is impenetrable as-it-is, due to the complete lack of structure.

Anyways, this would never be a single commit, but rather a series of incremental changes which maintain functionality with incremental refinement.

[rene-dev]

I had a go at this today, and after some help from @jepler stuff started to build.
There is a problem with a dependency for halcompile:
http://theory.stanford.edu/~amitp/yapps/
there is a python 3 compatible version, but its not available in Debian.
I will create a pull request later with what I have done so far.

[rene-dev]

my work in progress branch is here: #416