Support more (older) Python versions

[bittner]

Is your feature request related to a problem? Please describe.

Currently, the README states that only Python 3.10 is supported. Python 3.11 is not supported yet, and support for Python < 3.10 was apparently dropped.

Environments that cannot run Python 3.10 (typically banks and similar, large institutions) are hence unable to use Pynguin. Those organisations may need Pynguin, though, so that they can stop doing manual testing.

Describe the solution you'd like

I'd like to understand whether there is a hard technical reason for supporting only Python 3.10.

If that's not the case we should make the code as backward-compatible as possible, and have a test matrix that runs the test suite also against older Python versions.

Describe alternatives you've considered

I don't see any, for the moment.

Additional context

I work in a restricted corporate environment and try to beef up the test suite, so that we can demonstrate that automated tests actually make sense.

I run Pynguin 0.17.0 with Python 3.9 on Windows, and executing pynguin results in the following error:

pynguin (click to expand)

$ pynguin
Traceback (most recent call last): 
  File "D:\Program Files\Python39\lib\runpy.py", line 197, in _run_module_as_main 
    return _run_code(code, main_globals, None,
  File "D:\Program Files\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "D:\python\venv\Scripts\pynguin.exe\__main__.py", line 4, in <module>
  File "D:\python\venv\lib\site-packages\pynguin\__init__.py", line 9, in <module>
    import pynguin.generator as gen
  File "D:\python\venv\lib\site-packages\pynguin\generator.py", line 31, in <module>
    import pynguin.analyses.seeding.initialpopulationseeding as initpopseeding
  File "D:\python\venv\lib\site-packages\pynguin\analyses\seeding\initialpopulationseeding.py", line 17, in <module>
    import pynguin.ga.testcasechromosome as tcc
  File "D:\python\venv\lib\site-packages\pynguin\ga\testcasechromosome.py", line 13, in <module>
    import pynguin.ga.chromosome as chrom
  File "D:\python\venv\lib\site-packages\pynguin\ga\chromosome.py", line 13, in <module>
    import pynguin.ga.computations as ff
  File "D:\python\venv\lib\site-packages\pynguin\ga\computations.py", line 17, in <module>
    from pynguin.testcase.execution import ExecutionTrace
  File "D:\python\venv\lib\site-packages\pynguin\testcase\execution.py", line 428, in <module>
    class ExecutionTracer:
  File "D:\python\venv\lib\site-packages\pynguin\testcase\execution.py", line 463, in ExecutionTracer
    Compare.IN: lambda val1, val2: (
  File "D:\Program Files\Python39\lib\enum.py", line 429, in __getattr__
    raise AttributeError(name) from None

I'd be willing to make the necessary contributions that make Pynguin run with older Pythons, once I understand the situation better.

[stephanlukasczyk]

Thank you for your interest and the feature request. I agree that supporting more Python versions would be a very valuable goal; unfortunately, as Pynguin as currently my PhD project, I have to finish my PhD, and I am basically the only person who's working on it, I cannot add support for further versions currently.

So, let me explain what the underlying problem is: Pynguin heavily relies on two components of Python (or better CPython): one is the AST that got some changes over the years. While this might be a solvable issue, it still was a burden.

The other, more crucial, issue is that Pynguin instruments the bytecode generated and executed by the CPython interpreter. This bytecode changes with (almost) every new version of Python. It is furthermore an implementation detail of the CPython interpreter—thus Pynguin is not portable to other interpreters, e.g., PyPy. Updating this instrumentation is cumbersome, error prone, and time consuming. This is the reason why we dropped support for Python 3.8 with Pynguin 0.18.0.

Adding support for other Python versions than 3.10 is possible in theory; most work is related to the bytecode instrumentation (see pynguin.instrumentation.instrumentation). Some parts of the test-case execution (see pynguin.testcase.execution) might be affected, too. And, depending on the Python version(s) to support, changes in all AST-related parts might be necessary, too. While all this could be done, I unfortunately cannot do this, as mentioned before.
Personally, I hope that the new sys.monitoring API in Python 3.12 might make it easier to support future Python versions, once it's implemented in Pynguin for Python 3.12 support.

While this is very disappointing for the usage of Pynguin there might still be hope: while executing Pynguin itself requires Python 3.10, there is no real requirement for the subject under test. Thus, it should totally be possible to let Pynguin generate tests for a project that runs, e.g., on Python 3.8. The resulting test cases are not tied to a specific Python version; nor do they use any very recent features from PyTest.

I am happy to elaborate on this further if requested.