Fixing python versions allowed

[manuelsh]

Modifying CONTRIBUTING documentation to reflect the fact that one can use Python 3.8 or above but not Python 3.11. This is due to PyTorch not working in Python 3.11 yet. Otherwise contributors may start with the highest version of Python and get errors.

What does this PR do?

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

@stevhliu and @MKhalusova

[amyeroberts]

Hi @manuelsh, thanks for opening this PR!

I don't think we should add in this amendment at the moment c.f my comment on the diff

[amyeroberts]

I don't think this is correct

Pytorch docs state that for linux, mac and windows , it is recommended to use python 3.8 - 3.11 i.e. inclusive of 3.11. It doesn't explicitly state incompatibility with 3.12 or above.

Transformers is also not exclusive to pytorch usage - some users run with tensorflow or jax

[manuelsh]

Thanks for the quick answer, Amy. I had some issues running pip install -e .[dev] with python 3.11 (first related to cython, then with some dependencies) until I downgraded it to Python 3.10.

One can reproduce it by running it in a docker:

FROM python:3.11

# Install system dependencies
RUN apt-get update && apt-get install -y git 

# Set the working directory
WORKDIR /usr/src/app

# Clone the transformers repository
RUN git clone https://github.com/huggingface/transformers.git

# Set the working directory to the cloned repository
WORKDIR /usr/src/app/transformers

# Install development dependencies
RUN pip install -e .[dev]

the error I got was:

5.80 Collecting av==9.2.0 (from transformers==4.43.0.dev0)
25.82   Downloading av-9.2.0.tar.gz (2.4 MB)
26.04      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 11.8 MB/s eta 0:00:00
26.22   Installing build dependencies: started
31.02   Installing build dependencies: finished with status 'done'
31.03   Getting requirements to build wheel: started
31.44   Getting requirements to build wheel: finished with status 'error'
31.45   error: subprocess-exited-with-error
31.45   
31.45   × Getting requirements to build wheel did not run successfully.
31.45   │ exit code: 1
31.45   ╰─> [11 lines of output]
31.45       Package libavformat was not found in the pkg-config search path.
31.45       Perhaps you should add the directory containing `libavformat.pc'
31.45       to the PKG_CONFIG_PATH environment variable
31.45       Package 'libavformat', required by 'virtual:world', not found
31.45       Package 'libavcodec', required by 'virtual:world', not found
31.45       Package 'libavdevice', required by 'virtual:world', not found
31.45       Package 'libavutil', required by 'virtual:world', not found
31.45       Package 'libavfilter', required by 'virtual:world', not found
31.45       Package 'libswscale', required by 'virtual:world', not found
31.45       Package 'libswresample', required by 'virtual:world', not found
31.45       pkg-config could not find libraries ['avformat', 'avcodec', 'avdevice', 'avutil', 'avfilter', 'swscale', 'swresample']
31.45       [end of output]
31.45   
31.45   note: This error originates from a subprocess, and is likely not a problem with pip.
31.45 error: subprocess-exited-with-error

tried several things (e.g. install av and other packages independently) but didn't work until I downgraded python.

[amyeroberts]

@manuelsh Thanks for the detailed explanation! Did you see the same issues when installing with pip install -e . ? i.e. not the dev option? cc @ydshieh

[manuelsh]

No problem without the [dev] option. It runs smoothly. I tried also to change av package from ver 9.2.0 to 12.2.0 in the python.py file, but then it clashes with jaxlib package...

[amyeroberts]

OK, there's a wider issue here about compatibility with some of the libraries, but this is somewhat orthogonal to the python version. In this case, we can open an issue to track and try and find a set of libraries which work, but I don't think we should update the docs here as it can be misleading