Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ld: cannot find -lpython3.10 #275

Closed
2 of 5 tasks
joaodlf opened this issue Nov 17, 2021 · 6 comments · May be fixed by #909
Closed
2 of 5 tasks

ld: cannot find -lpython3.10 #275

joaodlf opened this issue Nov 17, 2021 · 6 comments · May be fixed by #909
Labels
bug Something isn't working

Comments

@joaodlf
Copy link

joaodlf commented Nov 17, 2021

Description:
Getting ld: cannot find -lpython3.10when installing some dependencies.

Platform:

  • Ubuntu
  • macOS
  • Windows

Runner type:

  • Hosted
  • Self-hosted

Tools version:

3.10.0

We are trying to move our action to a self hosted environment, this is how the file looks:

name: Deploy

on:
  push:
    branches:
      - master

jobs:
  build:
    runs-on: self-hosted
    container: ubuntu:20.04
    services:
      postgres:
        ...

    steps:
      - uses: actions/checkout@v1

      ...

      - name: Setup Python
        uses: actions/setup-python@v2
        with:
          python-version: 3.10.0

      - name: Install poetry
        run: |
          python -m pip install poetry==1.1.11

      - name: Configure poetry
        run: |
          python -m poetry config virtualenvs.in-project true

      - name: Upgrade pip
        run: |
          python -m poetry run pip install --upgrade pip

      - name: Cache the virtualenv
        uses: actions/cache@v2
        with:
          path: ./.venv
          key: ${{ runner.os }}-venv1-${{ hashFiles('**/poetry.lock') }}

      - name: Install dependencies
        run: |
          python -m poetry install

     ...

We've managed to get to the Install dependencies step, most of the dependencies are installing correctly, but when we get to uwsgi, we are getting the following:

2021-11-17T14:24:25.7315863Z       gcc -pthread -o /__w/project/project/.venv/bin/uwsgi -L/opt/hostedtoolcache/Python/3.10.0/x64/lib -Wl,-rpath,/opt/hostedtoolcache/Python/3.10.0/x64/lib core/utils.o core/protocol.o core/socket.o core/logging.o core/master.o core/master_utils.o core/emperor.o core/notify.o core/mule.o core/subscription.o core/stats.o core/sendfile.o core/async.o core/master_checks.o core/fifo.o core/offload.o core/io.o core/static.o core/websockets.o core/spooler.o core/snmp.o core/exceptions.o core/config.o core/setup_utils.o core/clock.o core/init.o core/buffer.o core/reader.o core/writer.o core/alarm.o core/cron.o core/hooks.o core/plugins.o core/lock.o core/cache.o core/daemons.o core/errors.o core/hash.o core/master_events.o core/chunked.o core/queue.o core/event.o core/signal.o core/strings.o core/progress.o core/timebomb.o core/ini.o core/fsmon.o core/mount.o core/metrics.o core/plugins_builder.o core/sharedarea.o core/rpc.o core/gateway.o core/loop.o core/cookie.o core/querystring.o core/rb_timers.o core/transformations.o core/uwsgi.o proto/base.o proto/uwsgi.o proto/http.o proto/fastcgi.o proto/scgi.o proto/puwsgi.o lib/linux_ns.o core/yaml.o core/dot_h.o core/config_py.o plugins/python/python_plugin.o plugins/python/pyutils.o plugins/python/pyloader.o plugins/python/wsgi_handlers.o plugins/python/wsgi_headers.o plugins/python/wsgi_subhandler.o plugins/python/web3_subhandler.o plugins/python/pump_subhandler.o plugins/python/gil.o plugins/python/uwsgi_pymodule.o plugins/python/profiler.o plugins/python/symimporter.o plugins/python/tracebacker.o plugins/python/raw.o plugins/gevent/gevent.o plugins/gevent/hooks.o plugins/ping/ping_plugin.o plugins/cache/cache.o plugins/nagios/nagios.o plugins/rrdtool/rrdtool.o plugins/carbon/carbon.o plugins/rpc/rpc_plugin.o plugins/corerouter/cr_common.o plugins/corerouter/cr_map.o plugins/corerouter/corerouter.o plugins/fastrouter/fastrouter.o plugins/http/http.o plugins/http/keepalive.o plugins/http/https.o plugins/http/spdy3.o plugins/ugreen/ugreen.o plugins/signal/signal_plugin.o plugins/syslog/syslog_plugin.o plugins/rsyslog/rsyslog_plugin.o plugins/logsocket/logsocket_plugin.o plugins/router_uwsgi/router_uwsgi.o plugins/router_redirect/router_redirect.o plugins/router_basicauth/router_basicauth.o plugins/zergpool/zergpool.o plugins/redislog/redislog_plugin.o plugins/mongodblog/mongodblog_plugin.o plugins/router_rewrite/router_rewrite.o plugins/router_http/router_http.o plugins/logfile/logfile.o plugins/router_cache/router_cache.o plugins/rawrouter/rawrouter.o plugins/router_static/router_static.o plugins/sslrouter/sslrouter.o plugins/spooler/spooler_plugin.o plugins/cheaper_busyness/cheaper_busyness.o plugins/symcall/symcall_plugin.o plugins/transformation_tofile/tofile.o plugins/transformation_gzip/gzip.o plugins/transformation_chunked/chunked.o plugins/transformation_offload/offload.o plugins/router_memcached/router_memcached.o plugins/router_redis/router_redis.o plugins/router_hash/router_hash.o plugins/router_expires/expires.o plugins/router_metrics/plugin.o plugins/transformation_template/tt.o plugins/stats_pusher_socket/plugin.o -lpthread -lm -rdynamic -ldl -lcrypt -lpthread -ldl -lutil -lm -lm -lpython3.10 -lcrypt
2021-11-17T14:24:25.7330403Z       /usr/bin/ld: cannot find -lpython3.10
2021-11-17T14:24:25.7331040Z       collect2: error: ld returned 1 exit status
2021-11-17T14:24:25.7331619Z       *** error linking uWSGI ***
2021-11-17T14:24:25.7332292Z       ----------------------------------------```

The step has the following log output (before it is run):

python -m poetry install
  shell: sh -e {0}
  env:
    pythonLocation: /__w/_tool/Python/3.10.0/x64
    LD_LIBRARY_PATH: /__w/_tool/Python/3.10.0/x64/lib

Any help would be appreciated! Have not been able to find much about this issue...

edit:
just tried downgrading python version to 3.8, but same situation with -lpython3.8.

@joaodlf joaodlf added bug Something isn't working needs triage labels Nov 17, 2021
@joaodlf
Copy link
Author

joaodlf commented Nov 18, 2021

Further investigation, I can see the following files in our system:

/opt/hostedtoolcache/Python/3.10.0/x64/lib/libpython3.10.so
/opt/hostedtoolcache/Python/3.10.0/x64/lib/libpython3.10.so.1.0

We are even running the runner as root (RUNNER_ALLOW_RUNASROOT=1 ./run.sh) to avoid any possible permissions issue.

Not sure if this is important (my knowledge over docker and containers is very limited), but the host machine is a Centos 8 box, which I know isn't supported, but I was hoping to circumvent that issue by setting the container to ubuntu:20.04.

@nikita-bykov
Copy link
Contributor

Hello @joaodlf, thanks for reporting! We will take a look.

@dsame
Copy link
Contributor

dsame commented Apr 15, 2022

Hello @joaodlf,

According to README it is necessary to set environment variable AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache in order the python to be installed in /opt/hostedtoolcache where your build expects to find the library

      - name: Setup Python
        uses: actions/setup-python@v2
        env:
          AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache
        with:
          python-version: 3.10.0

According to your note

I can see the following files in our system:

/opt/hostedtoolcache/Python/3.10.0/x64/lib/libpython3.10.so
/opt/hostedtoolcache/Python/3.10.0/x64/lib/libpython3.10.so.1.0

I assume you meant the host system while your actual build is executed within the docker container

jobs:
  build:
    runs-on: self-hosted
    container: ubuntu:20.04

This is why the setup-python installs the python binaries into /__w/_tool/Python/3.10.0/x64/lib/ within the docker filesystem

Please confirm AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache solves your problem.
NOTE: you may face an error during the setup-pyton task the binary not found - in this case please wait the PR #338 is merged

@marko-zivic-93
Copy link
Contributor

Hello @joaodlf
Due to the lack of response, I am going to close this issue :)
If you have any additional problems or concerns, please feel free to reopen it or to create a new one.

@gchamon
Copy link

gchamon commented Sep 5, 2023

@dsame I had this issue with a similar setup as OP and I can confirm, at least on my end, setting AGENT_TOOLSDIRECTORY to /opt/hostedtoolcache solved the issue for me. In order for me not to modify every single workflow file deployed in my organization, I decided to set the variable at the runner level, in the file /opt/actions-runner/.env.

@tyrken
Copy link

tyrken commented Jul 12, 2024

With our self-hosted runners (pods on EKS helped by https://github.com/actions/actions-runner-controller ) changing AGENT_TOOLSDIRECTORY didn't work for us - still getting gcc compilation errors about missing lpython3.9 library.

Instead we see this action already sets LD_LIBRARY_PATH to a custom value, e.g. /home/runner/_work/_tool/Python/3.9.19/x64/lib for an old app we're testing. While that works for running applications, that env-var isn't used for compilation. Setting LIBRARY_PATH to the same value as LD_LIBRARY_PATH does work.

It would be nice if this action was updated to set both natively...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants