"Check if the Actions runner has access to the GitHub Actions service" fails with 401 to https://docs.github.com/rest

[jasonsketchup]

Select Topic Area

Question

Body

This was previously working until the runner registration expired after two weeks unused.

I followed the steps for creating a new self-hosted runner associated with my repository on a MacBook M1 Pro running MacOS Sonoma 14.5 build 23F79. I have retried this multiple times with the same results. I have tried this on multiple networks (home, enterprise, VPN).

./run.sh reports...

√ Connected to GitHub

Current runner version: '2.317.0'
2024-06-28 16:56:27Z: Listening for Jobs

The runner appears in the GitHub repository Settings > Actions > Runners with status green and "Idle". However on running a job with matching tags "self-hosted, macOS, ARM64, [...]" (custom tags redacted) the workflow pauses on...

Requested labels: self-hosted, macOS, ARM64, [...]
Job defined at: SketchUp/mobile_qa/.github/workflows/main.yml@refs/heads/master
Waiting for a runner to pick up this job...

Following the steps at https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners/monitoring-and-troubleshooting-self-hosted-runners, I ran a diagnostic check with environment variable TOKEN from the setup steps (URL redated)...

% ./config.sh --check --url https://github.com/REDACTED --pat ${TOKEN}                     
**********************************************************************************************************************
**  Check:               Internet Connection
**  Description:         Check if the Actions runner has internet access.
**********************************************************************************************************************
**                                                                                                                  **
**                                            P A S S                                                               **
**                                                                                                                  **
**********************************************************************************************************************
** Log: /Users/runner/actions-runner/_diag/InternetCheck_20240628-170417-utc.log
**********************************************************************************************************************


**********************************************************************************************************************
**  Check:               GitHub Actions Connection
**  Description:         Check if the Actions runner has access to the GitHub Actions service.
**********************************************************************************************************************
**                                                                                                                  **
**                                            F A I L                                                               **
**                                                                                                                  **
**********************************************************************************************************************
** Log: /Users/runner/actions-runner/_diag/ActionsCheck_20240628-170417-utc.log
** Help Doc: https://github.com/actions/runner/blob/main/docs/checks/actions.md
**********************************************************************************************************************


**********************************************************************************************************************
**  Check:               Git Certificate/Proxy Validation
**  Description:         Check if the Git CLI can access GitHub.com or GitHub Enterprise Server.
**********************************************************************************************************************
**                                                                                                                  **
**                                            P A S S                                                               **
**                                                                                                                  **
**********************************************************************************************************************
** Log: /Users/runner/actions-runner/_diag/GitCheck_20240628-170417-utc.log
**********************************************************************************************************************


**********************************************************************************************************************
**  Check:               Node.js Certificate/Proxy Validation
**  Description:         Check if Node.js has access to GitHub.com or GitHub Enterprise Server.
**********************************************************************************************************************
**                                                                                                                  **
**                                            P A S S                                                               **
**                                                                                                                  **
**********************************************************************************************************************
** Log: /Users/runner/actions-runner/_diag/NodeJsCheck_20240628-170417-utc.log
**********************************************************************************************************************

Looking in the /Users/runner/actions-runner/_diag/ActionsCheck_20240628-170417-utc.log I see...

2024-06-28T17:04:18.5455210Z Http response body: {"message":"Bad credentials","documentation_url":"https://docs.github.com/rest","status":"401"}
2024-06-28T17:04:18.5455260Z ***************************************************************************************************************
2024-06-28T17:04:18.5456070Z [START Private.InternalDiagnostics.System.Net.Http - HandlerMessage]
2024-06-28T17:04:18.5456080Z 3137518
2024-06-28T17:04:18.5456090Z 0
2024-06-28T17:04:18.5456100Z 0
2024-06-28T17:04:18.5456100Z Dispose
2024-06-28T17:04:18.5456110Z Disposing pool.
2024-06-28T17:04:18.5456110Z [END Private.InternalDiagnostics.System.Net.Http - HandlerMessage]
2024-06-28T17:04:18.5455370Z Http request 'GET' to https://api.github.com/ failed with Unauthorized
2024-06-28T17:04:18.5456480Z ***************************************************************************************************************

All the troubleshooting checks found at https://github.com/actions/runner/blob/main/docs/checks/actions.md#what-is-this-check-for pass, including the GitHub API command...

% curl -v https://api.github.com/zen
* Host api.github.com:443 was resolved.
* IPv6: (none)
* IPv4: 140.82.113.6
*   Trying 140.82.113.6:443...
* Connected to api.github.com (140.82.113.6) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.github.com
*  start date: Mar  7 00:00:00 2024 GMT
*  expire date: Mar  7 23:59:59 2025 GMT
*  subjectAltName: host "api.github.com" matched cert's "*.github.com"
*  issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo ECC Domain Validation Secure Server CA
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://api.github.com/zen
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: api.github.com]
* [HTTP/2] [1] [:path: /zen]
* [HTTP/2] [1] [user-agent: curl/8.6.0]
* [HTTP/2] [1] [accept: */*]
> GET /zen HTTP/2
> Host: api.github.com
> User-Agent: curl/8.6.0
> Accept: */*
> 
< HTTP/2 200 
< server: istio-envoy
< date: Fri, 28 Jun 2024 17:26:37 GMT
< content-type: text/plain;charset=utf-8
< content-length: 15
< x-github-api-version-selected: 2022-11-28
< access-control-expose-headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset
< access-control-allow-origin: *
< strict-transport-security: max-age=31536000; includeSubdomains; preload
< x-frame-options: deny
< x-content-type-options: nosniff
< x-xss-protection: 0
< referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin
< content-security-policy: default-src 'none'
< vary: Accept-Encoding, Accept, X-Requested-With
< x-envoy-upstream-service-time: 20
< x-ratelimit-limit: 60
< x-ratelimit-remaining: 52
< x-ratelimit-reset: 1719596003
< x-ratelimit-resource: core
< x-ratelimit-used: 8
< accept-ranges: bytes
< x-github-request-id: D866:12BB72:96765D:1158DB1:667EF24D
< 
* Connection #0 to host api.github.com left intact
Encourage flow.% 

I am at a loss and would appreciate assistance/suggestions.

[jasonsketchup]

Reviewing https://docs.github.com/en/rest/authentication/authenticating-to-the-rest-api?apiVersion=2022-11-28#about-authentication, I tried (and failed) to authenticate to the GitHub REST API using the self-hosted runner token.

% curl --request GET \
--url "https://api.github.com/octocat" \
--header "Authorization: Bearer ${TOKEN}" \  
--header "X-GitHub-Api-Version: 2022-11-28"
{
  "message": "Bad credentials",
  "documentation_url": "https://docs.github.com/rest",
  "status": "401"
}

[jasonsketchup]

Pretty sure I am not spamming the GitHub REST API as I am performing checks manually from the command line, not scripting them. I am consistently getting a HTTP status code 401 and not generating thousands of requests per hour. Additionally, I am not seeing any other 4XX codes indicating this issue is rate limiting.

Performing this first thing today, I still get...

% curl --request GET --url "https://api.github.com/octocat" --header "Authorization: Bearer ${TOKEN}" --header "X-GitHub-Api-Version: 2022-11-28"
{
  "message": "Bad credentials",
  "documentation_url": "https://docs.github.com/rest",
  "status": "401"
}

Generating a new GitHub self-hosted runner token exported as ${TOKEN2}, this still fails the same way...

% curl --request GET --url "https://api.github.com/octocat" --header "Authorization: Bearer ${TOKEN2}" --header "X-GitHub-Api-Version: 2022-11-28" 
{
  "message": "Bad credentials",
  "documentation_url": "https://docs.github.com/rest",
  "status": "401"
}

I have visually verified the ${TOKEN} and ${TOKEN2} environment variables are not empty and performed a command line check...

% echo ${TOKEN} | wc -c
      30
% echo ${TOKEN2} | wc -c                                                                                                                               
      30

Again, this was working previously with the existing firewall settings. Furthermore, removing the Authorization header I get...

% curl --request GET --url "https://api.github.com/octocat" --header "X-GitHub-Api-Version: 2022-11-28"

               MMM.           .MMM
               MMMMMMMMMMMMMMMMMMM
               MMMMMMMMMMMMMMMMMMM      _________________________________________
              MMMMMMMMMMMMMMMMMMMMM    |                                         |
             MMMMMMMMMMMMMMMMMMMMMMM   | It's not fully shipped until it's fast. |
            MMMMMMMMMMMMMMMMMMMMMMMM   |_   _____________________________________|
            MMMM::- -:::::::- -::MMMM    |/
             MM~:~ 00~:::::~ 00~:~MM
        .. MMMMM::.00:::+:::.00::MMMMM ..
              .MM::::: ._. :::::MM.
                 MMMM;:::::;MMMM
          -MM        MMMMMMM
          ^  M+     MMMMMMMMM
              MMMMMMM MM MM MM
                   MM MM MM MM
                   MM MM MM MM
                .~~MM~MM~MM~MM~~.
             ~~~~MM:~MM~~~MM~:MM~~~~
            ~~~~~~==~==~~~==~==~~~~~~
             ~~~~~~==~==~==~==~~~~~~
                 :~==~==~==~==~~

[jasonsketchup]

Results are the same using the Postman application...