Issues creating scratch orgs in sf org create scratch

[sf-erush]

Summary

We are seeing issues starting yesterday when creating scratch orgs locally and in our integration pipelines. See error below.

sf version @salesforce/cli/2.0.2 darwin-arm64 node-v19.2.0

We are seeing issues creating scratch orgs with our devhub since yesterday. We have a pretty basic setup on a connected app and certificate keys (followed the documentation here)

Steps To Reproduce

FYI - We are using this out of a already existing 2GP manage package repo connected to our devhub org.
Set up a devhub
Set up a connected app with RSA keys
Get connected app consumer key
Create a scratch definition file
Run sf org login jwt cmd to auth to devhub. (this is succesful)
Run sf org create scratch -a jwtTest -y 1 -f default.definition.json. to create a scratch org (this fails)

  "code": 1,
  "actions": [
    "See https://developer.salesforce.com/docs/atlas.en-us.sfdx_dev.meta/sfdx_dev/sfdx_dev_scratch_orgs_error_codes.htm for information on error code C-1016."
  ],
  "context": "EnvCreateScratch",
  "commandName": "EnvCreateScratch",
  "data": {
    "username": "[email protected]",
    "orgId": "00D80000000ctx5EAA",
    "scratchOrgInfoId": "2SRPD00000008jx4AA"
  },
  "message": "We encountered a problem while attempting to configure and approve the Connected App for your org. Verify the Connected App configuration with your Salesforce admin.",
  "name": "RemoteOrgSignupFailed",
  "status": 1,
  "stack": "RemoteOrgSignupFailed: We encountered a problem while attempting to configure and approve the Connected App for your org. Verify the Connected App configuration with your Salesforce admin.\n    at checkScratchOrgInfoForErrors (/opt/homebrew/lib/node_modules/@salesforce/cli/node_modules/@salesforce/core/lib/org/scratchOrgErrorCodes.js:62:19)\n    at async pollForScratchOrgInfo (/opt/homebrew/lib/node_modules/@salesforce/cli/node_modules/@salesforce/core/lib/org/scratchOrgInfoApi.js:303:16)\n    at async scratchOrgCreate (/opt/homebrew/lib/node_modules/@salesforce/cli/node_modules/@salesforce/core/lib/org/scratchOrgCreate.js:160:17)\n    at async EnvCreateScratch.run (/opt/homebrew/lib/node_modules/@salesforce/cli/node_modules/@salesforce/plugin-org/lib/commands/org/create/scratch.js:39:72)\n    at async EnvCreateScratch._run (/opt/homebrew/lib/node_modules/@salesforce/cli/node_modules/@oclif/core/lib/command.js:117:22)\n    at async Config.runCommand (/opt/homebrew/lib/node_modules/@salesforce/cli/node_modules/@oclif/core/lib/config/config.js:329:25)\n    at async run (/opt/homebrew/lib/node_modules/@salesforce/cli/node_modules/@oclif/core/lib/main.js:89:16)",
  "exitCode": 1,
  "warnings": []
}```

### Expected result
Scratch Orgs Created

### Actual result
Getting Error

### System Information
mac terminal zsh or Jenkins sh script
```{
  "cliVersion": "@salesforce/cli/2.2.7",
  "architecture": "darwin-arm64",
  "nodeVersion": "node-v19.2.0",
  "osVersion": "Darwin 22.6.0",
  "shell": "zsh",
  "rootPath": "/opt/homebrew/lib/node_modules/@salesforce/cli",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 2.3.3 (core)",
    "@oclif/plugin-commands 2.2.20 (core)",
    "@oclif/plugin-help 5.2.14 (core)",
    "@oclif/plugin-not-found 2.3.34 (core)",
    "@oclif/plugin-plugins 3.1.8 (core)",
    "@oclif/plugin-search 0.0.21 (core)",
    "@oclif/plugin-update 3.1.28 (core)",
    "@oclif/plugin-version 1.3.7 (core)",
    "@oclif/plugin-warn-if-update-available 2.0.45 (core)",
    "@oclif/plugin-which 2.2.27 (core)",
    "@salesforce/cli 2.2.7 (core)",
    "apex 2.3.8 (core)",
    "auth 2.8.9 (core)",
    "data 2.5.3 (core)",
    "deploy-retrieve 1.16.3 (core)",
    "info 2.6.32 (core)",
    "limits 2.3.27 (core)",
    "login 1.2.22 (core)",
    "org 2.9.27 (core)",
    "packaging 1.16.8 (user)",
    "schema 2.3.21 (core)",
    "settings 1.4.21 (core)",
    "sobject 0.1.40 (core)",
    "source 2.10.27 (core)",
    "telemetry 2.2.3 (core)",
    "templates 55.5.7 (core)",
    "trust 2.4.34 (core)",
    "user 2.3.27 (core)"
  ]
}

[github-actions]

Hello @sf-erush 👋 None of the versions of sf you shared match the latest release.

Shared: 2.0.2, 2.0.2
Latest: 2.2.7

Update to the latest version of Salesforce CLI (docs) and confirm that you're still seeing your issue.
You can also try the rc and nightly releases! (docs)

After updating, share the full output of sf version --verbose --json

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[sf-erush]

  "cliVersion": "@salesforce/cli/2.2.7",
  "architecture": "darwin-arm64",
  "nodeVersion": "node-v19.2.0",
  "osVersion": "Darwin 22.6.0",
  "shell": "zsh",
  "rootPath": "/opt/homebrew/lib/node_modules/@salesforce/cli",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 2.3.3 (core)",
    "@oclif/plugin-commands 2.2.20 (core)",
    "@oclif/plugin-help 5.2.14 (core)",
    "@oclif/plugin-not-found 2.3.34 (core)",
    "@oclif/plugin-plugins 3.1.8 (core)",
    "@oclif/plugin-search 0.0.21 (core)",
    "@oclif/plugin-update 3.1.28 (core)",
    "@oclif/plugin-version 1.3.7 (core)",
    "@oclif/plugin-warn-if-update-available 2.0.45 (core)",
    "@oclif/plugin-which 2.2.27 (core)",
    "@salesforce/cli 2.2.7 (core)",
    "apex 2.3.8 (core)",
    "auth 2.8.9 (core)",
    "data 2.5.3 (core)",
    "deploy-retrieve 1.16.3 (core)",
    "info 2.6.32 (core)",
    "limits 2.3.27 (core)",
    "login 1.2.22 (core)",
    "org 2.9.27 (core)",
    "packaging 1.16.8 (user)",
    "schema 2.3.21 (core)",
    "settings 1.4.21 (core)",
    "sobject 0.1.40 (core)",
    "source 2.10.27 (core)",
    "telemetry 2.2.3 (core)",
    "templates 55.5.7 (core)",
    "trust 2.4.34 (core)",
    "user 2.3.27 (core)"
  ]
}```

[sf-erush]

Still seeing same issue once updating to latest cli version.

[sf-bbagley]

Followed up this morning with more testing and wanted to share some results

1. sf org login jwt

• used our connected app
• Successful auth
• Scratch org fails creation

2. /services/oauth2/token to get access token

• used our same connected app
• successful token retrieval
• used sf org login access-token
• Scratch org creation successful

Leads me to believe perhaps something is going on with the JWT sfdx flow.

[cristiand391]

I see in the internal Slack thread that this might be related to an expired connected app, please let me know us know if that's not the case.