Salesforce CLI Test Execution

[jhgihub0]

We have large orgs with large deployments. Each time we do a deployment for our large orgs, even if we deploy just 1 object, it runs all the tests. For sandboxes we can specify "NoTestRun", but that is not possible for PROD deployments. These tests can take hours to run. We also have many developers working on metadata updates, so it’s not always easy to get the list of test classes associated with the change being deployed, so running specified tests is not feasible.

It would be great if Salesforce CLI can use AI to determine what has changed in the metadata since the last deployment, and only run those metadata deployments and associated tests. There are some 3rd party CLI plugins that does that, but we have open source restrictions. Something official from Salesforce would be ideal.

Is anything like this in the works, or available for us to use? If not, can this be added into Salesforce CLI?

[jhgihub0]

I am not asking for a fix to a previous issue or new issues. I am looking for new functionality to allow teams to deploy faster while running tests. AI is everywhere in Salesforce, so it would be great if it can be used with the SF CLI to determine what metadata has changed, and only run those tests associated with the change.

[VivekMChawla]

@jhgihub0 - My understanding is that Apex tests are run by default in production only if your deployment includes Apex classes or triggers as long as the API version is 34.0 or higher. This sounds different from the behavior you describe where all tests run even if deploying a single custom object.

Regarding your ask for AI help to specify test classes to run, are you asking for the Salesforce CLI to...

1. Inspect the deployment and build a dependency graph identifying all Apex classes/triggers that touch the classes being deployed
2. Determine the test classes that provide code coverage all classes identified in the previous step
3. Automatically use that list of test classes as part of the deployment as if you had specified them via the --tests flag in your sf project deploy start command

Thanks for engaging on this idea.

[jhgihub0]

@VivekMChawla

Yes, it would be nice if CLI can do the steps you provided.

[mcarvin8]

We deploy with a package.xml and scan the package using a custom script to determine what SF CLI command to run.

If the package doesn't contain Apex, we just run sf project deploy start -x package.xml without any of the Test flags specified. Salesforce will scan the package/directory and confirm it has no Apex, thus no tests are required.

Ensure you do not use sf project deploy validate on non-apex packages/directories. This validate command will enforce Apex tests since validates are a precursor to quick deploys, which require Apex tests to run. I found this out the hard way. If you want to "validate" a non-apex package, you need to use --dry-run on sf project deploy start instead: sf project deploy start --dry-run

If the package contains Apex, then we run sf project deploy validate -x package.xml -l RunSpecifiedTests -t {testclasses} and follow that up with the quick-deployment.

So technically our CI/CD process involves two package scans (1 done by our script and 1 done by Salesforce), but this was the most reliable way we've found to separate Apex vs non-Apex deployments to ensure we optimize the deployment process.

If you want to avoid manual package creation, I would recommend https://github.com/scolladon/sfdx-git-delta to create delta packages based on the changed files but that is a 3rd party plugin. The guy who made that plugin does work for Salesforce though so I would advise for it and it's one of the most popular plugins.

[jhgihub0]

Thank you, but we not longer use the package.xml. We are using DX folder structure, and have too many changes going in to generate a custom package.xml for each deploy.

We run

sf project deploy start --target-org OrgAlias --wait 120 --test-level "NoTestRun" --source-dir ./force-app