Run only selective tests

[GagandeepSharma060]

Hi team , the filters thing that is mentioned in the documentation. Is it also possible if I am not using junit/java for my tests and if i just have the yaml files -> mention it in the specmatic.json and then run it through command line. I tried multiple ways but was not able to do so

"start": "cross-env-file -p ./token.json  specmatic test --testBaseURL=https://abc.com  --junitReportDir=./test-output && node cleanup"

This is the start command I am using, need urgent help

[GagandeepSharma060]

And when we write --filter-name "xyz" , where should this xyz be present for that endpoint (example) to be hit or not hit. Please provide example. It will be great as I am stuck for this implementation

@harikrishnan83 @nashjain @rakeshpai

[pranavgawri]

@GagandeepSharma060 thanks for sharing issue you are facing. We will update you shortly.

[GagandeepSharma060]

@pranavgawri found out that the filters are applied on the path name itself. It is great but can we have some tagging kind of mechanism like we have in Cypress.

Because filtering on the path names does not seem to be a great idea when dealing with a lot of services.

Just a thought.

[pranavgawri]

@GagandeepSharma060, Thank you for sharing your experience and suggestion. We're glad you found a workable solution with path name filtering.
Your idea about implementing a tagging system similar to Cypress is interesting and could indeed be beneficial for filtering tests across multiple services. We've noted this as a enhancement for future consideration.
We appreciate your feedback as it helps us improve Specmatic. Thanks.

[pranavgawri]

Hi @GagandeepSharma060 ,

We have enhanced our --filter option and I hope it should be able to help you better in your use case. Here is a quick overview:

The new filtering system provides powerful capabilities that align well with API contract testing:

1. Filter by HTTP Method and Path:

specmatic test --filter="METHOD=POST" --filter="PATH=/api/employees"

2. Filter by Status Codes (including pattern matching):

specmatic test --filter-not="STATUS=4xx"  # Excludes all client errors

3. Advanced Combinations:

specmatic test --filter="PATH=/api/employees" --filter="METHOD=POST" --filter-not="STATUS=400,401,403"

These filters work directly with your API contract structure, making it natural to organize and run your tests based on your API's design.

Please find the full details in our documentation here: https://specmatic.io/documentation/contract_tests.html#selectively-running-tests-in-ci

Let us know how this works for you!

Thanks,
Pranav

[GagandeepSharma060]

Thanks for the quick response and change @pranavgawri , will try it out and get back to you for feedback or any issues .

I believe this feature would help others too

Just a question, can we filter through description of the test .... as in that way it will give a huge flexibility

By description I mean we can have a dedicated field or use the fields like description or summary for a path

[pranavgawri]

Hi @GagandeepSharma060,

Thanks for the feedback about filtering by description/summary fields. I've opened a discussion thread to explore this further: Filtering Tests by Description/Summary Fields

Please take a look and share your thoughts there. Once you've reviewed the discussion, please feel free to go ahead and close this original issue.
Appreciate you helping us improve Specmatic!

Thanks,
Pranav

[GagandeepSharma060]

Hi @pranavgawri ...Thanks for the response. Justwanted to let you know that filtering by the Example name is not working for me..

Let me explain

--filter="EXAMLPE_NAME=ABC" is not working however. --filter-not="EXAMLPE_NAME=ABC" is working fine.

Can u please look into it

[pranavgawri]

Hi @GagandeepSharma060 ,

Thanks for sharing the issue you are facing. We are able to replicate it at our end, and will push the fix soon.

I will update you on the same.

Thanks,
Pranav