Provide Summary after tflint Command Run

[fracca]

When running the tflint command within a pipeline, I want a summary report on successful and failed runs.

The Summary report should include

• Count of Rules checked(for each enabled plugin)

If the report contains failures:

• Count of each error type (Errors: nnn, Warnings: nnn, etc...)

The reason for this is to confirm that all rules have been run. This provides confidence in the Linting.

[fracca]

Current JUNIT outputs show this only - (so may be a bug)
<?xml version="1.0" encoding="UTF-8"?> <testsuites> <testsuite tests="0" failures="0" time="0" name=""> <properties></properties> </testsuite> </testsuites>

If I deliberately break something I get:

<?xml version="1.0" encoding="UTF-8"?> <testsuites> <testsuite tests="1" failures="1" time="0" name=""> <properties></properties> <testcase classname="variables.tf" name="terraform_documented_variables" time="0"> <failure message="variables.tf:5,1-30: redacted_varvariable has no description" type="Notice">Notice:redacted_var variable has no description&#xA;Rule: terraform_documented_variables&#xA;Range: variables.tf:5,1-30</failure> </testcase> </testsuite> </testsuites>

[wata727]

Is this about JUnit formatter?

If so, this is the intended design. Rules (linter) and tests are different concepts and cannot be directly mapped to JUnit output format. For example, one rule may detect multiple issues. In that case, the sum of failures may be much larger than tests.

See also #1600 #1354

On the other hand, if you're talking about the default formatter, I think this request makes sense.

See also #1758

[fracca]

Yes, the original request around the default formatter is why I started this.

I was playing with the JUnit formatter to see if this would be acceptable to generate an appropriate test output for BitBucket Pipelines to latch onto.

I would be happy with #1758

[wata727]

Thank you for your clarification. Perhaps this can be achieved by changing the formatter/pretty.go.

tflint/formatter/pretty.go

Lines 24 to 30 in 1d58c87

	if len(issues) > 0 {
	fmt.Fprintf(f.Stdout, "%d issue(s) found:\n\n", len(issues))
	for _, issue := range issues.Sort() {
	f.prettyPrintIssueWithSource(issue, sources)
	}
	}

The count of rules is not passed to the formatter, so we'll have to fix it more upstream, but it shouldn't be too difficult.

[wata727]

Before getting into this, we should keep in mind that some people may want to empty the output if no issues. See #182
There is room to consider whether to add the --quiet option again or enable it with an option.

Also, there is currently no way to get a list of enabled rules, so we will need to extend the SDK to accomplish this.

[PatMyron]

hacky stopgap before proper support:
aws-cloudformation/cfn-lint#882 (comment)

tflint --format=sarif | grep -Eo '"ruleId.*' | sort | uniq -c | sort -nr
      9 "ruleId": "terraform_required_providers",
      3 "ruleId": "terraform_unused_declarations",