feat(misconf): Improve misconfig scanner logging experience

[simar7]

Description

Currently there are a few places where the misconfiguration engine (defsec, trivy-iac, et al.) don't necessarily integrate well with the Trivy logging options (for e.g. --quiet and --debug).

One instance of this problem is documented in a fix here. In this case, passing --debug didn't enable debug logging within the defsec engine, hence missing some important messages from the engine.

Action items

1. Need to make sure that logging options are disabling/enabling the logging options within the misconfiguration scanning code.
2. Logging within trivy-iac codebase must respect the options set by the Trivy logger.

[nikpivkin]

I think it's also worth fixing the logging in trivy-iac (ran with this patch #5550)

2023-11-10T13:25:19.762+0700    DEBUG   [misconf] 25:19.762136000 rego.scanner                     Overriding filesystem for policies!
2023-11-10T13:25:19.792+0700    DEBUG   [misconf] 25:19.792068000 rego.scanner                     Loaded 189 policies from disk.
2023-11-10T13:25:19.792+0700    DEBUG   [misconf] 25:19.792415000 rego.scanner                     Overriding filesystem for data!
2023-11-10T13:25:20.096+0700    DEBUG   [misconf] 25:20.096401000 helm.scanner                     Processing rendered chart file: templates/serviceaccount.yaml
2023-11-10T13:25:20.096+0700    DEBUG   [misconf] 25:20.096495000 rego.scanner                     Scanning 1 inputs...
2023-11-10T13:25:20.246+0700    DEBUG   [misconf] 25:20.246132000 helm.scanner                     Processing rendered chart file: templates/service.yaml
2023-11-10T13:25:20.246+0700    DEBUG   [misconf] 25:20.246235000 rego.scanner                     Scanning 1 inputs...
2023-11-10T13:25:20.409+0700    DEBUG   [misconf] 25:20.409735000 helm.scanner                     Processing rendered chart file: templates/deployment.yaml
2023-11-10T13:25:20.409+0700    DEBUG   [misconf] 25:20.409920000 rego.scanner                     Scanning 1 inputs...

[simar7]

I think it's also worth fixing the logging in trivy-iac (ran with this patch #5550)

2023-11-10T13:25:19.762+0700    DEBUG   [misconf] 25:19.762136000 rego.scanner                     Overriding filesystem for policies!
2023-11-10T13:25:19.792+0700    DEBUG   [misconf] 25:19.792068000 rego.scanner                     Loaded 189 policies from disk.
2023-11-10T13:25:19.792+0700    DEBUG   [misconf] 25:19.792415000 rego.scanner                     Overriding filesystem for data!
2023-11-10T13:25:20.096+0700    DEBUG   [misconf] 25:20.096401000 helm.scanner                     Processing rendered chart file: templates/serviceaccount.yaml
2023-11-10T13:25:20.096+0700    DEBUG   [misconf] 25:20.096495000 rego.scanner                     Scanning 1 inputs...
2023-11-10T13:25:20.246+0700    DEBUG   [misconf] 25:20.246132000 helm.scanner                     Processing rendered chart file: templates/service.yaml
2023-11-10T13:25:20.246+0700    DEBUG   [misconf] 25:20.246235000 rego.scanner                     Scanning 1 inputs...
2023-11-10T13:25:20.409+0700    DEBUG   [misconf] 25:20.409735000 helm.scanner                     Processing rendered chart file: templates/deployment.yaml
2023-11-10T13:25:20.409+0700    DEBUG   [misconf] 25:20.409920000 rego.scanner                     Scanning 1 inputs...

@nikpivkin What do you propose that we should change? Could you provide a small desired output?

[nikpivkin]

I forgot to clarify, I meant that the logs contain a large indentation between the scanner type and the message.

[simar7]

I forgot to clarify, I meant that the logs contain a large indentation between the scanner type and the message.

I think that's from here - it's something all logger prints should have or is it not? It seems fine to me as is.

[nikpivkin]

debug and quiet flags are supported by iac scanners. The aws scanner from trivy-aws does not support the quiet flag.

trivy aws --endpoint http://localhost:4566 --region us-east-1 --service s3 --update-cache -d -q
2024-03-15T17:28:20.068+0700    DEBUG   Looking for AWS credentials provider...
2024-03-15T17:28:20.070+0700    DEBUG   Looking up AWS caller identity...
2024-03-15T17:28:20.227+0700    DEBUG   Verified AWS credentials for account 000000000000!
2024-03-15T17:28:20.227+0700    DEBUG   Specific services were requested: [s3]...
2024-03-15T17:28:20.227+0700    DEBUG   scanning services: [s3]
2024-03-15T17:28:20.232+0700    DEBUG   Policies successfully loaded from disk
2024-03-15T17:28:20.239+0700    DEBUG   [aws] 28:20.239224000 aws-api.scanner.adapt.aws        Using region 'us-east-1'
2024-03-15T17:28:20.239+0700    DEBUG   [aws] 28:20.239245000 aws-api.scanner.adapt.aws        Using endpoint 'http://localhost:4566'
2024-03-15T17:28:20.239+0700    DEBUG   [aws] 28:20.239248000 aws-api.scanner.adapt.aws        Discovering caller identity...
2024-03-15T17:28:20.245+0700    DEBUG   [aws] 28:20.245388000 aws-api.scanner.adapt.aws        AWS account ID: 000000000000
2024-03-15T17:28:20.245+0700    DEBUG   [aws] 28:20.245412000 aws-api.scanner.adapt.aws        Preparing to run for all 32 registered services

https://github.com/aquasecurity/trivy/blob/main/pkg/cloud/aws/scanner/progress.go#L72

trivy aws --endpoint http://localhost:4566 --region us-east-1 --service s3 --update-cache -q
[1/32] Scanning accessanalyzer...
[2/32] Scanning api-gateway...
[3/32] Scanning athena...
[4/32] Scanning cloudfront...
[5/32] Scanning cloudtrail...
[6/32] Scanning cloudwatch...
[7/32] Scanning codebuild...
[8/32] Scanning documentdb...
[9/32] Scanning dynamodb...
[10/32] Scanning ec2...
[11/32] Scanning ecr...
[12/32] Scanning ecs...
[13/32] Scanning efs...