Skip to content

Commit

Permalink
feat(report): output plugin (#4863)
Browse files Browse the repository at this point in the history
Signed-off-by: knqyf263 <[email protected]>
Co-authored-by: DmitriyLewen <[email protected]>
  • Loading branch information
knqyf263 and DmitriyLewen authored Dec 4, 2023
1 parent 70078b9 commit 99c04c4
Show file tree
Hide file tree
Showing 30 changed files with 366 additions and 163 deletions.
2 changes: 1 addition & 1 deletion cmd/trivy/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ func run() error {
if !plugin.IsPredefined(runAsPlugin) {
return xerrors.Errorf("unknown plugin: %s", runAsPlugin)
}
if err := plugin.RunWithArgs(context.Background(), runAsPlugin, os.Args[1:]); err != nil {
if err := plugin.RunWithURL(context.Background(), runAsPlugin, plugin.RunOptions{Args: os.Args[1:]}); err != nil {
return xerrors.Errorf("plugin error: %w", err)
}
return nil
Expand Down
45 changes: 44 additions & 1 deletion docs/docs/advanced/plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,8 +182,51 @@ $ trivy myplugin
Hello from Trivy demo plugin!
```

## Plugin Types
Plugins are typically intended to be used as subcommands of Trivy,
but some plugins can be invoked as part of Trivy's built-in commands.
Currently, the following type of plugin is experimentally supported:

- Output plugins

### Output Plugins

!!! warning "EXPERIMENTAL"
This feature might change without preserving backwards compatibility.

Trivy supports "output plugins" which process Trivy's output,
such as by transforming the output format or sending it elsewhere.
For instance, in the case of image scanning, the output plugin can be called as follows:

```shell
$ trivy image --format json --output plugin=<plugin_name> [--output-plugin-arg <plugin_flags>] <image_name>
```

Since scan results are passed to the plugin via standard input, plugins must be capable of handling standard input.

!!! warning
To avoid Trivy hanging, you need to read all data from `Stdin` before the plugin exits successfully or stops with an error.

While the example passes JSON to the plugin, other formats like SBOM can also be passed (e.g., `--format cyclonedx`).

If a plugin requires flags or other arguments, they can be passed using `--output-plugin-arg`.
This is directly forwarded as arguments to the plugin.
For example, `--output plugin=myplugin --output-plugin-arg "--foo --bar=baz"` translates to `myplugin --foo --bar=baz` in execution.

An example of the output plugin is available [here](https://github.com/aquasecurity/trivy-output-plugin-count).
It can be used as below:

```shell
# Install the plugin first
$ trivy plugin install github.com/aquasecurity/trivy-output-plugin-count
# Call the output plugin in image scanning
$ trivy image --format json --output plugin=count --output-plugin-arg "--published-after 2023-10-01" debian:12
```

## Example
https://github.com/aquasecurity/trivy-plugin-kubectl
- https://github.com/aquasecurity/trivy-plugin-kubectl
- https://github.com/aquasecurity/trivy-output-plugin-count

[kubectl]: https://kubernetes.io/docs/tasks/extend-kubectl/kubectl-plugins/
[helm]: https://helm.sh/docs/topics/plugins/
Expand Down
29 changes: 28 additions & 1 deletion docs/docs/configuration/reporting.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Reporting

## Supported Formats
## Format
Trivy supports the following formats:

- Table
Expand Down Expand Up @@ -373,6 +373,33 @@ $ trivy image --format template --template "@/usr/local/share/trivy/templates/ht
### SBOM
See [here](../supply-chain/sbom.md) for details.

## Output
Trivy supports the following output destinations:

- File
- Plugin

### File
By specifying `--output <file_path>`, you can output the results to a file.
Here is an example:

```
$ trivy image --format json --output result.json debian:12
```

### Plugin
!!! warning "EXPERIMENTAL"
This feature might change without preserving backwards compatibility.

Plugins capable of receiving Trivy's results via standard input, called "output plugin", can be seamlessly invoked using the `--output` flag.

```
$ trivy <target> [--format <format>] --output plugin=<plugin_name> [--output-plugin-arg <plugin_flags>] <target_name>
```

This is useful for cases where you want to convert the output into a custom format, or when you want to send the output somewhere.
For more details, please check [here](../advanced/plugins.md#output-plugins).

## Converting
To generate multiple reports, you can generate the JSON report first and convert it to other formats with the `convert` subcommand.

Expand Down
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_aws.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@ trivy aws [flags]
--max-cache-age duration The maximum age of the cloud cache. Cached data will be requeried from the cloud provider if it is older than this. (default 24h0m0s)
--misconfig-scanners strings comma-separated list of misconfig scanners to use for misconfiguration scanning (default [azure-arm,cloudformation,dockerfile,helm,kubernetes,terraform,terraformplan])
-o, --output string output file name
--output-plugin-arg string [EXPERIMENTAL] output plugin arguments
--policy-bundle-repository string OCI registry URL to retrieve policy bundle from (default "ghcr.io/aquasecurity/trivy-policies:0")
--policy-namespaces strings Rego namespaces
--region string AWS Region to scan
Expand Down
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_config.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ trivy config [flags] DIR
--misconfig-scanners strings comma-separated list of misconfig scanners to use for misconfiguration scanning (default [azure-arm,cloudformation,dockerfile,helm,kubernetes,terraform,terraformplan])
--module-dir string specify directory to the wasm modules that will be loaded (default "$HOME/.trivy/modules")
-o, --output string output file name
--output-plugin-arg string [EXPERIMENTAL] output plugin arguments
--password strings password. Comma-separated passwords allowed. TRIVY_PASSWORD should be used for security reasons.
--policy-bundle-repository string OCI registry URL to retrieve policy bundle from (default "ghcr.io/aquasecurity/trivy-policies:0")
--policy-namespaces strings Rego namespaces
Expand Down
27 changes: 14 additions & 13 deletions docs/docs/references/configuration/cli/trivy_convert.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,20 @@ trivy convert [flags] RESULT_JSON
### Options

```
--compliance string compliance report to generate
--dependency-tree [EXPERIMENTAL] show dependency origin tree of vulnerable packages
--exit-code int specify exit code when any security issues are found
--exit-on-eol int exit with the specified code when the OS reaches end of service/life
-f, --format string format (table,json,template,sarif,cyclonedx,spdx,spdx-json,github,cosign-vuln) (default "table")
-h, --help help for convert
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignorefile string specify .trivyignore file (default ".trivyignore")
--list-all-pkgs enabling the option will output all packages regardless of vulnerability
-o, --output string output file name
--report string specify a report format for the output (all,summary) (default "all")
-s, --severity strings severities of security issues to be displayed (UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL) (default [UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL])
-t, --template string output template
--compliance string compliance report to generate
--dependency-tree [EXPERIMENTAL] show dependency origin tree of vulnerable packages
--exit-code int specify exit code when any security issues are found
--exit-on-eol int exit with the specified code when the OS reaches end of service/life
-f, --format string format (table,json,template,sarif,cyclonedx,spdx,spdx-json,github,cosign-vuln) (default "table")
-h, --help help for convert
--ignore-policy string specify the Rego file path to evaluate each vulnerability
--ignorefile string specify .trivyignore file (default ".trivyignore")
--list-all-pkgs enabling the option will output all packages regardless of vulnerability
-o, --output string output file name
--output-plugin-arg string [EXPERIMENTAL] output plugin arguments
--report string specify a report format for the output (all,summary) (default "all")
-s, --severity strings severities of security issues to be displayed (UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL) (default [UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL])
-t, --template string output template
```

### Options inherited from parent commands
Expand Down
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_filesystem.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ trivy filesystem [flags] PATH
--no-progress suppress progress bar
--offline-scan do not issue API requests to identify dependencies
-o, --output string output file name
--output-plugin-arg string [EXPERIMENTAL] output plugin arguments
--parallel int number of goroutines enabled for parallel scanning, set 0 to auto-detect parallelism (default 5)
--password strings password. Comma-separated passwords allowed. TRIVY_PASSWORD should be used for security reasons.
--policy-bundle-repository string OCI registry URL to retrieve policy bundle from (default "ghcr.io/aquasecurity/trivy-policies:0")
Expand Down
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_image.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ trivy image [flags] IMAGE_NAME
--no-progress suppress progress bar
--offline-scan do not issue API requests to identify dependencies
-o, --output string output file name
--output-plugin-arg string [EXPERIMENTAL] output plugin arguments
--parallel int number of goroutines enabled for parallel scanning, set 0 to auto-detect parallelism (default 5)
--password strings password. Comma-separated passwords allowed. TRIVY_PASSWORD should be used for security reasons.
--platform string set platform in the form os/arch if image is multi-platform capable
Expand Down
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ trivy kubernetes [flags] { cluster | all | specific resources like kubectl. eg:
--node-collector-namespace string specify the namespace in which the node-collector job should be deployed (default "trivy-temp")
--offline-scan do not issue API requests to identify dependencies
-o, --output string output file name
--output-plugin-arg string [EXPERIMENTAL] output plugin arguments
--parallel int number of goroutines enabled for parallel scanning, set 0 to auto-detect parallelism (default 5)
--password strings password. Comma-separated passwords allowed. TRIVY_PASSWORD should be used for security reasons.
--policy-bundle-repository string OCI registry URL to retrieve policy bundle from (default "ghcr.io/aquasecurity/trivy-policies:0")
Expand Down
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_repository.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ trivy repository [flags] (REPO_PATH | REPO_URL)
--no-progress suppress progress bar
--offline-scan do not issue API requests to identify dependencies
-o, --output string output file name
--output-plugin-arg string [EXPERIMENTAL] output plugin arguments
--parallel int number of goroutines enabled for parallel scanning, set 0 to auto-detect parallelism (default 5)
--password strings password. Comma-separated passwords allowed. TRIVY_PASSWORD should be used for security reasons.
--policy-bundle-repository string OCI registry URL to retrieve policy bundle from (default "ghcr.io/aquasecurity/trivy-policies:0")
Expand Down
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_rootfs.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ trivy rootfs [flags] ROOTDIR
--no-progress suppress progress bar
--offline-scan do not issue API requests to identify dependencies
-o, --output string output file name
--output-plugin-arg string [EXPERIMENTAL] output plugin arguments
--parallel int number of goroutines enabled for parallel scanning, set 0 to auto-detect parallelism (default 5)
--password strings password. Comma-separated passwords allowed. TRIVY_PASSWORD should be used for security reasons.
--policy-bundle-repository string OCI registry URL to retrieve policy bundle from (default "ghcr.io/aquasecurity/trivy-policies:0")
Expand Down
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_sbom.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ trivy sbom [flags] SBOM_PATH
--no-progress suppress progress bar
--offline-scan do not issue API requests to identify dependencies
-o, --output string output file name
--output-plugin-arg string [EXPERIMENTAL] output plugin arguments
--redis-ca string redis ca file location, if using redis as cache backend
--redis-cert string redis certificate file location, if using redis as cache backend
--redis-key string redis key file location, if using redis as cache backend
Expand Down
1 change: 1 addition & 0 deletions docs/docs/references/configuration/cli/trivy_vm.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ trivy vm [flags] VM_IMAGE
--no-progress suppress progress bar
--offline-scan do not issue API requests to identify dependencies
-o, --output string output file name
--output-plugin-arg string [EXPERIMENTAL] output plugin arguments
--parallel int number of goroutines enabled for parallel scanning, set 0 to auto-detect parallelism (default 5)
--policy-bundle-repository string OCI registry URL to retrieve policy bundle from (default "ghcr.io/aquasecurity/trivy-policies:0")
--redis-ca string redis ca file location, if using redis as cache backend
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ require (
github.com/masahiro331/go-mvn-version v0.0.0-20210429150710-d3157d602a08
github.com/masahiro331/go-vmdk-parser v0.0.0-20221225061455-612096e4bbbd
github.com/masahiro331/go-xfs-filesystem v0.0.0-20230608043311-a335f4599b70
github.com/mattn/go-shellwords v1.0.12
github.com/mitchellh/hashstructure/v2 v2.0.2
github.com/mitchellh/mapstructure v1.5.0
github.com/moby/buildkit v0.11.6
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -1326,6 +1326,8 @@ github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m
github.com/mattn/go-runewidth v0.0.14 h1:+xnbZSEeDbOIg5/mE6JF0w6n9duR1l3/WmbinWVwUuU=
github.com/mattn/go-runewidth v0.0.14/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-shellwords v1.0.3/go.mod h1:3xCvwCdWdlDJUrvuMn7Wuy9eWs4pE8vqg+NOMyg4B2o=
github.com/mattn/go-shellwords v1.0.12 h1:M2zGm7EW6UQJvDeQxo4T51eKPurbeFbe8WtebGE2xrk=
github.com/mattn/go-shellwords v1.0.12/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
github.com/mattn/go-sqlite3 v1.14.6/go.mod h1:NyWgC/yNuGj7Q9rpYnZvas74GogHl5/Z4A/KQRfk6bU=
github.com/mattn/go-sqlite3 v1.14.16 h1:yOQRA0RpS5PFz/oikGwBEqvAWhWg5ufRz4ETLjwpU1Y=
github.com/mattn/go-sqlite3 v1.14.16/go.mod h1:2eHXhiwb8IkHr+BDWZGa96P6+rkvnG63S2DGjv9HUNg=
Expand Down
3 changes: 1 addition & 2 deletions pkg/cloud/aws/commands/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,6 @@ func filterServices(opt *flag.Options) error {
}

func Run(ctx context.Context, opt flag.Options) error {

ctx, cancel := context.WithTimeout(ctx, opt.GlobalOptions.Timeout)
defer cancel()

Expand Down Expand Up @@ -168,7 +167,7 @@ func Run(ctx context.Context, opt flag.Options) error {
}

r := report.New(cloud.ProviderAWS, opt.Account, opt.Region, res, opt.Services)
if err := report.Write(r, opt, cached); err != nil {
if err := report.Write(ctx, r, opt, cached); err != nil {
return xerrors.Errorf("unable to write results: %w", err)
}

Expand Down
8 changes: 3 additions & 5 deletions pkg/cloud/report/report.go
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,8 @@ func (r *Report) Failed() bool {
}

// Write writes the results in the give format
func Write(rep *Report, opt flag.Options, fromCache bool) error {
output, cleanup, err := opt.OutputWriter()
func Write(ctx context.Context, rep *Report, opt flag.Options, fromCache bool) error {
output, cleanup, err := opt.OutputWriter(ctx)
if err != nil {
return xerrors.Errorf("failed to create output file: %w", err)
}
Expand All @@ -72,8 +72,6 @@ func Write(rep *Report, opt flag.Options, fromCache bool) error {

var filtered []types.Result

ctx := context.Background()

// filter results
for _, resultsAtTime := range rep.Results {
for _, res := range resultsAtTime.Results {
Expand Down Expand Up @@ -137,7 +135,7 @@ func Write(rep *Report, opt flag.Options, fromCache bool) error {

return nil
default:
return pkgReport.Write(base, opt)
return pkgReport.Write(ctx, base, opt)
}
}

Expand Down
3 changes: 2 additions & 1 deletion pkg/cloud/report/resource_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package report

import (
"bytes"
"context"
"testing"

"github.com/stretchr/testify/assert"
Expand Down Expand Up @@ -111,7 +112,7 @@ No problems detected.

output := bytes.NewBuffer(nil)
tt.options.SetOutputWriter(output)
require.NoError(t, Write(report, tt.options, tt.fromCache))
require.NoError(t, Write(context.Background(), report, tt.options, tt.fromCache))

assert.Equal(t, "AWS", report.Provider)
assert.Equal(t, tt.options.AWSOptions.Account, report.AccountID)
Expand Down
3 changes: 2 additions & 1 deletion pkg/cloud/report/result_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package report

import (
"bytes"
"context"
"strings"
"testing"

Expand Down Expand Up @@ -70,7 +71,7 @@ See https://avd.aquasec.com/misconfig/avd-aws-9999

output := bytes.NewBuffer(nil)
tt.options.SetOutputWriter(output)
require.NoError(t, Write(report, tt.options, tt.fromCache))
require.NoError(t, Write(context.Background(), report, tt.options, tt.fromCache))

assert.Equal(t, "AWS", report.Provider)
assert.Equal(t, tt.options.AWSOptions.Account, report.AccountID)
Expand Down
3 changes: 2 additions & 1 deletion pkg/cloud/report/service_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package report

import (
"bytes"
"context"
"github.com/aquasecurity/trivy/pkg/clock"
"testing"
"time"
Expand Down Expand Up @@ -322,7 +323,7 @@ Scan Overview for AWS Account

output := bytes.NewBuffer(nil)
tt.options.SetOutputWriter(output)
require.NoError(t, Write(report, tt.options, tt.fromCache))
require.NoError(t, Write(context.Background(), report, tt.options, tt.fromCache))

assert.Equal(t, "AWS", report.Provider)
assert.Equal(t, tt.options.AWSOptions.Account, report.AccountID)
Expand Down
4 changes: 2 additions & 2 deletions pkg/commands/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ func loadPluginCommands() []*cobra.Command {
Short: p.Usage,
GroupID: groupPlugin,
RunE: func(cmd *cobra.Command, args []string) error {
if err = p.Run(cmd.Context(), args); err != nil {
if err = p.Run(cmd.Context(), plugin.RunOptions{Args: args}); err != nil {
return xerrors.Errorf("plugin error: %w", err)
}
return nil
Expand Down Expand Up @@ -773,7 +773,7 @@ func NewPluginCommand() *cobra.Command {
Short: "Run a plugin on the fly",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
return plugin.RunWithArgs(cmd.Context(), args[0], args[1:])
return plugin.RunWithURL(cmd.Context(), args[0], plugin.RunOptions{Args: args[1:]})
},
},
&cobra.Command{
Expand Down
8 changes: 4 additions & 4 deletions pkg/commands/artifact/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ type Runner interface {
// Filter filter a report
Filter(ctx context.Context, opts flag.Options, report types.Report) (types.Report, error)
// Report a writes a report
Report(opts flag.Options, report types.Report) error
Report(ctx context.Context, opts flag.Options, report types.Report) error
// Close closes runner
Close(ctx context.Context) error
}
Expand Down Expand Up @@ -280,8 +280,8 @@ func (r *runner) Filter(ctx context.Context, opts flag.Options, report types.Rep
return report, nil
}

func (r *runner) Report(opts flag.Options, report types.Report) error {
if err := pkgReport.Write(report, opts); err != nil {
func (r *runner) Report(ctx context.Context, opts flag.Options, report types.Report) error {
if err := pkgReport.Write(ctx, report, opts); err != nil {
return xerrors.Errorf("unable to write results: %w", err)
}

Expand Down Expand Up @@ -451,7 +451,7 @@ func Run(ctx context.Context, opts flag.Options, targetKind TargetKind) (err err
return xerrors.Errorf("filter error: %w", err)
}

if err = r.Report(opts, report); err != nil {
if err = r.Report(ctx, opts, report); err != nil {
return xerrors.Errorf("report error: %w", err)
}

Expand Down
5 changes: 4 additions & 1 deletion pkg/commands/convert/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ import (
)

func Run(ctx context.Context, opts flag.Options) (err error) {
ctx, cancel := context.WithTimeout(ctx, opts.Timeout)
defer cancel()

f, err := os.Open(opts.Target)
if err != nil {
return xerrors.Errorf("file open error: %w", err)
Expand All @@ -37,7 +40,7 @@ func Run(ctx context.Context, opts flag.Options) (err error) {
}

log.Logger.Debug("Writing report to output...")
if err = report.Write(r, opts); err != nil {
if err = report.Write(ctx, r, opts); err != nil {
return xerrors.Errorf("unable to write results: %w", err)
}

Expand Down
Loading

0 comments on commit 99c04c4

Please sign in to comment.