feat: Add support for openEuler OS. #6475

wjunLu · 2024-04-10T02:21:16Z

Description

What's openEuler?

openEuler is an open source, free Linux distribution platform. The platform provides an open community for global developers to build an open, diversified, and architecture-inclusive software ecosystem. openEuler is also an innovative platform that
encourages everyone to propose new ideas, explore new approaches, and practice new solutions.

Learn more, please visit https://www.openeuler.org/en/

To support openEuler

Now, openEuler has 2,345,659 users, 18,072 contributors and 1,501 organization members(see https://datastat.openeuler.org/en/overview). It is necessary to support such a very mature open source operating system.

Discussion

Our discussion is here #6400

Related issues

Support security scanning for openEuler Linux #6400

Related PRs

[trivy-db ] (feat(openEuler): Add openEuler CVE database trivy-db#397)
[vuln-list-update ] (feat(openEuler): Add module for updating openEuler os cves vuln-list-update#284)

Remove this section if you don't have related PRs.

Self-test

I have tested the function to scan openEuler CVEs, see the following log

~# ./trivy image openeuler-22.03-lts-sp3:latest -d             
2024-04-09T19:53:05.437+0800    DEBUG   Severities: ["UNKNOWN" "LOW" "MEDIUM" "HIGH" "CRITICAL"]
2024-04-09T19:53:05.438+0800    DEBUG   Ignore statuses {"statuses": null}
2024-04-09T19:53:05.476+0800    DEBUG   cache dir:  /Users/luweijun/Library/Caches/trivy
2024-04-09T19:53:05.477+0800    DEBUG   DB update was skipped because the local DB is the latest
2024-04-09T19:53:05.477+0800    DEBUG   DB Schema: 2, UpdatedAt: 2024-04-09 06:11:37.740307479 +0000 UTC, NextUpdate: 2024-04-09 12:11:37.740306928 +0000 UTC, DownloadedAt: 2024-04-09 06:19:11.548928 +0000 UTC
2024-04-09T19:53:05.483+0800    INFO    Vulnerability scanning is enabled
2024-04-09T19:53:05.483+0800    DEBUG   Vulnerability type:  [os library]
2024-04-09T19:53:05.483+0800    INFO    Secret scanning is enabled
2024-04-09T19:53:05.483+0800    INFO    If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-04-09T19:53:05.483+0800    INFO    Please see also https://aquasecurity.github.io/trivy/dev/docs/scanner/secret/#recommendation for faster secret detection
2024-04-09T19:53:05.483+0800    DEBUG   Enabling misconfiguration scanners: [azure-arm cloudformation dockerfile helm kubernetes terraform terraformplan-json terraformplan-snapshot]
2024-04-09T19:53:05.626+0800    DEBUG   No secret config detected: trivy-secret.yaml
2024-04-09T19:53:05.628+0800    DEBUG   The nuget packages directory couldn't be found. License search disabled
2024-04-09T19:53:05.628+0800    DEBUG   No secret config detected: trivy-secret.yaml
2024-04-09T19:53:05.628+0800    DEBUG   Entering Inspect...
2024-04-09T19:53:05.628+0800    DEBUG   Image ID: sha256:1e1f6b42f136a4e5e2f5a8bd8c4036872da690fe4f626dc6b07617d2d08260a3
2024-04-09T19:53:05.628+0800    DEBUG   Diff IDs: [sha256:7e0958b5167da951982e27ec84f3ffa84bbab380dc71ce668fe9e305673ba44e]
2024-04-09T19:53:05.628+0800    DEBUG   Base Layers: []
2024-04-09T19:53:05.630+0800    DEBUG   Missing image ID in cache: sha256:1e1f6b42f136a4e5e2f5a8bd8c4036872da690fe4f626dc6b07617d2d08260a3
2024-04-09T19:53:05.630+0800    DEBUG   Running inspect...
2024-04-09T19:53:05.631+0800    DEBUG   Missing diff ID in cache: sha256:7e0958b5167da951982e27ec84f3ffa84bbab380dc71ce668fe9e305673ba44e
2024-04-09T19:53:13.793+0800    INFO    Family: openEuler, VersionID: 
2024-04-09T19:53:13.827+0800    INFO    Family: openEuler, VersionID: 22.03-LTS-SP3
2024-04-09T19:53:18.287+0800    INFO    License acquired from METADATA classifiers may be subject to additional terms for [gpg:1.16.0]
2024-04-09T19:53:18.288+0800    INFO    License acquired from METADATA classifiers may be subject to additional terms for [libcomps:0.1.19]
2024-04-09T19:53:18.380+0800    DEBUG   No secrets found in container image config
2024-04-09T19:53:18.454+0800    DEBUG   Finished inspect.
2024-04-09T19:53:18.457+0800    DEBUG   Exited from Inspect
2024-04-09T19:53:18.464+0800    INFO    Detected OS: openEuler
2024-04-09T19:53:18.466+0800    INFO    Detecting openEuler vulnerabilities [HERE]...
2024-04-09T19:53:18.466+0800    DEBUG   openEuler: os version: 22.03-LTS-SP3
2024-04-09T19:53:18.466+0800    DEBUG   openEuler: the number of packages: 137
2024-04-09T19:53:18.477+0800    INFO    Number of language-specific files: 1
2024-04-09T19:53:18.478+0800    INFO    Detecting python-pkg vulnerabilities...
2024-04-09T19:53:18.478+0800    DEBUG   Detecting library vulnerabilities, type: python-pkg, path: 

openeuler-22.03-lts-sp3:latest (openEuler 22.03-LTS-SP3)

Total: 13 (UNKNOWN: 0, LOW: 6, MEDIUM: 4, HIGH: 3, CRITICAL: 0)

Checklist

I've read the guidelines for contributing to this repository.
I've followed the conventions in the PR title.
I've added tests that prove my fix is effective or that my feature works.
I've updated the documentation with the relevant information (if needed).
I've added usage information (if the PR introduces new options)
I've included a "before" and "after" example to the description (if the PR is a user interface change).

github-actions · 2024-06-13T00:06:00Z

This PR is stale because it has been labeled with inactivity.

wjunLu · 2024-06-20T03:39:27Z

@knqyf263 Could you please run the tests first as you have time. Thank you very much!

wjunLu · 2024-06-24T06:25:49Z

The main errors occurred because aquasecurity/trivy-db#397 has not been merged, which thus should be reviewed first.

wjunLu · 2024-07-12T10:25:44Z

@DmitriyLewen I have updated this PR as we discussed in aquasecurity/trivy-db#397

2024-07-12T18:15:09+08:00       INFO    [openEuler] Detecting openEuler vulnerabilities...      os_version="20.03-LTS" pkg_num=352
2024-07-12T18:15:09+08:00       INFO    Number of language-specific files       num=1
2024-07-12T18:15:09+08:00       INFO    [python-pkg] Detecting vulnerabilities...
2024-07-12T18:15:09+08:00       WARN    This OS version is no longer supported by the distribution      family="openEuler" version="20.03-LTS"
2024-07-12T18:15:09+08:00       WARN    The vulnerability detection may be insufficient because security updates are not provided

pkg/fanal/analyzer/os/release/release.go

docs/docs/scanner/vulnerability.md

pkg/detector/ospkg/openeuler/testdata/fixtures/openeuler.yaml

DmitriyLewen · 2024-07-15T03:47:53Z

@wjunLu left notes. Take a look. please.
Also use go mod edit -replace github.com/aquasecurity/trivy-db=github.com/wjunLu/trivy-db@<commit_hash> + go mod tidy ( currently aquasecurity/trivy-db@21eae95) command and push into this PR.
This is necessary to make it easier to review this PR and run CI/CD tests.
After merge #397 you will remove replace and update trivy-db version.

wjunLu · 2024-07-16T06:22:27Z

@wjunLu left notes. Take a look. please. Also use go mod edit -replace github.com/aquasecurity/trivy-db=github.com/wjunLu/trivy-db@<commit_hash> + go mod tidy ( currently aquasecurity/trivy-db@21eae95) command and push into this PR. This is necessary to make it easier to review this PR and run CI/CD tests. After merge #397 you will remove replace and update trivy-db version.

As you mentioned before, I have changed code, the result looks good

… a vulnerability.

wjunLu · 2024-07-19T01:53:15Z

The code has beed changed with the latest trivy-db, testing result looks good:

github-actions · 2024-09-18T00:06:37Z

This PR is stale because it has been labeled with inactivity.

wjunLu · 2024-10-10T02:24:01Z

Hi @DmitriyLewen @knqyf263!
I'm so sad to see this PR has been closed, would you please take a final look at aquasecurity/trivy-db#397?

I'm looking forward to your reply.

DmitriyLewen · 2024-10-10T05:24:15Z

Hello @wjunLu
Sorry for the delay in reviewing.

We are focused on fixing bugs, fixing 429 error in Trivy-db and other high priority tasks.

Your PR is adding a new OS for scanning. These changes require attention and careful review.
When we restore stability to Trivy and have time to focus on your PR, we will do it.

Again, sorry for the wait and hope for your understanding

wjunLu · 2024-10-10T11:19:14Z

Hello @wjunLu Sorry for the delay in reviewing.

We are focused on fixing bugs, fixing 429 error in Trivy-db and other high priority tasks.

Your PR is adding a new OS for scanning. These changes require attention and careful review. When we restore stability to Trivy and have time to focus on your PR, we will do it.

Again, sorry for the wait and hope for your understanding

I get it!
Thanks for your reply, please feel free to left your questions and I will try my best to solve them.

github-actions · 2024-12-11T00:07:43Z

This PR is stale because it has been labeled with inactivity.

wjunLu · 2024-12-11T06:33:04Z

@DmitriyLewen @knqyf263 please take a look when you have time! Thank you!

pkg/detector/ospkg/openeuler/openeuler.go

markjens · 2024-12-11T07:51:52Z

pkg/detector/ospkg/openeuler/openeuler_test.go

+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			_ = dbtest.InitDB(t, tt.fixtures)
+			defer db.Close()


What is db here ? Where is it initialized ?
It looks like the result of dbtest.InitDB() call but it is ignored ...

Here, db represents the package github.com/aquasecurity/trivy-db/pkg/db.

Actually dbtest.InitDB() is equal to db.Init() which initializes a global *bolt.DB object, and this object can be closed by db.Close().

So, db.Close() is used to close the DB object initialized by dbtest.InitDB(). The details can be seen from

trivy/internal/dbtest/db.go

Line 34 in 4202c4b

require.NoError(t, db.Init(dbDir))

trivy/pkg/db/db.go

Line 36 in 4202c4b

Init = db.Init

Thank you for the explanation!

…ting allocation

Add support for openEuler OS.

5561890

wjunLu requested review from knqyf263 and DmitriyLewen as code owners April 10, 2024 02:21

wjunLu changed the title ~~Add support for openEuler OS.~~ feat: Add support for openEuler OS. Apr 11, 2024

wjunLu changed the title ~~feat: Add support for openEuler OS.~~ feat(openEuler): Add support for openEuler OS. Apr 13, 2024

wjunLu changed the title ~~feat(openEuler): Add support for openEuler OS.~~ feat: Add support for openEuler OS. Apr 13, 2024

wjunLu mentioned this pull request Jun 11, 2024

feat(openEuler): Add module for updating openEuler os cves aquasecurity/vuln-list-update#284

Merged

github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Jun 13, 2024

knqyf263 removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Jun 13, 2024

Merge branch 'main' into main

218dbe8

DmitriyLewen mentioned this pull request Jul 12, 2024

feat(openEuler): Add openEuler CVE database aquasecurity/trivy-db#397

Open

wjunLu added 3 commits July 12, 2024 17:33

Adapt openeuler_db

51cf0b6

Merge branch 'main' of https://github.com/wjunLu/trivy

b9a8fad

Bugfix

9db4c97

wjunLu and others added 2 commits July 12, 2024 18:30

Merge branch 'main' into main

0332491

Format code

e6344c3

DmitriyLewen reviewed Jul 15, 2024

View reviewed changes

pkg/fanal/analyzer/os/release/release.go Outdated Show resolved Hide resolved

docs/docs/scanner/vulnerability.md Outdated Show resolved Hide resolved

pkg/detector/ospkg/openeuler/testdata/fixtures/openeuler.yaml Outdated Show resolved Hide resolved

change code to adapt openeuler trivy-db

1705023

wjunLu added 2 commits July 18, 2024 00:56

Fix CI bugs

3385eed

Fix: src package versions are used for determining whether there is…

fedcbd8

… a vulnerability.

github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Sep 18, 2024

github-actions bot closed this Oct 8, 2024

DmitriyLewen reopened this Oct 10, 2024

DmitriyLewen removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Oct 10, 2024

wjunLu added 4 commits October 11, 2024 11:48

merge

cf9e78a

Merge remote-tracking branch 'upstream/main'

a1a2644

refresh go.mod

de4c47c

refresh docs

d91ed06

github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Dec 11, 2024

markjens reviewed Dec 11, 2024

View reviewed changes

Move vuln := types.DetectedVulnerability{} inside if to avoid was…

70c9871

…ting allocation

github-actions bot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and will be auto-closed. label Dec 12, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add support for openEuler OS. #6475

feat: Add support for openEuler OS. #6475

wjunLu commented Apr 10, 2024

github-actions bot commented Jun 13, 2024

wjunLu commented Jun 20, 2024

wjunLu commented Jun 24, 2024

wjunLu commented Jul 12, 2024

DmitriyLewen commented Jul 15, 2024 •

edited

Loading

wjunLu commented Jul 16, 2024

wjunLu commented Jul 19, 2024

github-actions bot commented Sep 18, 2024

wjunLu commented Oct 10, 2024

DmitriyLewen commented Oct 10, 2024

wjunLu commented Oct 10, 2024

github-actions bot commented Dec 11, 2024

wjunLu commented Dec 11, 2024

markjens Dec 11, 2024

wjunLu Dec 11, 2024

markjens Dec 12, 2024

feat: Add support for openEuler OS. #6475

Are you sure you want to change the base?

feat: Add support for openEuler OS. #6475

Conversation

wjunLu commented Apr 10, 2024

Description

What's openEuler?

To support openEuler

Discussion

Related issues

Related PRs

Self-test

Checklist

github-actions bot commented Jun 13, 2024

wjunLu commented Jun 20, 2024

wjunLu commented Jun 24, 2024

wjunLu commented Jul 12, 2024

DmitriyLewen commented Jul 15, 2024 • edited Loading

wjunLu commented Jul 16, 2024

wjunLu commented Jul 19, 2024

github-actions bot commented Sep 18, 2024

wjunLu commented Oct 10, 2024

DmitriyLewen commented Oct 10, 2024

wjunLu commented Oct 10, 2024

github-actions bot commented Dec 11, 2024

wjunLu commented Dec 11, 2024

markjens Dec 11, 2024

Choose a reason for hiding this comment

wjunLu Dec 11, 2024

Choose a reason for hiding this comment

markjens Dec 12, 2024

Choose a reason for hiding this comment

DmitriyLewen commented Jul 15, 2024 •

edited

Loading