Support downloading dependencies from S3

[johannespostler]

In our build setup, we use some internal dependencies that are published in an internal Maven repository hosted on S3. The configuration for these repositories looks like this [1]:

maven {
    setUrl("s3://my-bucket.s3.eu-central-1.amazonaws.com")
    credentials(AwsCredentials::class.java) {
        accessKey = project.s3AccessKey
        secretKey = project.s3SecretKey
    }
}

This leads to the following errors when executing spdxSbom:

org.spdx.library.InvalidSPDXAnalysisException: Invalid download location s3://my-bucket.s3.eu-central-1.amazonaws.com/org/package/org.package.dep/1.2.3/org.package.dep-1.2.3.jar. Must match the pattern ^(NONE|NOASSERTION|(((git|hg|svn|bzr)\+)?(http:\/\/www\.|https:\/\/www\.|http:\/\/|https:\/\/|ssh:\/\/|git:\/\/|svn:\/\/|sftp:\/\/|ftp:\/\/|file:\/\/|file:\/)?[a-zA-Z0-9\.\-\ \_]+(:[0-9]{1,5})?(\/.*)?)|(git\+git@[a-zA-Z0-9\.\-]+:[a-zA-Z0-9/\\.@\-]+)|(bzr\+lp:[a-zA-Z0-9\.\-]+))$

Apparently s3:// is not among the list of protocols allowed.

[1] https://docs.gradle.org/current/userguide/declaring_repositories.html#sec:repository-types

[loosebazooka]

Yeah I think this comes from the spdx-java-library. @goneall any ideas? Is this is a spdx limitation? Or should we add s3 as a valid protocol?

[goneall]

Yeah I think this comes from the spdx-java-library. @goneall any ideas? Is this is a spdx limitation? Or should we add s3 as a valid protocol?

It looks like the spec should support any URL (reference SPDX download location).

Should we allow any protocol? Looking at the list of URL schemes, there are a lot of informal protocols in use out there.

[loosebazooka]

Yeah I'm down for being as permissive as possible. But I think this change is in spdx-java-library so it's your call?

[goneall]

I'll compare notes in tomorrows SPDX tools call and see what the other library maintainers are allowing.

I created an issue to track in the library: spdx/Spdx-Java-Library#253