testgrid should support non-GCS stores for test results

[dims]

Can we please add support for things other than GCS buckets?

• plain old browsable web directories (http/https)
• S3 style object stores
  etc..

[dims]

cc @thockin @BenTheElder

[BenTheElder]

cc @michelle192837 @rmmh
I believe the plan is to open source testgrid including the update which @fejta has also worked on but I'm not sure where these efforts are at currently.

[fejta]

/area testgrid
/help

This sounds like a great task for someone outside of Google to take on!

[BenTheElder]

this would be the OSS updater which scrapes the results, though we're not migrated to this one yet.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[akutz]

/remove-lifecycle stale

Hi @BenTheElder,

Is the OSS updater still the appropriate location to which to add support for non-GCS storage? Or has Spyglass superseded that? I was thinking of taking on the task of adding S3 support to the updater either directly or via the Go Cloud Project.

[BenTheElder]

Spyglass is unrelated to testgrid. (they consume the same data but do different things, sypglass is targeted more at what gubernator job viewing does). I'll defer to @michelle192837 @fejta for testgrid.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[michelle192837]

/remove-lifecycle stale

This is blocked on open sourcing.

[midnightconman]

Can we have an update on this @michelle192837?

[michelle192837]

Yup, though not a clear one. This is still blocked on open sourcing, and I don't have a good ETA at the moment for it. :( You probably want to track #10409, #3323, and/or #3324 for when I've got better updates on this.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[michelle192837]

/remove-lifecycle rotten
/lifecycle frozen

[spiffxp]

FWIW, we're moving on open sourcing. It'll take a bit before we're in a position to accept contributions and see those land in testgrid.k8s.io, but we're getting there #10409 (comment)

[spiffxp]

/remove-help
I'm going to remove help since this is blocked at the moment, we should add back when it's feasible for a new contributor to accomplish