Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add .netrc support #82

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add .netrc support #82

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions MODULE.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ use_repo(
go_deps,
"com_github_bazelbuild_buildtools",
"com_github_crillab_gophersat",
"com_github_jdx_go_netrc",
"com_github_onsi_gomega",
"com_github_sassoftware_go_rpmutils",
"com_github_sirupsen_logrus",
Expand Down
1 change: 1 addition & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ require (
github.com/golang/protobuf v1.5.2 // indirect
github.com/google/go-cmp v0.5.9 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
github.com/jdx/go-netrc v1.0.0 // indirect
github.com/klauspost/compress v1.11.1 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/inconshreveable/mousetrap v1.0.1 h1:U3uMjPSQEBMNp1lFxmllqCPM6P5u/Xq7Pgzkat/bFNc=
github.com/inconshreveable/mousetrap v1.0.1/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/jdx/go-netrc v1.0.0 h1:QbLMLyCZGj0NA8glAhxUpf1zDg6cxnWgMBbjq40W0gQ=
github.com/jdx/go-netrc v1.0.0/go.mod h1:Gh9eFQJnoTNIRHXl2j5bJXA1u84hQWJWgGh569zF3v8=
github.com/klauspost/compress v1.11.1 h1:bPb7nMRdOZYDrpPMTA3EInUQrdgoBinqUuSwlGdKDdE=
github.com/klauspost/compress v1.11.1/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/onsi/ginkgo/v2 v2.7.0 h1:/XxtEV3I3Eif/HobnVx9YmJgk8ENdRsuUmM+fLCFNow=
Expand Down
1 change: 1 addition & 0 deletions pkg/repo/BUILD.bazel
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ go_library(
"//pkg/api",
"//pkg/api/bazeldnf",
"//pkg/rpm",
"@com_github_jdx_go_netrc//:go-netrc",
"@com_github_sirupsen_logrus//:logrus",
"@io_k8s_sigs_yaml//:yaml",
],
Expand Down
40 changes: 39 additions & 1 deletion pkg/repo/fetch.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,20 @@ package repo

import (
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"fmt"
"hash"
"io"
"net/http"
"net/url"
"os"
"os/user"
"path"
"path/filepath"
"strings"

"github.com/jdx/go-netrc"
"github.com/rmohr/bazeldnf/pkg/api"
"github.com/rmohr/bazeldnf/pkg/api/bazeldnf"
log "github.com/sirupsen/logrus"
Expand Down Expand Up @@ -229,6 +232,41 @@ func fileGet(filename string) (*http.Response, error) {
return resp, nil
}

var netrcCached *netrc.Netrc

func getNetrc() (*netrc.Netrc, error) {
if netrcCached != nil {
return netrcCached, nil
}
usr, err := user.Current()
if err != nil {
return nil, fmt.Errorf("getting current user: %w", err)
}
n, err := netrc.Parse(filepath.Join(usr.HomeDir, ".netrc"))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bazaldnf is consumed internally by the bazel rules. It seems like this would break hermeticity in that case?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT this part of bazeldnf runs in the repository rule / loading phase, which is "allowed" to poke at local files and perform network calls.
Also, this is reading the .netrc file to supply credentials to a http GET request. The GET request itself is also not hermetic.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moreso what I was getting at is that this dependency is opaque to everything that consumes bazeldnf itself. ie: how would you identify this as something that you build depends on?

Copy link
Author

@mkosiba mkosiba Dec 2, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bazel already defaults to reading ~/.netrc for the built in http_archive rules (see https://github.com/bazelbuild/bazel/blob/master/tools/build_defs/repo/http.bzl#L144 and https://github.com/bazelbuild/bazel/blob/master/tools/build_defs/repo/utils.bzl#L460 which show how it's implemented) without any additional configuration from the user. This is well established behaviour.

Actually, I had another look through the code and I'll go back on my previous statement. I don't think this code path runs as part of the build at all:

  • the bazeldnf target is only exposed as a user convenience, so you can bazel run it, and so we should not consider the entire bazeldnf command with all of its subcommands to be part of the build (though, yea, sure, someone could run it as part of their custom rule or a genrule, but that's on them),
  • the two rules that invoke bazeldnf are rpm2tar and tar2files, neither of which use the repo package, so they shouldn't hit this code path,
  • the rpm rules use Bazel's downloader to download the files.

In other words these changes only affect bazeldnf invocations that are performed by the user locally prior to the build (like bazeldnf rpmtree) and so build hermeticity concerns shouldn't come into play at all.

if err == nil {
netrcCached = n
}
return n, err
}

func httpGet(rawUrl string) (*http.Response, error) {
url, err := url.Parse(rawUrl)
if err != nil {
return nil, fmt.Errorf("parsing %s as a URL: %w", rawUrl, err)
}
netrc, err := getNetrc()
if err != nil {
return nil, fmt.Errorf("getting netrc: %w", err)
}
m := netrc.Machine(url.Hostname())
req, err := http.NewRequest("GET", rawUrl, nil)
if m != nil {
auth := m.Get("login") + ":" + m.Get("password")
req.Header.Add("Authorization", "Basic "+base64.StdEncoding.EncodeToString([]byte(auth)))
}
return http.DefaultClient.Do(req)
}

func (*getterImpl) Get(rawURL string) (*http.Response, error) {
u, err := url.Parse(rawURL)
if err != nil {
Expand All @@ -237,7 +275,7 @@ func (*getterImpl) Get(rawURL string) (*http.Response, error) {
if u.Scheme == "file" {
return fileGet(u.Path)
}
return http.Get(rawURL)
return httpGet(rawURL)
}

func toHex(hasher hash.Hash) string {
Expand Down