Skip to content
This repository has been archived by the owner on Jan 13, 2023. It is now read-only.

Fetching Organization Private Repos #149

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

betobrandao
Copy link

@betobrandao betobrandao commented Aug 9, 2018

As discussed in Issue #76 the current version of GitRob is not able to fetch private repositories of organizations.

This patch adds a new function that uses the GitHub API method Repositories.ListByOrg to retrieve a list of all the repositories from an organization including the private ones if the access token given has access to them.

After that, the CloneURL is set to the SSHURL attribute of the GitHub object , allowing the cloning of the repositories over ssh. In the future it's possible to check if the repository is public before setting the CloneURL, if public keeps using the HTTTPS URL, otherwise uses the SSHURL instead.
The selection of what function to use is based on the target type.

The command go fmt was ran on the core/github.go file, that's why the diff is showing so many modifications.

Added function GetRepositoriesFromOrganization for downloading of
private repo when target is an Organization using SSH URL


diff --git a/core/github.go b/core/github.go
index a194170..a32552b 100644
--- a/core/github.go
+++ b/core/github.go
@@ -91,6 +91,44 @@ func GetRepositoriesFromOwner(login *string, client *github.Client) ([]*GithubRe
        return allRepos, nil
 }

+func GetRepositoriesFromOrganization(login *string, client *github.Client) ([]*GithubRepository, error) {
+       var allRepos []*GithubRepository
+       loginVal := *login
+       ctx := context.Background()
+       opt := &github.RepositoryListByOrgOptions{
+               Type: "sources",
+       }
+
+       for {
+               repos, resp, err := client.Repositories.ListByOrg(ctx, loginVal, opt)
+               if err != nil {
+                       return allRepos, err
+               }
+               for _, repo := range repos {
+                       if !*repo.Fork {
+                               r := GithubRepository{
+                                       Owner:         repo.Owner.Login,
+                                       ID:            repo.ID,
+                                       Name:          repo.Name,
+                                       FullName:      repo.FullName,
+                                       CloneURL:      repo.SSHURL,
+                                       URL:           repo.HTMLURL,
+                                       DefaultBranch: repo.DefaultBranch,
+                                       Description:   repo.Description,
+                                       Homepage:      repo.Homepage,
+                               }
+                               allRepos = append(allRepos, &r)
+                       }
+               }
+               if resp.NextPage == 0 {
+                       break
+               }
+               opt.Page = resp.NextPage
+       }
+
+       return allRepos, nil
+}
+
 func GetOrganizationMembers(login *string, client *github.Client) ([]*GithubOwner, error) {
        var allMembers []*GithubOwner
        loginVal := *login
diff --git a/main.go b/main.go
--- a/main.go
+++ b/main.go
@@ -57,12 +57,18 @@ func GatherRepositories(sess *core.Session) {
   for i := 0; i < threadNum; i++ {
     go func() {
       for {
+        var repos []*core.GithubRepository
+       var err error
         target, ok := <-ch
         if !ok {
           wg.Done()
           return
         }
-        repos, err := core.GetRepositoriesFromOwner(target.Login, sess.GithubClient)
+       if *target.Type == "Organization" {
+               repos, err = core.GetRepositoriesFromOrganization(target.Login, sess.GithubClient)
+       } else {
+               repos, err = core.GetRepositoriesFromOwner(target.Login, sess.GithubClient)
+       }
         if err != nil {
           sess.Out.Error(" Failed to retrieve repositories from %s: %s\n", *target.Login, err)
         }



Added function GetRepositoriesFromOrganization for downloading of
private repo when target is an Organization using SSH URL
@njfox
Copy link

njfox commented Nov 28, 2018

Hey @betobrandao, I'm trying to build your fork of Gitrob with this patch for an engagement but I'm getting the following error:

$ go get github.com/betobrandao/gitrob
# github.com/betobrandao/gitrob
/home/nick/go/src/github.com/betobrandao/gitrob/main.go:68:16: undefined: core.GetRepositoriesFromOrganization

And trying to build manually:

$ go build -o gitrob
# _/home/nick/Documents/projects/gitrob
./main.go:68:16: undefined: core.GetRepositoriesFromOrganization

Am I doing anything wrong? I tried to fix it but I don't know Go so I wasn't able to get it to compile.

@tmatias
Copy link

tmatias commented Nov 28, 2018

@njfox that's because the import path in the source doesn't match the directory structure of your clone (/home/nick/go/src/github.com/betobrandao/gitrob, from the error message).

https://golang.org/doc/code.html fwiw, is a concise overview of the basic mechanics for working with go code

@@ -1,113 +1,151 @@
package core

import (
"context"
"context"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did you change the format? Wasn't it gofmt'd?

@njfox
Copy link

njfox commented Nov 28, 2018

Thanks @tmatias, that worked. The patch also seems to have worked for fetching private organization repositories.

@rayterrill
Copy link

Appreciate this work @betobrandao! Forked this repo as well and took much of your code, made a few more changes to get it working over HTTPS vs SSH using the same access token already provided. Thank you!

@irawanhd
Copy link

irawanhd commented Feb 6, 2020

@rayterrill @njfox
having issue with ssh failed, any ideas? git clone using ssh works well

Error cloning repository xx/xx: ssh: handshake failed: ssh: unable to authenticate, attempted methods [publickey none], no supported methods remain`

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants