diff --git a/apps/docs/docs/contribute/connect-data/airbyte.md b/apps/docs/docs/contribute/connect-data/airbyte.md index 3fe0d97d5..e1ca9707d 100644 --- a/apps/docs/docs/contribute/connect-data/airbyte.md +++ b/apps/docs/docs/contribute/connect-data/airbyte.md @@ -1,5 +1,5 @@ --- -title: Connect via Airbyte +title: 🏗️ Connect via Airbyte sidebar_position: 2 --- diff --git a/apps/docs/docs/contribute/connect-data/gcs.md b/apps/docs/docs/contribute/connect-data/gcs.md index c2832049b..335dd5ea5 100644 --- a/apps/docs/docs/contribute/connect-data/gcs.md +++ b/apps/docs/docs/contribute/connect-data/gcs.md @@ -1,5 +1,5 @@ --- -title: Connect via Google Cloud Storage (GCS) +title: 🏗️ Connect via Google Cloud Storage (GCS) sidebar_position: 4 --- diff --git a/apps/docs/docs/contribute/impact-models.md b/apps/docs/docs/contribute/impact-models.md index 79cc10743..20e02316e 100644 --- a/apps/docs/docs/contribute/impact-models.md +++ b/apps/docs/docs/contribute/impact-models.md @@ -36,7 +36,7 @@ Before you begin you'll need the following on your system: - Python Poetry >= 1.8 (see [here](https://pypi.org/project/poetry/) to install it) - git (see [here](https://github.com/git-guides/install-git) if you don't have it installed) - A GitHub account (see [here](https://github.com/join) to open a new account) -- BigQuery access (see [here](../get-started/#login-to-bigquery) if you don't have it setup already) +- BigQuery access (see [here](../get-started) if you don't have it setup already) ### Install `gcloud` diff --git a/apps/docs/docs/contribute/index.mdx b/apps/docs/docs/contribute/index.mdx index 0028d0b75..5cf2a5e75 100644 --- a/apps/docs/docs/contribute/index.mdx +++ b/apps/docs/docs/contribute/index.mdx @@ -18,37 +18,37 @@ There are a variety of ways you can contribute to OSO. This doc features some of -Update Project Data +Update Project Data oss-directory Add a new project or update info for an existing project. OSS Projects, Analysts, General Public -Add Funding Data +Add Funding Data oss-funding Add to our database of OSS funding via CSV upload. OSS Funders, Analysts -Connect Your Data +Connect Your Data oso Write a plugin or help us replicate your data in the OSO data warehouse. Data Engineers, Developers -Propose an Impact Data Model +Propose an Impact Data Model oso Submit a dbt data model for tracking open source impact metrics. Data Scientists, Analysts -Share Insights +Share Insights insights Contribute to our library of data visualizations and Jupyter notebooks. Data Scientists, Analysts -Join a Data Challenge +Join a Data Challenge insights Work on a specific data challenge and get paid for your contributions. Data Scientists, Analysts diff --git a/apps/docs/docs/contribute/share-insights.md b/apps/docs/docs/contribute/share-insights.md index 42144b8ab..2ff57cec9 100644 --- a/apps/docs/docs/contribute/share-insights.md +++ b/apps/docs/docs/contribute/share-insights.md @@ -19,7 +19,7 @@ Share your work analyzing and visualizing OSS data by contributing to the [Insig We've included some starter notebooks to help data scientists get going with OSO datasets. You can find templates and community notebooks [here](https://github.com/opensource-observer/insights/blob/main/community/notebooks). -Also check out our playbook for [doing data science](../integrate/data-science) with OSO data. +Also check out our playbook for [doing data science](../integrate/python-notebooks) with OSO data. If you've created a notebook that you think our community can learn from, submit a PR to the `./community/notebooks/` directory in the Insights repo. Please include markdown in your notebook to explain your work. diff --git a/apps/docs/docs/get-started/index.mdx b/apps/docs/docs/get-started/index.mdx index 3d5ceb921..e94dd8445 100644 --- a/apps/docs/docs/get-started/index.mdx +++ b/apps/docs/docs/get-started/index.mdx @@ -89,7 +89,8 @@ To explore all the OSO datasets available, see [here](https://console.cloud.goog Now that you're set up, there are many ways to contribute to OSO and integrate the data with your application: -- [Do Data Science](../integrate/data-science) over OSO data sets +- [BigQuery Studio Guide](../integrate/query-data) +- [Write Python notebooks](../integrate/python-notebooks) - [Propose an impact model](../contribute/impact-models) to run in our data pipeline - [Query the OSO API](../integrate/api) for metrics and impact vectors from your web app diff --git a/apps/docs/docs/integrate/3rd-party.md b/apps/docs/docs/integrate/3rd-party.md new file mode 100644 index 000000000..5a80cb67b --- /dev/null +++ b/apps/docs/docs/integrate/3rd-party.md @@ -0,0 +1,8 @@ +--- +title: 🏗️ Connect to 3rd Party Tools +sidebar_position: 5 +--- + +:::warning +Coming soon... This page is a work in progress. +::: diff --git a/apps/docs/docs/integrate/api.md b/apps/docs/docs/integrate/api.md index a6619dbe2..af3475003 100644 --- a/apps/docs/docs/integrate/api.md +++ b/apps/docs/docs/integrate/api.md @@ -1,17 +1,14 @@ --- title: Use the GraphQL API -sidebar_position: 1 +sidebar_position: 10 --- -:::info -The OSO API currently only allows read-only GraphQL queries. -This API should only be used to fetch data to integrate into a live application. -If you need to perform data science over a large dataset, see the guides on -[doing data science](./data-science) -and [downloading static data](./download-data). -::: - -The OSO GraphQL API serves impact metrics for OSS projects, collections, and artifacts. Access to the OSO GraphQL API is necessary for any integration with OSO datasets. +The OSO API currently only allows read-only GraphQL queries against OSO mart models +(e.g. impact metrics, project info). +This API should only be used to fetch data to integrate into a live application in production. +For data exploration, check out the guides on +[performing queries](./query-data.md) +and [Python notebooks](./python-notebooks.md). ## Generate an API key diff --git a/apps/docs/docs/integrate/embed.md b/apps/docs/docs/integrate/embed.md deleted file mode 100644 index 1f323ce85..000000000 --- a/apps/docs/docs/integrate/embed.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: 🏗️ Embed OSO Widgets -sidebar_position: 4 ---- - -:::warning -This page is a work in progress. -::: - -Excited for this feature? Tell us on the -[GitHub issue](https://github.com/opensource-observer/oso/issues/623) diff --git a/apps/docs/docs/integrate/fork-pipeline.md b/apps/docs/docs/integrate/fork-pipeline.md new file mode 100644 index 000000000..e09bdeb2e --- /dev/null +++ b/apps/docs/docs/integrate/fork-pipeline.md @@ -0,0 +1,8 @@ +--- +title: 🏗️ Fork the Data Pipeline +sidebar_position: 4 +--- + +:::warning +Coming soon... This page is a work in progress. +::: diff --git a/apps/docs/docs/integrate/index.md b/apps/docs/docs/integrate/index.md new file mode 100644 index 000000000..48ddfe9f3 --- /dev/null +++ b/apps/docs/docs/integrate/index.md @@ -0,0 +1,16 @@ +--- +title: Get OSO Data +sidebar_position: 0 +--- + +Open Source Observer is a fully open data pipeline for measuring the impact of open source efforts. +That means all source code, data, and infrastructure is publicly available for use. + +- [Get Started](../get-started): to setup your Google account for data access and run your first query +- [Data Overview](./overview): for an overview of all data available +- [BigQuery Studio Guide](./query-data): to quickly query and download any data +- [Python notebooks](./python-notebooks): to do more in-depth data science and processing +- [Fork the data pipeline](./fork-pipeline): to setup your own data pipeline off any OSO model +- [Connect OSO to 3rd Party tools](./3rd-party): like Hex.tech, Tableau, and Metabase +- [API access](./api): to integrate OSO metrics into a live production application +- [oss-directory](./oss-directory): to leverage [oss-directory](https://github.com/opensource-observer/oss-directory) data separate from OSO diff --git a/apps/docs/docs/integrate/index.mdx b/apps/docs/docs/integrate/index.mdx deleted file mode 100644 index 435af9e5d..000000000 --- a/apps/docs/docs/integrate/index.mdx +++ /dev/null @@ -1,15 +0,0 @@ ---- -title: Get OSO Data -sidebar_position: 0 ---- - -:::info -There are a number of ways to access OSO data. This doc features some of the most common use cases, which you can explore further via the links on the sidebar. -::: - - -- If you want to download a snapshot of the data, the easiest way is to download it directly from BigQuery. Check out our guide to [Get Started](../get-started). -- If you are trying to connect the latest OSO metrics into a live production application, then check out our [GraphQL API](./api). -- If you want to do data science over any dataset, check out this [guide](./data-science). -- If you want to just download the project info from OSS directory, we have [libraries and exports](./oss-directory) that you can use. - diff --git a/apps/docs/docs/integrate/oss-directory.md b/apps/docs/docs/integrate/oss-directory.md index f376fe9c1..f07db5842 100644 --- a/apps/docs/docs/integrate/oss-directory.md +++ b/apps/docs/docs/integrate/oss-directory.md @@ -1,6 +1,6 @@ --- title: Fetch Project Info -sidebar_position: 3 +sidebar_position: 12 --- :::info diff --git a/apps/docs/docs/integrate/overview/ethereum.png b/apps/docs/docs/integrate/overview/ethereum.png new file mode 100644 index 000000000..b6c33aecd Binary files /dev/null and b/apps/docs/docs/integrate/overview/ethereum.png differ diff --git a/apps/docs/docs/integrate/overview/farcaster.jpg b/apps/docs/docs/integrate/overview/farcaster.jpg new file mode 100644 index 000000000..c07038534 Binary files /dev/null and b/apps/docs/docs/integrate/overview/farcaster.jpg differ diff --git a/apps/docs/docs/integrate/overview/gitcoin.png b/apps/docs/docs/integrate/overview/gitcoin.png new file mode 100644 index 000000000..38272ba4d Binary files /dev/null and b/apps/docs/docs/integrate/overview/gitcoin.png differ diff --git a/apps/docs/docs/integrate/overview/github.png b/apps/docs/docs/integrate/overview/github.png new file mode 100644 index 000000000..d9f7224c0 Binary files /dev/null and b/apps/docs/docs/integrate/overview/github.png differ diff --git a/apps/docs/docs/integrate/overview/index.mdx b/apps/docs/docs/integrate/overview/index.mdx new file mode 100644 index 000000000..078b8c707 --- /dev/null +++ b/apps/docs/docs/integrate/overview/index.mdx @@ -0,0 +1,267 @@ +--- +title: Data Overview +sidebar_position: 1 +--- + +import Button from "../../../src/components/plasmic/Button"; +import OsoLogo from "./oso-primary.png"; +import GithubLogo from "./github.png"; +import EthereumLogo from "./ethereum.png"; +import SuperchainLogo from "./superchain.png"; +import FarcasterLogo from "./farcaster.jpg"; +import LensLogo from "./lens-protocol.png"; +import GitcoinLogo from "./gitcoin.png"; +import OpenrankLogo from "./openrank.png"; + + +## OSO Data Pipeline + + + +