docs: more reorganization of docs (#2643)

README.md

@@ -212,4 +212,4 @@ is released under Apache 2.0
 This repository does not contain data.
 Datasets may include material that may be subject to third party rights.
 For details on each dataset, see
-the [Data Overview](https://docs.opensource.observer/docs/integrate/overview/).
+the [Data Overview](https://docs.opensource.observer/docs/integrate/datasets/).

apps/docs/blog/2024-03-20-request-for-impact-metrics/index.md

@@ -17,7 +17,7 @@ If you'd like to get involved, here's what to do:
 1. Apply to [join the Data Collective](https://www.kariba.network/). It only takes a few minutes. We'll review, and reach out to schedule an onboarding call.
 2. Join our [Discord server](https://www.opensource.observer/discord) and say hello.
 3. Get inspiration from our Colab directory of [starter notebooks for impact metrics](https://drive.google.com/drive/folders/1I4exiLfZYgPwIGBEzuAMeGtJUhtS_DE7?usp=drive_link). We also have of them in our [Insights repo](https://github.com/opensource-observer/insights/tree/main/community/notebooks) if you prefer to run them locally.
-4. Consult [our docs](../../docs/) and especially our [impact metric spec](../../docs/guides/impact-metrics/) as you prepare your analysis.
+4. Consult [our docs](../../docs/) and especially our [impact metric spec](../../docs/references/impact-metrics/) as you prepare your analysis.
 
 The rest of this post includes more details on the types of impact metrics we're interested in.
 
@@ -62,7 +62,7 @@ _Metrics that attempt to differentiate onchain users based on behavior and trust
 
 This is a hot topic right now, with a number of projects attempting to create reputation systems for onchain users. We're integrating with many of the leading projects and bringing the data they generate into the OSO data warehouse. From there, there are all sorts of directions you can take the analysis!
 
-The **Onchain Activity** metric is a good starting point for new analysts. Have a look at our working definitions for [onchain users](../../docs/guides/impact-metrics/onchain) and help us create some better ones. The **Network Loyalty** is a spicy metric that will get even spicier as we add more networks to the OSO dataset!
+The **Onchain Activity** metric is a good starting point for new analysts. Have a look at our working definitions for [onchain users](../../docs/references/impact-metrics/onchain) and help us create some better ones. The **Network Loyalty** is a spicy metric that will get even spicier as we add more networks to the OSO dataset!
 
 | Impact Metric    | Description                                           | Comments                                   |
 | :--------------- | :---------------------------------------------------- | :----------------------------------------- |

apps/docs/blog/2024-06-29-impact-metrics-rf4-deep-dive/index.md

@@ -116,7 +116,7 @@ We're excited to see how this type of model evolves and is extended for future u
 
 ## Developing the impact metrics
 
-The following design principles guide the development and evolution of [impact metrics](../../docs/guides/impact-metrics/#principles):
+The following design principles guide the development and evolution of [impact metrics](../../docs/references/impact-metrics/#principles):
 
 - Verifiability: Metrics should be based on public data that can be independently verified. They should not rely on proprietary data sources or private APIs.
 - Reproducibility: Metrics should be easy to reproduce, simulate, and audit to ensure they are achieving the intended results. They should not have a "black box" element that makes them difficult to understand or replicate.

apps/docs/blog/2024-07-16-oso-data-portal/index.md

@@ -14,17 +14,17 @@ OSO Data Exchange on Google BigQuery.
 Here, we will publish every data set we have as live,
 up-to-date, and free to use datasets.
 In addition to sharing every model in the
-[OSO production data pipeline](https://docs.opensource.observer/docs/integrate/overview/#oso-production-data-pipeline),
+[OSO production data pipeline](../../docs/integrate/datasets#oso-production-data-pipeline),
 we are sharing source data for blocks/transactions/traces across
-[7 chains in the OP Superchain](https://docs.opensource.observer/docs/integrate/overview/#superchain-data)
+[7 chains in the OP Superchain](../../docs/integrate/datasets#optimism-data)
 (including Optimism, Base, Frax, Metal, Mode, PGN, Zora),
-[Gitcoin Data](https://docs.opensource.observer/docs/integrate/overview/#gitcoin-and-passport-data),
-and [OpenRank](https://docs.opensource.observer/docs/integrate/overview/#openrank-data).
+[Gitcoin Data](../../docs/integrate/datasets#gitcoin-and-passport-data),
+and [OpenRank](../../docs/integrate/datasets#openrank-data).
 This builds on the existing BigQuery public data ecosystem that includes
-[GitHub](https://docs.opensource.observer/docs/integrate/overview/#github-data),
-[Ethereum](https://docs.opensource.observer/docs/integrate/overview/#ethereum-data),
-[Farcaster](https://docs.opensource.observer/docs/integrate/overview/#farcaster-data),
-and [Lens](https://docs.opensource.observer/docs/integrate/overview/#lens-data) data.
+[GitHub](../../docs/integrate/datasets#github-data),
+[Ethereum](../../docs/integrate/datasets#ethereum-data),
+[Farcaster](../../docs/integrate/datasets#farcaster-data),
+and [Lens](../../docs/integrate/datasets#lens-data) data.
 To learn more, check out the data portal here:
 
 <p style={{"text-align": "center"}}>

apps/docs/blog/2024-08-12-war-for-public-goods/index.md

@@ -97,4 +97,4 @@ Once again, there are valuable lessons from baseball that we can apply:
 - **Embrace Competing Implementations**: As mentioned earlier, there is no single standard calculation for WAR. Even though the data and methodologies are public, there’s vibrant debate about which method is best. In an industry as dynamic as crypto, this diversity of approaches is a strength, encouraging innovation and continuous improvement.
 - **Engage the Research Community**: WAR models are like catnip for data enthusiasts. They offer endless opportunities for simulation, experimentation, and visualization. By providing well-documented, easy-to-use datasets and stoking interest in the results from protocol leaders and governance teams, we can create a garden of catnip for researchers and data nerds.
 
-At Open Source Observer, we want to build the equivalent of a sabermetrics for open source impact. If you have ideas for a WAR model, check out our existing library of [data](../../docs/integrate/overview) and [metrics](../../docs/guides/impact-metrics/), and come say hi on our [Discord](https://www.opensource.observer/discord).
+At Open Source Observer, we want to build the equivalent of a sabermetrics for open source impact. If you have ideas for a WAR model, check out our existing library of [data](../../docs/integrate/datasets) and [metrics](../../docs/references/impact-metrics/), and come say hi on our [Discord](https://www.opensource.observer/discord).

apps/docs/docs/contribute-data/bigquery/_category_.json

@@ -1,6 +1,6 @@
 {
   "label": "Connect via BigQuery",
-  "position": 1,
+  "position": 2,
   "link": {
     "type": "doc",
     "id": "index"

apps/docs/docs/contribute-data/bigquery/index.md

@@ -8,7 +8,9 @@ trivially easy to integrate any public dataset into
 the OSO data pipeline, provided the dataset exists in
 the US multi-region.
 
-If the dataset needs to be replicated, see our guide on
+If you want OSO to host a copy of
+the dataset in the US multi-region,
+see our guide on
 [BigQuery Data Transfer Service](./replication.md).
 
 ## Make the data available in the US region

apps/docs/docs/contribute-data/quickstart.md

@@ -1,6 +1,6 @@
 ---
 title: Dagster Quickstart
-sidebar_position: 2
+sidebar_position: 1
 ---
 
 # Dagster quickstart

apps/docs/docs/contribute-models/challenges/2024-04-05-data-challenge-01.md

@@ -72,7 +72,7 @@ Here’s what you need to do to participate in this (and future) data challenges
 
 1. Join the [Kariba Data Collective](https://www.kariba.network/). Participation is open to anyone in the data collective (free to join, but we review applications).
 2. Join our Discord and receive a **_data-collective_** role; ask questions and share code snippets in the **_#data-challenges_** channel.
-3. Bookmark [our docs and tutorials](../../guides/impact-metrics/) for creating impact metrics. Also make sure to browse our [Colab Notebooks](https://drive.google.com/drive/folders/1mzqrSToxPaWhsoGOR-UVldIsaX1gqP0F) for examples and inspiration.
+3. Bookmark [our docs and tutorials](../../references/impact-metrics/) for creating impact metrics. Also make sure to browse our [Colab Notebooks](https://drive.google.com/drive/folders/1mzqrSToxPaWhsoGOR-UVldIsaX1gqP0F) for examples and inspiration.
 4. Submit your impact metric(s) by opening an issue on our Insight repo [here](https://github.com/opensource-observer/insights/issues/new/choose).
 
 Every impact metric submission should include:

apps/docs/docs/contribute-models/challenges/2024-07-30-openrank.md

@@ -34,7 +34,7 @@ We've queued up a few ideas to get you started:
 
 - **Protocol Rank**. This [notebook](https://github.com/opensource-observer/insights/blob/main/community/data_challenges/openrank/OpenRank_ProtocolRank.ipynb) uses the same logic as the Repo Rank notebook, but narrows the curation space to only include projects that have deployed contracts on an Ethereum Layer 2.
 
-- **Gitcoin Project Rank**. This [notebook](https://github.com/opensource-observer/insights/blob/main/community/data_challenges/openrank/OpenRank_GitcoinProjectRank.ipynb) builds on top of our recently-added [Gitcoin Grants dataset](https://docs.opensource.observer/docs/integrate/overview/#gitcoin-and-passport-data), which includes a history of every peer-to-peer donation on the Gitcoin platform. It also taps the Farcaster dataset to identify usernames associated with donors.
+- **Gitcoin Project Rank**. This [notebook](https://github.com/opensource-observer/insights/blob/main/community/data_challenges/openrank/OpenRank_GitcoinProjectRank.ipynb) builds on top of our recently-added [Gitcoin Grants dataset](../../integrate/datasets/index.mdx#gitcoin-and-passport-data), which includes a history of every peer-to-peer donation on the Gitcoin platform. It also taps the Farcaster dataset to identify usernames associated with donors.
 
 ### Requirements
 

apps/docs/docs/contribute-models/index.mdx

@@ -17,42 +17,6 @@ There are a variety of ways you can contribute to OSO. This doc features some of
     </tr>
   </thead>
   <tbody>
-    <tr>
-      <td>
-        <a href="./project-data">Update Project Data</a>
-      </td>
-      <td>
-        <a href="https://github.com/opensource-observer/oss-directory">
-          oss-directory
-        </a>
-      </td>
-      <td>Add a new project or update info for an existing project.</td>
-      <td>OSS Projects, Analysts, General Public</td>
-    </tr>
-    <tr>
-      <td>
-        <a href="./connect-data/funding-data">Add Funding Data</a>
-      </td>
-      <td>
-        <a href="https://github.com/opensource-observer/oss-funding">
-          oss-funding
-        </a>
-      </td>
-      <td>Add to our database of OSS funding via CSV upload.</td>
-      <td>OSS Funders, Analysts</td>
-    </tr>
-    <tr>
-      <td>
-        <a href="./connect-data">Connect Your Data</a>
-      </td>
-      <td>
-        <a href="https://github.com/opensource-observer/oso">oso</a>
-      </td>
-      <td>
-        Write a plugin or help us replicate your data in the OSO data warehouse.
-      </td>
-      <td>Data Engineers, Developers</td>
-    </tr>
     <tr>
       <td>
         <a href="./data-models">Propose an Impact Data Model</a>

apps/docs/docs/contribute-models/share-insights.md

@@ -31,7 +31,7 @@ Contribute your own data visualization templates by adding them to the `./visual
 
 ### Impact Metrcs
 
-We're building scripts and analysis frameworks for community-generated impact metrics. Submissions must be replicable with data sourced entirely from the OSO data warehouse and should follow the follow the [Impact Metrics Specification](../guides/impact-metrics/).
+We're building scripts and analysis frameworks for community-generated impact metrics. Submissions must be replicable with data sourced entirely from the OSO data warehouse and should follow the follow the [Impact Metrics Specification](../references/impact-metrics/).
 
 In most cases, impact vectors will be reviewed and recognized via one of our Data Challenges, with separate instructions for showcasing work and submitting pull requests.
 

apps/docs/docs/get-started/api.mdx

@@ -9,7 +9,7 @@ import TabItem from '@theme/TabItem';
 The [OSO](https://www.opensource.observer) API serves 
 queries on metrics and metadata about open source projects.
 It is a [GraphQL](https://graphql.org/) API backed by the
-[OSO data pipeline](../guides/architecture.md).
+[OSO data pipeline](../references/architecture.md).
 
 Let's make your first query in under five minutes.
 

apps/docs/docs/get-started/bigquery.mdx

@@ -70,7 +70,7 @@ Click on the following link to subscribe to the OSO production dataset:
 
 Create a linked dataset in your own GCP project.
 
-![link dataset](../integrate/overview/bigquery_subscribe.png)
+![link dataset](../integrate/datasets/bigquery_subscribe.png)
 
 ## Make your first query
 
@@ -96,8 +96,8 @@ The results will appear in a table at the bottom of the console.
 The console will help you complete your query as you type, and will also provide you with a preview of the results and computation time. You can save your queries, download the results, and even make simple visualizations directly from the console.
 
 :::tip
-To explore all the OSO datasets available, see the
-[Data Overview](../integrate/overview/index.mdx).
+To explore all the OSO datasets available, see the 
+[Data Overview](../integrate/datasets/index.mdx).
 
 - **oso_production** contains all production data. This can be quite large depending on the dataset.
 - **oso_playground** contains only the last 2 weeks for every dataset. We recommend using this for development and testing.

apps/docs/docs/guides/ops/_category_.json

@@ -1,5 +1,5 @@
 {
-  "label": "🏗️ Operations Guides",
+  "label": "🏗️ Operations",
   "position": 8,
   "link": {
     "type": "doc",

apps/docs/docs/guides/style-guide.md

@@ -1,6 +1,6 @@
 ---
 title: Code Style Guide
-sidebar_position: 6
+sidebar_position: 1
 ---
 
 ## dbt Models

apps/docs/docs/integrate/3rd-party.mdx

@@ -12,7 +12,7 @@ OSO datasets and models are public and can be accessed on BigQuery. This allows
 First, we need to subscribe to an OSO dataset in your own
 Google Cloud account.
 You can see all of our available datasets in the
-[Data Overview](./overview/index.mdx).
+[Data Overview](./datasets/index.mdx).
 
 We recommend starting with the OSO production data pipeline here:
 

apps/docs/docs/integrate/api.md

@@ -10,22 +10,6 @@ If you want access to the full dataset for data exploration, check out the guide
 [performing queries](./query-data.mdx)
 and [Python notebooks](./python-notebooks.md).
 
-## How to Generate an API key
-
-First, go to [www.opensource.observer](https://www.opensource.observer) and create a new account.
-
-If you already have an account, log in. Then create a new personal API key:
-
-1. Go to [Account settings](https://www.opensource.observer/app/settings)
-2. In the "API Keys" section, click "+ New"
-3. Give your key a label - this is just for you, usually to describe a key's purpose.
-4. You should see your brand new key. **Immediately** save this value, as you'll **never** see it again after refreshing the page.
-5. Click "Create" to save the key.
-
-**You can create as many keys as you like.**
-
-![generate API key](./generate-api-key.png)
-
 ## GraphQL Endpoint
 
 All API requests are sent to the following URL:
@@ -34,25 +18,26 @@ All API requests are sent to the following URL:
 https://www.opensource.observer/api/v1/graphql
 ```
 
+### GraphQL Explorer
+
 You can navigate to our
 [public GraphQL explorer](https://www.opensource.observer/graphql)
 to explore the schema and execute test queries.
 
-## How to Authenticate
+![GraphQL explorer](./api-explorer.gif)
 
-In order to authenticate with the API service, you have to use the `Authorization` HTTP header and `Bearer` authentication on all HTTP requests, like so:
+:::tip
+As shown in the video, you must "Inline Variables" in order for queries to run in the explorer.
+:::
 
-```js
-const headers = {
-  Authorization: `Bearer ${DEVELOPER_API_KEY}`,
-};
-```
+The GraphQL schema is automatically generated from [`oso/dbt/models/marts`](https://github.com/opensource-observer/oso/tree/main/dbt/models/marts). Any dbt model defined there will automatically be exported to our GraphQL API. See the guide on [adding DBT models](../contribute-models/data-models.md) for more information on contributing to our marts models.
 
-:::tip
-Our API opens a small rate limit for anonymous queries without authentication. Feel free to use for low volume queries.
+:::warning
+Our data pipeline is under heavy development and all table schemas are subject to change until we introduce versioning to marts models.
+Please join us on [Discord](https://www.opensource.observer/discord) to stay up to date on updates.
 :::
 
-## Example
+### Example queries
 
 This query will fetch the first 10 projects in
 [oss-directory](https://github.com/opensource-observer/oss-directory).
@@ -85,22 +70,40 @@ query GetCodeMetrics {
 }
 ```
 
-## GraphQL Explorer
+## Authentication
 
-You can navigate to our [public GraphQL explorer](https://www.opensource.observer/graphql) to explore the schema and execute test queries.
+All requests to the API must be authenticated.
+OSO is currently supported by generous grants that
+allow us to provide this as a free service for the
+community.
+We will monitor usage of models, both to ensure quality of service
+and to deprecate models that are no longer used.
 
-![GraphQL explorer](./api-explorer.gif)
+### Generate an API key
 
-:::tip
-As shown in the video, you must "Inline Variables" in order for queries to run in the explorer.
-:::
+First, go to [www.opensource.observer](https://www.opensource.observer) and create a new account.
 
-The GraphQL schema is automatically generated from [`oso/dbt/models/marts`](https://github.com/opensource-observer/oso/tree/main/dbt/models/marts). Any dbt model defined there will automatically be exported to our GraphQL API. See the guide on [adding DBT models](../contribute-models/data-models.md) for more information on contributing to our marts models.
+If you already have an account, log in. Then create a new personal API key:
 
-:::warning
-Our data pipeline is under heavy development and all table schemas are subject to change until we introduce versioning to marts models.
-Please join us on [Discord](https://www.opensource.observer/discord) to stay up to date on updates.
-:::
+1. Go to [Account settings](https://www.opensource.observer/app/settings)
+2. In the "API Keys" section, click "+ New"
+3. Give your key a label - this is just for you, usually to describe a key's purpose.
+4. You should see your brand new key. **Immediately** save this value, as you'll **never** see it again after refreshing the page.
+5. Click "Create" to save the key.
+
+**You can create as many keys as you like.**
+
+![generate API key](./generate-api-key.png)
+
+### How to Authenticate
+
+In order to authenticate with the API service, you have to use the `Authorization` HTTP header and `Bearer` authentication on all HTTP requests, like so:
+
+```js
+const headers = {
+  Authorization: `Bearer ${DEVELOPER_API_KEY}`,
+};
+```
 
 ## Rate Limits
 

apps/docs/docs/integrate/overview/index.mdx → apps/docs/docs/integrate/datasets/index.mdx

@@ -1,5 +1,5 @@
 ---
-title: Data Overview
+title: Access Public Datasets
 sidebar_position: 1
 ---
 
@@ -18,9 +18,9 @@ import ArbitrumLogo from "./arbitrum.png";
 import EasLogo from "./eas.png";
 import OcLogo from "./open-collective.png";
 
-First, you need to set up your BigQuery account. You can do this by going to the
-[Get Started](../../get-started/index.md)
-page.
+OSO shares every dataset on BigQuery Analytics Hub.
+To get started,
+[set up your BigQuery account](../../get-started/bigquery.mdx).
 
 ## OSO Data Exchange on Analytics Hub
 
@@ -87,7 +87,7 @@ in a live production application._
 
 From source data, we produce a "universal event table", currently stored at
 [`int_events`](https://models.opensource.observer/#!/model/model.opensource_observer.int_events).
-Each event consists of an [event_type](../../guides/event.md)
+Each event consists of an [event_type](../../references/event.md)
 (e.g. a git commit or contract invocation),
 [to/from artifacts](../../guides/oss-directory/artifact.md),
 a timestamp, and an amount.