diff --git a/_index.ee.md b/_index.ee.md deleted file mode 100644 index 368965a..0000000 --- a/_index.ee.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -title: "Overcoming Bottlenecks in RDS & Enhancing Development Workflows with Neon " # MODIFY THIS TO BE THE TITLE OF YOUR WORKSHOP -chapter: true -weight: 1 ---- - -# AWS Workshop: Overcoming RDS Bottlenecks & Enhancing Development Workflows with Neon -
-![Partner Logo](/images/AWS_Logo.png) -
- -## Welcome to Neon! - -In this workshop you will learn how to boost your development speed with Neon and AWS RDS - -- We'll show you how to speed up your development process by integrating **Neon** with your existing **AWS RDS Postgres** setup. If you don't have an existing RDS Postgres setup, we'll walk you through how to set one up and migrate your staging data to Neon. -- While RDS is great for production, it has limitations when it comes to development. Neon, a serverless Postgres solution with features like **database branching**, provides a more flexible and developer-friendly environment for testing and development. With Neon, you can improve your workflows and collaboration without making disruptive changes to your production database. - - - -
-
- -{{% notice warning %}} -The examples and sample code provided in this workshop are intended to be consumed as instructional content. These will help you understand how various AWS services can be architected to build a solution while demonstrating best practices along the way. These examples are not intended for use in production environments. -{{% /notice %}} \ No newline at end of file diff --git a/content/1_ModuleOne/12_CohostedWorkshopPrerequisites.md b/content/1_ModuleOne/12_CohostedWorkshopPrerequisites.md index d069648..cd7b842 100644 --- a/content/1_ModuleOne/12_CohostedWorkshopPrerequisites.md +++ b/content/1_ModuleOne/12_CohostedWorkshopPrerequisites.md @@ -47,9 +47,9 @@ Neon is built on an innovative [branch-based architecture](https://neon.tech/bl - Once everything is set up, you’ll be able to create additional branches from the main branch to [duplicate your development environment in a second](https://neon.tech/blog/how-to-copy-large-postgres-databases-in-seconds), without additional storage costs, as many times as you need. - Every engineer on your team can have their own dev branch, facilitating parallel development without adding overheads in database management or costs. -
-🚨 Once you’ve set up your Neon account and your project, **create a database in the main branch, and make a note of the connection string.** You will need this in the next step. -
+{{% notice warning %}} +🚨 Once you’ve set up your Neon account and your project, **create a database in the main branch, and make a note of the connection string.** You will need this in the next steps. +{{% /notice %}} #### Select Your Project -> Connection Details -> Connection String @@ -63,11 +63,9 @@ Neon is built on an innovative [branch-based architecture](https://neon.tech/bl ### 3. GitHub repository access to Actions and Secrets -- **Directions for preparing the GitHub Repository**
- - **Create a GitHub repository** if you don't have one already, as this will store the automation scripts and workflows for the synchronization.
- [GitHub Directions For Creating a Repository Here] +First you will need to **make sure you have a GitHub Account**, if you do not have one already, you can sign up for one [here.](https://github.com/signup)
- - **Add necessary secrets** to GitHub Secrets: You’ll need the connection URLs for both your RDS production database and your Neon database (these can be added to GitHub Secrets under `PROD_DATABASE_URL` and `DEV_DATABASE_URL`). + **Create a GitHub repository** if you don't have one already, as this will store the automation scripts and workflows for the synchronization.
- [GitHub Directions For Adding Secrets] \ No newline at end of file +- **Add necessary secrets** to GitHub Secrets: You’ll need the connection URLs for both your RDS production database and your Neon database (these can be added to GitHub Secrets under `PROD_DATABASE_URL` and `DEV_DATABASE_URL`). Directions on how to add secrets to GitHub can be found [here.](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions?tool=webui) diff --git a/content/2_ModuleTwo/21_CohostedNeonTwinDeploySetUp.md b/content/2_ModuleTwo/21_CohostedNeonTwinDeploySetUp.md index 7039e5a..3f7885a 100644 --- a/content/2_ModuleTwo/21_CohostedNeonTwinDeploySetUp.md +++ b/content/2_ModuleTwo/21_CohostedNeonTwinDeploySetUp.md @@ -11,8 +11,96 @@ weight: 1 # MODIFY THIS VALUE TO REFLECT THE ORDERING OF THE MODULES If you're already running RDS in production, migrating a live database can be a real challenge. The good news is, you don’t have to move everything over to enjoy Neon's advantages in development speed and cost efficiency for non-production databases. +![Neon Twin](/static/images/neontwin.jpg) + With a Neon Twin—a synchronized copy of your RDS data in Neon—there’s a simple, automated way to keep your development environment up-to-date. This copy updates automatically each night using ```pg_dump/restore ``` and ```GitHub Actions```, so your development data stays fresh without any heavy lifting. -## Submodule One Heading + + +## Create a Neon Twin in your GitHub Actions workflow: +### 1. Create a workflow YAML file: +Add a new workflow file in your GitHub repository at ```.github/workflows/create-neon-twin.yml.``` This file will define how the synchronization happens. + + +```yaml +name: Create Neon Twin + +on: + schedule: + - cron: '0 0 * * *' # Runs every day at midnight + workflow_dispatch: # Allows manual trigger + +env: + PROD_DATABASE_URL: ${{ secrets.PROD_DATABASE_URL }} # RDS connection string + DEV_DATABASE_URL: ${{ secrets.DEV_DATABASE_URL }} # Neon connection string + PG_VERSION: '16' + +jobs: + dump-and-restore: + runs-on: ubuntu-latest + + steps: + - name: Install PostgreSQL + run: | + sudo apt update + yes '' | sudo /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh + sudo apt install -y postgresql-${{ env.PG_VERSION }} + + - name: Dump from RDS and Restore to Neon + run: | + /usr/lib/postgresql/${{ env.PG_VERSION }}/bin/pg_dump "${{ env.PROD_DATABASE_URL }}" -Fc -f "${{ github.workspace }}/prod-dump-file.dump" + /usr/lib/postgresql/${{ env.PG_VERSION }}/bin/pg_restore -d "${{ env.DEV_DATABASE_URL }}" --clean --no-owner --no-acl --if-exists "${{ github.workspace }}/prod-dump-file.dump" + +``` + +### 2. Understanding the Workflow Steps + +#### Workflow Components + +| Component | Description | +|-----------|-------------| +| **Cron** | Runs nightly at midnight (Eastern Time) using POSIX cron syntax | +| **workflow_dispatch** | Enables manual workflow triggering during development | + +#### Environment Variables + +| Variable | Description | +|----------|-------------| +| **PROD_DATABASE_URL** | Connection string for production database (stored in GitHub Secrets) | +| **DEV_DATABASE_URL** | Connection string for Neon Twin (stored in GitHub Secrets) | +| **PG_VERSION** | Specifies the Postgres version to install | + +#### Database Operations + +| Command | Flag | Description | +|---------|------|-------------| +| **pg_dump** | `-Fc` | Uses custom format for dump file | +| | `-f` | Specifies dump file path and name | +| **pg_restore** | `--clean` | Removes existing objects before restore | +| | `--if-exists` | Prevents errors for non-existent objects | +| | `--no-owner` | Skips original ownership restoration | +| | `--no-acl` | Skips original permissions restoration | +| | `-d` | Specifies target database using DEV_DATABASE_URL | + +> **Note**: Environment variables mentioned above must be configured in your GitHub repository's secrets for the Action to work properly. + + + +### 2. Configuring secrets in GitHub Actions + +In the workshop prerequisites module you will have created a GitHub repository and added secrets to the repository. + +- In your GitHub Actions workflow (e.g., in `.github/workflows/create-neon-twin.yml`), reference these secrets like this: + + ```yaml + yaml + Copy code + env: + PROD_DATABASE_URL: ${{ secrets.PROD_DATABASE_URL }} + DEV_DATABASE_URL: ${{ secrets.DEV_DATABASE_URL }} + + ``` +This syntax ensures that the sensitive values, such as database credentials or API keys, are securely injected into your workflow at runtime without hardcoding them into the file. + diff --git a/content/2_ModuleTwo/_index.md b/content/2_ModuleTwo/_index.md index a42b180..145eefd 100644 --- a/content/2_ModuleTwo/_index.md +++ b/content/2_ModuleTwo/_index.md @@ -1,11 +1,11 @@ --- -title: " Module One " # MODIFY THIS TITLE IF APPLICABLE +title: " Set Up Instructions " # MODIFY THIS TITLE IF APPLICABLE chapter: true weight: 2 --- -# Addressing RDS Development Challenges and Unlocking Neon's Potential -## Overcoming Bottlenecks & Enhancing Workflows with Neon +!-- MODIFY THIS HEADING --> +# Overcoming Bottlenecks & Enhancing Dev/Test Workflows with Neon AWS RDS for PostgreSQL is a solid, established solution that has been instrumental in bringing Postgres into modern workplaces. While Neon can handle mission-critical workloads, its real strength lies in optimizing the developer workflow so your developers can build faster and optimize costs. ## RDS in Development: Overview of AWS RDS Bottlenecks @@ -31,4 +31,4 @@ Neon’s key features and benefits, focusing on how it improves development and - **Instant Environment Provisioning:** Learn how Neon enables the quick setup of isolated environments for each team member, speeding up development and reducing bottlenecks. - **Database Branching:** Explore how Neon’s database branching feature provides isolated, production-like environments for safe experimentation and testing. - **CI/CD Integration with GitHub Actions:** Understand how Neon integrates seamlessly with CI/CD workflows, allowing for efficient automation and testing pipelines. -- Discover how Neon’s scale-to-zero functionality helps save on infrastructure costs by automatically scaling down environments when not in use. \ No newline at end of file +- **Scale-to-zero:** Discover how Neon’s scale-to-zero functionality helps save on infrastructure costs by automatically scaling down environments when not in use. diff --git a/content/2_ModuleTwo/image-1.png b/content/2_ModuleTwo/image-1.png new file mode 100644 index 0000000..57449d7 Binary files /dev/null and b/content/2_ModuleTwo/image-1.png differ diff --git a/content/2_ModuleTwo/image.png b/content/2_ModuleTwo/image.png new file mode 100644 index 0000000..57449d7 Binary files /dev/null and b/content/2_ModuleTwo/image.png differ diff --git a/static/images/neontwin.jpg b/static/images/neontwin.jpg new file mode 100644 index 0000000..60344f0 Binary files /dev/null and b/static/images/neontwin.jpg differ