Link SWOT Datum Tutorial to Cookbook

Add ingredients for PO.DAAC Cookbook to tutorial and connect in all the pieces.
podaac · Aug 22, 2024 · ca2a571 · ca2a571
1 parent aa8554c
commit ca2a571
Show file tree

Hide file tree

Showing 4 changed files with 83 additions and 39 deletions.
diff --git a/_quarto.yml b/_quarto.yml
@@ -172,6 +172,8 @@ website:
                         href: notebooks/GIS/SWOT_datetime_GIS.ipynb
                   - section: "Transform Data"
                     contents:
+                      - text: "Datum Conversion"
+                        href: quarto_text/SWOT_Datum.qmd
                       - text: "River Time Series"
                         href: notebooks/datasets/Hydrocron_SWOT_timeseries_examples.ipynb
                       - text: "Lake Time Series"

diff --git a/notebooks/datasets/SWOT_USGS_Comparison.ipynb b/notebooks/datasets/SWOT_USGS_Comparison.ipynb
@@ -1,12 +1,22 @@
 {
  "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "![](https://img.shields.io/badge/Community-Contribution-%20?color=grey&labelColor=yellow)\n",
+    "\n",
+    "> From the PO.DAAC Cookbook, to access the GitHub version of the notebook, follow [this link](https://github.com/podaac/tutorials/blob/master/notebooks/datasets/SWOT_USGS_Comparison.ipynb). "
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "# Compare SWOT water surface elevation with USGS gage heights\n",
+    "### Datum Transformation Tutorial\n",
     "\n",
-    "#### *Author: Katie McQuillan and George Allen, Virginia Tech*\n",
+    "#### *Authors: Katie McQuillan and George Allen, Virginia Tech*\n",
     "\n",
     "## Summary \n",
     "This notebook showcases how to transform the horizontal and vertical coordinates of USGS gage heights to match SWOT LakeSP water surface elevation using GDAL. \n",
@@ -20,17 +30,12 @@
     "\n",
     "- GDAL must be installed (https://gdal.org/)\n",
     "\n",
-    "### 2. Earthdata Login\n",
-    "\n",
-    "An Earthdata Login account is required to access data, as well as discover restricted data, from the NASA Earthdata system. Thus, to access NASA data, you need Earthdata Login. Please visit https://urs.earthdata.nasa.gov to register and manage your Earthdata Login account. This account is free to create and only takes a moment to set up. \n",
-    "\n",
-    "\n",
     "### Import libraries"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -53,9 +58,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "1. USGS Lake/Reservoir Water Surface Elevation dataset can be acccessed using the DataRetrieval python module with the parameter code 62615. \n",
+    "1. USGS Lake/Reservoir Water Surface Elevation dataset can be acccessed using the [DataRetrieval](https://github.com/DOI-USGS/dataretrieval-python) python module with the parameter code 62615. \n",
+    "\n",
+    "2. SWOT LakeSP dataset can be accessed using the [Earthaccess](https://earthaccess.readthedocs.io/en/latest/) python module or the [PO.DAAC Data Downloader](https://podaac.github.io/tutorials/quarto_text/DataSubscriberDownloader.html). \n",
     "\n",
-    "2. SWOT LakeSP dataset can be accessed using the EarthAccess python module or the PO.DAAC Data Downloader. \n",
+    "### Datum Transformation to Compare with SWOT Data\n",
     "\n",
     "Cotemporal SWOT LakeSP and USGS observations were used to form the dataset used for comparisons including X lakes with gages. To directly compare SWOT and USGS datasets, the USGS horizontal and vertical coordiantes must be transformed to match the SWOT datums. Datums for each dataset are noted in Table 1. \n",
     "\n",
@@ -69,6 +76,7 @@
    "metadata": {},
    "source": [
     "Table 1. SWOT and USGS datum information\n",
+    "\n",
     "| Data source | Horizontal Datum | Reference Ellipsoid | Vertical Datum | EPSG Code |\n",
     "| --- | --- | --- | --- | --- |\n",
     "| SWOT | ITRF2014 | WGS84 | EGM2008 | EPSG:9057+EPSG:3855 |\n",
@@ -113,14 +121,6 @@
       "        AUTHORITY[\"EPSG\",\"5703\"]],\n",
       "    AUTHORITY[\"EPSG\",\"6349\"]]\n"
      ]
-    },
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "c:\\Users\\kmcquil\\anaconda3\\envs\\swot-wind\\lib\\site-packages\\osgeo\\osr.py:410: FutureWarning: Neither osr.UseExceptions() nor osr.DontUseExceptions() has been explicitly called. In GDAL 4.0, exceptions will be enabled by default.\n",
-      "  warnings.warn(\n"
-     ]
     }
    ],
    "source": [
@@ -210,7 +210,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 25,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [
     {
@@ -219,24 +219,14 @@
      "text": [
       "The number of cotemporal USGS and SWOT observations = 425\n"
      ]
-    },
-    {
-     "data": {
-      "text/plain": [
-       "CompletedProcess(args='cd c:\\\\Users\\\\kmcquil\\\\Documents\\\\tutorials\\\\notebooks\\\\resources && gdaltransform -s_coord_epoch \"2010.0\" -t_coord_epoch \"2010.0\" -s_srs \"EPSG:6349\" -t_srs \"EPSG:9057+EPSG:3855\" < \"gdal_in.txt\" > \"gdal_out.txt\"', returncode=0)"
-      ]
-     },
-     "execution_count": 25,
-     "metadata": {},
-     "output_type": "execute_result"
     }
    ],
    "source": [
     "# Change working directory to tutorials folder\n",
     "os.chdir('.')\n",
     "\n",
     "# Open the dataset of cotemporal SWOT and USGS observations\n",
-    "swot_usgs_df = pd.read_csv(\"notebooks/resources/usgs_swot_merged_example.csv\", index_col=0)\n",
+    "swot_usgs_df = pd.read_csv(\"../resources/usgs_swot_merged_example.csv\", index_col=0)\n",
     "\n",
     "# How many cotemporal observations? \n",
     "print('The number of cotemporal USGS and SWOT observations = ' + str(swot_usgs_df.shape[0]))\n",
@@ -258,7 +248,7 @@
     "]\n",
     "\n",
     "# Save the combined column to a text file\n",
-    "in_gdal[\"out\"].to_csv(\"notebooks/resources/gdal_in.txt\", header=False, index=False)"
+    "in_gdal[\"out\"].to_csv(\"../resources/gdal_in.txt\", header=False, index=False)"
    ]
   },
   {
@@ -270,12 +260,24 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 7,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "CompletedProcess(args='cd /home/jovyan/tutorials/notebooks//resources && gdaltransform -s_coord_epoch \"2010.0\" -t_coord_epoch \"2010.0\" -s_srs \"EPSG:6349\" -t_srs \"EPSG:9057+EPSG:3855\" < \"gdal_in.txt\" > \"gdal_out.txt\"', returncode=0)"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
    "source": [
     "# Write the gdal command and run in the shell \n",
-    "cd_command = \"cd \" + os.getcwd() + \"\\\\notebooks\\\\resources && \"\n",
+    "os.chdir('../')\n",
+    "cd_command = \"cd \" + os.getcwd() + \"//resources && \"\n",
     "gdal_command = 'gdaltransform -s_coord_epoch \"2010.0\" -t_coord_epoch \"2010.0\" -s_srs \"EPSG:6349\" -t_srs \"EPSG:9057+EPSG:3855\" < \"gdal_in.txt\" > \"gdal_out.txt\"'\n",
     "subprocess.run(cd_command + gdal_command, shell=True)\n"
    ]
@@ -284,12 +286,12 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Calculate error statistics"
+    "#### Calculate error statistics between USGS datum-transformed data and SWOT data"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 28,
+   "execution_count": 11,
    "metadata": {},
    "outputs": [
     {
@@ -303,7 +305,7 @@
    ],
    "source": [
     "# Merge back with the original data \n",
-    "out_gdal = pd.read_csv(\"notebooks/resources/gdal_out.txt\", header=None)\n",
+    "out_gdal = pd.read_csv(\"resources/gdal_out.txt\", header=None)\n",
     "out_gdal = out_gdal.rename(columns={0: \"result\"})\n",
     "out_gdal[[\"usgs_long\", \"usgs_lat\", \"usgs_X_62615_00000_egm0_meters\"]] = out_gdal[\"result\"].str.split(\" \", expand=True)\n",
     "swot_usgs_df[\"usgs_X_62615_00000_egm0_meters\"] = out_gdal[\"usgs_X_62615_00000_egm0_meters\"].astype(float)\n",
@@ -320,7 +322,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "swot-wind",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -334,9 +336,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.18"
+   "version": "3.10.14"
   }
  },
  "nbformat": 4,
- "nbformat_minor": 2
+ "nbformat_minor": 4
 }
diff --git a/quarto_text/SWOT.qmd b/quarto_text/SWOT.qmd
@@ -1,6 +1,17 @@
 ---
 title: SWOT
 subtitle: SWOT Data Tutorials
+jupyter:
+  jupytext:
+    text_representation:
+      extension: .qmd
+      format_name: quarto
+      format_version: '1.0'
+      jupytext_version: 1.16.1
+  kernelspec:
+    display_name: Python 3 (ipykernel)
+    language: python
+    name: python3
 ---
 
 ## SWOT Background
@@ -134,6 +145,8 @@ In [Earthdata Search GUI](https://search.earthdata.nasa.gov/search):
 
 ### **Transform**
 
+### [Datum Conversion to Compare with SWOT Data](SWOT_Datum.qmd)
+
 #### [HiTIDE subsetter for Sea Surface Height Products](https://hitide.podaac.earthdatacloud.nasa.gov/) - select KaRIn instrument in `sensors`, see video tutorial [here](https://www.youtube.com/watch?v=S8MIHltlwWA)
 
 #### [Hydrocron: Time series API](../notebooks/datasets/Hydrocron_SWOT_timeseries_examples.ipynb) - Currently for rivers, see [Hydrocron documentation](https://podaac.github.io/hydrocron/intro.html) and more description under tools below. [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.11176233.svg)](https://doi.org/10.5281/zenodo.11176233)
@@ -216,3 +229,4 @@ This workshop focuses on the SWOT Hydrology datasets including river and lake ve
 [https://podaac.github.io/2022-SWOT-Ocean-Cloud-Workshop/](https://podaac.github.io/2022-SWOT-Ocean-Cloud-Workshop/){.uri}
 
 The goal of the workshop was to enable the (oceanography) science team to be ready for processing and handling the large volumes of SWOT SSH data in the cloud. Learning objectives focus on how to access the [simulated SWOT L2 SSH data](https://podaac.jpl.nasa.gov/announcements/2022-01-31-Release-simulated-SWOT-SSH-version1-datasets) from Earthdata Cloud either by downloading or accessing the data on the cloud.
+
diff --git a/quarto_text/SWOT_Datum.qmd b/quarto_text/SWOT_Datum.qmd
@@ -0,0 +1,26 @@
+---
+title: Datum Conversion for SWOT Data
+subtitle: Community Tutorials
+jupyter:
+  jupytext:
+    text_representation:
+      extension: .qmd
+      format_name: quarto
+      format_version: '1.0'
+      jupytext_version: 1.16.1
+  kernelspec:
+    display_name: Python 3 (ipykernel)
+    language: python
+    name: python3
+---
+
+# Datum Conversion for SWOT Data
+
+Please first see the [SWOT Data User Handbook](https://archive.podaac.earthdata.nasa.gov/podaac-ops-cumulus-docs/web-misc/swot_mission_docs/D-109532_SWOT_UserHandbook_20240502.pdf?_ga=2.76536725.1816513343.1724259518-1354658737.1715875596) sections 3.3.3 on the reference ellipsoid and 11.3.1 on the geoid. In addition, section 3.1.23 gives detailed information about Height with section 3.1.24 elaborating on Sea Surface Height Anomalies and 3.1.25 elaborating on Water Surface Elevation over inland waters.
+
+As a summary, SWOT data is referenced to the WGS84 reference ellipsoid as a first step, and in a second step referenced to the EGM2008 geoid that references WGS84. To get a direct conversion with another dataset, it is easiest to transform all local data to the EGM2008 geoid. We recognize this conversion is different globally and depending on the data being used to compare, but we have collected a few community examples of direct comparisons to share below:
+
+1. Comparing USGS gage data and SWOT data in the United States - [Python Notebook](../notebooks/datasets/SWOT_USGS_Comparison.ipynb)
+2. Converting the SWOT Frame and Geoid in Canada - [Google Colab Section from Canadian Colleagues](https://colab.research.google.com/drive/1we0a24gBKbs-CFMVBfRIUm4SSW8RdwmA#scrollTo=cu09EYYRxxd4)
+
+If you have any questions for your specific region, please ask in the [Earthdata Forum](https://forum.earthdata.nasa.gov/viewforum.php?f=7) and experts should be able to help.