0.1.x (#69)

* Slight update to __init__.py with version number plus test of new versioning branch (#58) * Update pyproject.toml * Update __init__.py * Update update.txt * Small changes from dev-kja (#59) * Update pyproject.toml * Update __init__.py * new testing notebook, new notebook directory, update __init__.py again 1. Created a notebooks folder to hold example and testing notebooks 2. Create a notebook to test on West Walker River (the entirely floating chronology with some weird series codes) 3. Removed the os.chdir in __init__.py so it doesn't leave the working directory upon importing dplPy * various 1. removed line in xdate.py that was printing all the ranges - OK for debugging but not necessary for operational use? 2. removed update.txt 3. continued to work on WWR notebook examples * Update floating_chronology_example.ipynb * Updates to README to reflect current stage of project including 2 new assets (#60) * Update pyproject.toml * Update __init__.py * new testing notebook, new notebook directory, update __init__.py again 1. Created a notebooks folder to hold example and testing notebooks 2. Create a notebook to test on West Walker River (the entirely floating chronology with some weird series codes) 3. Removed the os.chdir in __init__.py so it doesn't leave the working directory upon importing dplPy * various 1. removed line in xdate.py that was printing all the ranges - OK for debugging but not necessary for operational use? 2. removed update.txt 3. continued to work on WWR notebook examples * Update floating_chronology_example.ipynb * update readme and assets * README update (#61) * Update pyproject.toml * Update __init__.py * new testing notebook, new notebook directory, update __init__.py again 1. Created a notebooks folder to hold example and testing notebooks 2. Create a notebook to test on West Walker River (the entirely floating chronology with some weird series codes) 3. Removed the os.chdir in __init__.py so it doesn't leave the working directory upon importing dplPy * various 1. removed line in xdate.py that was printing all the ranges - OK for debugging but not necessary for operational use? 2. removed update.txt 3. continued to work on WWR notebook examples * Update floating_chronology_example.ipynb * update readme and assets * Update README.md * primarily a change to detrend plus some organization and clean up (#62) * Update pyproject.toml * Update __init__.py * new testing notebook, new notebook directory, update __init__.py again 1. Created a notebooks folder to hold example and testing notebooks 2. Create a notebook to test on West Walker River (the entirely floating chronology with some weird series codes) 3. Removed the os.chdir in __init__.py so it doesn't leave the working directory upon importing dplPy * various 1. removed line in xdate.py that was printing all the ranges - OK for debugging but not necessary for operational use? 2. removed update.txt 3. continued to work on WWR notebook examples * Update floating_chronology_example.ipynb * update readme and assets * Update README.md * modify detrend to pass correct period to spline plus some organization * Update README.md * Update README.md (#63) * Dev ife (#64) * xdate works for overall series correlation * Added code for creating bins and dividing series into segments * Cleaning up and commenting related to xdate * series_corr works but is inefficient * WIP changes * Added comments, updated working jupyter notebook * Changes since start of fall semester * variance stabiliization produces accurate values * Unit tests for readers, summary, stats and tbrm * Added unit tests for detrend and chron * Added tests for chron_stabilized, series_corr and writers * Fixed merge conflicts * v0.1 release * Fixing the rwl reader and writer problem * Create pypi_release.yml * push for workflow (#54) * Update pypi_release.yml * Update pyproject.toml * xdate works for overall series correlation * series_corr works but is inefficient * WIP changes * Added comments, updated working jupyter notebook * Changes since start of fall semester * variance stabiliization produces accurate values * Unit tests for readers, summary, stats and tbrm * Added tests for chron_stabilized, series_corr and writers * v0.1 release * Attempt at fixing dependency issues * Create pypi_release.yml * Update pypi_release.yml * Modification to readers plus small changes to dplpy and pyproject (#55) * mostly change to readers for RWL files 1. change readers to handle series IDs of different lengths and the potential for negative (B.C.) years 2. removed directory change in __init__ so that default is path where the notebook or script is * Update __init__.py * Update readers.py --------- Co-authored-by: Michele Cosi <[email protected]> * Revert "Modification to readers plus small changes to dplpy and pyproject (#55)" This reverts commit 68b9aa1. * Update pyproject.toml (#57) * Update update.txt * Github workflow for unit testing * Added github workflow to run integration tests * Fixed issue that caused integs to fail in previous commit * Changed write function in writers.py to writers * Fixed deprecation warning for \d regex expression --------- Co-authored-by: Ifeoluwa Ale <[email protected]> Co-authored-by: cosimichele <[email protected]> Co-authored-by: Ifeoluwa Ale <[email protected]> Co-authored-by: Michele Cosi <[email protected]> Co-authored-by: Kevin Anchukaitis <[email protected]> * updated .py modules with additional help descriptions, updated copywrite year * updated help docs to match numpy documentation format * updated dplpy.readme() * Dev ife (#65) * xdate works for overall series correlation * Added code for creating bins and dividing series into segments * Cleaning up and commenting related to xdate * series_corr works but is inefficient * WIP changes * Added comments, updated working jupyter notebook * Changes since start of fall semester * variance stabiliization produces accurate values * Unit tests for readers, summary, stats and tbrm * Added unit tests for detrend and chron * Added tests for chron_stabilized, series_corr and writers * Fixed merge conflicts * v0.1 release * Fixing the rwl reader and writer problem * Create pypi_release.yml * push for workflow (#54) * Update pypi_release.yml * Update pyproject.toml * xdate works for overall series correlation * series_corr works but is inefficient * WIP changes * Added comments, updated working jupyter notebook * Changes since start of fall semester * variance stabiliization produces accurate values * Unit tests for readers, summary, stats and tbrm * Added tests for chron_stabilized, series_corr and writers * v0.1 release * Attempt at fixing dependency issues * Create pypi_release.yml * Update pypi_release.yml * Modification to readers plus small changes to dplpy and pyproject (#55) * mostly change to readers for RWL files 1. change readers to handle series IDs of different lengths and the potential for negative (B.C.) years 2. removed directory change in __init__ so that default is path where the notebook or script is * Update __init__.py * Update readers.py --------- Co-authored-by: Michele Cosi <[email protected]> * Revert "Modification to readers plus small changes to dplpy and pyproject (#55)" This reverts commit 68b9aa1. * Update pyproject.toml (#57) * Update update.txt * Github workflow for unit testing * Added github workflow to run integration tests * Fixed issue that caused integs to fail in previous commit * Changed write function in writers.py to writers * Fixed deprecation warning for \d regex expression * Fix for PerformanceWarning due to fragmented dataframe * Fix for xdate_plot bugs --------- Co-authored-by: Ifeoluwa Ale <[email protected]> Co-authored-by: cosimichele <[email protected]> Co-authored-by: Ifeoluwa Ale <[email protected]> Co-authored-by: Michele Cosi <[email protected]> Co-authored-by: Kevin Anchukaitis <[email protected]> * Updated developer instructions (#67) * Updates for release of 0.1.2 (#68) * Updated developer instructions * Changes for release of 0.1.2 --------- Co-authored-by: Kevin Anchukaitis <[email protected]> Co-authored-by: Michele Cosi <[email protected]> Co-authored-by: Ifeoluwa Ale <[email protected]> Co-authored-by: cosimichele <[email protected]> Co-authored-by: Ifeoluwa Ale <[email protected]> Co-authored-by: Tyson Lee Swetnam <[email protected]>
OpenDendro · Jan 4, 2024 · b25f55d · b25f55d
1 parent 84197e3
commit b25f55d
Show file tree

Hide file tree

Showing 45 changed files with 9,208 additions and 282 deletions.
diff --git a/.github/workflows/pypi_release.yml b/.github/workflows/pypi_release.yml
@@ -1,11 +1,12 @@
 # Build the package and publish it to PyPI after tests pass.
 name: Publish to PyPI
 on:
-  push:
+  workflow_run:
+    workflows: ["run_tests"]
+    types:
+      - completed
     branches:
-      - main
-    tags:
-      - "*"
+      - 0.1.2
 
 jobs:
   publish:

diff --git a/.github/workflows/run_tests.yml b/.github/workflows/run_tests.yml
@@ -0,0 +1,31 @@
+name: Python package
+
+on: [push]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"] # Goal is to run tests for different python versions
+
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Display Python version
+        run: python -c "import sys; print(sys.version)"
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r ${{ matrix.python-version }}-requirements.txt
+          pip install pytest pytest-cov
+      - name: Run unit tests
+        run: |
+          pytest tests/unit --cov=dplpy
+      - name: Run integ tests
+        run: |
+          pytest tests/integs -rA
diff --git a/.gitignore b/.gitignore
@@ -136,3 +136,4 @@ dmypy.json
 # GitHub Copilot
 "GitHub Copilot"
 tests/data/.DS_Store
+notebooks/spline_testing.ipynb
diff --git a/3.10-requirements.txt b/3.10-requirements.txt
@@ -0,0 +1,6 @@
+numpy==1.22.4
+statsmodels==0.13.5
+matplotlib==3.8.0
+csaps==1.1.0
+pandas==2.0.0
+scipy==1.11.3
diff --git a/3.11-requirements.txt b/3.11-requirements.txt
@@ -0,0 +1,6 @@
+numpy==1.23.2
+statsmodels==0.13.5
+matplotlib==3.8.0
+csaps==1.1.0
+pandas==2.0.0
+scipy==1.11.3
diff --git a/3.12-requirements.txt b/3.12-requirements.txt
@@ -0,0 +1,6 @@
+numpy==1.26.0
+statsmodels==0.14.0
+matplotlib==3.8.0
+csaps==1.1.0
+pandas==2.0.0
+scipy==1.11.3
diff --git a/README.md b/README.md
@@ -1,49 +1,79 @@
-# dplPy
-The Dendrochronology Program Library for Python
+ <p align="center">
+ <img src="docs/assets/dplpy.png" width="175"> 
+
+# dplPy -the Dendrochronology Program Library in Python
+The Dendrochronology Program Library (DPL) in Python has its roots in both the [original FORTRAN program](https://www.ltrr.arizona.edu/software.html) created by the [legendary Richard Holmes](https://arizona.aws.openrepository.com/handle/10150/262569?show=full) and the subsequent R Project package by Andy Bunn, [dplR](https://github.com/OpenDendro/dplR).  Our aim is to provide researchers working with tree-ring data the necessary tools in open-source environments, promoting open science practices, enhancing rigor and transparency in dendrochronology, and eventually allowing reproducible research entirely in a single programming language.
+
+ The development of dplPy is supported by a grant from the Paleoclimate program of the US National Science Foundation (AGS-2054516) to Andy Bunn, Kevin Anchukaitis, Ed Cook, and Tyson Swetnam.
+<br>
+
 
 ---
 
 
 ## Index
 
-- [Issues](#issues)
-- [Requirements](#requirements)
-- [Building Environment](#building-environment)
-- [Using jupyter](#using-jupyter)
-    - [Linux, macOS](#linux-macos)
+- [dplPy - the Dendrochronology Program Library in Python](#dplpy---the-dendrochronology-program-library-in-python)
+  - [Index](#index)
+  - [Requirements](#requirements)
+  - [Current Version and Changelog](#current-version-and-changelog)
+  - [Installation](#installation)
+  - [Building directly from Github](#building-directly-from-github)
+  - [Using VSCode in your operating system](#using-vscode-in-your-operating-system)
+    - [Linux or MacOS](#linux-or-macos)
     - [Windows](#windows)
-- [Functionalities and usage](#functionalities-and-usage)
-    - [Loading data](#loading-data)
-    - [Data Summary](#data-summary)
-    - [Data Stastics](#data-stastics)
-    - [Data Report](#data-report)
+  - [Functionalities and Usage](#functionalities-and-usage)
+    - [Loading data using  `readers`](#loading-data-using--readers)
+    - [Data Summary from `summary`](#data-summary-from-summary)
+    - [Data Stastics from `stats`](#data-stastics-from-stats)
+    - [Data Report from `report`](#data-report-from-report)
     - [Plotting](#plotting)
+    - [Detrending using `detrend`](#detrending-using-detrend)
     - [Autoregressive (AR) modeling](#autoregressive-ar-modeling)
-    - [Detrending](#detrending)
-    - [Chronology](#chronology)
-
----
-
-## Issues
-
-We're using [ZenHub](https://app.zenhub.com/workspaces/opendendro-60ec698d8790d700171ceee8/board?repos=385244315) to manage our [GitHub Issues](https://github.com/opendendro/dplpy/issues)
+    - [Build a chronology with `chron`](#build-a-chronology-with-chron)
+    - [Crossdate with `xdate`](#crossdate-with-xdate)
 
 ---
 
 ## Requirements
 
-:warning: **Note**: DplPy has been successfully tested on Ubuntu 20, Ubuntu 22, macOS (Intel, M2).
-
+- Python (>=3.10)
 - Conda ([Anaconda](https://docs.anaconda.com/anaconda/install/index.html) or [Miniconda](https://docs.conda.io/projects/continuumio-conda/en/latest/user-guide/install/index.html))
 - (Suggested) [Mamba](https://mamba.readthedocs.io/en/latest/installation.html)
 - (Suggested) [VSCode](https://code.visualstudio.com/)
 
-## Building Environment
+Under the hood, dplPy uses `numpy`, `pandas`, `matplotlib`, `statsmodels`, `scipy`, and `csaps`.
+
+:warning: dplPy has been successfully tested thus far on Ubuntu 20, Ubuntu 22, macOS (Intel and M2). Other operating systems may experience unexpected errors or conflicts.  Please let the developers know. 
+
+## Current Version and Changelog
+
+dplPy is currently at version `v0.1.1` - Changes and new functions are currently merged into the `0.1.x` branch.
+
+## Installation
+
+dplPy is now available to [install via pip](https://pypi.org/project/dplpy/):
+
+```
+pip install dplpy
+```
+
+You can install a conda virtual environment using the [environment.yml for the project](https://github.com/OpenDendro/dplPy/blob/main/environment.yml):
+
+```
+$ conda env create -f environment.yml     
+```
+
+---
 
-> :warning: **it is recommended to _NOT_ use GitHub Codespaces (as of Mar 2022)**
+
+## Building directly from Github
+
+You can still still install dplPy firectly from Github if you wish:
 
 1\. Clone and change directory to this repository
 
+
 ```
 $ git clone https://github.com/OpenDendro/dplPy.git
 $ cd dplPy
@@ -69,36 +99,25 @@ $ conda activate dplpy
 
 Your environment should be successfully built.
 
-4\. Your python environment should be able to import `numpy`, `pandas`, `matplotlib`, `statsmodels` and `csaps`:
-
-![env_3](docs/assets/env_3.png)
+4\. Your python environment should be able to import `numpy`, `pandas`, `matplotlib`, `statsmodels` and `csaps`.
 
 ---
 
-## Using Jupyter
+## Using VSCode in your operating system
 
-The Conda enviroment is essential as it provides will all necessary packages. To execute the code, use Jupyter Notebook.
-
-:warning: **Note**: if using Jupyter from the terminal, you need to ensure that the kernel is findable by doing the following command once the environment is active:
-
-```
-python -m ipykernel install --user --name dplpy --display-name "Python (dplpy)"
-```
-
-### Linux, MacOS
+### Linux or MacOS
 
 1\. In your VSCode terminal, activate the conda environment with `conda activate dplpy3`.
-2\. Open a Jupyer Notebook (`<file>.ipynb`) and select the `dplpy3` Kernel when prompted (or from the top right of your screen).
-This will automatically load the environment we created.
+
+2\. Open a Jupyer Notebook (`<file>.ipynb`) and select the `dplpy3` Kernel when prompted (or from the top right of your screen). This will automatically load the environment we created.
 
 ### Windows
 
 In VSCode:
 
 1\. In your VSCode terminal window, activate the conda environment with `conda activate dplpy3`.
-2\. In the same terminal window, start a Jupyter Notebook with `jupyter notebook`. Jupyter will then return URLs that you can copy; *Copy* one of these URLs. 
 
-![ipynb_env1](docs/assets/ipynb_env1.jpg)
+2\. In the same terminal window, start a Jupyter Notebook with `jupyter notebook`. Jupyter will then return URLs that you can copy; *Copy* one of these URLs. 
 
 3\. Open a Jupyter Notebook (`<file>.ipynb`) and from the **bottom right** of the VSCode screen, click **Jupyter Server**;
 
@@ -114,15 +133,22 @@ A dropdown menu will open from the top of the screen: select Existing and *paste
 
 ## Functionalities and Usage
 
-Import the DplPy tool with
+Import the dplPy tool with
 
 ```
 import dplpy as dpl
 ```
 
-This will load the necessary functions.
+or alternatively:
+
+```
+import dplpy 
+```
+
+This will load the package and its functions.
+
 
-### Loading data
+### Loading data using  `readers`
 
 - Description: reads data from supported file types (`csv` and `rwl`) and stores them in a dataframe.
 - Options: 
@@ -132,7 +158,7 @@ This will load the necessary functions.
     >>> data = dpl.readers("/path/to/file.rwl", header=True)
     ```
 
-### Data Summary
+### Data Summary from `summary`
 
 - Description: generates a summary of each series recorded in `rwl`  and `csv` format files
 - Usage Example:
@@ -142,7 +168,7 @@ This will load the necessary functions.
     >>> dpl.summary(data)
     ```
 
-### Data Stastics
+### Data Stastics from `stats`
 
 - Description: generates summary statistics for `rwl`  and `csv` format files
 - Usage Example:
@@ -152,7 +178,7 @@ This will load the necessary functions.
     >>> dpl.stats(data)
     ```
 
-### Data Report
+### Data Report from `report`
 
 - Description: generates a report about absent rings in the data set
 - Usage Example:
@@ -181,23 +207,8 @@ This will load the necessary functions.
     >>> dpl.plot(data[[SERIES_1, SERIES_2, SERIES_3]], type="spag")
     ```
 
-### Autoregressive (AR) modeling 
-
-- Description: ontains methods that fit series to autoregressive models and perform functions related to AR modeling.
-- Functions:
-    - `autoreg(data['Name of series'], max_lag)`: returns parameters of best fit AR model with maxlag of 5 (default) or other specified number
-    - `ar_func(data['Name of series'], max_lag)`: returns residuals plus mean of best fit from AR models with max lag of either 5 (default) or specified number
-- Options:
-    - `max_lag`: default 5, can be specified to user's needs.
-- Usage Example:
-    ```
-    >>> dpl.autoreg(data[SERIES_1])
-    # or
-    >>> dpl.ar_func(data[SERIES_2], max_lag=7)
-    ```
-
-### Detrending
-
+### Detrending using `detrend`
+
 - Description: Detrends a given series or data frame, first by fitting data to curve(s), and then by calculating residuals or differences compared to the original data.
 - Options:
     - `fit="spline"`: default detrending method.
@@ -218,7 +229,23 @@ This will load the necessary functions.
     >>> dpl.detrend(data[[SERIES_1, SERIES_2, SERIES_3]], fit="Hugershoff", method="difference")
     ```
 
-### Chronology
+
+### Autoregressive (AR) modeling 
+
+- Description: ontains methods that fit series to autoregressive models and perform functions related to AR modeling.
+- Functions:
+    - `autoreg(data['Name of series'], max_lag)`: returns parameters of best fit AR model with maxlag of 5 (default) or other specified number
+    - `ar_func(data['Name of series'], max_lag)`: returns residuals plus mean of best fit from AR models with max lag of either 5 (default) or specified number
+- Options:
+    - `max_lag`: default 5, can be specified to user's needs.
+- Usage Example:
+    ```
+    >>> dpl.autoreg(data[SERIES_1])
+    # or
+    >>> dpl.ar_func(data[SERIES_2], max_lag=7)
+    ```
+
+### Build a chronology with `chron`
 
 - Description: creates a mean value chronology for a dataset, typically the ring width indices of a detrended series. **Note: input data has to be detrended first.**
 - Options:
@@ -233,3 +260,6 @@ This will load the necessary functions.
     # Perform chronology
     >>> dpl.chron(rwi_data, biweight=False, plot=False)
     ```
+### Crossdate with `xdate`
+- Description: evaluate the dating accuracy of a set of tree-ring measurements
+- Options: