Skip to content

Commit

Permalink
Vector data catalog (#36)
Browse files Browse the repository at this point in the history
* fix reading the the vector catalog

* fix reading the the vector catalog

* update vector catalog

* refactor

* refactor

* ignore ppt files

* ignore era5 precipitation file

* update checklist files

* update checklist files
  • Loading branch information
MAfarrag authored Feb 5, 2023
1 parent a5afa58 commit 28bd2ef
Show file tree
Hide file tree
Showing 16 changed files with 361 additions and 304 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -155,3 +155,6 @@ examples/Copy of RETRIEVE-ALL-METADATA-WITH-ACCOUNT-INFO.ipynb
tests/data/asci_write_test.asc
tests/data/poligonized.geojson
tests/mo/*
examples/coords.csv
*.pptx
examples/monthly_precipitation_amount_1hour_Accumulation_2022.05.01.00.00.00.tif
151 changes: 90 additions & 61 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,85 +1,114 @@
fail_fast: true
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: end-of-file-fixer
name: "[py - check] validate yaml"
- id: trailing-whitespace
name: "[file - format] trim trailing whitespace"
args: [ --markdown-linebreak-ext=md ]
- id: check-added-large-files
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.1.0
hooks:
- id: end-of-file-fixer
name: "[py - check] validate yaml"
- id: trailing-whitespace
name: "[file - format] trim trailing whitespace"
args: [ --markdown-linebreak-ext=md ]
- id: check-added-large-files
name: "[file - check] large file"
args: [ --maxkb=5000 ]
- id: check-docstring-first
- id: check-docstring-first
name: "[py - check] docstring first"
files: /examples
types : [file, python ]
- id: check-json
- id: check-json
name: "[json - check] validate json"
- id: check-merge-conflict
- id: check-merge-conflict
name: "[git - check] merge conflict"
- id: debug-statements
- id: debug-statements
name: "[py - check] debug statements"
- id: detect-private-key
- id: detect-private-key
name: "[cred - check] private keys"
- id: fix-encoding-pragma
- id: fix-encoding-pragma
name: "[file - format] coding pragma"
args: [ --remove ]
- id: mixed-line-ending
- id: mixed-line-ending
name: "[file - format] mixed line ending"
args: [ --fix=auto ]
- id: pretty-format-json
- id: pretty-format-json
name: "[json - format] pretty json"
args: [ --autofix,
--indent=4,
--no-sort-keys ]
- id: requirements-txt-fixer
- id: requirements-txt-fixer
name: "[reqs - format] fix requirements.txt"
- id: check-yaml
- id: check-yaml
name: "[yaml - check] validate yaml"
- repo: https://github.com/PyCQA/docformatter
rev: v1.4
hooks:
- id: docformatter
name: "[py - format] docformatter"
args: [ -i, --wrap-summaries, "0" ]
# - repo: https://github.com/pre-commit/mirrors-isort
# rev: v5.10.1
# hooks:
# - id: isort
# name: "[py - format] isort"
- repo: https://github.com/PyCQA/docformatter
rev: v1.4
hooks:
- id: docformatter
name: "[py - format] docformatter"
args: [ -i, --wrap-summaries, "0" ]

- repo: https://github.com/PyCQA/pydocstyle
rev: 6.1.1
hooks:
- id: pydocstyle
name: "[py - check] pydocstyle"
files: ^Hapi/
- repo: https://github.com/PyCQA/pydocstyle
rev: 6.1.1
hooks:
- id: pydocstyle
name: "[py - check] pydocstyle"
files: ^Hapi/

- repo: https://gitlab.com/pycqa/flake8
rev: 3.8.4
hooks:
- id: flake8
name: "[py - check] flake8"
language_version: python3.9
exclude: ^(examples/|tests/)
- repo: https://gitlab.com/pycqa/flake8
rev: 4.0.1
hooks:
- id: flake8
name: "[py - check] flake8"
language_version: python3.9
exclude: ^(examples/|tests/)

- repo: https://github.com/psf/black
rev: 22.8.0
hooks:
- id: black
- repo: https://github.com/pre-commit/mirrors-isort
rev: v5.7.0
hooks:
- id: isort
name: "[py - format] isort"
- repo: https://github.com/ambv/black
rev: 22.8.0
hooks:
- id: black
name: "[py - format] black"
language_version: python3.9
- repo: https://github.com/psf/black
rev: 22.8.0
hooks:
- id: black
# - repo: https://github.com/ambv/black
# rev: 22.8.0
# hooks:
# - id: black
# name: "[py - format] black"
# language_version: python3.9
- repo: https://github.com/lovesegfault/beautysh
rev: v6.2.1
hooks:
- id: beautysh
name: "[bash - format] beautysh"

- repo: local
hooks:
- id: pytest-check
name: pytest-check
entry: pytest
language: system
pass_filenames: false
always_run: true
# pre-commit-shell: Checks shell scripts against shellcheck.
- repo: https://github.com/detailyang/pre-commit-shell
rev: v1.0.6
hooks:
- id: shell-lint
name: "[bash - lint] shell-lint"

- repo: https://github.com/rlindsgaard/pre-commit-commit-msg-hooks
rev: 0.1.0
hooks:
- id: check-description-max-length
name: "[bash - format] check-description-max-length"
- id: check-second-line-empty
name: "[bash - format] check-second-line-empty"
- id: check-summary-capitalized
name: "[bash - format] check-summary-capitalized"
- id: check-summary-imperative
name: "[bash - format] check-summary-imperative"
- id: check-summary-max-length
name: "[bash - format] check-summary-max-length"
- id: check-summary-punctuation
name: "[bash - format] check-summary-punctuation"

- repo: local
hooks:
- id: pytest-check
name: pytest-check
entry: pytest -vvv
language: system
pass_filenames: false
always_run: true
5 changes: 5 additions & 0 deletions HISTORY.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,3 +46,8 @@ History
* refactor code
* add documentation
* fix creating memory driver with compression in _createDataset

0.3.3 (2023-02-06)
------------------
* fix bug in reading the ogr drivers catalog for the vector class
* fix bug in creating rasterLike in the asciiToRaster method
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@
Current release info
====================

| Name | Downloads | Version | Platforms |
|----------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| --- |
| [![Conda Recipe](https://img.shields.io/badge/recipe-pyramids-green.svg)](https://anaconda.org/conda-forge/pyramids) | [![Conda Downloads](https://img.shields.io/conda/dn/conda-forge/pyramids.svg)](https://anaconda.org/conda-forge/pyramids) [![Downloads](https://pepy.tech/badge/pyramids-gis)](https://pepy.tech/project/pyramids-gis) [![Downloads](https://pepy.tech/badge/pyramids-gis/month)](https://pepy.tech/project/pyramids-gis) [![Downloads](https://pepy.tech/badge/pyramids-gis/week)](https://pepy.tech/project/pyramids-gis) ![PyPI - Downloads](https://img.shields.io/pypi/dd/pyramids-gis?color=blue&style=flat-square) ![GitHub all releases](https://img.shields.io/github/downloads/MAfarrag/pyramids/total) ![GitHub release (latest by date)](https://img.shields.io/github/downloads/MAfarrag/pyramids/0.2.1/total) | [![Conda Version](https://img.shields.io/conda/vn/conda-forge/pyramids.svg)](https://anaconda.org/conda-forge/pyramids) [![PyPI version](https://badge.fury.io/py/pyramids-gis.svg)](https://badge.fury.io/py/pyramids-gis) | [![Conda Platforms](https://img.shields.io/conda/pn/conda-forge/pyramids.svg)](https://anaconda.org/conda-forge/pyramids) [![Join the chat at https://gitter.im/Hapi-Nile/Hapi](https://badges.gitter.im/Hapi-Nile/Hapi.svg)](https://gitter.im/Hapi-Nile/Hapi?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) |
| Name | Downloads | Version | Platforms |
|----------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| --- |
| [![Conda Recipe](https://img.shields.io/badge/recipe-pyramids-green.svg)](https://anaconda.org/conda-forge/pyramids) | [![Conda Downloads](https://img.shields.io/conda/dn/conda-forge/pyramids.svg)](https://anaconda.org/conda-forge/pyramids) [![Downloads](https://pepy.tech/badge/pyramids-gis)](https://pepy.tech/project/pyramids-gis) [![Downloads](https://pepy.tech/badge/pyramids-gis/month)](https://pepy.tech/project/pyramids-gis) [![Downloads](https://pepy.tech/badge/pyramids-gis/week)](https://pepy.tech/project/pyramids-gis) ![PyPI - Downloads](https://img.shields.io/pypi/dd/pyramids-gis?color=blue&style=flat-square) | [![Conda Version](https://img.shields.io/conda/vn/conda-forge/pyramids.svg)](https://anaconda.org/conda-forge/pyramids) [![PyPI version](https://badge.fury.io/py/pyramids-gis.svg)](https://badge.fury.io/py/pyramids-gis) | [![Conda Platforms](https://img.shields.io/conda/pn/conda-forge/pyramids.svg)](https://anaconda.org/conda-forge/pyramids) [![Join the chat at https://gitter.im/Hapi-Nile/Hapi](https://badges.gitter.im/Hapi-Nile/Hapi.svg)](https://gitter.im/Hapi-Nile/Hapi?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) |

pyramids - GIS utility package
=====================================================================
Expand All @@ -48,7 +48,7 @@ Installing pyramids
Installing `pyramids` from the `conda-forge` channel can be achieved by:

```
conda install -c conda-forge pyramids=0.3.2
conda install -c conda-forge pyramids=0.3.3
```

It is possible to list all of the versions of `pyramids` available on your platform with:
Expand All @@ -66,7 +66,7 @@ pip install git+https://github.com/Serapieum-of-alex/pyramids
## pip
to install the last release you can easly use pip
```
pip install pyramids-gis==0.3.2
pip install pyramids-gis==0.3.3
```

Quick start
Expand Down
10 changes: 8 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@
# "sphinx.ext.doctest",
# "sphinx.ext.autosectionlabel",
"numpydoc",
"nbsphinx"
"nbsphinx",
]

autosectionlabel_prefix_document = True
Expand Down Expand Up @@ -216,7 +216,13 @@
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
("index", "pyramids-gis.tex", "pyramids-gis Documentation", "Mostafa Farrag", "report")
(
"index",
"pyramids-gis.tex",
"pyramids-gis Documentation",
"Mostafa Farrag",
"report",
)
]

# The name of an image file (relative to this directory) to place at the top of
Expand Down
2 changes: 1 addition & 1 deletion docs/environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,4 @@ dependencies:
- typing-extensions==3.10.*
- pip:
- pydata-sphinx-theme
- nbsphinx
- nbsphinx
2 changes: 1 addition & 1 deletion docs/spatial_indexing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -153,4 +153,4 @@ Returns:
>>> 1 POLYGON ((107.74262 15.06330, 106.41168 2.7703...
>>> 2 POLYGON ((94.36893 -13.83685, 95.22669 -2.5046...
>>> 3 POLYGON ((37.16428 29.33908, 31.19369 18.24201...
>>> dtype: geometry
>>> dtype: geometry
4 changes: 2 additions & 2 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ channels:
- conda-forge
dependencies:
- python >=3.9,<3.11
- numpy ==1.24.1
- numpy >=1.24.1
- pip >=22.3.1
- gdal >=3.5.3
- affine >=2.3.1
Expand All @@ -19,5 +19,5 @@ dependencies:
- PyYAML >=6.0
- loguru >=0.6.0
- pytest >=7.2.0
- pytest-cov ==4.0.0
- pytest-cov >=4.0.0
- requests >=2.28.1
Binary file modified examples/data/fill_raster_saved.tif
Binary file not shown.
Binary file modified examples/data/raster_like_saved.tif
Binary file not shown.
14 changes: 0 additions & 14 deletions mkdocs.yml

This file was deleted.

6 changes: 3 additions & 3 deletions pyramids/convert.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import shutil
import tempfile
import uuid
from typing import Any, Dict, Union
from typing import Any, Union

import geopandas as gpd
import netCDF4
Expand All @@ -26,7 +26,7 @@ class Convert:
"""Convert data from one form to another."""

def __init__(self):
self.vector_catalog: Dict[str, str] = Vector.getCatalog()
# self.vector_catalog: Dict[str, str] = Vector.getCatalog()
pass

@staticmethod
Expand Down Expand Up @@ -144,7 +144,7 @@ def asciiToRaster(
ASCIIRows == RasterRows and ASCIIColumns == RasterColumns
), " Data in both ASCII file and Raster file should have the same number of row and columns"

Raster.rasterLike(src, ASCIIValues, save_path, pixel_type)
Raster.rasterLike(src, ASCIIValues, save_path)
elif epsg is not None:
assert (
type(epsg) == int
Expand Down
Loading

0 comments on commit 28bd2ef

Please sign in to comment.