Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Al objectdetection #85

Open
wants to merge 129 commits into
base: hdc_activelearning2
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
7dd01a5
Update README.md
annmary-roy Aug 3, 2022
942984b
updated
abhinavchobey Aug 25, 2022
485290d
Merge pull request #5 from abhinavchobey/master
annmary-roy Aug 25, 2022
f631dcb
Adding the exetrnal presentations
annmary-roy Aug 26, 2022
e386884
Added the pdf version
annmary-roy Aug 26, 2022
b8ec135
Removed pylint errors
abhinavchobey Aug 29, 2022
235a581
Merge pull request #7 from abhinavchobey/master
annmary-roy Aug 31, 2022
860e293
documentation changes to add more context about the project
annmary-roy Sep 1, 2022
46906e4
Fixed 7 more pylint errors.
abhinavchobey Sep 2, 2022
53c384a
Merge branch 'master' of https://github.com/abhinavchobey/cmf
abhinavchobey Sep 2, 2022
9d736cf
Merge pull request #12 from abhinavchobey/master
annmary-roy Sep 8, 2022
5ed95bd
Update README.md
annmary-roy Sep 8, 2022
a25749c
Update README.md
annmary-roy Sep 8, 2022
51ca4ed
Update to Readme - Added the slack channel
annmary-roy Sep 8, 2022
a2bc753
Update README.md
annmary-roy Sep 8, 2022
920ceaa
uniquified metrics uuid - lightning logger debugging
Sep 9, 2022
385818c
Uniqify Metrics Name - Pytorch Lightning Logger
atripathy86 Sep 9, 2022
81b5cb7
Merge branch 'metrics_fix' of github.com:HewlettPackard/cmf into metr…
atripathy86 Sep 9, 2022
fe07ef1
Reeverted the print statemenst from pylint check in
Sep 12, 2022
4a8b7c9
Revert "Reeverted the print statemenst from pylint check in"
annmary-roy Sep 12, 2022
f01c056
Reverted the print statement from pylint fixes
annmary-roy Sep 12, 2022
d69db79
Creating a seperate branch for capturing the cmf commits
annmary-roy Sep 12, 2022
86f96fb
Added the pre checks for cmf
annmary-roy Sep 12, 2022
8bafdf2
Merge remote-tracking branch 'origin/metrics_fix'
annmary-roy Sep 13, 2022
f37cca8
Added code to update an existing execution
annmary-roy Sep 13, 2022
4be7e84
Pylint fixes
annmary-roy Sep 14, 2022
ba8a711
Update README.md
annmary-roy Sep 16, 2022
b1aa90b
Fixes for pylint
annmary-roy Sep 19, 2022
e0ba735
Added code to get the parent executions for an artifact and bug fix f…
annmary-roy Sep 21, 2022
e417065
Pylint fixes
annmary-roy Sep 27, 2022
6b94e59
Merge branch 'pylint_fixes'
annmary-roy Sep 27, 2022
e40e782
Update README.md
annmary-roy Oct 20, 2022
8e8bcfb
Docker fixes
annmary-roy Oct 20, 2022
c139187
Merge branch 'master' of https://github.com/HewlettPackard/cmf
annmary-roy Oct 21, 2022
4c0536c
Fixes to docker files
annmary-roy Oct 21, 2022
8829942
Fix for branch_name
annmary-roy Oct 24, 2022
3e22c6b
Modified the jupyter notebook
annmary-roy Nov 1, 2022
b528fd2
Update README.md
annmary-roy Nov 14, 2022
6746bca
First commit of the `.gitignore` file.
sergey-serebryakov Nov 20, 2022
96326c7
Merge pull request #23 from sergey-serebryakov/feature/gitignore
annmary-roy Nov 21, 2022
e525abe
MKDocs-based documentation (#24)
sergey-serebryakov Nov 29, 2022
ca87e94
Updated readme to fix a broken link
annmary-roy Nov 29, 2022
3bc5ca7
GitHub workflow to build and deploy docs to GitHub pages (#26)
sergey-serebryakov Nov 29, 2022
a21f9f4
BugFix to change the main branch name for the "Build CMF Docs & Deplo…
sergey-serebryakov Nov 29, 2022
158f127
BigFix: use single quotes in GitHub workflow expressions (#28)
sergey-serebryakov Nov 29, 2022
488a0a0
Updated the documentation for docker (#29)
annmary-roy Nov 29, 2022
28d11a6
Documentation fix for broken links and link to docker documentation (…
annmary-roy Nov 29, 2022
84a8d0c
Bug fix (#31)
annmary-roy Nov 30, 2022
24674a1
Upgraded ml metadata from 1.3.0 to 1.11.0 (#32)
annmary-roy Nov 30, 2022
9bf2392
Updated the documentation for python supportability (#33)
annmary-roy Nov 30, 2022
69ed23a
Docker documentation
annmary-roy Nov 28, 2022
189fb63
Update README.md
annmary-roy Nov 28, 2022
e0abee1
Added pypi dependency files (#34)
annmary-roy Dec 1, 2022
d8a465a
Env example updated
annmary-roy Dec 2, 2022
5dd7d32
Merge branch 'master' of https://github.com/HewlettPackard/cmf
annmary-roy Dec 2, 2022
519de1a
documentation fix (#36)
annmary-roy Dec 2, 2022
cefc153
fix in docker compose to export neo4j (#37)
annmary-roy Dec 2, 2022
9cd7720
Ann origin/master (#38)
annmary-roy Dec 2, 2022
7df8b8b
Merge the updated query tester (#39)
annmary-roy Dec 2, 2022
0d7071c
Restored base python
annmary-roy Dec 2, 2022
0ae48dc
Change sklearn to scikit-learn, also fix neo4j docker name and python…
annmary-roy Dec 2, 2022
00237b4
nb_conda added to the base image of the docker (#40)
annmary-roy Dec 2, 2022
64f1a22
Added dvc import in cmf.py without that in jupyter lab environment ge…
annmary-roy Dec 2, 2022
edb6be9
Merge pull request #42 from annmary-roy/requirements-fix
atripathy86 Dec 2, 2022
58e98e1
Merge branch 'master' of https://github.com/HewlettPackard/cmf
annmary-roy Dec 5, 2022
7220f60
Update README.md
annmary-roy Dec 5, 2022
5688a8a
Merge branch 'master' of https://github.com/HewlettPackard/cmf
annmary-roy Dec 5, 2022
0ed821f
Restored api.md
annmary-roy Dec 5, 2022
93a1c9e
Restored API.md -Workaround for docs not displaying properly in githu…
annmary-roy Dec 5, 2022
2770e70
Merge branch 'master' of https://github.com/HewlettPackard/cmf
annmary-roy Dec 5, 2022
dd712a5
Added a missed out file in active_learning example (#54)
annmary-roy Dec 6, 2022
9e579af
Unifying the custom properties name (#46)
annmary-roy Dec 6, 2022
d8865bf
Readme fix- Pointing to the public github pages (#53)
annmary-roy Dec 6, 2022
17ea424
Active learning- Bug fix and added comments (#58)
annmary-roy Dec 6, 2022
694bd2e
Merge branch 'master' of https://github.com/HewlettPackard/cmf
annmary-roy Dec 12, 2022
660872c
Fixed exposed names
annmary-roy Dec 14, 2022
eb88def
fixed exposed names
annmary-roy Dec 14, 2022
9c7abec
Merge branch 'master' of https://github.com/HewlettPackard/cmf
annmary-roy Dec 14, 2022
e9519e2
neo4j defaults
annmary-roy Dec 14, 2022
4e4e06b
Modified dvc.yaml
annmary-roy Aug 3, 2022
4812d12
First draft of miaod from Yuan et al integrated with dvc run
martin-foltin Dec 8, 2022
118e6a2
Fixed dvc.yaml
martin-foltin Dec 9, 2022
9b9d4d9
metadata ingest
annmary-roy Dec 12, 2022
9d75ce8
Added missing import
annmary-roy Dec 12, 2022
7544c08
Added initialize and sample_env file in the al_object_detection example
annmary-roy Dec 12, 2022
cc84bb2
Added our work to README.md
martin-foltin Dec 14, 2022
b69eaf8
Fixed README.md
martin-foltin Dec 14, 2022
72e73d3
Merge branch 'al_objectdetection' of https://github.com/HewlettPackar…
annmary-roy Dec 14, 2022
9912240
Added code for full dataset train
annmary-roy Mar 10, 2023
7a30d02
Merge remote-tracking branch 'ann-origin/al_objectdetection' into al_…
annmary-roy Mar 10, 2023
fb15635
Full training- python code and dvc.yaml
annmary-roy Mar 10, 2023
a656663
Merge pull request #83 from annmary-roy/al_objectdetection_full_train
martin-foltin Mar 10, 2023
5bc7db1
changed paths and names to match hdc dataset
Mar 10, 2023
0d4f06b
add changes for class names
Mar 10, 2023
4a398c4
training run only
Mar 10, 2023
4cb9d38
fix run cycles
Mar 10, 2023
a0462e8
add latest changes
Mar 14, 2023
30cd10d
Merge branch 'hdc_activelearning2' into al_objectdetection
annmary-roy Mar 20, 2023
43ee079
Merge branch 'al_objectdetection' of https://github.com/HewlettPackar…
annmary-roy Mar 30, 2023
f095e21
Clamping the loss and claming the gradients
annmary-roy Mar 30, 2023
4402e00
Merge branch 'al_objectdetection' of https://github.com/HewlettPackar…
annmary-roy Apr 20, 2023
42e6b1a
random selection strategy
annmary-roy Apr 21, 2023
d350280
push latest config settings
May 19, 2023
a33fcce
add epoch based runner changes and unlabeled in MIAOD head
May 19, 2023
638d505
distributed and default train settings
May 19, 2023
27e34e8
Added fix for stop iteration and deprecated libraries
annmary-roy Jun 14, 2023
c9a480b
add no dummy to folder name
Jun 14, 2023
e4d2f6e
Single cycle workflow for production runs. Added cycle.yaml, al_cycle…
martin-foltin Jun 29, 2023
21d7a99
Support image path mapping files in csv format
martin-foltin Jun 29, 2023
85be843
Support file numbers with leading zeros
martin-foltin Jun 29, 2023
178ee4f
Added support for datesets with no annotation files for unlabeled sam…
martin-foltin Jul 6, 2023
f7e7045
Added first revision of code that outputs hints guiding annotator whe…
martin-foltin Oct 13, 2023
32b29d5
Modified dvc.yaml and cycle.yaml to support plotting of bounding box …
martin-foltin Oct 13, 2023
fb745c5
Resolved bug in mmdet/utils/hdc/active_datasets.py
martin-foltin Oct 13, 2023
161474d
yaml
Oct 19, 2023
85b0099
previous changes before new active learning visualization
Oct 19, 2023
e8a6bcb
ReCreate dvc.yaml
orozda Oct 19, 2023
52b4142
Create model.py
orozda Nov 2, 2023
d4eaedc
Change the corrdinates from text file to json
Nov 27, 2023
6106808
Changed png files to jpg
Nov 29, 2023
e56ca00
Merge pull request #135 from HewlettPackard/TextToJson
rishabhsharma22 Nov 29, 2023
8d58028
Added rotation agumentations
Dec 7, 2023
207ec66
original yaml and new yaml
Jan 10, 2024
24d3c79
changes to yaml
Jan 25, 2024
374f0c5
delete lock file
Jan 25, 2024
1bd4dbd
commenting out of rotation code
Jan 30, 2024
5a82274
Intersection Over Ground Truth added in bbox_overlap methods
Jan 30, 2024
82d8a0b
Fixed XML_Style to work with Albumentations
Apr 12, 2024
b90cf34
cmf and dvc integration (#170)
rishabhsharma22 May 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions .github/workflows/deploy_docs_to_gh_pages.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Deploy CMF mkdocs-based documentation to GitHub pages. The CMF docs need to be built first (because we automatically
# build API documentation pages).
# https://github.com/peaceiris/actions-gh-pages

name: Build CMF Docs & Deploy to GitHub pages

on:
push:
branches:
- master
paths:
- '.github/workflows/deploy_docs_to_gh_pages.yaml' # The workflow file itself.
- 'docs/**' # Documentation files
- '!docs/_src/**' # but exclude this one (raw resources for docs).
- 'cmflib/cmf.py' # Public API documentation.

jobs:
deploy-docs-to-gh-pages:
# Do not run on forked repositories.
if: github.repository_owner == 'HewlettPackard'
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Checkout Repository
uses: actions/checkout@v3

- name: Setup Python Environment
uses: actions/setup-python@v3
with:
python-version: '3.8'

- name: Install Python Dependencies
run: |
python -m pip install --upgrade pip
pip install -r docs/requirements.txt

- name: Build Docs
working-directory: './'
# First - build documentation, then remove a directory that contains raw files for documentation assets, and
# these raw files are not required for running the CMF documentation site.
run: |
mkdocs build --theme material --site-dir ../site/
rm -r ../site/_src

- name: Deploy Docts To GitHub Pages
# This step will deploy the generated documentation to `gh-pages` branch.
uses: peaceiris/[email protected]
with:
user_name: 'github-actions[bot]'
user_email: 'github-actions[bot]@users.noreply.github.com'
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ../site
allow_empty_commit: true
88 changes: 88 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
./build/
mlcommons_box/build
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.pytest_cache/

# Translations
*.mo
*.pot

# Sphinx documentation
docs/_build/

# PyBuilder
target/

# Jupyter Notebook
.ipynb_checkpoints
*.ipynb
# pyenv
.python-version

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# mkdocs documentation
/site

# mypy
.mypy_cache/

# PyCharm, IDEA
.idea

49 changes: 30 additions & 19 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,50 +12,61 @@ FROM jupyter/tensorflow-notebook
#FROM nvcr.io/partners/chainer:4.0.0b1

#Following: https://jupyter-docker-stacks.readthedocs.io/en/latest/using/recipes.html#
ARG NB_USER
ARG NB_USER
ARG NB_UID
ARG NB_GID


# name your environment and choose the python version
ARG conda_env=python37
ARG py_ver=3.7
#ARG conda_env=python37
#ARG py_ver=3.7

#https://stackoverflow.com/questions/53004311/how-to-add-conda-environment-to-jupyter-lab
#https://gist.github.com/James-Leslie/734babcbc1dd3f5fbc828af773922497
RUN conda install -c conda-forge nb_conda_kernels

# you can add additional libraries you want mamba to install by listing them below the first line and ending with "&& \"
RUN mamba create --quiet --yes -p "${CONDA_DIR}/envs/${conda_env}" python=${py_ver} ipython ipykernel && \
mamba clean --all -f -y
#RUN mamba create --quiet --yes -p "${CONDA_DIR}/envs/${conda_env}" python=${py_ver} ipython ipykernel && \
# mamba clean --all -f -y

#RUN conda activate ${conda_env}

# create Python kernel and link it to jupyter
RUN "${CONDA_DIR}/envs/${conda_env}/bin/python" -m ipykernel install --user --name="${conda_env}" && \
fix-permissions "${CONDA_DIR}" && \
fix-permissions "/home/${NB_USER}"
#RUN "${CONDA_DIR}/envs/${conda_env}/bin/python" -m ipykernel install --user --name="${conda_env}" && \
# fix-permissions "${CONDA_DIR}" && \
# fix-permissions "/home/${NB_USER}"

# any additional pip installs can be added by uncommenting the following line
#RUN "${CONDA_DIR}/envs/${conda_env}/bin/pip" install --quiet --no-cache-dir

# if you want this environment to be the default one, uncomment the following line:
RUN echo "conda activate ${conda_env}" >> "${HOME}/.bashrc"

#RUN apt-get update; apt-get install -y build-essential

USER ${NB_USER}

RUN "${CONDA_DIR}/envs/${conda_env}/bin/pip" install --quiet --no-cache-dir 'flake8==3.9.2' && \
fix-permissions "${CONDA_DIR}" && \
fix-permissions "/home/${NB_USER}" && \
mkdir /home/${NB_USER}/cmflib/
#RUN "${CONDA_DIR}/envs/${conda_env}/bin/pip" install --quiet --no-cache-dir 'flake8==3.9.2' && \

# fix-permissions "${CONDA_DIR}" && \
# fix-permissions "/home/${NB_USER}" && \
RUN mkdir /home/${NB_USER}/cmflib/

COPY --chown=${NB_UID}:${NB_GID} cmflib/Requirements.txt /home/${NB_USER}/cmflib/
RUN "${CONDA_DIR}/envs/${conda_env}/bin/pip" install --no-cache-dir --requirement /home/${NB_USER}/cmflib/Requirements.txt && \
COPY --chown=${NB_UID}:${NB_GID} Requirements.txt /home/${NB_USER}/cmflib/
RUN "${CONDA_DIR}/bin/pip" install --no-cache-dir --requirement /home/${NB_USER}/cmflib/Requirements.txt && \
fix-permissions "${CONDA_DIR}" && \
fix-permissions "/home/${NB_USER}"

COPY --chown=${NB_UID}:${NB_GID} cmflib/cmflib /home/${NB_USER}/cmflib/cmflib
COPY --chown=${NB_UID}:${NB_GID} cmflib/setup.py /home/${NB_USER}/cmflib/setup.py
RUN cd /home/${NB_USER}/cmflib && "${CONDA_DIR}/envs/${conda_env}/bin/pip" install --no-cache . && \
COPY --chown=${NB_UID}:${NB_GID} cmflib /home/${NB_USER}/cmflib/cmflib
COPY --chown=${NB_UID}:${NB_GID} setup.py /home/${NB_USER}/cmflib/setup.py
RUN cd /home/${NB_USER}/cmflib && "${CONDA_DIR}/bin/pip" install --no-cache . && \
fix-permissions "${CONDA_DIR}" && \
fix-permissions "/home/${NB_USER}"

#ENV PYTHONPATH "${PYTHONPATH}:/home/${NB_USER}/cmflib"

COPY --chown=${NB_UID}:${NB_GID} cmflib/example-get-started /home/${NB_USER}/example-get-started
COPY --chown=${NB_UID}:${NB_GID} examples/example-get-started /home/${NB_USER}/example-get-started


# if you want this environment to be the default one, uncomment the following line:
RUN echo "conda activate ${conda_env}" >> "${HOME}/.bashrc"

Loading