diff --git a/.github/workflows/docs-ci.yml b/.github/workflows/docs-ci.yml index 2b10c392..e4c5cd3c 100644 --- a/.github/workflows/docs-ci.yml +++ b/.github/workflows/docs-ci.yml @@ -13,10 +13,10 @@ jobs: steps: - name: Checkout code - uses: actions/checkout@v2 + uses: actions/checkout@v3 - name: Set up Python ${{ matrix.python-version }} - uses: actions/setup-python@v2 + uses: actions/setup-python@v4 with: python-version: ${{ matrix.python-version }} diff --git a/.github/workflows/pypi-release.yml b/.github/workflows/pypi-release.yml index 22315ff0..4ebe10d1 100644 --- a/.github/workflows/pypi-release.yml +++ b/.github/workflows/pypi-release.yml @@ -24,9 +24,9 @@ jobs: runs-on: ubuntu-20.04 steps: - - uses: actions/checkout@master + - uses: actions/checkout@v3 - name: Set up Python - uses: actions/setup-python@v1 + uses: actions/setup-python@v4 with: python-version: 3.9 diff --git a/.readthedocs.yml b/.readthedocs.yml index 1b71cd9e..97ffa418 100644 --- a/.readthedocs.yml +++ b/.readthedocs.yml @@ -5,6 +5,16 @@ # Required version: 2 +build: + os: ubuntu-22.04 + tools: + python: "3.11" + +# Build PDF & ePub +formats: + - epub + - pdf + # Where the Sphinx conf.py file is located sphinx: configuration: docs/source/conf.py diff --git a/docs/source/conf.py b/docs/source/conf.py index d5435e75..abad1f97 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -17,7 +17,7 @@ # -- Project information ----------------------------------------------------- -project = "nexb-skeleton" +project = "PurlDB" copyright = "nexB Inc. and others." author = "AboutCode.org authors and contributors" @@ -67,8 +67,8 @@ html_context = { "display_github": True, "github_user": "nexB", - "github_repo": "nexb-skeleton", - "github_version": "develop", # branch + "github_repo": "purldb", + "github_version": "main", # branch "conf_py_path": "/docs/source/", # path in the checkout to the docs root } @@ -95,3 +95,9 @@ .. role:: img-title-para """ + +# -- Options for LaTeX output ------------------------------------------------- + +latex_elements = { + 'classoptions': ',openany,oneside' +} \ No newline at end of file diff --git a/docs/source/contribute/contrib_doc.rst b/docs/source/contribute/contrib_doc.rst deleted file mode 100644 index 13882e10..00000000 --- a/docs/source/contribute/contrib_doc.rst +++ /dev/null @@ -1,314 +0,0 @@ -.. _contrib_doc_dev: - -Contributing to the Documentation -================================= - -.. _contrib_doc_setup_local: - -Setup Local Build ------------------ - -To get started, create or identify a working directory on your local machine. - -Open that directory and execute the following command in a terminal session:: - - git clone https://github.com/nexB/skeleton.git - -That will create an ``/skeleton`` directory in your working directory. -Now you can install the dependencies in a virtualenv:: - - cd skeleton - ./configure --docs - -.. note:: - - In case of windows, run ``configure --docs`` instead of this. - -Now, this will install the following prerequisites: - -- Sphinx -- sphinx_rtd_theme (the format theme used by ReadTheDocs) -- docs8 (style linter) - -These requirements are already present in setup.cfg and `./configure --docs` installs them. - -Now you can build the HTML documents locally:: - - source venv/bin/activate - cd docs - make html - -Assuming that your Sphinx installation was successful, Sphinx should build a local instance of the -documentation .html files:: - - open build/html/index.html - -.. note:: - - In case this command did not work, for example on Ubuntu 18.04 you may get a message like “Couldn’t - get a file descriptor referring to the console”, try: - - :: - - see build/html/index.html - -You now have a local build of the AboutCode documents. - -.. _contrib_doc_share_improvements: - -Share Document Improvements ---------------------------- - -Ensure that you have the latest files:: - - git pull - git status - -Before commiting changes run Continious Integration Scripts locally to run tests. Refer -:ref:`doc_ci` for instructions on the same. - -Follow standard git procedures to upload your new and modified files. The following commands are -examples:: - - git status - git add source/index.rst - git add source/how-to-scan.rst - git status - git commit -m "New how-to document that explains how to scan" - git status - git push - git status - -The Scancode-Toolkit webhook with ReadTheDocs should rebuild the documentation after your -Pull Request is Merged. - -Refer the `Pro Git Book `_ available online for Git tutorials -covering more complex topics on Branching, Merging, Rebasing etc. - -.. _doc_ci: - -Continuous Integration ----------------------- - -The documentations are checked on every new commit through Travis-CI, so that common errors are -avoided and documentation standards are enforced. Travis-CI presently checks for these 3 aspects -of the documentation : - -1. Successful Builds (By using ``sphinx-build``) -2. No Broken Links (By Using ``link-check``) -3. Linting Errors (By Using ``Doc8``) - -So run these scripts at your local system before creating a Pull Request:: - - cd docs - ./scripts/sphinx_build_link_check.sh - ./scripts/doc8_style_check.sh - -If you don't have permission to run the scripts, run:: - - chmod u+x ./scripts/doc8_style_check.sh - -.. _doc_style_docs8: - -Style Checks Using ``Doc8`` ---------------------------- - -How To Run Style Tests -^^^^^^^^^^^^^^^^^^^^^^ - -In the project root, run the following commands:: - - $ cd docs - $ ./scripts/doc8_style_check.sh - -A sample output is:: - - Scanning... - Validating... - docs/source/misc/licence_policy_plugin.rst:37: D002 Trailing whitespace - docs/source/misc/faq.rst:45: D003 Tabulation used for indentation - docs/source/misc/faq.rst:9: D001 Line too long - docs/source/misc/support.rst:6: D005 No newline at end of file - ======== - Total files scanned = 34 - Total files ignored = 0 - Total accumulated errors = 326 - Detailed error counts: - - CheckCarriageReturn = 0 - - CheckIndentationNoTab = 75 - - CheckMaxLineLength = 190 - - CheckNewlineEndOfFile = 13 - - CheckTrailingWhitespace = 47 - - CheckValidity = 1 - -Now fix the errors and run again till there isn't any style error in the documentation. - -What is Checked? -^^^^^^^^^^^^^^^^ - -PyCQA is an Organization for code quality tools (and plugins) for the Python programming language. -Doc8 is a sub-project of the same Organization. Refer this `README `_ for more details. - -What is checked: - - - invalid rst format - D000 - - lines should not be longer than 100 characters - D001 - - - RST exception: line with no whitespace except in the beginning - - RST exception: lines with http or https URLs - - RST exception: literal blocks - - RST exception: rst target directives - - - no trailing whitespace - D002 - - no tabulation for indentation - D003 - - no carriage returns (use UNIX newlines) - D004 - - no newline at end of file - D005 - -.. _doc_interspinx: - -Interspinx ----------- - -ScanCode toolkit documentation uses `Intersphinx `_ -to link to other Sphinx Documentations, to maintain links to other Aboutcode Projects. - -To link sections in the same documentation, standart reST labels are used. Refer -`Cross-Referencing `_ for more information. - -For example:: - - .. _my-reference-label: - - Section to cross-reference - -------------------------- - - This is the text of the section. - - It refers to the section itself, see :ref:`my-reference-label`. - -Now, using Intersphinx, you can create these labels in one Sphinx Documentation and then referance -these labels from another Sphinx Documentation, hosted in different locations. - -You just have to add the following in the ``conf.py`` file for your Sphinx Documentation, where you -want to add the links:: - - extensions = [ - 'sphinx.ext.intersphinx' - ] - - intersphinx_mapping = {'aboutcode': ('https://aboutcode.readthedocs.io/en/latest/', None)} - -To show all Intersphinx links and their targets of an Intersphinx mapping file, run:: - - python -msphinx.ext.intersphinx https://aboutcode.readthedocs.io/en/latest/objects.inv - -.. WARNING:: - - ``python -msphinx.ext.intersphinx https://aboutcode.readthedocs.io/objects.inv`` will give - error. - -This enables you to create links to the ``aboutcode`` Documentation in your own Documentation, -where you modified the configuration file. Links can be added like this:: - - For more details refer :ref:`aboutcode:doc_style_guide`. - -You can also not use the ``aboutcode`` label assigned to all links from aboutcode.readthedocs.io, -if you don't have a label having the same name in your Sphinx Documentation. Example:: - - For more details refer :ref:`doc_style_guide`. - -If you have a label in your documentation which is also present in the documentation linked by -Intersphinx, and you link to that label, it will create a link to the local label. - -For more information, refer this tutorial named -`Using Intersphinx `_. - -.. _doc_style_conv: - -Style Conventions for the Documentaion --------------------------------------- - -1. Headings - - (`Refer `_) - Normally, there are no heading levels assigned to certain characters as the structure is - determined from the succession of headings. However, this convention is used in Python’s Style - Guide for documenting which you may follow: - - # with overline, for parts - - * with overline, for chapters - - =, for sections - - -, for subsections - - ^, for sub-subsections - - ", for paragraphs - -2. Heading Underlines - - Do not use underlines that are longer/shorter than the title headline itself. As in: - - :: - - Correct : - - Extra Style Checks - ------------------ - - Incorrect : - - Extra Style Checks - ------------------------ - -.. note:: - - Underlines shorter than the Title text generates Errors on sphinx-build. - - -3. Internal Links - - Using ``:ref:`` is advised over standard reStructuredText links to sections (like - ```Section title`_``) because it works across files, when section headings are changed, will - raise warnings if incorrect, and works for all builders that support cross-references. - However, external links are created by using the standard ```Section title`_`` method. - -4. Eliminate Redundancy - - If a section/file has to be repeated somewhere else, do not write the exact same section/file - twice. Use ``.. include: ../README.rst`` instead. Here, ``../`` refers to the documentation - root, so file location can be used accordingly. This enables us to link documents from other - upstream folders. - -5. Using ``:ref:`` only when necessary - - Use ``:ref:`` to create internal links only when needed, i.e. it is referenced somewhere. - Do not create references for all the sections and then only reference some of them, because - this created unnecessary references. This also generates ERROR in ``restructuredtext-lint``. - -6. Spelling - - You should check for spelling errors before you push changes. `Aspell `_ - is a GNU project Command Line tool you can use for this purpose. Download and install Aspell, - then execute ``aspell check `` for all the files changed. Be careful about not - changing commands or other stuff as Aspell gives prompts for a lot of them. Also delete the - temporary ``.bak`` files generated. Refer the `manual `_ for more - information on how to use. - -7. Notes and Warning Snippets - - Every ``Note`` and ``Warning`` sections are to be kept in ``rst_snippets/note_snippets/`` and - ``rst_snippets/warning_snippets/`` and then included to eliminate redundancy, as these are - frequently used in multiple files. - -Converting from Markdown ------------------------- - -If you want to convert a ``.md`` file to a ``.rst`` file, this `tool `_ -does it pretty well. You'd still have to clean up and check for errors as this contains a lot of -bugs. But this is definitely better than converting everything by yourself. - -This will be helpful in converting GitHub wiki's (Markdown Files) to reStructuredtext files for -Sphinx/ReadTheDocs hosting. diff --git a/docs/source/index.rst b/docs/source/index.rst index eb63717b..b9dfad4d 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,12 +1,7 @@ -Welcome to nexb-skeleton's documentation! +Welcome to PurlDB documentation! ========================================= -.. toctree:: - :maxdepth: 2 - :caption: Contents: - - skeleton-usage - contribute/contrib_doc +.. include:: ../../README.rst Indices and tables ================== diff --git a/docs/source/skeleton-usage.rst b/docs/source/skeleton-usage.rst deleted file mode 100644 index cde23dcd..00000000 --- a/docs/source/skeleton-usage.rst +++ /dev/null @@ -1,160 +0,0 @@ -Usage -===== -A brand new project -------------------- -.. code-block:: bash - - git init my-new-repo - cd my-new-repo - git pull git@github.com:nexB/skeleton - - # Create the new repo on GitHub, then update your remote - git remote set-url origin git@github.com:nexB/your-new-repo.git - -From here, you can make the appropriate changes to the files for your specific project. - -Update an existing project ---------------------------- -.. code-block:: bash - - cd my-existing-project - git remote add skeleton git@github.com:nexB/skeleton - git fetch skeleton - git merge skeleton/main --allow-unrelated-histories - -This is also the workflow to use when updating the skeleton files in any given repository. - -Customizing ------------ - -You typically want to perform these customizations: - -- remove or update the src/README.rst and tests/README.rst files -- set project info and dependencies in setup.cfg -- check the configure and configure.bat defaults - -Initializing a project ----------------------- - -All projects using the skeleton will be expected to pull all of it dependencies -from thirdparty.aboutcode.org/pypi or the local thirdparty directory, using -requirements.txt and/or requirements-dev.txt to determine what version of a -package to collect. By default, PyPI will not be used to find and collect -packages from. - -In the case where we are starting a new project where we do not have -requirements.txt and requirements-dev.txt and whose dependencies are not yet on -thirdparty.aboutcode.org/pypi, we run the following command after adding and -customizing the skeleton files to your project: - -.. code-block:: bash - - ./configure - -This will initialize the virtual environment for the project, pull in the -dependencies from PyPI and add them to the virtual environment. - - -Generating requirements.txt and requirements-dev.txt ----------------------------------------------------- - -After the project has been initialized, we can generate the requirements.txt and -requirements-dev.txt files. - -Ensure the virtual environment is enabled. - -.. code-block:: bash - - source venv/bin/activate - -To generate requirements.txt: - -.. code-block:: bash - - python etc/scripts/gen_requirements.py -s venv/lib/python/site-packages/ - -Replace \ with the version number of the Python being used, for example: -``venv/lib/python3.6/site-packages/`` - -To generate requirements-dev.txt after requirements.txt has been generated: - -.. code-block:: bash - - ./configure --dev - python etc/scripts/gen_requirements_dev.py -s venv/lib/python/site-packages/ - -Note: on Windows, the ``site-packages`` directory is located at ``venv\Lib\site-packages\`` - -.. code-block:: bash - - python .\\etc\\scripts\\gen_requirements.py -s .\\venv\\Lib\\site-packages\\ - .\configure --dev - python .\\etc\\scripts\\gen_requirements_dev.py -s .\\venv\\Lib\\site-packages\\ - - -Collecting and generating ABOUT files for dependencies ------------------------------------------------------- - -Ensure that the dependencies used by ``etc/scripts/fetch_thirdparty.py`` are installed: - -.. code-block:: bash - - pip install -r etc/scripts/requirements.txt - -Once we have requirements.txt and requirements-dev.txt, we can fetch the project -dependencies as wheels and generate ABOUT files for them: - -.. code-block:: bash - - python etc/scripts/fetch_thirdparty.py -r requirements.txt -r requirements-dev.txt - -There may be issues with the generated ABOUT files, which will have to be -corrected. You can check to see if your corrections are valid by running: - -.. code-block:: bash - - python etc/scripts/check_thirdparty.py -d thirdparty - -Once the wheels are collected and the ABOUT files are generated and correct, -upload them to thirdparty.aboutcode.org/pypi by placing the wheels and ABOUT -files from the thirdparty directory to the pypi directory at -https://github.com/nexB/thirdparty-packages - - -Usage after project initialization ----------------------------------- - -Once the ``requirements.txt`` and ``requirements-dev.txt`` have been generated -and the project dependencies and their ABOUT files have been uploaded to -thirdparty.aboutcode.org/pypi, you can configure the project as needed, typically -when you update dependencies or use a new checkout. - -If the virtual env for the project becomes polluted, or you would like to remove -it, use the ``--clean`` option: - -.. code-block:: bash - - ./configure --clean - -Then you can run ``./configure`` again to set up the project virtual environment. - -To set up the project for development use: - -.. code-block:: bash - - ./configure --dev - -To update the project dependencies (adding, removing, updating packages, etc.), -update the dependencies in ``setup.cfg``, then run: - -.. code-block:: bash - - ./configure --clean # Remove existing virtual environment - source venv/bin/activate # Ensure virtual environment is activated - python etc/scripts/gen_requirements.py -s venv/lib/python/site-packages/ # Regenerate requirements.txt - python etc/scripts/gen_requirements_dev.py -s venv/lib/python/site-packages/ # Regenerate requirements-dev.txt - pip install -r etc/scripts/requirements.txt # Install dependencies needed by etc/scripts/bootstrap.py - python etc/scripts/fetch_thirdparty.py -r requirements.txt -r requirements-dev.txt # Collect dependency wheels and their ABOUT files - -Ensure that the generated ABOUT files are valid, then take the dependency wheels -and ABOUT files and upload them to thirdparty.aboutcode.org/pypi. diff --git a/setup.cfg b/setup.cfg index 681b6fe7..12485874 100644 --- a/setup.cfg +++ b/setup.cfg @@ -72,7 +72,7 @@ testing = black mock -docs= - Sphinx==6.2.1 - sphinx-rtd-theme>=0.5.0 - doc8>=0.8.1 +docs = + Sphinx==5.0.2 + sphinx-rtd-theme==1.0.0 + doc8==0.11.2 \ No newline at end of file