diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 4b404d4a..f258e7ea 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.3","generation_timestamp":"2024-05-17T12:42:55","documenter_version":"1.4.1"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.3","generation_timestamp":"2024-05-17T13:41:19","documenter_version":"1.4.1"}} \ No newline at end of file diff --git a/dev/10-full-guide/index.html b/dev/10-full-guide/index.html index cd3f3f4f..3c9ac57c 100644 --- a/dev/10-full-guide/index.html +++ b/dev/10-full-guide/index.html @@ -8,4 +8,4 @@ pre-commit install # Future commits can't be directly to main unless you use -n

It is common to have some pre-commit issues due to your package's name length triggering JuliaFormatter.

Create a repo on GitHub and push your code to it.

Info

The actions will run and you will see errors in the documentation and linting. Do not despair.

Documentation

Go to your package setting on Github and find the "Actions" tab, the "General" link. On that page, find the "Workflow permissions" and change the selection to "Read and write permissions", and enable "Allow GitHub Actions can create and approve pull requests". This will allow the documentation workflow to work for development.

Go to the Actions page, click the failing Docs workflow and click on "re-run all jobs". It should pass now.

Now, go to your package setting on GitHub and find the "Pages" link. You should see an option to set the Source to "Deploy from a branch", and select the branch to be "gh-pages" and to deploy from the "/ (root)".

After circa 1 minute, you can check that the documentation was built properly.

Info

At this point, you should have passing workflows.

  1. Tests should have been passing from the start.
  2. Lint was fixed when we pushed the code to GitHub.
  3. Docs was fixed with the permissions change.

You will still need to set a DOCUMENTER_KEY to build the documentation from the tags automatically when using TagBot (which we do by default). Do the following:

pkg> activate --temp
 pkg> add DocumenterTools
 julia> using DocumenterTools
-julia> DocumenterTools.genkeys(user="UserName", repo="PackageName.jl")

Follow the instruction in the terminal.

Add key for Copier.yml workflow (or delete it)

You can reapply the template in the future. This is normally a manual job, specially because normally there are conflicts. That being said, we are experimenting with having a workflow that automatically checks whether there are updates to the template and reapplies it. A Pull Request is created with the result.

Warning

This is optional, and in development, so you might want to delete the workflow instead.

If you decide to use, here are the steps to set it up:

  1. Create a Personal Access Token to be used by the Copier workflow.
  2. Go to https://github.com/settings/tokens.
  3. Create a token with "Content", "Pull-request", and "Workflows" permissions.
  4. Copy the Token.
  5. Go to your YOUR_PACKAGE_URL/settings/secrets/actions.
  6. Create a "New repository secret" named COPIER_PAT.

CITATION.cff and Zenodo deposition

Update your CITATION.cff file with correct information. You can use cffinit to generate it easily.

Before releasing, enable Zenodo integration at https://zenodo.org/account/settings/github/ to automatically generate a deposition of your package, i.e., archive a version on Zenodo and generate a DOI.

Update README.md

  1. Update the badges
  2. Add a description

Enable discussions

Enable GitHub discussions.

+julia> DocumenterTools.genkeys(user="UserName", repo="PackageName.jl")

Follow the instruction in the terminal.

Add key for Copier.yml workflow (or delete it)

You can reapply the template in the future. This is normally a manual job, specially because normally there are conflicts. That being said, we are experimenting with having a workflow that automatically checks whether there are updates to the template and reapplies it. A Pull Request is created with the result.

Warning

This is optional, and in development, so you might want to delete the workflow instead.

If you decide to use, here are the steps to set it up:

  1. Create a Personal Access Token to be used by the Copier workflow.
  2. Go to https://github.com/settings/tokens.
  3. Create a token with "Content", "Pull-request", and "Workflows" permissions.
  4. Copy the Token.
  5. Go to your YOUR_PACKAGE_URL/settings/secrets/actions.
  6. Create a "New repository secret" named COPIER_PAT.

CITATION.cff and Zenodo deposition

Update your CITATION.cff file with correct information. You can use cffinit to generate it easily.

Before releasing, enable Zenodo integration at https://zenodo.org/account/settings/github/ to automatically generate a deposition of your package, i.e., archive a version on Zenodo and generate a DOI.

Update README.md

  1. Update the badges
  2. Add a description

Enable discussions

Enable GitHub discussions.

diff --git a/dev/20-explanation/index.html b/dev/20-explanation/index.html index cc8e3b3b..3b038d8a 100644 --- a/dev/20-explanation/index.html +++ b/dev/20-explanation/index.html @@ -1,2 +1,2 @@ -Explanation · COPIERTemplate.jl

Explanation

In this section, we hope to explain the motivation for the package, and what is inside the template. Some things might have been obvious when creating the package and not at the moment, so feel free to create issues to ask, or suggest, clarifications.

The engine, the project generator, and the template

Let me start by marking some names clearer.

  • The template is the collection of files and folders written with some placeholders. For instance, the link to a GitHub project will be something like https://github.com/{{ PackageOwner }}/{{ PackageName }}.jl.
  • The engine is the tool that converts the template into the end result, by changing the placeholders into the actual values that we want.
  • The project generator is the tool that interacts with the user to get the placeholder values and give to the engine.

Comparison with existing solutions

Julia has a very good package generator called PkgTemplates.jl, so why did we create another one?

The short answer is that we want a more streamlined development experience, a template more focused on best practices, and the ability to keep reusing the template whenever new tools and ideas are implemented.

In more details, first, see the differences in the parts of the project in the table below:

COPIERTemplate.jlPkgTemplates.jl
TemplatePart of the packagePart of the package
EngineJinjaMustache
Project generatorcopier, with some wrappers in the packagePart of the package

Now, we can split this into three comparisons.

Template differences

The template differences are mostly due to opinion and contributions and it should be easy to translate files from one template to the other. We are heavily inspired by PkgTemplates.jl, as we used it for many years, but we made some changes in the hopes of improving software sustainability, package maintainability and code quality (which we just overtly simplify as best practices). As such, our current differences (as of the time of writing) are:

  • We have more best practices tools, such as pre-commit, configuration for linters and formatters for Julia, Markdown, TOML, YAML and JSON, CITATION.cff, Lint GitHub workflow, .editorconfig file, issues and pull requests templates, etc.
  • We focus on the main use cases (GitHub and GitHub actions), so we have much less options.

Engine differences

We can't say much about these, since we don't know or care in details.

Project generator differences

PkgTemplates.jl is a project generator. This means that if you want to programmatically create templates inside Julia, this is the best solution. The questions (user interface) are implemented by the package, which then translates that into the answers for the engine. Disclaimer: We haven't worked on the package, this information is based on the docs.

We use copier as project generator. It is an external Python tool, so we also include some wrappers in the package to use it from Julia without the need to explicitly install it. Copier has many features, so we recommend that you check their comparisons pages for more information.

Most notably, the feature that made us choose copier in the first place has the ability to applied and reapplied to existing projects. This means that existing packages can benefit from all best practices that we provide. Furthermore, they can keep reaping benefits when we create new versions of the template.

Template details

Let's dive into the details of the template now.

Basic package structure

This is the basic structure of a package:

  • PackageName.jl/
    • src/
      • PackageName.jl
    • test/
      • Project.toml
      • runtests.jl
    • LICENSE.md
    • Project.toml
    • README.md

With the exception of test/Project.toml, all other files are requirements to register a package.

Documentation

On top of the basic structure, we add some Documenter.jl structure.

  • docs/
    • src/
      • 90-contributing.md
      • 90-developer.md
      • 90-reference.md
      • index.md
    • make.jl
    • Project.toml

Brief explanation of the details:

  • The Project.toml, make.jl and src/index.md are the basic structure.
  • docs/src/90-contributing.md: Sometimes added as CONTRIBUTING.md, it explains how contributors can get involved in the project.
  • docs/src/90-developer.md: Sometimes added as README.dev.md or DEVELOPER.md, it explains how to setup your local environment and other information relevant for developers only.
  • docs/src/90-reference.md is the API reference page, which include an @autodocs.

One noteworthy aspect of our make.jl, is that we include some code to automatically generate the list of pages. Create a file in the form ##-name.md, where ## is a two-digit number, and it will be automatically added to the pages list.

Linting and Formatting

The most important file related to linting and formatting is .pre-commit-config.yaml, which is the configuration for pre-commit. It defines a list of linters and formatters for Julia, Markdown, TOML, YAML, and JSON.

It requires installing pre-commit (I recommend installing it globally with pipx). Installing pre-commit (pre-commit install) will make sure that it runs the relevant hooks before committing. Furthermore, if you run pre-commit run -a, it runs all hooks.

Some hooks in the .pre-commit-config.yaml file have configuration files of their own: .JuliaFormatter.toml, .markdownlint.json, .markdown-link-config.json, and .yamllint.yml.

Also slightly related, is the .editorconfig file, which tells your editor, if you install the correct plugin, how to format some things.

GitHub Workflows

We have a few workflows, with plans to expand in the future:

  • CompatHelper.yml: Should be well known by now. It checks that your Project.toml compat entries are up-to-date.
  • Copier.yml: This will periodically check the template for updates. If there are updates, this action creates a pull request updating your repo.
  • Docs.yml: Build the docs. Only runs when relevant files change.
  • Lint.yml: Run the linter and formatter through the command pre-commit run -a.
  • TagBot.yml: Create GitHub releases automatically after your new release is merged on the Registry.
  • Test.yml: Run the tests.

Issues and PR templates

We include issues and PR templates for GitHub (see .github/). These provide a starting point to your project management.

Other files

  • .cirrus.yml: For Cirrus CI, which we use solely for FreeBSD testing.
  • CITATION.cff: Instead of the more classic .bib, we use .cff, which serves a better purpose of providing the metadata of the package. CFF files have been adopted by GitHub, so you can generate a BibTeX entry by clicking on "Cite this repository" on the repository's main page. CFF files have also been adopted by Zenodo to provide the metadata of your deposition.
  • CODE_OF_CONDUCT.md: A code of conduct file from Contributor Covenant.
+Explanation · COPIERTemplate.jl

Explanation

In this section, we hope to explain the motivation for the package, and what is inside the template. Some things might have been obvious when creating the package and not at the moment, so feel free to create issues to ask, or suggest, clarifications.

The engine, the project generator, and the template

Let me start by marking some names clearer.

  • The template is the collection of files and folders written with some placeholders. For instance, the link to a GitHub project will be something like https://github.com/{{ PackageOwner }}/{{ PackageName }}.jl.
  • The engine is the tool that converts the template into the end result, by changing the placeholders into the actual values that we want.
  • The project generator is the tool that interacts with the user to get the placeholder values and give to the engine.

Comparison with existing solutions

Julia has a very good package generator called PkgTemplates.jl, so why did we create another one?

The short answer is that we want a more streamlined development experience, a template more focused on best practices, and the ability to keep reusing the template whenever new tools and ideas are implemented.

In more details, first, see the differences in the parts of the project in the table below:

COPIERTemplate.jlPkgTemplates.jl
TemplatePart of the packagePart of the package
EngineJinjaMustache
Project generatorcopier, with some wrappers in the packagePart of the package

Now, we can split this into three comparisons.

Template differences

The template differences are mostly due to opinion and contributions and it should be easy to translate files from one template to the other. We are heavily inspired by PkgTemplates.jl, as we used it for many years, but we made some changes in the hopes of improving software sustainability, package maintainability and code quality (which we just overtly simplify as best practices). As such, our current differences (as of the time of writing) are:

  • We have more best practices tools, such as pre-commit, configuration for linters and formatters for Julia, Markdown, TOML, YAML and JSON, CITATION.cff, Lint GitHub workflow, .editorconfig file, issues and pull requests templates, etc.
  • We focus on the main use cases (GitHub and GitHub actions), so we have much less options.

Engine differences

We can't say much about these, since we don't know or care in details.

Project generator differences

PkgTemplates.jl is a project generator. This means that if you want to programmatically create templates inside Julia, this is the best solution. The questions (user interface) are implemented by the package, which then translates that into the answers for the engine. Disclaimer: We haven't worked on the package, this information is based on the docs.

We use copier as project generator. It is an external Python tool, so we also include some wrappers in the package to use it from Julia without the need to explicitly install it. Copier has many features, so we recommend that you check their comparisons pages for more information.

Most notably, the feature that made us choose copier in the first place has the ability to applied and reapplied to existing projects. This means that existing packages can benefit from all best practices that we provide. Furthermore, they can keep reaping benefits when we create new versions of the template.

Template details

Let's dive into the details of the template now.

Basic package structure

This is the basic structure of a package:

  • PackageName.jl/
    • src/
      • PackageName.jl
    • test/
      • Project.toml
      • runtests.jl
    • LICENSE.md
    • Project.toml
    • README.md

With the exception of test/Project.toml, all other files are requirements to register a package.

Documentation

On top of the basic structure, we add some Documenter.jl structure.

  • docs/
    • src/
      • 90-contributing.md
      • 90-developer.md
      • 90-reference.md
      • index.md
    • make.jl
    • Project.toml

Brief explanation of the details:

  • The Project.toml, make.jl and src/index.md are the basic structure.
  • docs/src/90-contributing.md: Sometimes added as CONTRIBUTING.md, it explains how contributors can get involved in the project.
  • docs/src/90-developer.md: Sometimes added as README.dev.md or DEVELOPER.md, it explains how to setup your local environment and other information relevant for developers only.
  • docs/src/90-reference.md is the API reference page, which include an @autodocs.

One noteworthy aspect of our make.jl, is that we include some code to automatically generate the list of pages. Create a file in the form ##-name.md, where ## is a two-digit number, and it will be automatically added to the pages list.

Linting and Formatting

The most important file related to linting and formatting is .pre-commit-config.yaml, which is the configuration for pre-commit. It defines a list of linters and formatters for Julia, Markdown, TOML, YAML, and JSON.

It requires installing pre-commit (I recommend installing it globally with pipx). Installing pre-commit (pre-commit install) will make sure that it runs the relevant hooks before committing. Furthermore, if you run pre-commit run -a, it runs all hooks.

Some hooks in the .pre-commit-config.yaml file have configuration files of their own: .JuliaFormatter.toml, .markdownlint.json, .markdown-link-config.json, and .yamllint.yml.

Also slightly related, is the .editorconfig file, which tells your editor, if you install the correct plugin, how to format some things.

GitHub Workflows

We have a few workflows, with plans to expand in the future:

  • CompatHelper.yml: Should be well known by now. It checks that your Project.toml compat entries are up-to-date.
  • Copier.yml: This will periodically check the template for updates. If there are updates, this action creates a pull request updating your repo.
  • Docs.yml: Build the docs. Only runs when relevant files change.
  • Lint.yml: Run the linter and formatter through the command pre-commit run -a.
  • TagBot.yml: Create GitHub releases automatically after your new release is merged on the Registry.
  • Test.yml: Run the tests.

Issues and PR templates

We include issues and PR templates for GitHub (see .github/). These provide a starting point to your project management.

Other files

  • .cirrus.yml: For Cirrus CI, which we use solely for FreeBSD testing.
  • CITATION.cff: Instead of the more classic .bib, we use .cff, which serves a better purpose of providing the metadata of the package. CFF files have been adopted by GitHub, so you can generate a BibTeX entry by clicking on "Cite this repository" on the repository's main page. CFF files have also been adopted by Zenodo to provide the metadata of your deposition.
  • CODE_OF_CONDUCT.md: A code of conduct file from Contributor Covenant.
diff --git a/dev/90-contributing/index.html b/dev/90-contributing/index.html index 420dce73..63394148 100644 --- a/dev/90-contributing/index.html +++ b/dev/90-contributing/index.html @@ -1,2 +1,2 @@ -Contributing · COPIERTemplate.jl

Contributing guidelines

First of all, thanks for the interest!

We welcome all kinds of contribution, including, but not limited to code, documentation, examples, configuration, issue creating, etc.

Be polite and respectful and follow the code of conduct.

Bug reports and discussions

If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.

Working on an issue

If you found an issue that interests you, comment on that issue what your plans are. If the solution to the issue is clear, you can immediately create a pull request (see below). Otherwise, say what your proposed solution is and wait for a discussion around it.

Tip

Feel free to ping us after a few days if there are no responses.

If your solution involves code (or something that requires running the package locally), check the developer documentation. Otherwise, you can use the GitHub interface directly to create your pull request.

+Contributing · COPIERTemplate.jl

Contributing guidelines

First of all, thanks for the interest!

We welcome all kinds of contribution, including, but not limited to code, documentation, examples, configuration, issue creating, etc.

Be polite and respectful and follow the code of conduct.

Bug reports and discussions

If you think you found a bug, feel free to open an issue. Focused suggestions and requests can also be opened as issues. Before opening a pull request, start an issue or a discussion on the topic, please.

Working on an issue

If you found an issue that interests you, comment on that issue what your plans are. If the solution to the issue is clear, you can immediately create a pull request (see below). Otherwise, say what your proposed solution is and wait for a discussion around it.

Tip

Feel free to ping us after a few days if there are no responses.

If your solution involves code (or something that requires running the package locally), check the developer documentation. Otherwise, you can use the GitHub interface directly to create your pull request.

diff --git a/dev/90-developer/index.html b/dev/90-developer/index.html index 931d1963..6d908ee6 100644 --- a/dev/90-developer/index.html +++ b/dev/90-developer/index.html @@ -3,4 +3,4 @@ pipx install pre-commit

With pre-commit installed, activate it as a pre-commit hook:

pre-commit install

To run the linting and formatting manually, enter the command below:

pre-commit run -a

Now, you can only commit if all the pre-commit tests pass.

First time clone

If this is the first time you work with this repository, follow the instructions below to clone the repository.

  1. Fork this repo

  2. Clone your repo (this will create a git remote called origin)

  3. Add this repo as a remote:

    git remote add orgremote https://github.com/abelsiqueira/COPIERTemplate.jl

Working on a new issue

  1. Fetch from the JSO remote and fast-forward your local main

    git fetch orgremote
 git switch main
 git merge --ff-only orgremote/main

  2. Branch from main to address the issue (see below for naming)

    git switch -c 42-add-answer-universe

  3. Push the new local branch to your personal remote repository

    git push -u origin 42-add-answer-universe

  4. Create a pull request to merge your remote branch into the org main.

Branch naming

Commit message

Before creating a pull request

Tips

Testing local changes

To test you local modifications, you can run copier with the --vcs-ref HEAD flag and point to your local clone. This will use the latest changes, including uncommitted modifications (i.e., the dirty state). What I normally do is this:

cd $(mktemp -d) # Go to a tmp folder
-copier copy --vcs-ref HEAD /path/to/clone/ pkg # Clone dirty clone into pkg
+copier copy --vcs-ref HEAD /path/to/clone/ pkg # Clone dirty clone into pkg diff --git a/dev/90-reference/index.html b/dev/90-reference/index.html index 8a924b3c..6625e66e 100644 --- a/dev/90-reference/index.html +++ b/dev/90-reference/index.html @@ -1,2 +1,2 @@ -Reference · COPIERTemplate.jl

Reference

Contents

Index

COPIERTemplate.generate — Function
generate(path, generate_missing_uuid = true; kwargs...)

Runs the copy command of copier with the COPIERTemplate template. Even though the template is available offline through this template, this uses the github URL to allow updating.

The keyword arguments are passed directly to the run_copy function of copier. If generate_missing_uuid is true and there is no kwargs[:data]["PackageUUID"], then a UUID is generated for the package.

source
+Reference · COPIERTemplate.jl

Reference

Contents

Index

COPIERTemplate.generate — Function
generate(path, generate_missing_uuid = true; kwargs...)

Runs the copy command of copier with the COPIERTemplate template. Even though the template is available offline through this template, this uses the github URL to allow updating.

The keyword arguments are passed directly to the run_copy function of copier. If generate_missing_uuid is true and there is no kwargs[:data]["PackageUUID"], then a UUID is generated for the package.

source
diff --git a/dev/index.html b/dev/index.html index 1ae22761..0a1b33e1 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,3 +1,3 @@ Home · COPIERTemplate.jl

COPIERTemplate - Copier OPInionated Evolving Reusable Template

COPIERTemplate.jl

Welcome to the documentation of COPIERTemplate.jl. This package provides a template in the copier engine for a Julia package. Furthermore, it provides a wrapper around convenience calls to that package.

The main features of this package/template are:

  • It provides a curated (opinionated) list of tools and best practices for Julia package development;
  • It can be applied and reapplied to existing packages, allowing the updates in the template to be imported into the package;

Using

To fully benefit from the template, there are some steps to be done before and after you generate your package. Check the full guide for more details.

However, if you kinda know what you need to do, this is the TL;DR:

julia> using COPIERTemplate
-julia> COPIERTemplate.generate("YourPackage.jl")

Or, alternatively, using copier, run

copier copy https://github.com/abelsiqueira/COPIERTemplate.jl YourPackage.jl

I really recommend checking the full guide, though.

To understand more about our motivation and what the template provides, check the explanation page.

Getting and providing help

I hope you find this package useful. If you have any questions, requests, or comments, check the issues and discussion pages.

If you would like to get involved in the COPIERTemplate growth, please check our contributing guide. We welcome contributions of many types, including coding, reviewing, creating issues, creating tutorials, interacting with users, etc. Make sure to follow our code of conduct.

If your interest is in developing the package, check the development guide as well.

+julia> COPIERTemplate.generate("YourPackage.jl")

Or, alternatively, using copier, run

copier copy https://github.com/abelsiqueira/COPIERTemplate.jl YourPackage.jl

I really recommend checking the full guide, though.

To understand more about our motivation and what the template provides, check the explanation page.

Getting and providing help

I hope you find this package useful. If you have any questions, requests, or comments, check the issues and discussion pages.

If you would like to get involved in the COPIERTemplate growth, please check our contributing guide. We welcome contributions of many types, including coding, reviewing, creating issues, creating tutorials, interacting with users, etc. Make sure to follow our code of conduct.

If your interest is in developing the package, check the development guide as well.