calcipy is a Python package that implements best practices such as code style (linting, auto-fixes), documentation, CI/CD, and logging. Like the calcium carbonate in hard coral, packages can be built on the calcipy foundation.
calcipy
calcipy has some configurability, but is tailored for my particular use cases. If you want the same sort of functionality, there are a number of alternatives to consider:
Calcipy needs a few static files managed using copier and a template project: kyleking/calcipy_template
You can quickly use the template to create a new project or add calcipy to an existing one:
# Install copier. pipx is recommended\npipx install copier\n\n# To create a new project\ncopier copy gh:KyleKing/calcipy_template new_project\ncd new_project\n\n# Or convert/update an existing one\ncd my_project\ncopier copy gh:KyleKing/calcipy_template .\ncopier update\n
Additionally, calcipy can be run as a CLI application without adding the package as a dependency.
Quick Start:
# For the CLI, only install a few of the extras which can be used from a few different CLI commands\npipx install 'calcipy[flake8,lint,tags]'\n\n# Use 'tags' to create a CODE_TAG_SUMMARY of the specified directory\ncalcipy-tags tags --help\ncalcipy-tags tags --base-dir=~/path/to/my_project\n\n# You can list all provided CLI commands with\npipx list\n
venvs are in ~/.local/pipx/venvs\napps are exposed on your $PATH at ~/.local/bin\n package calcipy 1.4.0, installed using Python 3.11.4\n - calcipy\n - calcipy-lint\n - calcipy-pack\n - calcipy-tags\n - calcipy-types\n
Note: the CLI output below is compressed for readability, but you can try running each of these commands locally to see the most up-to-date documentation and the full set of options. The \u201cUsage\u201d, \u201cCore options\u201d, and \u201cGlobal Task Options\u201d are the same for each subsequent command, so they are excluded for brevity.
> calcipy-lint\nUsage: calcipy-lint [--core-opts] <subcommand> [--subcommand-opts] ...\n\nCore options:\n\n --complete Print tab-completion candidates for given parse remainder.\n --hide=STRING Set default value of run()'s 'hide' kwarg.\n --print-completion-script=STRING Print the tab-completion script for your preferred shell (bash|zsh|fish).\n --prompt-for-sudo-password Prompt user at start of session for the sudo.password config value.\n --write-pyc Enable creation of .pyc files.\n -d, --debug Enable debug output.\n -D INT, --list-depth=INT When listing tasks, only show the first INT levels.\n -e, --echo Echo executed commands before running.\n -f STRING, --config=STRING Runtime configuration file to use.\n -F STRING, --list-format=STRING Change the display format used when listing tasks. Should be one of: flat (default), nested, json.\n -h [STRING], --help[=STRING] Show core or per-task help and exit.\n -l [STRING], --list[=STRING] List available tasks, optionally limited to a namespace.\n -p, --pty Use a pty when executing shell commands.\n -R, --dry Echo commands instead of running.\n -T INT, --command-timeout=INT Specify a global command execution timeout, in seconds.\n -V, --version Show version and exit.\n -w, --warn-only Warn, instead of failing, when shell commands fail.\n\nSubcommands:\n\n lint.autopep8 Run autopep8.\n lint.check (lint) Run ruff as check-only.\n lint.fix Run ruff and apply fixes.\n lint.flake8 Run flake8.\n lint.pre-commit Run pre-commit.\n lint.pylint Run pylint.\n lint.security Attempt to identify possible security vulnerabilities.\n lint.watch Run ruff as check-only.\n\nGlobal Task Options:\n\n *file_args List of Paths available globally to all tasks. Will resolve paths with working_dir\n --keep-going Continue running tasks even on failure\n --working_dir=STRING Set the cwd for the program. Example: \"../run --working-dir .. lint test\"\n -v,-vv,-vvv Globally configure logger verbosity (-vvv for most verbose)\n\n> calcipy-pack\n\nSubcommands:\n\n pack.check-licenses Check licenses for compatibility with `licensecheck`.\n pack.install-extras Run poetry install with all extras.\n pack.lock Ensure poetry.lock is up-to-date.\n pack.publish Build the distributed format(s) and publish.\n\n> calcipy-tags\n\nSubcommands:\n\n tags.collect-code-tags (tags) Create a `CODE_TAG_SUMMARY.md` with a table for TODO- and FIXME-style code comments.\n\n> calcipy-types\n\nSubcommands:\n\n types.mypy Run mypy.\n types.pyright Run pyright.\n
calcipy can also be used as a pre-commit task by adding the below snippet to your pre-commit file:
pre-commit
repos:\n - repo: https://github.com/KyleKing/calcipy\n rev: main\n hooks:\n - id: tags\n - id: lint-fix\n - id: types\n
See the Open Issues and/or the CODE_TAG_SUMMARY. For release history, see the CHANGELOG.
Open Issues
We welcome pull requests! For your pull request to be accepted smoothly, we suggest that you first open a GitHub issue to discuss your idea. For resources on getting started with the code base, see the below documentation:
We follow the Contributor Covenant Code of Conduct.
We try to reasonably meet most aspects of the \u201cOpenSSF scorecard\u201d from Open Source Insights
If you have any security issue to report, please contact the project maintainers privately. You can reach us at dev.act.kyle@gmail.com.
LICENSE
nest invoke-specific code in tasks/
drop pytest-html and conftest completely
use sqlite3 for pre-commit concurrent doit access
bump Python to 3.8.4 and drop 3.7
Found code tags for TODO (6), PLANNED (1)
git clone https://github.com/kyleking/calcipy.git\ncd calcipy\npoetry install --sync\npoetry run calcipy-pack pack.install-extras\n\n# See the available tasks\npoetry run calcipy\n# Or use a local 'run' file (so that 'calcipy' can be extended)\n./run\n\n# Run the default task list (lint, auto-format, test coverage, etc.)\n./run main\n\n# Make code changes and run specific tasks as needed:\n./run lint.fix test\n
For testing, create an account on TestPyPi. Replace ... with the API token generated on TestPyPi or PyPi respectively
...
poetry config repositories.testpypi https://test.pypi.org/legacy/\npoetry config pypi-token.testpypi ...\n\n./run main pack.publish --to-test-pypi\n# If you didn't configure a token, you will need to provide your username and password to publish\n
To publish to the real PyPi
poetry config pypi-token.pypi ...\n./run release\n\n# Or for a pre-release\n./run release --suffix=rc\n
calcipy/__init__.py
calcipy/can_skip.py
calcipy/check_for_stale_packages/__init__.py
calcipy/check_for_stale_packages/_check_for_stale_packages.py
calcipy/cli.py
calcipy/code_tag_collector/__init__.py
calcipy/code_tag_collector/_collector.py
calcipy/dot_dict/__init__.py
calcipy/dot_dict/_dot_dict.py
calcipy/experiments/__init__.py
calcipy/experiments/bump_programmatically.py
calcipy/experiments/check_duplicate_test_names.py
calcipy/file_search.py
calcipy/invoke_helpers.py
calcipy/md_writer/__init__.py
calcipy/md_writer/_writer.py
calcipy/noxfile/__init__.py
calcipy/noxfile/_noxfile.py
calcipy/scripts.py
calcipy/tasks/__init__.py
calcipy/tasks/_invoke.py
calcipy/tasks/all_tasks.py
calcipy/tasks/cl.py
calcipy/tasks/defaults.py
calcipy/tasks/doc.py
calcipy/tasks/executable_utils.py
calcipy/tasks/lint.py
calcipy/tasks/nox.py
calcipy/tasks/pack.py
calcipy/tasks/stale.py
calcipy/tasks/tags.py
calcipy/tasks/test.py
calcipy/tasks/types.py
Generated on: 2023-11-08
calcipy 1.0.0
calcipy v1 was a complete rewrite to switch from doit to invoke:
v1
doit
invoke
dodo.py
doit excelled at clearly delineated task output and run summary, but invoke isn\u2019t designed that way. I would like to improve the CLI output, but the benefits are worth this tradeoff.
calcipy v0 was built on doit and thus required a dodo.py file. I began adding cement to support a separate CLI for calcipy installed with pipx, but that required a lot of boilerplate code. With doit, the string command needed to be complete at task evaluation rather than runtime, so globbing files couldn\u2019t be resolved lazily.
v0
cement
pipx
While refactoring, the global configuration was mostly removed (DoitGlobals) along with a few tasks, but the main functionality is still present. Any project dependent on calcipy will need substantial changes. The easiest way to start migrating is to run copier copy gh:KyleKing/calcipy_template . for calcipy_template
DoitGlobals
copier copy gh:KyleKing/calcipy_template .
It turns out that switching to invoke appears to have only saved 100ms
> hyperfine -m 20 --warmup 5 \"poetry run python -c 'print(1)'\"\nBenchmark 1: poetry run python -c 'print(1)'\nTime (mean \u00b1 \u03c3): 377.9 ms \u00b1 3.1 ms [User: 235.0 ms, System: 61.8 ms]\nRange (min \u2026 max): 372.7 ms \u2026 384.0 ms 20 runs\n> hyperfine -m 20 --warmup 5 ./run\nBenchmark 1: ./run\nTime (mean \u00b1 \u03c3): 936.0 ms \u00b1 26.9 ms [User: 1548.2 ms, System: 1687.7 ms]\nRange (min \u2026 max): 896.4 ms \u2026 1009.4 ms 20 runs\n> hyperfine -m 20 --warmup 5 \"poetry run calcipy_tags\"\nBenchmark 1: poetry run calcipy_tags\nTime (mean \u00b1 \u03c3): 618.5 ms \u00b1 29.7 ms [User: 1536.8 ms, System: 1066.2 ms]\nRange (min \u2026 max): 578.2 ms \u2026 694.9 ms 20 runs\n> hyperfine -m 20 --warmup 5 \"poetry run doit list\"\nBenchmark 1: poetry run doit list\nTime (mean \u00b1 \u03c3): 1.002 s \u00b1 0.015 s [User: 1.643 s, System: 1.682 s]\nRange (min \u2026 max): 0.974 s \u2026 1.023 s 20 runs\n
Additionally, the major decrease in dependencies will make install and update actions much faster. With the recommended extras installed, calcipy-v1 has 124 dependencies (with all extras, 164) vs. calcipy-v0\u2019s 259. Counted with: cat .calcipy_packaging.lock | jq 'keys' | wc -l
calcipy-v1
calcipy-v0
cat .calcipy_packaging.lock | jq 'keys' | wc -l
Accounting for code extracted to corallium, the overall number of lines decreased from 1772 to 1550 or only 12%, while increasing the CLI and pre-commit capabilities.
corallium
~/calcipy-v0 > cloc calcipy\n-------------------------------------------------------------------------------\nLanguage files blank comment code\n-------------------------------------------------------------------------------\nPython 26 942 1075 1772\n-------------------------------------------------------------------------------\nSUM: 26 942 1075 1772\n-------------------------------------------------------------------------------\n~/calcipy > cloc calcipy\n-------------------------------------------------------------------------------\nLanguage files blank comment code\n-------------------------------------------------------------------------------\nPython 27 454 438 1185\n-------------------------------------------------------------------------------\nSUM: 27 454 438 1185\n-------------------------------------------------------------------------------\n~/corallium > cloc corallium\n-------------------------------------------------------------------------------\nLanguage files blank comment code\n-------------------------------------------------------------------------------\nPython 7 176 149 365\n-------------------------------------------------------------------------------\nSUM: 7 176 149 365\n-------------------------------------------------------------------------------\n\n~/calcipy > cloc tests\n-------------------------------------------------------------------------------\nLanguage files blank comment code\n-------------------------------------------------------------------------------\nYAML 2 0 0 580\nPython 19 176 68 578\nJSON 2 0 0 60\nMarkdown 3 9 10 8\nText 1 0 0 2\n-------------------------------------------------------------------------------\nSUM: 27 185 78 1228\n-------------------------------------------------------------------------------\n~/calcipy-v0 > cloc tests\n-------------------------------------------------------------------------------\nLanguage files blank comment code\n-------------------------------------------------------------------------------\nJSON 30 0 0 762\nYAML 2 0 0 580\nPython 24 314 186 578\nMarkdown 3 9 10 8\n-------------------------------------------------------------------------------\nSUM: 59 323 196 1928\n-------------------------------------------------------------------------------\n~/corallium > cloc tests\n-------------------------------------------------------------------------------\nLanguage files blank comment code\n-------------------------------------------------------------------------------\nPython 6 36 15 69\nMarkdown 1 1 0 2\n-------------------------------------------------------------------------------\nSUM: 7 37 15 71\n-------------------------------------------------------------------------------\n
I would like to restore the doit task summary, but invoke\u2019s architecture doesn\u2019t really make this possible. The --continue option was extremely useful, but that also might not be achievable.
--continue
> poetry run doit run\n. format_recipes > [\n Python: function format_recipes\n]\n\n2023-02-19 10:40:23.954 | INFO | recipes.formatter:_write_toc:287 - Creating TOC for: ./recipes/docs/breakfast\n2023-02-19 10:40:23.957 | INFO | recipes.formatter:_write_toc:287 - Creating TOC for: ./recipes/docs/rice\n2023-02-19 10:40:23.959 | INFO | recipes.formatter:_write_toc:287 - Creating TOC for: ./recipes/docs/meals\n2023-02-19 10:40:23.964 | INFO | recipes.formatter:_write_toc:287 - Creating TOC for: ./recipes/docs/seafood\n2023-02-19 10:40:23.967 | INFO | recipes.formatter:_write_toc:287 - Creating TOC for: ./recipes/docs/pizza\n2023-02-19 10:40:23.969 | INFO | recipes.formatter:_write_toc:287 - Creating TOC for: ./recipes/docs/poultry\n2023-02-19 10:40:23.972 | INFO | recipes.formatter:_write_toc:287 - Creating TOC for: ./recipes/docs/sushi\n. collect_code_tags > [\n Python: function write_code_tag_file\n]\n\n. cl_write > [\n Cmd: poetry run cz changelog\n Python: function _move_cl\n]\n\n. lock > [\n Cmd: poetry lock --no-update\n]\n\nResolving dependencies...\n. nox_coverage > [\n Cmd: poetry run nox --error-on-missing-interpreters --session coverage\n]\n\n...\n\ndoit> Summary:\ndoit> format_recipes was successful\ndoit> collect_code_tags was successful\ndoit> cl_write was successful\ndoit> lock was successful\ndoit> nox_coverage was successful\ndoit> auto_format was successful\ndoit> document was successful\ndoit> check_for_stale_packages was successful\ndoit> pre_commit_hooks failed (red)\ndoit> lint_project was not run\ndoit> static_checks was not run\ndoit> security_checks was not run\ndoit> check_types was not run\n
We use Commitizen to manage both an auto-generated Changelog and incrementing the release version following semver. For both of these automated outputs to work well, please follow the Conventional Commits style, which is described in more detail below.
type(scope): description
!
refactor!: drop support for Node 6
fix : PATCH // feat : MINOR // BREAKING CHANGE : MAJOR
build(poetry): bump requests to v3
style(#32): add missing type annotations
git rebase -i
fix(roles): bug in admin role permissions
feat(ui): implement new button design
build(pip): upgrade package to remove vulnerabilities
refactor: file structure to improve code readability
perf(cli): rewrite methods
feat(api): endpoints to implement new customer dashboard
Personal Guide
<\u2013 Links \u2013>
Auto-generated with pylint-pyreverse
pylint-pyreverse
Full Size
calcipy.
configure_runtime_type_checking_mode()\n
Optionally configure runtime type checking mode globally.
def configure_runtime_type_checking_mode() -> None: # pragma: no cover\n \"\"\"Optionally configure runtime type checking mode globally.\"\"\"\n rtc_mode = _RuntimeTypeCheckingModes.from_environment()\n\n if rtc_mode is not _RuntimeTypeCheckingModes.OFF:\n from beartype.roar import BeartypeClawDecorWarning # noqa: PLC0415\n\n beartype_this_package(conf=BeartypeConf(\n warning_cls_on_decorator_exception=(\n None if rtc_mode is _RuntimeTypeCheckingModes.ERROR else BeartypeClawDecorWarning\n ),\n ))\n
Support can-skip logic from Make.
can_skip(*, prerequisites, targets)\n
Generic make-style task skipping logic based on file mtime.
mtime
Example use with Invoke, but can be used anywhere:
@task\ndef test(ctx: Context) -> None:\n if can_skip(prerequisites=[*Path('src').rglob('*.py')], targets=[Path('.coverage.xml')]):\n return # Exit early\n\n ... # Task code\n
@beartype\ndef can_skip(*, prerequisites: List[Path], targets: List[Path]) -> bool:\n \"\"\"Generic make-style task skipping logic based on file `mtime`.\n\n Example use with Invoke, but can be used anywhere:\n\n ```py\n @task\n def test(ctx: Context) -> None:\n if can_skip(prerequisites=[*Path('src').rglob('*.py')], targets=[Path('.coverage.xml')]):\n return # Exit early\n\n ... # Task code\n ```\n\n \"\"\"\n if not (ts_prerequisites := [pth.stat().st_mtime for pth in prerequisites]):\n raise ValueError('Required files do not exist', prerequisites)\n\n ts_targets = [pth.stat().st_mtime for pth in targets]\n if ts_targets and min(ts_targets) > max(ts_prerequisites):\n logger.warning('Skipping because targets are newer', targets=targets)\n return True\n return False\n
dont_skip(*, prerequisites, targets)\n
To use for testing with mock; always returns False.
@beartype\ndef dont_skip(*, prerequisites: List[Path], targets: List[Path]) -> bool:\n \"\"\"To use for testing with mock; always returns False.\"\"\"\n logger.debug('Mocking can_skip', prerequisites=prerequisites, targets=targets)\n return False\n
Extend Invoke for Calcipy.
Bases: Config
Config
Opinionated Config with better defaults.
class CalcipyConfig(Config):\n \"\"\"Opinionated Config with better defaults.\"\"\"\n\n @staticmethod\n def global_defaults() -> Dict: # type: ignore[type-arg] # pragma: no cover\n \"\"\"Override the global defaults.\"\"\"\n invoke_defaults = Config.global_defaults()\n calcipy_defaults = {\n 'run': {\n 'echo': True,\n 'echo_format': '\\033[2;3;37mRunning: {command}\\033[0m',\n 'pty': use_pty(),\n },\n }\n return merge_dicts(invoke_defaults, calcipy_defaults)\n
staticmethod
global_defaults()\n
Override the global defaults.
@staticmethod\ndef global_defaults() -> Dict: # type: ignore[type-arg] # pragma: no cover\n \"\"\"Override the global defaults.\"\"\"\n invoke_defaults = Config.global_defaults()\n calcipy_defaults = {\n 'run': {\n 'echo': True,\n 'echo_format': '\\033[2;3;37mRunning: {command}\\033[0m',\n 'pty': use_pty(),\n },\n }\n return merge_dicts(invoke_defaults, calcipy_defaults)\n
start_program(pkg_name, pkg_version, module=None, collection=None)\n
Run the customized Invoke Program.
FYI: recommendation is to extend the core_args method, but this won\u2019t parse positional arguments: https://docs.pyinvoke.org/en/stable/concepts/library.html#modifying-core-parser-arguments
core_args
@beartype\ndef start_program( # noqa: CAC001\n pkg_name: str,\n pkg_version: str,\n module: Optional[ModuleType] = None,\n collection: Optional[Union[Collection, InvokeCollection]] = None,\n) -> None: # pragma: no cover\n \"\"\"Run the customized Invoke Program.\n\n FYI: recommendation is to extend the `core_args` method, but this won't parse positional arguments:\n https://docs.pyinvoke.org/en/stable/concepts/library.html#modifying-core-parser-arguments\n\n \"\"\"\n # Manipulate 'sys.argv' to hide arguments that invoke can't parse\n _gto = GlobalTaskOptions()\n sys_argv: List[str] = sys.argv[:1]\n last_argv = ''\n for argv in sys.argv[1:]:\n if not last_argv.startswith('-') and Path(argv).is_file():\n _gto.file_args.append(Path(argv))\n # Check for CLI flags\n elif argv in {'-v', '-vv', '-vvv', '--verbose'}:\n _gto.verbose = argv.count('v')\n elif argv == '--keep-going':\n _gto.keep_going = True\n # Check for CLI arguments with values\n elif last_argv == '--working-dir':\n _gto.working_dir = Path(argv).resolve()\n elif argv != '--working-dir':\n sys_argv.append(argv)\n last_argv = argv\n _gto.file_args = [\n _f if _f.is_absolute() else Path.cwd() / _f\n for _f in _gto.file_args\n ]\n sys.argv = sys_argv\n\n class _CalcipyConfig(CalcipyConfig):\n\n gto: GlobalTaskOptions = _gto\n\n if module and collection:\n raise ValueError('Only one of collection or module can be specified')\n\n _CalcipyProgram(\n name=pkg_name,\n version=pkg_version,\n namespace=Collection.from_module(module) if module else collection,\n config_class=_CalcipyConfig,\n ).run()\n
task(*dec_args, **dec_kwargs)\n
Marks wrapped callable object as a valid Invoke task.
def task(*dec_args: Any, **dec_kwargs: Any) -> Callable: # type: ignore[type-arg]\n \"\"\"Marks wrapped callable object as a valid Invoke task.\"\"\"\n def wrapper(func: Any) -> Callable: # type: ignore[type-arg]\n # Attach arguments for Task\n setattr(func, TASK_ARGS_ATTR, dec_args)\n setattr(func, TASK_KWARGS_ATTR, dec_kwargs)\n # Attach public attributes from invoke that are expected\n func.help = dec_kwargs.pop('help', {})\n\n def _with_kwargs(**extra_kwargs: Any) -> Callable: # type: ignore[type-arg] # nosem\n \"\"\"Support building partial tasks.\"\"\"\n if extra_kwargs:\n # Set a unique name when 'extra_kwargs' was provided\n # https://github.com/pyinvoke/invoke/blob/07b836f2663bb073a7bcef3d6c454e1dc6b867ae/invoke/tasks.py#L81-L104\n encoded = b64encode(str(extra_kwargs).encode())\n func.__name__ = f'{func.__name__}_{encoded.decode().rstrip(\"=\")}'\n\n @wraps(func) # nosem\n def _with_kwargs_inner(*args: Any, **kwargs: Any) -> Any:\n return func(*args, **kwargs, **extra_kwargs)\n return _with_kwargs_inner\n\n func.with_kwargs = _with_kwargs\n\n @wraps(func) # nosem\n def _inner(*args: Any, **kwargs: Any) -> Any:\n return func(*args, **kwargs)\n\n return _inner\n\n # Handle the case when the decorator is called without arguments\n if (\n len(dec_args) == 1\n and callable(dec_args[0])\n and not hasattr(dec_args[0], TASK_ARGS_ATTR)\n ):\n return wrapper(dec_args[0])\n\n return wrapper\n
Find Files.
find_project_files(path_project, ignore_patterns)\n
Find project files in git version control.
Note: uses the relative project directory and verifies that each file exists
path_project
Path to the project directory
TYPE: Path
Path
ignore_patterns
glob ignore patterns
TYPE: List[str]
List[str]
List[Path]
Dict[str, List[Path]]: where keys are the suffix (without leading dot) and values the list of paths
@beartype\ndef find_project_files(path_project: Path, ignore_patterns: List[str]) -> List[Path]:\n \"\"\"Find project files in git version control.\n\n > Note: uses the relative project directory and verifies that each file exists\n\n Args:\n path_project: Path to the project directory\n ignore_patterns: glob ignore patterns\n\n Returns:\n Dict[str, List[Path]]: where keys are the suffix (without leading dot) and values the list of paths\n\n \"\"\"\n file_paths = []\n rel_filepaths = _get_all_files(cwd=path_project)\n filtered_rel_files = _filter_files(rel_filepaths=rel_filepaths, ignore_patterns=ignore_patterns)\n for rel_file in filtered_rel_files:\n path_file = path_project / rel_file\n if path_file.is_file():\n file_paths.append(path_file)\n else: # pragma: no cover\n logger.warning('Could not find the specified file', path_file=path_file)\n return file_paths\n
find_project_files_by_suffix(path_project, *, ignore_patterns=None)\n
TYPE: Optional[List[str]] DEFAULT: None
Optional[List[str]]
None
Dict[str, List[Path]]
@beartype\ndef find_project_files_by_suffix(\n path_project: Path, *, ignore_patterns: Optional[List[str]] = None,\n) -> Dict[str, List[Path]]:\n \"\"\"Find project files in git version control.\n\n > Note: uses the relative project directory and verifies that each file exists\n\n Args:\n path_project: Path to the project directory\n ignore_patterns: glob ignore patterns\n\n Returns:\n Dict[str, List[Path]]: where keys are the suffix (without leading dot) and values the list of paths\n\n \"\"\"\n file_lookup: Dict[str, List[Path]] = defaultdict(list)\n for path_file in find_project_files(path_project, ignore_patterns or []):\n file_lookup[path_file.suffix.lstrip('.')].append(path_file)\n return dict(file_lookup)\n
Invoke Helpers.
get_doc_subdir(path_project=None)\n
Retrieve the documentation directory from the copier answer file.
Path to the project directory with contains .copier-answers.yml
.copier-answers.yml
TYPE: Optional[Path] DEFAULT: None
Optional[Path]
to the source documentation directory
@beartype\ndef get_doc_subdir(path_project: Optional[Path] = None) -> Path:\n \"\"\"Retrieve the documentation directory from the copier answer file.\n\n Args:\n path_project: Path to the project directory with contains `.copier-answers.yml`\n\n Returns:\n Path: to the source documentation directory\n\n \"\"\"\n path_copier = (path_project or get_project_path()) / COPIER_ANSWERS\n doc_dir = read_yaml_file(path_copier).get('doc_dir', 'docs')\n return path_copier.parent / doc_dir / 'docs' # type: ignore[no-any-return]\n
cached
get_project_path()\n
Retrieve the cwd.
cwd
@lru_cache(maxsize=1)\ndef get_project_path() -> Path:\n \"\"\"Retrieve the `cwd`.\"\"\"\n return Path.cwd()\n
run(ctx, *run_args, **run_kwargs)\n
Wrap invoke.run to run within the working_dir.
working_dir
@beartype\ndef run(ctx: Context, *run_args: Any, **run_kwargs: Any) -> Optional[Result]:\n \"\"\"Wrap invoke.run to run within the `working_dir`.\"\"\"\n working_dir = '.'\n with suppress(AttributeError):\n working_dir = ctx.config.gto.working_dir\n\n with ctx.cd(working_dir):\n return ctx.run(*run_args, **run_kwargs)\n
use_pty()\n
Returns False on Windows and some CI environments.
@lru_cache(maxsize=1)\ndef use_pty() -> bool:\n \"\"\"Returns False on Windows and some CI environments.\"\"\"\n if platform.system() == 'Windows':\n return False\n return not environ.get('GITHUB_ACTION')\n
Start the command line program.
start()\n
def start() -> None: # pragma: no cover\n \"\"\"Run the customized Invoke Program.\"\"\"\n from .tasks import all_tasks # noqa: PLC0415\n start_program(__pkg_name__, __version__, all_tasks)\n
start_docs()\n
Run CLI with only the cl and doc namespaces.
def start_docs() -> None: # pragma: no cover\n \"\"\"Run CLI with only the cl and doc namespaces.\"\"\"\n from .tasks import cl, doc # noqa: PLC0415\n _start_subset([cl, doc])\n
start_lint()\n
Run CLI with only the lint namespace.
def start_lint() -> None: # pragma: no cover\n \"\"\"Run CLI with only the lint namespace.\"\"\"\n from .tasks import lint # noqa: PLC0415\n _start_subset([lint])\n
start_pack()\n
Run CLI with only the pack namespace.
def start_pack() -> None: # pragma: no cover\n \"\"\"Run CLI with only the pack namespace.\"\"\"\n from .tasks import pack # noqa: PLC0415\n _start_subset([pack])\n
start_tags()\n
Run CLI with only the tags namespace.
def start_tags() -> None: # pragma: no cover\n \"\"\"Run CLI with only the tags namespace.\"\"\"\n from .tasks import tags # noqa: PLC0415\n _start_subset([tags])\n
start_test()\n
Run CLI with only the test namespace.
def start_test() -> None: # pragma: no cover\n \"\"\"Run CLI with only the test namespace.\"\"\"\n from .tasks import test # noqa: PLC0415\n _start_subset([test])\n
start_types()\n
Run CLI with only the types namespace.
def start_types() -> None: # pragma: no cover\n \"\"\"Run CLI with only the types namespace.\"\"\"\n from .tasks import types # noqa: PLC0415\n _start_subset([types])\n
Check for stale packages.
module-attribute
CALCIPY_CACHE = Path('.calcipy_packaging.lock')\n
Path to the packaging lock file.
check_for_stale_packages(*, stale_months, path_lock=LOCK, path_cache=CALCIPY_CACHE)\n
Read the cached packaging information.
stale_months
cutoff in months for when a package might be stale enough to be a risk
TYPE: int
int
@beartype\ndef check_for_stale_packages(*, stale_months: int, path_lock: Path = LOCK, path_cache: Path = CALCIPY_CACHE) -> bool:\n \"\"\"Read the cached packaging information.\n\n Args:\n stale_months: cutoff in months for when a package might be stale enough to be a risk\n\n \"\"\"\n packages = _read_packages(path_lock)\n cached_packages = _read_cache(path_cache)\n if cached_packages and can_skip.can_skip(prerequisites=[path_lock], targets=[path_cache]):\n packages = [*cached_packages.values()]\n else:\n packages = _collect_release_dates(packages, cached_packages)\n _write_cache(packages, path_cache)\n return _packages_are_stale(packages, stale_months=stale_months)\n
Collect code tags and output for review in a single location.
CODE_TAG_RE = '((^|\\\\s|\\\\(|\"|\\\\\\')(?P<tag>{tag})(:| -)([^\\\\r\\\\n]))(?P<text>.+)'\n
Default code tag regex with tag and text matching groups.
tag
text
Requires formatting with list of tags: CODE_TAG_RE.format(tag='|'.join(tag_list))
CODE_TAG_RE.format(tag='|'.join(tag_list))
Commonly, the tag_list could be COMMON_CODE_TAGS
tag_list
COMMON_CODE_TAGS
COMMON_CODE_TAGS = ['FIXME', 'TODO', 'PLANNED', 'HACK', 'REVIEW', 'TBD', 'DEBUG']\n
Most common code tags.
FYI and NOTE are excluded to not be tracked in the Code Summary.
SKIP_PHRASE = 'calcipy_skip_tags'\n
String that indicates the file should be excluded from the tag search.
github_blame_url(clone_uri)\n
Format the blame URL.
clone_uri
git remote URI
TYPE: str
str
repo_url
@beartype\ndef github_blame_url(clone_uri: str) -> str:\n \"\"\"Format the blame URL.\n\n Args:\n clone_uri: git remote URI\n\n Returns:\n str: `repo_url`\n\n \"\"\"\n # Could be ssh or http (with or without .git)\n # > git@github.com:KyleKing/calcipy.git\n # > https://github.com/KyleKing/calcipy.git\n if matches := re.compile(_GITHUB_ORIGIN).match(clone_uri):\n github_url = 'https://github.com/'\n return f\"{github_url}{matches['owner']}/{matches['repository']}\"\n return ''\n
write_code_tag_file(\n path_tag_summary, paths_source, base_dir, regex=\"\", tags=\"\", header=\"# Task Summary\\n\\nAuto-Generated by `calcipy`\"\n)\n
Create the code tag summary file.
path_tag_summary
Path to the output file
paths_source
list of source files to parse
TYPE: List[Path]
base_dir
base directory relative to the searched files
regex
compiled regular expression. Expected to have matching groups (tag, text). Default is CODE_TAG_RE with tags from tag_order
(tag, text)
TYPE: str DEFAULT: ''
''
tags
subset of all tags to include in the report and specified order. Default is COMMON_CODE_TAGS
header
header text
TYPE: str DEFAULT: '# Task Summary\\n\\nAuto-Generated by `calcipy`'
'# Task Summary\\n\\nAuto-Generated by `calcipy`'
@beartype\ndef write_code_tag_file(\n path_tag_summary: Path,\n paths_source: List[Path],\n base_dir: Path,\n regex: str = '',\n tags: str = '',\n header: str = '# Task Summary\\n\\nAuto-Generated by `calcipy`',\n) -> None:\n \"\"\"Create the code tag summary file.\n\n Args:\n path_tag_summary: Path to the output file\n paths_source: list of source files to parse\n base_dir: base directory relative to the searched files\n regex: compiled regular expression. Expected to have matching groups `(tag, text)`.\n Default is CODE_TAG_RE with tags from tag_order\n tags: subset of all tags to include in the report and specified order. Default is COMMON_CODE_TAGS\n header: header text\n\n \"\"\"\n tag_order = [_t.strip() for _t in tags.split(',') if _t] or COMMON_CODE_TAGS\n matcher = (regex or CODE_TAG_RE).format(tag='|'.join(tag_order))\n\n matches = _search_files(paths_source, re.compile(matcher))\n if report := _format_report(\n base_dir, matches, tag_order=tag_order,\n ).strip():\n path_tag_summary.write_text(f'{header}\\n\\n{report}\\n\\n<!-- {SKIP_PHRASE} -->\\n')\n logger.text('Created Code Tag Summary', path_tag_summary=path_tag_summary)\n elif path_tag_summary.is_file():\n path_tag_summary.unlink()\n
Dotted dictionary for consistent interface.
Consider moving to Corallium, but I don\u2019t have any uses for it yet.
DdictType = Union[Dict[str, Any], Box]\n
Return type from ddict().
ddict()
ddict(**kwargs)\n
Return a dotted dictionary that can also be accessed normally.
python-box
cleverdict
munch
bunch
ddict
**kwargs
keyword arguments formatted into dictionary
TYPE: Dict[str, Any] DEFAULT: {}
Dict[str, Any]
{}
DdictType
dotted dictionary
TYPE: DdictType
@beartype\ndef ddict(**kwargs: Dict[str, Any]) -> DdictType:\n \"\"\"Return a dotted dictionary that can also be accessed normally.\n\n - Currently uses `python-box`\n - Could consider `cleverdict` which had updates as recently as 2022\n - There are numerous other variations that haven't been updated since 2020, such as `munch`, `bunch`, `ddict`\n\n Args:\n **kwargs: keyword arguments formatted into dictionary\n\n Returns:\n DdictType: dotted dictionary\n\n \"\"\"\n return Box(kwargs)\n
Experiment with bumping the git tag using griffe.
griffe
bump_tag(*, pkg_name, tag, tag_prefix)\n
Make a SemVer minor bump using griffe if there were any breaking changes.
Major versions must be bumped manually
@beartype\ndef bump_tag(*, pkg_name: str, tag: str, tag_prefix: str) -> str:\n \"\"\"Make a SemVer minor bump using `griffe` if there were any breaking changes.\n\n Major versions must be bumped manually\n\n \"\"\"\n previous = griffe.load_git(pkg_name, ref=tag)\n current = griffe.load(pkg_name)\n\n breakages = [*griffe.find_breaking_changes(previous, current)]\n for breakage in breakages:\n try:\n logger.text(breakage._explain_oneline()) # noqa: SLF001\n except BuiltinModuleError: # noqa: PERF203\n logger.warning(str(breakage))\n except Exception:\n logger.exception(str(breakage))\n\n try:\n ver = semver.Version.parse(tag.replace(tag_prefix, ''))\n except ValueError:\n logger.exception('Failed to parse tag', tag=tag)\n return ''\n new_ver = ver.bump_minor() if any(breakages) else ver.bump_patch()\n return f'{tag_prefix}{new_ver}'\n
Experiment with checking for duplicate test names.
run(test_path)\n
Check for duplicates in the test suite.
Inspired by: https://stackoverflow.com/a/67840804/3219667
@beartype\ndef run(test_path: Path) -> List[str]: # noqa: C901,CAC001 # pylint: disable=too-complex\n \"\"\"Check for duplicates in the test suite.\n\n Inspired by: https://stackoverflow.com/a/67840804/3219667\n\n \"\"\"\n summary = set()\n duplicates = []\n\n for path_test in test_path.rglob('test_*.py'): # pylint: disable=too-many-nested-blocks\n logger.info(path_test.as_posix())\n parsed_ast = ast.parse(path_test.read_text())\n\n for node in parsed_ast.body:\n if isinstance(node, ast.FunctionDef):\n if node.name in summary and node.name.startswith('test_'):\n duplicates.append(node.name)\n summary.add(node.name)\n _show_info(node)\n elif isinstance(node, ast.ClassDef):\n logger.info('Class name', name=node.name)\n for method in node.body:\n if isinstance(method, ast.FunctionDef):\n _show_info(method)\n\n for node in ast.walk(parsed_ast): # type: ignore[assignment]\n if (\n isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef))\n and node.name not in summary\n ):\n logger.info('Found new function(s) through walking')\n _show_info(node)\n summary.add(node.name)\n\n if duplicates:\n logger.error('Found Duplicates', duplicates=duplicates)\n return duplicates\n
Markdown Machine.
HandlerLookupT = Dict[str, Callable[[str, Path], List[str]]]\n
Handler Lookup.
write_autoformatted_md_sections(handler_lookup=None, paths_md=None)\n
Populate the auto-formatted sections of markdown files with user-configured logic.
@beartype\ndef write_autoformatted_md_sections(\n handler_lookup: Optional[HandlerLookupT] = None,\n paths_md: Optional[List[Path]] = None,\n) -> None:\n \"\"\"Populate the auto-formatted sections of markdown files with user-configured logic.\"\"\"\n _lookup: HandlerLookupT = handler_lookup or {\n 'COVERAGE ': _handle_coverage,\n 'SOURCE_FILE=': _handle_source_file,\n }\n\n paths = paths_md or find_project_files_by_suffix(get_project_path()).get('md') or []\n for path_md in paths:\n logger.text_debug('Processing', path_md=path_md)\n if md_lines := _ReplacementMachine().parse(read_lines(path_md), _lookup, path_md):\n path_md.write_text('\\n'.join(md_lines) + '\\n', encoding='utf-8')\n
nox-poetry configuration file.
Useful snippets from docs
poetry run nox -l\npoetry run nox --list-sessions\n\npoetry run nox -s build_check-3.8 build_dist-3.8 tests-3.8\npoetry run nox --session tests-3.11\n\npoetry run nox --python 3.8\n\npoetry run nox -k \"not build_check and not build_dist\"\n
Useful nox snippets
# Example conditionally skipping a session\nif not session.interactive:\n session.skip('Cannot run detect-secrets audit in non-interactive shell')\n\n# Install pinned version\nsession.install('detect-secrets==1.0.3')\n\n# Example capturing STDOUT into a file (could do the same for stderr)\npath_stdout = Path('.stdout.txt').resolve()\nwith open(path_stdout, 'w') as out:\n session.run(*shlex.split('echo Hello World!'), stdout=out)\n
build_check(session)\n
Check that the built output meets all checks.
@nox_poetry_session(python=_get_pythons()[-1:], reuse_venv=True)\ndef build_check(session: NPSession) -> None: # pragma: no cover\n \"\"\"Check that the built output meets all checks.\"\"\"\n # Build sdist and fix return URI, which will have file://...#egg=calcipy\n sdist_uri = session.poetry.build_package(distribution_format=DistributionFormat.SDIST)\n path_sdist = Path(url2pathname(urlparse(sdist_uri).path))\n logger.text_debug('Fixed sdist URI', sdist_uri=sdist_uri, path_sdist=path_sdist)\n # Check with pyroma\n session.install('pyroma>=4.0', '--upgrade')\n # required for \"poetry.core.masonry.api\" build backend\n session.run('python', '-m', 'pip', 'install', 'poetry>=1.3', stdout=True)\n session.run('pyroma', '--file', path_sdist.as_posix(), '--min=9', stdout=True)\n
build_dist(session)\n
Build and test the project files within a controlled environment for repeatability.
@nox_session(python=_get_pythons()[-1:], reuse_venv=False)\ndef build_dist(session: Union[NoxSession, NPSession]) -> None: # pragma: no cover\n \"\"\"Build and test the project files within a controlled environment for repeatability.\"\"\"\n dist_path = Path('dist')\n if_found_unlink(dist_path)\n\n # Support 'corallium' by re-implementing \"session.poetry.build_package()\", from:\n # https://github.com/cjolowicz/nox-poetry/blob/5772b66ebff8d5a3351a08ed402d3d31e48be5f8/src/nox_poetry/sessions.py#L233-L255\n # https://github.com/cjolowicz/nox-poetry/blob/5772b66ebff8d5a3351a08ed402d3d31e48be5f8/src/nox_poetry/poetry.py#L111-L154\n output = session.run(*shlex.split('poetry build --format=wheel --no-ansi'),\n external=True, silent=True, stderr=None)\n output = cast(str, output)\n wheel = dist_path / output.split()[-1]\n path_wheel = wheel.resolve().as_uri()\n\n logger.text('Created wheel', path_wheel=path_wheel)\n # Install the wheel and check that imports without any of the optional dependencies\n session.install(path_wheel)\n session.run(*shlex.split('python scripts/check_imports.py'), stdout=True)\n
tests(session)\n
Run doit test task for specified python versions.
@nox_session(python=_get_pythons(), reuse_venv=True)\ndef tests(session: Union[NoxSession, NPSession]) -> None: # pragma: no cover\n \"\"\"Run doit test task for specified python versions.\"\"\"\n _install_local(session, ['ddict', 'doc', 'lint', 'nox', 'stale', 'tags', 'test'])\n session.run(*shlex.split('pytest ./tests'), stdout=True, env={'RUNTIME_TYPE_CHECKING_MODE': 'WARNING'})\n
Bases: Collection
Collection
class Collection(InvokeCollection):\n\n @classmethod\n def from_module(\n cls,\n module: ModuleType,\n name: Optional[str] = None,\n config: Optional[Dict[str, Any]] = None,\n loaded_from: Optional[str] = None,\n auto_dash_names: Optional[bool] = None,\n ) -> 'InvokeCollection':\n \"\"\"Extend search for a namespace, Task, or deferred task.\"\"\"\n collection = super().from_module(\n module=module,\n name=name,\n config=config,\n loaded_from=loaded_from,\n auto_dash_names=auto_dash_names,\n )\n\n # If tasks were not loaded from a namespace or otherwise found\n if not collection.task_names:\n # Look for any decorated, but deferred \"Tasks\"\n for task in (fxn for fxn in vars(module).values() if hasattr(fxn, TASK_ARGS_ATTR)):\n collection.add_task(task)\n\n return collection\n\n def add_task(\n self,\n task: DeferedTask,\n name: Optional[str] = None,\n aliases: Optional[Tuple[str, ...]] = None,\n default: Optional[bool] = None,\n ) -> None:\n \"\"\"Extend for deferred tasks.\"\"\"\n super().add_task(task=_build_task(task), name=name, aliases=aliases, default=default)\n
add_task(task, name=None, aliases=None, default=None)\n
Extend for deferred tasks.
def add_task(\n self,\n task: DeferedTask,\n name: Optional[str] = None,\n aliases: Optional[Tuple[str, ...]] = None,\n default: Optional[bool] = None,\n) -> None:\n \"\"\"Extend for deferred tasks.\"\"\"\n super().add_task(task=_build_task(task), name=name, aliases=aliases, default=default)\n
classmethod
from_module(module, name=None, config=None, loaded_from=None, auto_dash_names=None)\n
Extend search for a namespace, Task, or deferred task.
@classmethod\ndef from_module(\n cls,\n module: ModuleType,\n name: Optional[str] = None,\n config: Optional[Dict[str, Any]] = None,\n loaded_from: Optional[str] = None,\n auto_dash_names: Optional[bool] = None,\n) -> 'InvokeCollection':\n \"\"\"Extend search for a namespace, Task, or deferred task.\"\"\"\n collection = super().from_module(\n module=module,\n name=name,\n config=config,\n loaded_from=loaded_from,\n auto_dash_names=auto_dash_names,\n )\n\n # If tasks were not loaded from a namespace or otherwise found\n if not collection.task_names:\n # Look for any decorated, but deferred \"Tasks\"\n for task in (fxn for fxn in vars(module).values() if hasattr(fxn, TASK_ARGS_ATTR)):\n collection.add_task(task)\n\n return collection\n
Bases: BaseModel
BaseModel
Global Task Options.
class GlobalTaskOptions(BaseModel):\n \"\"\"Global Task Options.\"\"\"\n\n working_dir: Path = Field(default_factory=Path.cwd)\n \"\"\"Working directory for the program to use globally.\"\"\"\n\n file_args: List[Path] = Field(default_factory=list)\n \"\"\"List of Paths to modify.\"\"\"\n\n verbose: PositiveInt = Field(default=0, le=3)\n \"\"\"Verbosity level.\"\"\"\n\n keep_going: bool = False\n \"\"\"Continue task execution regardless of failure.\"\"\"\n
class-attribute
instance-attribute
file_args = Field(default_factory=list)\n
List of Paths to modify.
keep_going = False\n
Continue task execution regardless of failure.
verbose = Field(default=0, le=3)\n
Verbosity level.
working_dir = Field(default_factory=Path.cwd)\n
Working directory for the program to use globally.
Tasks can be imported piecemeal or imported in their entirety from here.
TaskList = List[Union[Call, DeferedTask]]\n
List of wrapped or normal task functions.
main(_ctx)\n
Main task pipeline.
@task(post=with_progress(_MAIN_TASKS))\ndef main(_ctx: Context) -> None:\n \"\"\"Main task pipeline.\"\"\"\n
other(_ctx)\n
Run tasks that are otherwise not exercised in main.
@task(post=with_progress(_OTHER_TASKS)) # pyright: ignore[reportGeneralTypeIssues]\ndef other(_ctx: Context) -> None:\n \"\"\"Run tasks that are otherwise not exercised in main.\"\"\"\n
progress(_ctx, *, index, total)\n
Progress Task.
@task(\n help={\n 'index': 'Current index (0-indexed)',\n 'total': 'Total steps',\n },\n show_task_info=False,\n)\ndef progress(_ctx: Context, *, index: int, total: int) -> None:\n \"\"\"Progress Task.\"\"\"\n logger.text('Progress', is_header=True, index=index + 1, total=total)\n
release(ctx, *, suffix=None)\n
Release pipeline.
@task(\n help=cl.bump.help,\n post=with_progress(\n [ # pyright: ignore[reportGeneralTypeIssues]\n pack.lock,\n doc.build,\n doc.deploy,\n pack.publish,\n ],\n offset=1,\n ),\n)\ndef release(ctx: Context, *, suffix: cl.SuffixT = None) -> None:\n \"\"\"Release pipeline.\"\"\"\n cl.bumpz(ctx, suffix=suffix)\n
summary(_ctx, *, message)\n
Summary Task.
@task(\n help={\n 'message': 'String message to display',\n },\n show_task_info=False,\n)\ndef summary(_ctx: Context, *, message: str) -> None:\n \"\"\"Summary Task.\"\"\"\n logger.text(message, is_header=True)\n
with_progress(items, offset=0)\n
Inject intermediary \u2018progress\u2019 tasks.
items
list of tasks
TYPE: Any
Any
offset
Optional offset to shift counters
TYPE: int DEFAULT: 0
0
@beartype\ndef with_progress(items: Any, offset: int = 0) -> TaskList:\n \"\"\"Inject intermediary 'progress' tasks.\n\n Args:\n items: list of tasks\n offset: Optional offset to shift counters\n\n \"\"\"\n task_items = [_build_task(_t) for _t in items]\n message = 'Running tasks: ' + ', '.join([str(_t.__name__) for _t in task_items])\n tasks: TaskList = [summary.with_kwargs(message=message)]\n\n total = len(task_items) + offset\n for idx, item in enumerate(task_items):\n tasks += [progress.with_kwargs(index=idx + offset, total=total), item]\n return tasks\n
Changelog CLI.
SuffixT = Optional[Literal['alpha', 'beta', 'rc']]\n
Prerelease Suffix Type.
bump(ctx, *, suffix=None)\n
Bumps project version based on commits & settings in pyproject.toml.
@task(\n pre=[write],\n help={\n 'suffix': 'Specify prerelease suffix for version bump (alpha, beta, rc)',\n },\n)\ndef bump(ctx: Context, *, suffix: SuffixT = None) -> None:\n \"\"\"Bumps project version based on commits & settings in pyproject.toml.\"\"\"\n bumpz(ctx, suffix=suffix)\n
bumpz(ctx, *, suffix=None)\n
@beartype\ndef bumpz(ctx: Context, *, suffix: SuffixT = None) -> None:\n \"\"\"Bumps project version based on commits & settings in pyproject.toml.\"\"\"\n check_installed(ctx, executable='gh', message=GH_MESSAGE)\n\n opt_cz_args = f' --prerelease={suffix}' if suffix else ''\n run(ctx, f'{python_dir()}/cz bump{opt_cz_args} --annotated-tag --no-verify --gpg-sign')\n\n run(ctx, 'git push origin --tags --no-verify')\n\n get_last_tag = 'git tag --list --sort=-creatordate | head -n 1'\n opt_gh_args = ' --prerelease' if suffix else ''\n run(ctx, f'gh release create --generate-notes $({get_last_tag}){opt_gh_args}')\n
write(ctx)\n
Write a Changelog file with the raw Git history.
Resources:
List[DoitAction]: doit actions
@task()\ndef write(ctx: Context) -> None:\n \"\"\"Write a Changelog file with the raw Git history.\n\n Resources:\n\n - https://keepachangelog.com/en/1.0.0/\n - https://www.conventionalcommits.org/en/v1.0.0/\n - https://writingfordevelopers.substack.com/p/how-to-write-a-commit-message\n - https://chris.beams.io/posts/git-commit/\n - https://semver.org/\n - https://calver.org/\n\n Returns:\n List[DoitAction]: doit actions\n\n \"\"\"\n run(ctx, f'{python_dir()}/cz changelog')\n path_cl = get_project_path() / 'CHANGELOG.md'\n if not path_cl.is_file():\n msg = f'Could not locate the changelog at: {path_cl}'\n raise FileNotFoundError(msg)\n path_cl.replace(get_doc_subdir() / path_cl.name)\n
Calcipy-Invoke Defaults.
from_ctx(ctx, group, key)\n
Safely extract the value from the context or the defaults.
Instead of ctx.tests.out_dir use from_ctx(ctx, 'test', 'out_dir')
ctx.tests.out_dir
from_ctx(ctx, 'test', 'out_dir')
@beartype\ndef from_ctx(ctx: Context, group: str, key: str) -> str:\n \"\"\"Safely extract the value from the context or the defaults.\n\n Instead of `ctx.tests.out_dir` use `from_ctx(ctx, 'test', 'out_dir')`\n\n \"\"\"\n with suppress(KeyError):\n return str(ctx.config[group][key])\n return str(DEFAULTS[group][key])\n
new_collection()\n
Initialize a collection with the combination of merged and project-specific defaults.
@beartype\ndef new_collection() -> Collection:\n \"\"\"Initialize a collection with the combination of merged and project-specific defaults.\"\"\"\n ns = Collection('')\n\n # Merge default and user configuration\n ns.configure(DEFAULTS)\n config_path = Path('.calcipy.json')\n if config_path.is_file():\n ns.configure(json.loads(config_path.read_text(encoding='utf-8')))\n\n return ns\n
Document CLI.
build(ctx)\n
Build documentation with mkdocs.
@task()\ndef build(ctx: Context) -> None:\n \"\"\"Build documentation with mkdocs.\"\"\"\n auto_doc_path = get_doc_subdir().parent / 'modules'\n write_autoformatted_md_sections()\n delete_dir(auto_doc_path)\n _diagram_task(ctx, auto_doc_path)\n\n # Find and trim trailing whitespace\n for path_md in auto_doc_path.rglob('*.md'):\n trim_trailing_whitespace(path_md)\n\n run(ctx, f'{python_dir()}/mkdocs build --site-dir {get_out_dir()}')\n
deploy(ctx)\n
Deploy docs to the Github gh-pages branch.
gh-pages
@task()\ndef deploy(ctx: Context) -> None:\n \"\"\"Deploy docs to the Github `gh-pages` branch.\"\"\"\n if _is_mkdocs_local(): # pragma: no cover\n raise NotImplementedError('Not yet configured to deploy documentation without \"use_directory_urls\"')\n\n with suppress(UnexpectedExit):\n run(ctx, 'pre-commit uninstall') # To prevent pre-commit failures when mkdocs calls push\n run(ctx, f'{python_dir()}/mkdocs gh-deploy --force')\n with suppress(UnexpectedExit):\n run(ctx, 'pre-commit install') # Restore pre-commit\n
get_out_dir()\n
Retrieve the mkdocs-specified site directory.
@beartype\ndef get_out_dir() -> Path:\n \"\"\"Retrieve the mkdocs-specified site directory.\"\"\"\n mkdocs_config = read_yaml_file(get_project_path() / MKDOCS_CONFIG)\n return Path(mkdocs_config.get('site_dir', 'releases/site'))\n
watch(ctx)\n
Serve local documentation for local editing.
@task()\ndef watch(ctx: Context) -> None:\n \"\"\"Serve local documentation for local editing.\"\"\"\n if _is_mkdocs_local(): # pragma: no cover\n path_doc_index = get_out_dir() / 'index.html'\n open_in_browser(path_doc_index)\n else: # pragma: no cover\n webbrowser.open('http://localhost:8000')\n run(ctx, f'{python_dir()}/mkdocs serve --dirtyreload')\n
Utilities for working in calcipy\u2019s python environment.
check_installed(ctx, executable, message)\n
If the required executable isn\u2019t present, raise a clear user error.
@beartype\ndef check_installed(ctx: Context, executable: str, message: str) -> None:\n \"\"\"If the required executable isn't present, raise a clear user error.\"\"\"\n res = run(ctx, f'which {executable}', warn=True, hide=True)\n if not res or res.exited == 1:\n raise RuntimeError(message)\n
python_dir()\n
Runs an executable from the currently active Python directory.
@lru_cache(maxsize=1)\ndef python_dir() -> str:\n \"\"\"Runs an executable from the currently active Python directory.\"\"\"\n return str(resolve_python().parent)\n
python_m()\n
Return the active python path and -m flag.
-m
@lru_cache(maxsize=1)\ndef python_m() -> str:\n \"\"\"Return the active python path and `-m` flag.\"\"\"\n return f'{resolve_python()} -m'\n
resolve_python()\n
Resolve the user\u2019s Python path based on sys.
sys
@lru_cache(maxsize=1)\ndef resolve_python() -> Path:\n \"\"\"Resolve the user's Python path based on `sys`.\"\"\"\n python_path = Path(sys.executable)\n with suppress(ValueError):\n return python_path.relative_to(Path.cwd())\n return python_path\n
Lint CLI.
autopep8(ctx)\n
Run autopep8.
FYI: This is temporary until ruff implements white space rules https://github.com/charliermarsh/ruff/issues/970
@task()\ndef autopep8(ctx: Context) -> None:\n \"\"\"Run autopep8.\n\n FYI: This is temporary until ruff implements white space rules\n https://github.com/charliermarsh/ruff/issues/970\n\n \"\"\"\n cli_args = '--aggressive --recursive --in-place --max-line-length=120'\n _inner_task(ctx, command='autopep8', cli_args=cli_args)\n
check(ctx)\n
Run ruff as check-only.
@task(default=True)\ndef check(ctx: Context) -> None:\n \"\"\"Run ruff as check-only.\"\"\"\n _inner_task(ctx, command='ruff check')\n
fix(ctx)\n
Run ruff and apply fixes.
@task(pre=[autopep8])\ndef fix(ctx: Context) -> None:\n \"\"\"Run ruff and apply fixes.\"\"\"\n _inner_task(ctx, command='ruff check', cli_args='--fix')\n
flake8(ctx)\n
Run flake8.
@task()\ndef flake8(ctx: Context) -> None:\n \"\"\"Run flake8.\"\"\"\n _inner_task(ctx, command='flake8', run_as_module=False)\n
pre_commit(ctx, *, no_update=False)\n
Run pre-commit.
@task(\n help={\n 'no_update': 'Skip updating the pre-commit hooks',\n },\n)\ndef pre_commit(ctx: Context, *, no_update: bool = False) -> None:\n \"\"\"Run pre-commit.\"\"\"\n check_installed(ctx, executable='pre-commit', message=PRE_COMMIT_MESSAGE)\n\n run(ctx, 'pre-commit install')\n if not no_update:\n run(ctx, 'pre-commit autoupdate')\n\n all_hook_stages = [\n 'commit', 'merge-commit', 'push', 'prepare-commit-msg', 'commit-msg', 'post-checkout',\n 'post-commit', 'post-merge', 'post-rewrite', 'manual',\n ]\n stages_cli = ' '.join(f'--hook-stage {stg}' for stg in all_hook_stages)\n run(ctx, f'pre-commit run --all-files {stages_cli}')\n
pylint(ctx, *, report=False)\n
Run pylint.
@task(\n help={\n 'report': 'if provided, show the pylint summary report',\n },\n)\ndef pylint(ctx: Context, *, report: bool = False) -> None:\n \"\"\"Run pylint.\"\"\"\n cli_args = '--report=y' if report else ''\n _inner_task(ctx, command='pylint', cli_args=cli_args)\n
security(ctx)\n
Attempt to identify possible security vulnerabilities.
@task()\ndef security(ctx: Context) -> None:\n \"\"\"Attempt to identify possible security vulnerabilities.\"\"\"\n logger.warning('Note: Selectively override bandit with \"# nosec\"', is_header=True)\n pkg_name = read_package_name()\n run(ctx, f'{python_dir()}/bandit --recursive {pkg_name} -s B101')\n\n check_installed(ctx, executable='semgrep', message=SEMGREP_MESSAGE)\n logger.warning('Note: Selectively override semgrep with \"# nosem\"', is_header=True)\n # See additional semgrep rules at:\n # https://semgrep.dev/explore\n # https://github.com/returntocorp/semgrep-rules/tree/develop/python\n # https://awesomeopensource.com/project/returntocorp/semgrep-rules?categorypage=45\n semgrep_configs = ' '.join([ # noqa: FLY002\n '--config=p/ci',\n '--config=p/default',\n '--config=p/security-audit',\n '--config=r/bash',\n '--config=r/contrib',\n '--config=r/fingerprints',\n '--config=r/generic',\n '--config=r/json',\n '--config=r/python',\n '--config=r/terraform',\n '--config=r/yaml',\n '--exclude-rule=yaml.github-actions.security.third-party-action-not-pinned-to-commit-sha.third-party-action-not-pinned-to-commit-sha',\n ])\n run(ctx, f'semgrep ci --autofix {semgrep_configs}')\n
@task()\ndef watch(ctx: Context) -> None:\n \"\"\"Run ruff as check-only.\"\"\"\n _inner_task(ctx, command='ruff check', cli_args='--watch --show-source')\n
Nox CLI.
noxfile(ctx, *, session='')\n
Run nox from the local noxfile.
@task(\n default=True,\n help={\n 'session': 'Optional session to run',\n },\n)\ndef noxfile(ctx: Context, *, session: str = '') -> None:\n \"\"\"Run nox from the local noxfile.\"\"\"\n cli_args = ['--session', session] if session else []\n run(ctx, f'{python_dir()}/nox --error-on-missing-interpreters {\" \".join(cli_args)}')\n
Packaging CLI.
bump_tag(ctx, *, tag, tag_prefix='', pkg_name='')\n
Example for calcipy:
./run pack.bump-tag --tag=\"$(git tag -l \"*\" | sort | head -n 5 | tail -n 1)\" --tag-prefix=\"\"\n
@task(\n help={\n 'tag': 'Last tag, can be provided with `--tag=\"$(git tag -l \"v*\" | sort | tail -n 1)\"`',\n 'tag_prefix': 'Optional tag prefix, such as \"v\"',\n 'pkg_name': 'Optional package name. If not provided, will read the poetry pyproject.toml file',\n },\n)\ndef bump_tag(ctx: Context, *, tag: str, tag_prefix: str = '', pkg_name: str = '') -> None: # noqa: ARG001\n \"\"\"Experiment with bumping the git tag using `griffe`.\n\n Example for `calcipy`:\n\n ```sh\n ./run pack.bump-tag --tag=\"$(git tag -l \"*\" | sort | head -n 5 | tail -n 1)\" --tag-prefix=\"\"\n ```\n\n \"\"\"\n if not tag:\n raise ValueError('tag must not be empty')\n if not pkg_name:\n poetry_config = file_helpers.read_pyproject()['tool']['poetry']\n pkg_name = poetry_config['name']\n\n new_version = bump_programmatically.bump_tag(\n pkg_name=pkg_name,\n tag=tag,\n tag_prefix=tag_prefix,\n )\n logger.text(new_version)\n
check_licenses(ctx)\n
Check licenses for compatibility with licensecheck.
licensecheck
@task()\ndef check_licenses(ctx: Context) -> None:\n \"\"\"Check licenses for compatibility with `licensecheck`.\"\"\"\n res = run(ctx, 'which licensecheck', warn=True, hide=True)\n if not res or res.exited == 1:\n logger.warning('`licensecheck` not found. installing with pipx')\n run(ctx, 'pipx install licensecheck')\n run(ctx, 'licensecheck')\n
install_extras(ctx)\n
Run poetry install with all extras.
@task()\ndef install_extras(ctx: Context) -> None:\n \"\"\"Run poetry install with all extras.\"\"\"\n poetry_config = file_helpers.read_pyproject()['tool']['poetry']\n extras = (poetry_config.get('extras') or {}).keys()\n run(ctx, ' '.join(['poetry install --sync', *[f'--extras={ex}' for ex in extras]]))\n
lock(ctx)\n
Ensure poetry.lock is up-to-date.
@task()\ndef lock(ctx: Context) -> None:\n \"\"\"Ensure poetry.lock is up-to-date.\"\"\"\n if can_skip.can_skip(prerequisites=[PROJECT_TOML], targets=[LOCK]):\n return # Exit early\n\n run(ctx, 'poetry lock --no-update')\n
publish(ctx, *, to_test_pypi=False)\n
Build the distributed format(s) and publish.
@task(\n help={\n 'to_test_pypi': 'Publish to the TestPyPi repository',\n },\n)\ndef publish(ctx: Context, *, to_test_pypi: bool = False) -> None:\n \"\"\"Build the distributed format(s) and publish.\"\"\"\n run(ctx, f'{python_dir()}/nox --error-on-missing-interpreters --session build_dist build_check')\n\n cmd = 'poetry publish'\n if to_test_pypi:\n cmd += ' --repository testpypi'\n run(ctx, cmd)\n
Stale Packages CLI.
check_for_stale_packages(ctx, *, stale_months=48)\n
Identify stale dependencies.
@task(\n default=True,\n help={\n 'stale_months': 'Cutoff in months for when a package may be stale enough to be a risk',\n },\n)\ndef check_for_stale_packages(ctx: Context, *, stale_months: int = 48) -> None:\n \"\"\"Identify stale dependencies.\"\"\"\n cfsp(stale_months=stale_months)\n run(ctx, f'{python_dir()}/pip-check --cmd=\"poetry run pip\" --hide-unchanged')\n
Code Tag Collector CLI.
collect_code_tags(ctx, base_dir='.', filename=None, tag_order='', regex='', ignore_patterns='')\n
Create a CODE_TAG_SUMMARY.md with a table for TODO- and FIXME-style code comments.
CODE_TAG_SUMMARY.md
@task(\n default=True,\n help={\n 'base_dir': 'Working Directory',\n 'filename': 'Code Tag Summary Filename',\n 'tag_order': 'Ordered list of code tags to locate (Comma-separated)',\n 'regex': 'Custom Code Tag Regex. Must contain \"{tag}\"',\n 'ignore_patterns': 'Glob patterns to ignore files and directories when searching (Comma-separated)',\n },\n)\ndef collect_code_tags(\n ctx: Context,\n base_dir: str = '.',\n filename: Optional[str] = None,\n tag_order: str = '',\n regex: str = '',\n ignore_patterns: str = '',\n) -> None:\n \"\"\"Create a `CODE_TAG_SUMMARY.md` with a table for TODO- and FIXME-style code comments.\"\"\"\n pth_base_dir = Path(base_dir).resolve()\n path_tag_summary = get_doc_subdir() / (filename or from_ctx(ctx, 'tags', 'filename'))\n patterns = (ignore_patterns or from_ctx(ctx, 'tags', 'ignore_patterns')).split(',')\n paths_source = find_project_files(pth_base_dir, ignore_patterns=[*filter(lambda item: item, patterns)])\n\n write_code_tag_file(\n path_tag_summary=path_tag_summary,\n paths_source=paths_source,\n base_dir=pth_base_dir,\n regex=regex,\n tags=tag_order,\n header='# Collected Code Tags',\n )\n
Test CLI.
check(_ctx)\n
Run pytest checks, such as identifying .
@task()\ndef check(_ctx: Context) -> None:\n \"\"\"Run pytest checks, such as identifying .\"\"\"\n if duplciates := check_duplicate_test_names.run(Path('tests')):\n raise RuntimeError(f'Duplicate test names found ({duplciates}). See above for details.') # noqa: EM102\n
coverage(ctx, *, min_cover=0, out_dir=None, view=False)\n
Generate useful coverage outputs after running pytest.
Creates coverage.json used in doc.build
coverage.json
doc.build
@task(\n help={\n 'min_cover': 'Fail if coverage less than threshold',\n 'out_dir': 'Optional path to coverage directory. Typically \".cover\" or \"releases/tests\"',\n 'view': 'If True, open the created files',\n },\n)\ndef coverage(ctx: Context, *, min_cover: int = 0, out_dir: Optional[str] = None, view: bool = False) -> None:\n \"\"\"Generate useful coverage outputs after running pytest.\n\n Creates `coverage.json` used in `doc.build`\n\n \"\"\"\n pkg_name = read_package_name()\n _inner_task(ctx, cli_args='', min_cover=min_cover,\n command=f'coverage run --branch --source={pkg_name} --module pytest')\n\n cov_dir = Path(out_dir or from_ctx(ctx, 'test', 'out_dir'))\n cov_dir.mkdir(exist_ok=True, parents=True)\n print() # noqa: T201\n for cli_args in (\n 'report --show-missing', # Write to STDOUT\n f'html --directory={cov_dir}', # Write to HTML\n 'json', # Create coverage.json file for \"_handle_coverage\"\n ):\n run(ctx, f'{python_dir()}/coverage {cli_args}')\n\n if view: # pragma: no cover\n open_in_browser(cov_dir / 'index.html')\n
pytest(ctx, *, keyword='', marker='', min_cover=0)\n
Run pytest with default arguments.
Additional arguments can be set in the environment variable \u2018PYTEST_ADDOPTS\u2019
@task(\n default=True,\n help={\n 'min_cover': 'Fail if coverage less than threshold',\n **KM_HELP,\n },\n)\ndef pytest(ctx: Context, *, keyword: str = '', marker: str = '', min_cover: int = 0) -> None:\n \"\"\"Run pytest with default arguments.\n\n Additional arguments can be set in the environment variable 'PYTEST_ADDOPTS'\n\n \"\"\"\n pkg_name = read_package_name()\n durations = '--durations=25 --durations-min=\"0.1\"'\n _inner_task(ctx,\n cli_args=f' --cov={pkg_name} --cov-branch --cov-report=term-missing {durations}',\n keyword=keyword, marker=marker, min_cover=min_cover)\n
step(ctx, *, keyword='', marker='')\n
Run pytest optimized to stop on first error.
@task(help=KM_HELP)\ndef step(ctx: Context, *, keyword: str = '', marker: str = '') -> None:\n \"\"\"Run pytest optimized to stop on first error.\"\"\"\n _inner_task(ctx, cli_args=_STEPWISE_ARGS, keyword=keyword, marker=marker)\n
watch(ctx, *, keyword='', marker='')\n
Run pytest with polling and optimized to stop on first error.
@task(help=KM_HELP)\ndef watch(ctx: Context, *, keyword: str = '', marker: str = '') -> None:\n \"\"\"Run pytest with polling and optimized to stop on first error.\"\"\"\n _inner_task(ctx, cli_args=_STEPWISE_ARGS, keyword=keyword, marker=marker, command='ptw . --now')\n
Types CLI.
mypy(ctx)\n
Run mypy.
@task()\ndef mypy(ctx: Context) -> None:\n \"\"\"Run mypy.\"\"\"\n _inner_task(ctx, command=f'{python_dir()}/mypy')\n
pyright(ctx)\n
Run pyright.
@task()\ndef pyright(ctx: Context) -> None:\n \"\"\"Run pyright.\"\"\"\n check_installed(ctx, executable='pyright', message=PYRIGHT_MESSAGE)\n _inner_task(ctx, command='pyright')\n