diff --git a/.flake8 b/.flake8 index 82316f1..fc55304 100644 --- a/.flake8 +++ b/.flake8 @@ -2,6 +2,7 @@ exclude = .git, __pycache__ # Justifications for each ignored error or warning: +# * E252: equals for a default argument should look distinct from assignment. # * E302: always requiring two blank lines rather than one between top-level items is too annoying and nitpicky. # * E402: we want imports for tests to go between the main code and the tests. # * W503, W504: these give false positives for the style of avoiding \ at the end of each line by using parens. @@ -9,6 +10,6 @@ exclude = .git, __pycache__ # References: # - https://flake8.pycqa.org/en/latest/user/error-codes.html # - https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes -ignore = E302, E402, W503, W504 +ignore = E252, E302, E402, W503, W504 max-line-length = 120 diff --git a/.github/workflows/flake8.yml b/.github/workflows/lints.yml similarity index 76% rename from .github/workflows/flake8.yml rename to .github/workflows/lints.yml index 6c3d4ce..fb0c0ec 100644 --- a/.github/workflows/flake8.yml +++ b/.github/workflows/lints.yml @@ -1,4 +1,4 @@ -name: flake8 +name: lints on: pull_request @@ -16,8 +16,14 @@ jobs: - name: Install poetry run: pip install --user poetry + - name: Run poetry check + run: poetry check + - name: Install dependencies run: poetry install --no-root - name: Run flake8 run: poetry run flake8 + + - name: Run pyanalyze + run: poetry run pyanalyze simtfl diff --git a/README.md b/README.md index 6c4f99d..483d10c 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,7 @@ Note the caveats: *experimental*, *simulator*, *research*, *potential*. Design documentation is under the `doc/` directory: -* [Programming patterns for use of simpy](doc/patterns.md). +* [Programming patterns for use of simpy and type annotations](doc/patterns.md). You can also generate API documentation by running `./gendoc.sh`. This assumes that you have run `poetry install` as shown above. The starting point for the @@ -35,12 +35,12 @@ generated documentation is . ## Contributing -Please use `./check.sh` before submitting a PR. This currently runs `flake8` -and the unit tests locally. +Please use `./check.sh` before submitting a PR. This currently runs `flake8`, +`pyanalyze`, and the unit tests locally. -You can use `./check.sh -k ` to run `flake8` and then only tests -with names matching the given substring. This will not suppress output to -stdout or stderr (but `./check.sh -bk ` will). +You can use `./check.sh -k ` to run `flake8`, `pyanalyze`, and then +only tests with names matching the given substring. This will not suppress +output to stdout or stderr (but `./check.sh -bk ` will). To see other options for running unit tests, use `poetry run python -m unittest -h`. diff --git a/check.sh b/check.sh index 9db28b2..a471a66 100755 --- a/check.sh +++ b/check.sh @@ -3,9 +3,15 @@ set -eu cd -P -- "$(dirname -- "$(command -v -- "$0")")" +echo Running poetry check... +poetry check + echo Running flake8... poetry run flake8 +echo Running pyanalyze... +poetry run pyanalyze simtfl + echo echo Running unit tests... args="${*:---buffer}" diff --git a/doc/patterns.md b/doc/patterns.md index b8831eb..8b343db 100644 --- a/doc/patterns.md +++ b/doc/patterns.md @@ -6,8 +6,9 @@ processes are implemented as generators, so that the library can simulate timeouts and asynchronous communication (typically faster than real time). We use the convention of putting "(process)" in the doc comment of these -functions. They either must use the `yield` construct, *or* return the -result of calling another "(process)" function (not both). +functions. They are also annotated with a `ProcessEffect` return type. +They either must use the `yield` construct, *or* return the result of +calling another "(process)" function (not both). Objects that implement processes typically hold the `simpy.Environment` in an instance variable `self.env`. @@ -18,3 +19,67 @@ statements, `return f()` can be used as an optimization.) A "(process)" function that does nothing should `return skip()`, using `simtfl.util.skip`. + + +# Type annotations + +The codebase is written to use static type annotations and to pass the +[pyanalyze](https://pyanalyze.readthedocs.io/en/latest/faq.html) static +analyzer. + +The default annotation for argument and return types is `Any`. This works +well for interoperating with libraries that don't use static typing, but +please don't rely on it for code in this project. It is better to add +explicit `Any` annotations in the few cases where that is needed. This +means that functions and methods that do not return a value should be +annotated with `-> None`. + +`pyanalyze` has some limitations and is not claimed to be fully sound, +but it does a pretty good job in practice; the result feels pretty much +like a statically typed variant of Python. Importing the code it checks +allows it to be more compatible with some Python idioms. The following +workarounds for its limitations may be needed: + +* It is sometimes unable to see that a `None` value cannot occur in a + particular context. In that case, adding an assertion that the value + `is not None` may help. + +* There is no easy way to precisely check uses of `*args` or `**kwargs`. + +* If two files have mutually dependent types, they may end up circularly + importing each other, which Python does not support. This is more + likely for types than for implementation code. There are several + possible workarounds: + + * merge the files; + * move part of one file that is causing the circularity into the + other; + * create an abstract base class for the type that is being used + circularly (with methods that raise `NotImplementedError`), and + use that as the type instead of the concrete subclass. + +* Adding type annotations might require exposing internal classes that + would otherwise be intentionally hidden. Since this hiding is normally + only possible by convention (e.g. using underscore-prefixed names) + in any case, that does not really cause any problem. Please just + refrain from directly using the constructors of such classes. + +As is often the case for static typing in other languages, it typically +works best to use more general types for arguments and more specific +types for results. + +Each source file should have `from __future__ import annotations` at +the top. This (usually) allows types defined in the same file to be +referenced before their definition, without needing the workaround of +writing such types as string literals. The preferred style is for this +line to be immediately followed by other imports that are needed for +type annotations. + + +# Flake8 + +We also use [flake8](https://flake8.pycqa.org/en/latest/) to encourage +a consistent coding style. However, if you disagree with a particular +error or warning produced by `flake8` and have a good justification for +why it should not be enforced, please just add it to the `ignore` list +in `.flake8`, and document the justification there. diff --git a/poetry.lock b/poetry.lock index c12b531..1dbc1d0 100644 --- a/poetry.lock +++ b/poetry.lock @@ -1,5 +1,179 @@ # This file is automatically @generated by Poetry 1.4.1 and should not be changed by hand. +[[package]] +name = "aenum" +version = "3.1.15" +description = "Advanced Enumerations (compatible with Python's stdlib Enum), NamedTuples, and NamedConstants" +category = "dev" +optional = false +python-versions = "*" +files = [ + {file = "aenum-3.1.15-py2-none-any.whl", hash = "sha256:27b1710b9d084de6e2e695dab78fe9f269de924b51ae2850170ee7e1ca6288a5"}, + {file = "aenum-3.1.15-py3-none-any.whl", hash = "sha256:e0dfaeea4c2bd362144b87377e2c61d91958c5ed0b4daf89cb6f45ae23af6288"}, + {file = "aenum-3.1.15.tar.gz", hash = "sha256:8cbd76cd18c4f870ff39b24284d3ea028fbe8731a58df3aa581e434c575b9559"}, +] + +[[package]] +name = "ast-decompiler" +version = "0.7.0" +description = "Python module to decompile AST to Python code" +category = "dev" +optional = false +python-versions = ">=3.6" +files = [ + {file = "ast_decompiler-0.7.0-py3-none-any.whl", hash = "sha256:5ebd37ba129227484daff4a15dd6056d87c488fa372036dd004ee84196b207d3"}, + {file = "ast_decompiler-0.7.0.tar.gz", hash = "sha256:efc3a507e5f8963ec7b4b2ce2ea693e3755c2f52b741c231bc344a4526738337"}, +] + +[[package]] +name = "asynq" +version = "1.5.1" +description = "Quora's asynq library" +category = "dev" +optional = false +python-versions = "*" +files = [ + {file = "asynq-1.5.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:9b32dc05ee205901d4d1bc136fd69652b590b79e9f34c7d371ae4d804bb13443"}, + {file = "asynq-1.5.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:54bbb27169d274047646aa1ef5179d445039e5da69b972059c4379cb63fbf99d"}, + {file = "asynq-1.5.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f2a809aa5d5968f45900599edbc3de6a74a3312f7c6e02e9fee6704b76675643"}, + {file = "asynq-1.5.1-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:21930da219677de6ae8a648f5bb2e6fca22bf111337c0f8cccee2c199bee5273"}, + {file = "asynq-1.5.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:bea141057e23a8a9ec382832983c4d72b08eb9cac613d63481af8731725e7d7b"}, + {file = "asynq-1.5.1-cp310-cp310-win32.whl", hash = "sha256:315481be6b403b19e355ed28c3c6c0b47afc9c4d1fa30a8b4e2d6e1da573a6a7"}, + {file = "asynq-1.5.1-cp310-cp310-win_amd64.whl", hash = "sha256:94e854a237d07b1892b851003439534a7ae6c8f640c78d19101ee30ea98bf0f9"}, + {file = "asynq-1.5.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:7aaf135693d2db7db034d265616ecbe627df2a7ceac58a0be111f488acd72df9"}, + {file = "asynq-1.5.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b63aef73898af7ee2d67397e2fddf723d8b114772f9f144453d1646394be0b43"}, + {file = "asynq-1.5.1-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:58766ffb80bd2544e935e6f1a47e6cc281003d71ae2eed2d656da60f87296f48"}, + {file = "asynq-1.5.1-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:66e2e1b283957c2bf21b98e9e8d6b1efd97821711bb4054025a3713a4dd4b576"}, + {file = "asynq-1.5.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:3d699ca115f66731f8378bcae28309349aa6374d3379ac53a9b48e59f2ad5320"}, + {file = "asynq-1.5.1-cp311-cp311-win32.whl", hash = "sha256:909d0e9395a15248cc5465ebb06efaecd3c1101fb9332e78e9f71ba1b2d28fdd"}, + {file = "asynq-1.5.1-cp311-cp311-win_amd64.whl", hash = "sha256:2d24b0230ee7cd565ef79a0a1b53a8f3fd95c2cf28cdb908e733faa160511075"}, + {file = "asynq-1.5.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:4c2cce53ce7719a298d9631cd077a69776fc7e91fd6e09a93817c03153f0506b"}, + {file = "asynq-1.5.1-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f8e3aef44474f7f011bac97e1a92e255f201fc588464fb5f42513bc1de311b07"}, + {file = "asynq-1.5.1-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8ab7e4903499304477b43272809c5512b39551b5794df473038aa0cd3139d30c"}, + {file = "asynq-1.5.1-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:ef4baaa71d0e9c7b89b6f8282df06193d3846770ba8d2b042cb3f2ed8aa94d8f"}, + {file = "asynq-1.5.1-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:ccfa6bc026e286fd5eb07ca18ca0afee4d0c09c726605f66fa32102bc9c85ecb"}, + {file = "asynq-1.5.1-cp36-cp36m-win32.whl", hash = "sha256:5d75b7f07d54b3a72a196309215023fece4aa810ed7a2acaa8defd9e5011e3e1"}, + {file = "asynq-1.5.1-cp36-cp36m-win_amd64.whl", hash = "sha256:210365fef54fc7a93f06bdd9ae4e7029537aa3528f4e3d00ea77b4430961e208"}, + {file = "asynq-1.5.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:819842445c8ca418d303ae31bcdca7adb83afd54ea823a358306c376395ce06d"}, + {file = "asynq-1.5.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f20994fc3b3ed762d8c62197b508d8a437f7f535fef395e9f33748a6b2bf8535"}, + {file = "asynq-1.5.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a9592611b3317561617d51b41f45a745d64eefe2324f9380c8cff78100b94521"}, + {file = "asynq-1.5.1-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:6a5843014e7b59a02cd20ced5e777870fab593d1107a145cb176540fc2c4ad56"}, + {file = "asynq-1.5.1-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:7bcd6b10ae015c70a37cede9bd5dec05c30468552760c336ccef4f430c47e392"}, + {file = "asynq-1.5.1-cp37-cp37m-win32.whl", hash = "sha256:487fba1aa6179d9af29f4ffe5e8f99044432e318d95501bb7cd7e406e1615df1"}, + {file = "asynq-1.5.1-cp37-cp37m-win_amd64.whl", hash = "sha256:07f3e46d71de0c6379d4ec73d482cac30ac2ff42ba942089272c0512f87e0470"}, + {file = "asynq-1.5.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:45507c91514a73127ef904ee81672c4ad869e865c84abe9d751ba06fa8af8915"}, + {file = "asynq-1.5.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:775e8cc8bd2046a256f688641de3b93799dfcbc26bf454929ca47f3bfbc2aa85"}, + {file = "asynq-1.5.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:89bc8050755c61bef6a01db0ba3dc3ec80978f52bb4b05e8a2ba214e8ecb9520"}, + {file = "asynq-1.5.1-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:bf2fe2af6978e4c57a54838fb7d64d8051edc9c4154101d4b62106974cedb96f"}, + {file = "asynq-1.5.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:839500d3b504ff43b0d153526fb726682384ab8bada6fd84ae462d63170bb527"}, + {file = "asynq-1.5.1-cp38-cp38-win32.whl", hash = "sha256:c5c139dbfa4cdecb411fd7298ec275fd6ce0e6845ef91dfb213162ed9f761694"}, + {file = "asynq-1.5.1-cp38-cp38-win_amd64.whl", hash = "sha256:80f3d4d2cc9d5453f7d57570ae8a186e2cd491f76acc9085bcc2008511f72c0e"}, + {file = "asynq-1.5.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:cf848faf162ad655058ae0e99929750fc75d59d390e6b96dabd482a4ef4b7798"}, + {file = "asynq-1.5.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:12ec21dee0027d9783c3d44641304e72ac3928fb293e227874cfdab76cec74b1"}, + {file = "asynq-1.5.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:089833f67f178f93cfbc3ced48300591197b8afd51b83bb2e27c97b3689a57a6"}, + {file = "asynq-1.5.1-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:d38ea10eb99c60bcccfbc6b1bd88a3edf7ff7db98a1cb3a166e181f7d0650964"}, + {file = "asynq-1.5.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:7b5431242f09e7b6da47d0df1c56b6533d49acc1c84f23f4adaf0424f12e5295"}, + {file = "asynq-1.5.1-cp39-cp39-win32.whl", hash = "sha256:040e1d02fce358b1622cef41ba1f1e07bacce7736d4419c9e4ae096d5112638a"}, + {file = "asynq-1.5.1-cp39-cp39-win_amd64.whl", hash = "sha256:0f1146c2a91fc7cbfdc73036fcd4aa434121c04619509747d6ef952b669c540b"}, + {file = "asynq-1.5.1-pp37-pypy37_pp73-macosx_10_9_x86_64.whl", hash = "sha256:3210c05349d5582b51b23da69b3e5d1a5d11256782eff318327ebebb95d745ca"}, + {file = "asynq-1.5.1-pp37-pypy37_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:eb98e69916bc2c3e27f25634c4734a481a56778e6cd2bd176861fcb5dc38a988"}, + {file = "asynq-1.5.1-pp37-pypy37_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7880e44d7daef4eace2168afd763c95950bab3e54150ba803948939e2c927a25"}, + {file = "asynq-1.5.1-pp37-pypy37_pp73-win_amd64.whl", hash = "sha256:46b3b262c0f60901a4ce23daf4ba82b024cdff547e04fdcb00719a95b01a8c37"}, + {file = "asynq-1.5.1-pp38-pypy38_pp73-macosx_10_9_x86_64.whl", hash = "sha256:779a66c9440fe2e0d709b37406fe3977ecda66ae41f880b8cf34f79e0df1b451"}, + {file = "asynq-1.5.1-pp38-pypy38_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c9733a90510fddeb31976d3dfa449721ca8bd9caaf6d3d5fb0bc04032b094aa4"}, + {file = "asynq-1.5.1-pp38-pypy38_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:53dd5b7ea6ce911a3967485bbc8e3c435dc1008e9f31ea97f57804617828eea7"}, + {file = "asynq-1.5.1-pp38-pypy38_pp73-win_amd64.whl", hash = "sha256:40fefa839fe2cfbcd072c7249c0c4149af75d1d48573f3fc1e01c6b2bb6169f1"}, + {file = "asynq-1.5.1-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:a7be45e9336f52a9329cde19ece904fd93611da3602f5a98fbf119406dc6f340"}, + {file = "asynq-1.5.1-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6d5bbd5b68daf3e1a669c540cdc7a91a2ac9c49729ffae06d1790358759a1025"}, + {file = "asynq-1.5.1-pp39-pypy39_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:2cc72527d002fab4bca0d560e82e440ae69d42e393c59747ad2c8a2eb972aaa8"}, + {file = "asynq-1.5.1-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:8774eb6a042b1a011f1f7c3db9f555e83929184139fe2bfa1490a893819b04e5"}, + {file = "asynq-1.5.1.tar.gz", hash = "sha256:d1ae6a9b4934e821e22a08369512756d0edb9339436ae805b5fc815dacc4421e"}, +] + +[package.dependencies] +Cython = ">=0.27.1" +pygments = "*" +qcore = "*" + +[[package]] +name = "codemod" +version = "1.0.0" +description = "Codemod is a tool/library to assist you with large-scale codebase refactors that can be partially automated but still require human oversight and occasional intervention. Codemod was developed at Facebook and released as open source." +category = "dev" +optional = false +python-versions = "*" +files = [ + {file = "codemod-1.0.0.tar.gz", hash = "sha256:06e8c75f2b45210dd8270e30a6a88ae464b39abd6d0cab58a3d7bfd1c094e588"}, +] + +[[package]] +name = "cython" +version = "3.0.6" +description = "The Cython compiler for writing C extensions in the Python language." +category = "dev" +optional = false +python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*" +files = [ + {file = "Cython-3.0.6-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:0fcdfbf6fc7d0bd683d55e617c3d5a5f25b28ce8b405bc1e89054fc7c52a97e5"}, + {file = "Cython-3.0.6-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ccbee314f8d15ee8ddbe270859dda427e1187123f2c7c41526d1f260eee6c8f7"}, + {file = "Cython-3.0.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:14b992f36ffa1294921fca5f6488ea192fadd75770dc64fa25975379382551e9"}, + {file = "Cython-3.0.6-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:2ca2e90a75d405070f3c41e701bb8005892f14d42322f1d8fd00a61d660bbae7"}, + {file = "Cython-3.0.6-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:4121c1160bc1bd8828546e8ce45906bd9ff27799d14747ce3fbbc9d67efbb1b8"}, + {file = "Cython-3.0.6-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:519814b8f80869ee5f9ee2cb2363e5c310067c0298cbea291c556b22da1ef6ae"}, + {file = "Cython-3.0.6-cp310-cp310-win32.whl", hash = "sha256:b029d8c754ef867ab4d67fc2477dde9782bf0409cb8e4024a7d29cf5aff37530"}, + {file = "Cython-3.0.6-cp310-cp310-win_amd64.whl", hash = "sha256:2262390f453eedf600e084b074144286576ed2a56bb7fbfe15ad8d9499eceb52"}, + {file = "Cython-3.0.6-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:dfe8c7ac60363769ed8d91fca26398aaa9640368ab999a79b0ccb5e788d3bcf8"}, + {file = "Cython-3.0.6-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9e31a9b18ec6ce57eb3479df920e6093596fe4ba8010dcc372720040386b4bdb"}, + {file = "Cython-3.0.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bca2542f1f34f0141475b13777df040c31f2073a055097734a0a793ac3a4fb72"}, + {file = "Cython-3.0.6-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b24c1c38dad4bd85e142ccbe2f88122807f8d5a75352321e1e4baf2b293df7c6"}, + {file = "Cython-3.0.6-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:dc4b4e76c1414584bb55465dfb6f41dd6bd27fd53fb41ddfcaca9edf00c1f80e"}, + {file = "Cython-3.0.6-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:805a2c532feee09aeed064eaeb7b6ee35cbab650569d0a3756975f3cc4f246cf"}, + {file = "Cython-3.0.6-cp311-cp311-win32.whl", hash = "sha256:dcdb9a177c7c385fe0c0709a9a6790b6508847d67dcac76bb65a2c7ea447efe5"}, + {file = "Cython-3.0.6-cp311-cp311-win_amd64.whl", hash = "sha256:b8640b7f6503292c358cef925df5a69adf230045719893ffe20ad98024fdf7ae"}, + {file = "Cython-3.0.6-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:16b3b02cc7b3bc42ee1a0118b1465ca46b0f3fb32d003e6f1a3a352a819bb9a3"}, + {file = "Cython-3.0.6-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:11e1d9b153573c425846b627bef52b3b99cb73d4fbfbb136e500a878d4b5e803"}, + {file = "Cython-3.0.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:85a7a406f78c2f297bf82136ff5deac3150288446005ed1e56552a9e3ac1469f"}, + {file = "Cython-3.0.6-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:88be4fbc760de8f313df89ca8256098c0963c9ec72f3aa88538384b80ef1a6ef"}, + {file = "Cython-3.0.6-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:ea2e5a7c503b41618bfb10e4bc610f780ab1c729280531b5cabb24e05aa21cf2"}, + {file = "Cython-3.0.6-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:2d296b48e1410cab50220a28a834167f2d7ac6c0e7de12834d66e42248a1b0f6"}, + {file = "Cython-3.0.6-cp312-cp312-win32.whl", hash = "sha256:7f19e99c6e334e9e30dfa844c3ca4ac09931b94dbba406c646bde54687aed758"}, + {file = "Cython-3.0.6-cp312-cp312-win_amd64.whl", hash = "sha256:9cae02e26967ffb6503c6e91b77010acbadfb7189a5a11d6158d634fb0f73679"}, + {file = "Cython-3.0.6-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:cb6a54543869a5b0ad009d86eb0ebc0879fab838392bfd253ad6d4f5e0f17d84"}, + {file = "Cython-3.0.6-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8d2d9e53bf021cc7a5c7b6b537b5b5a7ba466ba7348d498aa17499d0ad12637e"}, + {file = "Cython-3.0.6-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:05d15854b2b363b35c755d22015c1c2fc590b8128202f8c9eb85578461101d9c"}, + {file = "Cython-3.0.6-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e5548316497a3b8b2d9da575ea143476472db90dee73c67def061621940f78ae"}, + {file = "Cython-3.0.6-cp36-cp36m-musllinux_1_1_aarch64.whl", hash = "sha256:9b853e0855e4b3d164c05b24718e5e2df369e5af54f47cb8d923c4f497dfc92c"}, + {file = "Cython-3.0.6-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:2c77f97f462a40a319dda7e28c1669370cb26f9175f3e8f9bab99d2f8f3f2f09"}, + {file = "Cython-3.0.6-cp36-cp36m-win32.whl", hash = "sha256:3ac8b6734f2cad5640f2da21cd33cf88323547d07e445fb7453ab38ec5033b1f"}, + {file = "Cython-3.0.6-cp36-cp36m-win_amd64.whl", hash = "sha256:8dd5f5f3587909ff71f0562f50e00d4b836c948e56e8f74897b12f38a29e41b9"}, + {file = "Cython-3.0.6-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:9c0472c6394750469062deb2c166125b10411636f63a0418b5c36a60d0c9a96a"}, + {file = "Cython-3.0.6-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:97081932c8810bb99cb26b4b0402202a1764b58ee287c8b306071d2848148c24"}, + {file = "Cython-3.0.6-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e781b3880dfd0d4d37983c9d414bfd5f26c2141f6d763d20ef1964a0a4cb2405"}, + {file = "Cython-3.0.6-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:ef88c46e91e21772a5d3b6b1e70a6da5fe098154ad4768888129b1c05e93bba7"}, + {file = "Cython-3.0.6-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:a38b9e7a252ec27dbc21ee8f00f09a896e88285eebb6ed99207b2ff1ea6af28e"}, + {file = "Cython-3.0.6-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:4975cdaf720d29288ec225b76b4f4471ff03f4f8b51841ba85d6587699ab2ad5"}, + {file = "Cython-3.0.6-cp37-cp37m-win32.whl", hash = "sha256:9b89463ea330318461ca47d3e49b5f606e7e82446b6f37e5c19b60392439674c"}, + {file = "Cython-3.0.6-cp37-cp37m-win_amd64.whl", hash = "sha256:0ca8f379b47417bfad98faeb14bf8a3966fc92cf69f8aaf7635cf6885e50d001"}, + {file = "Cython-3.0.6-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:b3dda1e80eb577b9563cee6cf31923a7b88836b9f9be0043ec545b138b95d8e8"}, + {file = "Cython-3.0.6-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:267e34e9a96f98c379100ef4192994a311678fb5c9af34c83ba5230223577581"}, + {file = "Cython-3.0.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:345d9112fde4ae0347d656f58591fd52017c61a19779c95423bb38735fe4a401"}, + {file = "Cython-3.0.6-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:25da0e51331ac12ff16cd858d1d836e092c984e1dc45d338166081d3802297c0"}, + {file = "Cython-3.0.6-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:eebbf09089b4988b9f398ed46f168892e32fcfeec346b15954fdd818aa103456"}, + {file = "Cython-3.0.6-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:e3ed0c125556324fa49b9e92bea13be7b158fcae6f72599d63c8733688257788"}, + {file = "Cython-3.0.6-cp38-cp38-win32.whl", hash = "sha256:86e1e5a5c9157a547d0a769de59c98a1fc5e46cfad976f32f60423cc6de11052"}, + {file = "Cython-3.0.6-cp38-cp38-win_amd64.whl", hash = "sha256:0d45a84a315bd84d1515cd3571415a0ee0709eb4e2cd4b13668ede928af344a7"}, + {file = "Cython-3.0.6-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:a8e788e64b659bb8fe980bc37da3118e1f7285dec40c5fb293adabc74d4205f2"}, + {file = "Cython-3.0.6-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9a77a174c7fb13d80754c8bf9912efd3f3696d13285b2f568eca17324263b3f7"}, + {file = "Cython-3.0.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1074e84752cd0daf3226823ddbc37cca8bc45f61c94a1db2a34e641f2b9b0797"}, + {file = "Cython-3.0.6-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:49d5cae02d56e151e1481e614a1af9a0fe659358f2aa5eca7a18f05aa641db61"}, + {file = "Cython-3.0.6-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:0b94610fa49e36db068446cfd149a42e3246f38a4256bbe818512ac181446b4b"}, + {file = "Cython-3.0.6-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:fabb2d14dd71add618a7892c40ffec584d1dae1e477caa193778e52e06821d83"}, + {file = "Cython-3.0.6-cp39-cp39-win32.whl", hash = "sha256:ce442c0be72ab014c305399d955b78c3d1e69d5a5ce24398122b605691b69078"}, + {file = "Cython-3.0.6-cp39-cp39-win_amd64.whl", hash = "sha256:8a05f79a0761fc76c42e945e5a9cb5d7986aa9e8e526fdf52bd9ca61a12d4567"}, + {file = "Cython-3.0.6-py2.py3-none-any.whl", hash = "sha256:5921a175ea20779d4443ef99276cfa9a1a47de0e32d593be7679be741c9ed93b"}, + {file = "Cython-3.0.6.tar.gz", hash = "sha256:399d185672c667b26eabbdca420c98564583798af3bc47670a8a09e9f19dd660"}, +] + [[package]] name = "flake8" version = "6.1.0" @@ -17,6 +191,25 @@ mccabe = ">=0.7.0,<0.8.0" pycodestyle = ">=2.11.0,<2.12.0" pyflakes = ">=3.1.0,<3.2.0" +[[package]] +name = "importlib-resources" +version = "6.1.1" +description = "Read resources from Python packages" +category = "dev" +optional = false +python-versions = ">=3.8" +files = [ + {file = "importlib_resources-6.1.1-py3-none-any.whl", hash = "sha256:e8bf90d8213b486f428c9c39714b920041cb02c184686a3dee24905aaa8105d6"}, + {file = "importlib_resources-6.1.1.tar.gz", hash = "sha256:3893a00122eafde6894c59914446a512f728a0c1a45f9bb9b63721b6bacf0b4a"}, +] + +[package.dependencies] +zipp = {version = ">=3.1.0", markers = "python_version < \"3.10\""} + +[package.extras] +docs = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "rst.linker (>=1.9)", "sphinx (<7.2.5)", "sphinx (>=3.5)", "sphinx-lint"] +testing = ["pytest (>=6)", "pytest-black (>=0.3.7)", "pytest-checkdocs (>=2.4)", "pytest-cov", "pytest-enabler (>=2.2)", "pytest-mypy (>=0.9.1)", "pytest-ruff", "zipp (>=3.17)"] + [[package]] name = "jinja2" version = "3.1.2" @@ -137,6 +330,31 @@ pygments = ">=2.12.0" [package.extras] dev = ["black", "hypothesis", "mypy", "pygments (>=2.14.0)", "pytest", "pytest-cov", "pytest-timeout", "ruff", "tox", "types-pygments"] +[[package]] +name = "pyanalyze" +version = "0.11.0" +description = "A static analyzer for Python" +category = "dev" +optional = false +python-versions = ">=3.8" +files = [ + {file = "pyanalyze-0.11.0-py3-none-any.whl", hash = "sha256:f2e5c1023eca53f7825c64ef44721f667810f8f086f817df22711337bb9ab683"}, + {file = "pyanalyze-0.11.0.tar.gz", hash = "sha256:0258648e2c919f849343cc942c61556fff189b03dea563f2b4c48620c4ca4ab5"}, +] + +[package.dependencies] +aenum = ">=2.2.3" +ast-decompiler = ">=0.4.0" +asynq = "*" +codemod = "*" +qcore = ">=0.5.1" +tomli = ">=1.1.0" +typeshed-client = ">=2.1.0" +typing-extensions = ">=4.1.0" + +[package.extras] +tests = ["annotated-types", "attrs", "mypy-extensions", "pydantic", "pytest"] + [[package]] name = "pycodestyle" version = "2.11.1" @@ -163,32 +381,153 @@ files = [ [[package]] name = "pygments" -version = "2.16.1" +version = "2.17.2" description = "Pygments is a syntax highlighting package written in Python." category = "dev" optional = false python-versions = ">=3.7" files = [ - {file = "Pygments-2.16.1-py3-none-any.whl", hash = "sha256:13fc09fa63bc8d8671a6d247e1eb303c4b343eaee81d861f3404db2935653692"}, - {file = "Pygments-2.16.1.tar.gz", hash = "sha256:1daff0494820c69bc8941e407aa20f577374ee88364ee10a98fdbe0aece96e29"}, + {file = "pygments-2.17.2-py3-none-any.whl", hash = "sha256:b27c2826c47d0f3219f29554824c30c5e8945175d888647acd804ddd04af846c"}, + {file = "pygments-2.17.2.tar.gz", hash = "sha256:da46cec9fd2de5be3a8a784f434e4c4ab670b4ff54d605c4c2717e9d49c4c367"}, ] [package.extras] plugins = ["importlib-metadata"] +windows-terminal = ["colorama (>=0.4.6)"] + +[[package]] +name = "qcore" +version = "1.10.0" +description = "Quora's core utility library" +category = "dev" +optional = false +python-versions = "*" +files = [ + {file = "qcore-1.10.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:230d5139ed85ee7befe1291e36cd9e9f83b14437fad34c01ca08a51fc5b4e07f"}, + {file = "qcore-1.10.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:658469d6150d9be77812f2f5c1d3333a1c59b744acb8072172a44d2bc876ec89"}, + {file = "qcore-1.10.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:3c3d8477c17153326571fcc60e74b0f337fda4e8c0a72e4347e0a82a22c70211"}, + {file = "qcore-1.10.0-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:9648a93350023dc514a04a94a7a48d32683493e8d7481cca4803d3773d3d3d49"}, + {file = "qcore-1.10.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:ae8accf69ed503fb41e76a3e5846b0e3f41d6e0a374c2891ea46a57958039439"}, + {file = "qcore-1.10.0-cp310-cp310-win32.whl", hash = "sha256:77f4beeb717aec0ad90c8b1f9e30d8cb599d682d1f3ac574707ffa54709d3942"}, + {file = "qcore-1.10.0-cp310-cp310-win_amd64.whl", hash = "sha256:d2bfd706ecf78763580e46911a7b01e32741d3f51eb52e51a12235120573c3a6"}, + {file = "qcore-1.10.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:aad41cf2c751d96720b026585427e4764955320f72c581ef270e6415f2284230"}, + {file = "qcore-1.10.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f369f4761e1c171d91c721701225219776440ae93c0f0bf2a65d2ab774ff7070"}, + {file = "qcore-1.10.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:675ef17fef24b2853c169cdb3c5da00aa93d7f6dd3590b6c5a48fd6984d51192"}, + {file = "qcore-1.10.0-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:0d1b3ca6cc5aa9d413814e088102a2f5680b89da30dada45d2d854fb78a00833"}, + {file = "qcore-1.10.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:6da2119af6a5d59a304cf2dfb9beaa450e54aa79b239840ea4047ecae6659cbb"}, + {file = "qcore-1.10.0-cp311-cp311-win32.whl", hash = "sha256:6a7fcfabc2be07d728f0f507bba8bc79df0895f0ecf2f19f7b0f70928318880a"}, + {file = "qcore-1.10.0-cp311-cp311-win_amd64.whl", hash = "sha256:a3e317e0406edb0c6b776d37dc5c6891cd50ec9adda285b379cd2098a6dbb6de"}, + {file = "qcore-1.10.0-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:cefb20a1ab5ca5c34a86dc2b4f52f12cc909ba574912aef13b2f738912429a9d"}, + {file = "qcore-1.10.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1b003fb8116fd815a6258e79b0eb5d77743faabd4c51d0142904241bb97b7115"}, + {file = "qcore-1.10.0-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:b290c99d1fa4c103b27de2c631db57d705eb0c7bf935158a5335b9529604951b"}, + {file = "qcore-1.10.0-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:abc664b95281a671b0fddabbd875d449d1ec616a05fc71d53caed8f2130ebc02"}, + {file = "qcore-1.10.0-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:f948a5a3064bf7d8f59e94b891e519718d01bafc0e9f3850017cb4b935d21040"}, + {file = "qcore-1.10.0-cp36-cp36m-win32.whl", hash = "sha256:34d55e604ee02ec1235248dca427d88ac06ccc28fe3cba618cceaf0d4fac595c"}, + {file = "qcore-1.10.0-cp36-cp36m-win_amd64.whl", hash = "sha256:3b30b6ffe2067e21c882ed3093538f0361dc800de63e0e15a099cf71c08dfd62"}, + {file = "qcore-1.10.0-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:123b0605634155f7fb3fd88b0120d7f05c2dbe714cbbf3097eaa59f6e28f3bbc"}, + {file = "qcore-1.10.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:23e0e8173330b5433586ba6362a57c3424ae90add7a967c5ab11a3768492873f"}, + {file = "qcore-1.10.0-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:02d7d6f3c9583403a16d0bd1905dc3ccb3edbca3529b1c2da8b09ef17599aa00"}, + {file = "qcore-1.10.0-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:44c2309531e67b9b9531eb9318940a8cb6cd9a4bca1b0ae01f79925527c77790"}, + {file = "qcore-1.10.0-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:8ef47b985a75cabbac656b38643b6c17114359b9a290cdf0e7e5bd9031b7672e"}, + {file = "qcore-1.10.0-cp37-cp37m-win32.whl", hash = "sha256:91d742174f7beff7b967ddf80253fa95f055a2063d69c73d40d4d928cec3f72a"}, + {file = "qcore-1.10.0-cp37-cp37m-win_amd64.whl", hash = "sha256:21b2952625a781465e67789fe687819d704800fd32ad19493f3225031869a0a0"}, + {file = "qcore-1.10.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:044008dbe127bf2d17a618210d607c73a0894a887185ff871b9d45f755469a6b"}, + {file = "qcore-1.10.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:071e933bbabea344ae9062af892f60351a5eec30f8f16230aa334bd17d82f09e"}, + {file = "qcore-1.10.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:8037cc36d2e64590a9974375e88047225dc8f00297d40cf581e38a765b9cd88d"}, + {file = "qcore-1.10.0-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:79402feea121bdae1ba595e199c681c6bbcef19211a79cfcad744519b872c44f"}, + {file = "qcore-1.10.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:0fe05dbc40483cf53c348faff7e1463c4c7465907072342c330c9663f1435dd7"}, + {file = "qcore-1.10.0-cp38-cp38-win32.whl", hash = "sha256:d96feb04f6521bde1eae9ae6c4b62bed47e9c84adb8c28b1dbc592e8bfa5f5ce"}, + {file = "qcore-1.10.0-cp38-cp38-win_amd64.whl", hash = "sha256:65dc62d408f340b9bb0a3d0dd2a506218dd614439a63898c1777c0b0000b517e"}, + {file = "qcore-1.10.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:c0bf625f48d74f1e4f4550df5e79626a60051ed2e3ec413a7eeec91a35225636"}, + {file = "qcore-1.10.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1f347126bdc32d8c571a390df579c1abbea01c210f8a8c3d37d956ba01e53c60"}, + {file = "qcore-1.10.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:511fa1a71bbb359b4105a4afeb577a043681b5cb93d877909421e9f7a3c595b2"}, + {file = "qcore-1.10.0-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:d45dddc8b02f4d968b034438b7be08ca95dc799465d5fdfc6d5a525ebaf96aeb"}, + {file = "qcore-1.10.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:df87f1a2c8e69fe9b916047383949919a445574b68400f59c888a4456df1110d"}, + {file = "qcore-1.10.0-cp39-cp39-win32.whl", hash = "sha256:48ffdc18f930a34d22c466fd0c6ce1832848c6d82f77caab13d5c6788ca348b2"}, + {file = "qcore-1.10.0-cp39-cp39-win_amd64.whl", hash = "sha256:566b1d3bd6771c7db9ba7188ab3daa1e9cf9290d8d52f20e7132e1c2d95979fd"}, + {file = "qcore-1.10.0-pp37-pypy37_pp73-macosx_10_9_x86_64.whl", hash = "sha256:50cc977da46c5dad7a98b3983a62e19100cec8043e8c24ff7e1a3f83630ed970"}, + {file = "qcore-1.10.0-pp37-pypy37_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:37b264a0b83f513d4fd1b12bbb8339146ae1893d18f9476e3df562f50620d2bf"}, + {file = "qcore-1.10.0-pp37-pypy37_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:72555913df64b0d3487f5d3156d7a2a618eb19f91e67ff50e9237915d8b6ed0b"}, + {file = "qcore-1.10.0-pp37-pypy37_pp73-win_amd64.whl", hash = "sha256:3e619573e2f0b8e16cd1f43ff4801d6cab541031be48ae1711c645445ea11755"}, + {file = "qcore-1.10.0-pp38-pypy38_pp73-macosx_10_9_x86_64.whl", hash = "sha256:68d5aec5186799fcbe651c383b489c27ba9c23624ea293ce7904fab36c0fe8c8"}, + {file = "qcore-1.10.0-pp38-pypy38_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:cfbb183a804f986d37a3f035dfd53d536271d020b4636e85ee7cfa30ca7a83e9"}, + {file = "qcore-1.10.0-pp38-pypy38_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:97e564ae11157c88708042aaf3a615bafcde258b6f18832f1605ef5297fd6d5b"}, + {file = "qcore-1.10.0-pp38-pypy38_pp73-win_amd64.whl", hash = "sha256:8b1737f71952a1f184d946200af0f50b93339228fd19e4f995693b003d7ad3fc"}, + {file = "qcore-1.10.0-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:fd756b471b74009bf4d403aa7f0c97a82a14a50288cfa9b547acb3563b84790d"}, + {file = "qcore-1.10.0-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:dd729c2e01d0da7acc77f32f471cbe2d0777b883419546055bf8d579b5b0a73d"}, + {file = "qcore-1.10.0-pp39-pypy39_pp73-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ff5fa19dffac882e564a9e2e0314c1b5373747b9aad0054f3570673f7e8f71ed"}, + {file = "qcore-1.10.0-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:0fbc0d69a3b973641e520fc9d139753dad971978a13361d808e878499eeea010"}, + {file = "qcore-1.10.0.tar.gz", hash = "sha256:88c3dcc0f1b4843eae063334fb4b278c4660c36db6382ffa98bc1150be71e8a1"}, +] [[package]] name = "simpy" -version = "4.0.2" +version = "4.1.1" description = "Event discrete, process based simulation for Python." category = "main" optional = false -python-versions = ">=3.6" +python-versions = ">=3.8" +files = [ + {file = "simpy-4.1.1-py3-none-any.whl", hash = "sha256:7c5ae380240fd2238671160e4830956f8055830a8317edf5c05e495b3823cd88"}, + {file = "simpy-4.1.1.tar.gz", hash = "sha256:06d0750a7884b11e0e8e20ce0bc7c6d4ed5f1743d456695340d13fdff95001a6"}, +] + +[[package]] +name = "tomli" +version = "2.0.1" +description = "A lil' TOML parser" +category = "dev" +optional = false +python-versions = ">=3.7" +files = [ + {file = "tomli-2.0.1-py3-none-any.whl", hash = "sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc"}, + {file = "tomli-2.0.1.tar.gz", hash = "sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f"}, +] + +[[package]] +name = "typeshed-client" +version = "2.4.0" +description = "A library for accessing stubs in typeshed." +category = "dev" +optional = false +python-versions = ">=3.7" +files = [ + {file = "typeshed_client-2.4.0-py3-none-any.whl", hash = "sha256:5358cab27cf2d7b1cd1e77dd92a3ac3cd9cd31df9eb2e958bd280a38160a3219"}, + {file = "typeshed_client-2.4.0.tar.gz", hash = "sha256:b4e4e3e40dca91ce1a667d2eb0eb350a0a2c0d80e18a232d18857aa61bed3492"}, +] + +[package.dependencies] +importlib-resources = ">=1.4.0" + +[[package]] +name = "typing-extensions" +version = "4.8.0" +description = "Backported and Experimental Type Hints for Python 3.8+" +category = "dev" +optional = false +python-versions = ">=3.8" +files = [ + {file = "typing_extensions-4.8.0-py3-none-any.whl", hash = "sha256:8f92fc8806f9a6b641eaa5318da32b44d401efaac0f6678c9bc448ba3605faa0"}, + {file = "typing_extensions-4.8.0.tar.gz", hash = "sha256:df8e4339e9cb77357558cbdbceca33c303714cf861d1eef15e1070055ae8b7ef"}, +] + +[[package]] +name = "zipp" +version = "3.17.0" +description = "Backport of pathlib-compatible object wrapper for zip files" +category = "dev" +optional = false +python-versions = ">=3.8" files = [ - {file = "simpy-4.0.2-py2.py3-none-any.whl", hash = "sha256:603cdf4299e396c9f16b10806e749decb0d08a7e72e0c26f9eb9762b9bde29cc"}, - {file = "simpy-4.0.2.tar.gz", hash = "sha256:6d8adc0229df6b02fb7e26dcd1338703b4f4f63f167a5ac2a7213cb80aba4484"}, + {file = "zipp-3.17.0-py3-none-any.whl", hash = "sha256:0e923e726174922dce09c53c59ad483ff7bbb8e572e00c7f7c46b88556409f31"}, + {file = "zipp-3.17.0.tar.gz", hash = "sha256:84e64a1c28cf7e91ed2078bb8cc8c259cb19b76942096c8d7b84947690cabaf0"}, ] +[package.extras] +docs = ["furo", "jaraco.packaging (>=9.3)", "jaraco.tidelift (>=1.4)", "rst.linker (>=1.9)", "sphinx (<7.2.5)", "sphinx (>=3.5)", "sphinx-lint"] +testing = ["big-O", "jaraco.functools", "jaraco.itertools", "more-itertools", "pytest (>=6)", "pytest-black (>=0.3.7)", "pytest-checkdocs (>=2.4)", "pytest-cov", "pytest-enabler (>=2.2)", "pytest-ignore-flaky", "pytest-mypy (>=0.9.1)", "pytest-ruff"] + [metadata] lock-version = "2.0" -python-versions = "^3.10" -content-hash = "d9f1f22d9f90f2ad4d00ec2785eee468e847dc1c8c4015615febe6094edf3e62" +python-versions = "^3.9.2" +content-hash = "7a2dfd4ab1e5ffb3daf67698df7cb1c9969cf7963b9c74d9c1b6b31bc9e01f6d" diff --git a/pyproject.toml b/pyproject.toml index b503d83..f4d1e94 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -13,10 +13,12 @@ simpy = "^4" [tool.poetry.dev-dependencies] flake8 = "^6" pdoc = "^14" +pyanalyze = "^0.11" [tool.poetry.scripts] demo = "simtfl.demo:run" bc-demo = "simtfl.bc.demo:run" +pyanalyze = "pyanalyze.__main__:main" [build-system] requires = ["poetry-core"] diff --git a/simtfl/bc/__init__.py b/simtfl/bc/__init__.py index c7c7110..a9e4180 100644 --- a/simtfl/bc/__init__.py +++ b/simtfl/bc/__init__.py @@ -15,9 +15,14 @@ actual privacy properties. """ -from collections import deque + +from __future__ import annotations +from typing import Iterable, Optional +from collections.abc import Sequence from dataclasses import dataclass from enum import Enum, auto + +from collections import deque from itertools import chain, islice from sys import version_info @@ -34,7 +39,7 @@ class BCTransaction: @dataclass(frozen=True) class _TXO: - tx: 'BCTransaction' + tx: BCTransaction index: int value: int @@ -51,8 +56,14 @@ class _Note(Unique): """ value: int - def __init__(self, transparent_inputs, transparent_output_values, shielded_inputs, - shielded_output_values, fee, anchor=None, issuance=0): + def __init__(self, + transparent_inputs: Sequence[BCTransaction._TXO], + transparent_output_values: Sequence[int], + shielded_inputs: Sequence[BCTransaction._Note], + shielded_output_values: Sequence[int], + fee: int, + anchor: Optional[BCContext]=None, + issuance: int=0): """ Constructs a `BCTransaction` with the given transparent inputs, transparent output values, anchor, shielded inputs, shielded output values, fee, and @@ -78,8 +89,6 @@ def __init__(self, transparent_inputs, transparent_output_values, shielded_input assert fee >= 0 or coinbase assert issuance == 0 or coinbase assert all((v >= 0 for v in chain(transparent_output_values, shielded_output_values))) - assert all((isinstance(txin, self._TXO) for txin in transparent_inputs)) - assert all((isinstance(note, self._Note) for note in shielded_inputs)) assert ( sum((txin.value for txin in transparent_inputs)) + sum((note.value for note in shielded_inputs)) @@ -88,7 +97,8 @@ def __init__(self, transparent_inputs, transparent_output_values, shielded_input + sum(shielded_output_values) + fee ) - assert anchor is None if len(shielded_inputs) == 0 else anchor.can_spend(shielded_inputs) + assert anchor is None if len(shielded_inputs) == 0 else ( + anchor is not None and anchor.can_spend(shielded_inputs)) self.transparent_inputs = transparent_inputs self.transparent_outputs = [self._TXO(self, i, v) @@ -99,23 +109,23 @@ def __init__(self, transparent_inputs, transparent_output_values, shielded_input self.anchor = anchor self.issuance = issuance - def transparent_input(self, index): + def transparent_input(self, index: int) -> BCTransaction._TXO: """Returns the transparent input TXO with the given index.""" return self.transparent_inputs[index] - def transparent_output(self, index): + def transparent_output(self, index: int) -> BCTransaction._TXO: """Returns the transparent output TXO with the given index.""" return self.transparent_outputs[index] - def shielded_input(self, index): + def shielded_input(self, index: int) -> BCTransaction._Note: """Returns the shielded input note with the given index.""" return self.shielded_inputs[index] - def shielded_output(self, index): + def shielded_output(self, index: int) -> BCTransaction._Note: """Returns the shielded output note with the given index.""" return self.shielded_outputs[index] - def is_coinbase(self): + def is_coinbase(self) -> bool: """ Returns `True` if this is a coinbase transaction (it has no inputs). """ @@ -140,27 +150,27 @@ class BCContext: def __init__(self): """Constructs an empty `BCContext`.""" - self.transactions = deque() # of BCTransaction - self.utxo_set = set() # of BCTransaction._TXO + self.transactions: deque[BCTransaction] = deque() + self.utxo_set: set[BCTransaction._TXO] = set() # Since dicts are insertion-ordered, this models the sequence in which # notes are committed as well as their spentness. - self.notes = {} # Note -> Spent | Unspent + self.notes: dict[BCTransaction._Note, Spentness] = {} self.total_issuance = 0 - def committed_notes(self): + def committed_notes(self) -> list[(BCTransaction._Note, Spentness)]: """ Returns a list of (`Note`, `Spentness`) for notes added to this context, preserving the commitment order. """ return list(self.notes.items()) - def can_spend(self, tospend): + def can_spend(self, tospend: Iterable[BCTransaction._Note]) -> bool: """Can all of the notes in `tospend` be spent in this context?""" return all((self.notes.get(note) == Spentness.Unspent for note in tospend)) - def _check(self, tx): + def _check(self, tx: BCTransaction) -> tuple[bool, set[BCTransaction._TXO]]: """ Checks whether `tx` is valid. To avoid recomputation, this returns a pair of the validity, and the set of transparent inputs of `tx`. @@ -169,11 +179,11 @@ def _check(self, tx): valid = txins.issubset(self.utxo_set) and self.can_spend(tx.shielded_inputs) return (valid, txins) - def is_valid(self, tx): + def is_valid(self, tx: BCTransaction) -> bool: """Is `tx` valid in this context?""" return self._check(tx)[0] - def add_if_valid(self, tx): + def add_if_valid(self, tx: BCTransaction) -> bool: """ If `tx` is valid in this context, add it to the context and return `True`. Otherwise leave the context unchanged and return `False`. @@ -194,7 +204,7 @@ def add_if_valid(self, tx): return valid - def copy(self): + def copy(self) -> BCContext: """Returns an independent copy of this `BCContext`.""" ctx = BCContext() ctx.transactions = self.transactions.copy() @@ -207,7 +217,11 @@ def copy(self): class BCBlock: """A block in a best-chain protocol.""" - def __init__(self, parent, added_score, transactions, allow_invalid=False): + def __init__(self, + parent: Optional[BCBlock], + added_score: int, + transactions: Sequence[BCTransaction], + allow_invalid: bool=False): """ Constructs a `BCBlock` with the given parent block, score relative to the parent, and sequence of transactions. `transactions` must not be modified @@ -215,22 +229,23 @@ def __init__(self, parent, added_score, transactions, allow_invalid=False): If `allow_invalid` is set, the block need not be valid. Use `parent=None` to construct the genesis block. """ - assert all((isinstance(tx, BCTransaction) for tx in transactions)) self.parent = parent - self.score = (0 if parent is None else self.parent.score) + added_score + self.score = added_score + if self.parent is not None: + self.score += self.parent.score self.transactions = transactions self.hash = BlockHash() if not allow_invalid: self.assert_noncontextually_valid() - def assert_noncontextually_valid(self): + def assert_noncontextually_valid(self) -> None: """Assert that non-contextual consensus rules are satisfied for this block.""" assert len(self.transactions) > 0 assert self.transactions[0].is_coinbase() assert not any((tx.is_coinbase() for tx in islice(self.transactions, 1, None))) assert sum((tx.fee for tx in self.transactions)) == 0 - def is_noncontextually_valid(self): + def is_noncontextually_valid(self) -> bool: """Are non-contextual consensus rules satisfied for this block?""" try: self.assert_noncontextually_valid() @@ -260,7 +275,7 @@ class BCProtocol: class TestBC(unittest.TestCase): - def test_basic(self): + def test_basic(self) -> None: ctx = BCContext() coinbase_tx0 = BCTransaction([], [10], [], [], 0, issuance=10) self.assertTrue(ctx.add_if_valid(coinbase_tx0)) diff --git a/simtfl/bc/demo.py b/simtfl/bc/demo.py index 6c8929d..a78bb88 100644 --- a/simtfl/bc/demo.py +++ b/simtfl/bc/demo.py @@ -1,7 +1,14 @@ +""" +This demo just runs the `simtfl.bc` unit tests for now. +""" + + +from __future__ import annotations + import unittest -def run(): +def run() -> None: """ Runs the demo. """ diff --git a/simtfl/bft/__init__.py b/simtfl/bft/__init__.py index 2f2d138..da82aea 100644 --- a/simtfl/bft/__init__.py +++ b/simtfl/bft/__init__.py @@ -11,7 +11,10 @@ """ -def two_thirds_threshold(n): +from __future__ import annotations + + +def two_thirds_threshold(n: int) -> int: """ Calculate the notarization threshold used in most permissioned BFT protocols: `ceiling(n * 2/3)`. @@ -26,7 +29,7 @@ class PermissionedBFTBase: It is also used as a base class for other BFT block and proposal classes. """ - def __init__(self, n, t): + def __init__(self, n: int, t: int): """ Constructs a genesis block for a permissioned BFT protocol with `n` nodes, of which at least `t` must sign each proposal. @@ -35,7 +38,7 @@ def __init__(self, n, t): self.t = t self.parent = None - def last_final(self): + def last_final(self) -> PermissionedBFTBase: """ Returns the last final block in this block's ancestor chain. For the genesis block, this is itself. @@ -55,7 +58,7 @@ class PermissionedBFTBlock(PermissionedBFTBase): BFT blocks are taken to be notarized, and therefore valid, by definition. """ - def __init__(self, proposal): + def __init__(self, proposal: PermissionedBFTProposal): """Constructs a `PermissionedBFTBlock` for the given proposal.""" super().__init__(proposal.n, proposal.t) @@ -69,13 +72,13 @@ def last_final(self): This should be overridden by subclasses; the default implementation will (inefficiently) just return the genesis block. """ - return self.parent.last_final() + return self if self.parent is None else self.parent.last_final() class PermissionedBFTProposal(PermissionedBFTBase): """A proposal for a BFT protocol.""" - def __init__(self, parent): + def __init__(self, parent: PermissionedBFTBase): """ Constructs a `PermissionedBFTProposal` with the given parent `PermissionedBFTBlock`. The parameters are determined by the parent @@ -85,14 +88,14 @@ def __init__(self, parent): self.parent = parent self.signers = set() - def assert_valid(self): + def assert_valid(self) -> None: """ Assert that this proposal is valid. This does not assert that it is notarized. This should be overridden by subclasses. """ pass - def is_valid(self): + def is_valid(self) -> bool: """Is this proposal valid?""" try: self.assert_valid() @@ -100,7 +103,7 @@ def is_valid(self): except AssertionError: return False - def assert_notarized(self): + def assert_notarized(self) -> None: """ Assert that this proposal is notarized. A `PermissionedBFTProposal` is notarized iff it is valid and has at least the threshold number of @@ -109,7 +112,7 @@ def assert_notarized(self): self.assert_valid() assert len(self.signers) >= self.t - def is_notarized(self): + def is_notarized(self) -> bool: """Is this proposal notarized?""" try: self.assert_notarized() @@ -117,7 +120,7 @@ def is_notarized(self): except AssertionError: return False - def add_signature(self, index): + def add_signature(self, index: int) -> None: """ Record that the node with the given `index` has signed this proposal. If the same node signs more than once, the subsequent signatures are @@ -127,17 +130,19 @@ def add_signature(self, index): assert len(self.signers) <= self.n +__all__ = ['two_thirds_threshold', 'PermissionedBFTBase', 'PermissionedBFTBlock', 'PermissionedBFTProposal'] + import unittest class TestPermissionedBFT(unittest.TestCase): - def test_basic(self): + def test_basic(self) -> None: # Construct the genesis block. genesis = PermissionedBFTBase(5, 2) current = genesis self.assertEqual(current.last_final(), genesis) - for i in range(2): + for _ in range(2): proposal = PermissionedBFTProposal(current) proposal.assert_valid() self.assertTrue(proposal.is_valid()) @@ -159,7 +164,7 @@ def test_basic(self): current = PermissionedBFTBlock(proposal) self.assertEqual(current.last_final(), genesis) - def test_assertions(self): + def test_assertions(self) -> None: genesis = PermissionedBFTBase(5, 2) proposal = PermissionedBFTProposal(genesis) self.assertRaises(AssertionError, PermissionedBFTBlock, proposal) diff --git a/simtfl/bft/streamlet/__init__.py b/simtfl/bft/streamlet/__init__.py index ef8d531..42c7174 100644 --- a/simtfl/bft/streamlet/__init__.py +++ b/simtfl/bft/streamlet/__init__.py @@ -6,6 +6,10 @@ """ +from __future__ import annotations +from typing import Optional +from collections.abc import Sequence + from .. import PermissionedBFTBase, PermissionedBFTBlock, PermissionedBFTProposal, \ two_thirds_threshold @@ -13,28 +17,27 @@ class StreamletProposal(PermissionedBFTProposal): """An adapted-Streamlet proposal.""" - def __init__(self, parent, epoch): + def __init__(self, parent: StreamletBlock | StreamletGenesis, epoch: int): """ Constructs a `StreamletProposal` with the given parent `StreamletBlock`, for the given `epoch`. The parameters are determined by the parent block. """ - assert isinstance(parent, StreamletBlock) or isinstance(parent, StreamletGenesis) super().__init__(parent) self.epoch = epoch - def __repr__(self): + def __repr__(self) -> str: return "StreamletProposal(parent=%r, epoch=%r)" % (self.parent, self.epoch) class StreamletGenesis(PermissionedBFTBase): """An adapted-Streamlet genesis block.""" - def __init__(self, n): + def __init__(self, n: int): """Constructs a genesis block for adapted-Streamlet with `n` nodes.""" super().__init__(n, two_thirds_threshold(n)) self.epoch = None - def __repr__(self): + def __repr__(self) -> str: return "StreamletGenesis(n=%r)" % (self.n,) @@ -46,13 +49,12 @@ class StreamletBlock(PermissionedBFTBlock): `StreamletBlock`s are taken to be notarized, and therefore valid, by definition. """ - def __init__(self, proposal): + def __init__(self, proposal: StreamletProposal): """Constructs a `StreamletBlock` for the given proposal.""" - assert isinstance(proposal, StreamletProposal) super().__init__(proposal) self.epoch = proposal.epoch - def last_final(self): + def last_final(self) -> StreamletBlock | StreamletGenesis: """ Returns the last final block in this block's ancestor chain. In Streamlet this is the middle block of the last group of three @@ -72,7 +74,7 @@ def last_final(self): return middle (first, middle, last) = (first.parent, first, middle) - def __repr__(self): + def __repr__(self) -> str: return "StreamletBlock(proposal=%r)" % (self.proposal,) @@ -81,7 +83,7 @@ def __repr__(self): class TestStreamlet(unittest.TestCase): - def test_simple(self): + def test_simple(self) -> None: """ Very simple example. @@ -89,7 +91,7 @@ def test_simple(self): """ self._test_last_final([0, 1, 2], [0, 0, 2]) - def test_figure_1(self): + def test_figure_1(self) -> None: """ Figure 1: Streamlet finalization example (without the invalid 'X' proposal). @@ -109,7 +111,7 @@ def test_figure_1(self): """ self._test_last_final([0, 0, 1, None, 2, 5, 6], [0, 0, 0, 0, 0, 0, 6]) - def test_complex(self): + def test_complex(self) -> None: """ Safety Violation: due to three simultaneous properties: @@ -123,7 +125,7 @@ def test_complex(self): """ self._test_last_final([0, 0, 1, None, 2, 5, 6, 3, 8, 9], [0, 0, 0, 0, 0, 0, 6, 0, 0, 9]) - def _test_last_final(self, parent_map, final_map): + def _test_last_final(self, parent_map: Sequence[Optional[int]], final_map: Sequence[int]) -> None: """ This test constructs a tree of proposals with structure determined by `parent_map`, and asserts `block.last_final()` matches the structure @@ -146,7 +148,9 @@ def _test_last_final(self, parent_map, final_map): blocks.append(None) continue - proposal = StreamletProposal(blocks[parent_epoch], epoch) + parent = blocks[parent_epoch] + assert parent is not None + proposal = StreamletProposal(parent, epoch) proposal.assert_valid() self.assertTrue(proposal.is_valid()) self.assertFalse(proposal.is_notarized()) diff --git a/simtfl/bft/streamlet/node.py b/simtfl/bft/streamlet/node.py index b4c78c9..c63845d 100644 --- a/simtfl/bft/streamlet/node.py +++ b/simtfl/bft/streamlet/node.py @@ -2,12 +2,17 @@ An adapted-Streamlet node. """ + +from __future__ import annotations +from typing import Any + from ...node import SequentialNode from ...message import PayloadMessage -from ...util import skip +from ...util import skip, ProcessEffect from . import StreamletGenesis, StreamletBlock, StreamletProposal + class Echo(PayloadMessage): """ An echo of another message. Streamlet requires nodes to broadcast each received @@ -19,16 +24,15 @@ class Echo(PayloadMessage): class StreamletNode(SequentialNode): """A Streamlet node.""" - def __init__(self, genesis): + def __init__(self, genesis: StreamletGenesis): """ Constructs a Streamlet node with parameters taken from the given `genesis` block (an instance of `StreamletGenesis`). """ - assert isinstance(genesis, StreamletGenesis) self.genesis = genesis self.voted_epoch = -1 - def handle(self, sender, message): + def handle(self, sender: int, message: Any) -> ProcessEffect: """ (process) Message handler for a Streamlet node: * `Echo` messages are unwrapped and treated like the original message. @@ -51,7 +55,7 @@ def handle(self, sender, message): else: yield from super().handle(sender, message) - def handle_proposal(self, proposal): + def handle_proposal(self, proposal: StreamletProposal) -> ProcessEffect: """ (process) If we already voted in the epoch specified by the same proposal, ignore it. """ @@ -62,3 +66,6 @@ def handle_proposal(self, proposal): return skip() return skip() + + def handle_block(self, block: StreamletBlock) -> ProcessEffect: + raise NotImplementedError diff --git a/simtfl/demo.py b/simtfl/demo.py index fbc01d0..001f490 100644 --- a/simtfl/demo.py +++ b/simtfl/demo.py @@ -2,12 +2,16 @@ A simple demo of message passing. """ + +from __future__ import annotations + from simpy import Environment from .logging import PrintLogger -from .message import PayloadMessage +from .message import Message, PayloadMessage from .network import Network -from .node import PassiveNode, SequentialNode +from .node import Node, SequentialNode +from .util import ProcessEffect class Ping(PayloadMessage): @@ -24,11 +28,11 @@ class Pong(PayloadMessage): pass -class PingNode(PassiveNode): +class PingNode(Node): """ A node that sends pings. """ - def run(self): + def run(self) -> ProcessEffect: """ (process) Sends two Ping messages to every node. """ @@ -43,7 +47,7 @@ class PongNode(SequentialNode): """ A node that responds to pings sequentially. """ - def handle(self, sender, message): + def handle(self, sender: int, message: Message) -> ProcessEffect: """ (process) Handles a Ping message by sending back a Pong message with the same payload. @@ -55,12 +59,12 @@ def handle(self, sender, message): yield from super().handle(sender, message) -def run(): +def run() -> None: """ Runs the demo. """ network = Network(Environment(), delay=4, logger=PrintLogger()) - for i in range(10): + for _ in range(10): network.add_node(PongNode()) network.add_node(PingNode()) diff --git a/simtfl/logging.py b/simtfl/logging.py index ed46c0f..4502872 100644 --- a/simtfl/logging.py +++ b/simtfl/logging.py @@ -2,24 +2,29 @@ Utility classes for logging. """ + +from __future__ import annotations +from typing import Any, Optional, TextIO +from numbers import Number + import sys -class NullLogger: +class Logger: """A logger that does nothing.""" - def header(self): + def header(self) -> None: """Do not print a header.""" pass - def log(self, now, ident, event, detail): + def log(self, now: Number, ident: int, event: Any, detail: str) -> None: """Do not log.""" pass -class PrintLogger: +class PrintLogger(Logger): """A logger that prints to a stream.""" - def __init__(self, out=None): + def __init__(self, out: Optional[TextIO]=None): """ Constructs a `PrintLogger` that prints to `out` (by default `sys.stdout`). """ @@ -27,11 +32,11 @@ def __init__(self, out=None): out = sys.stdout self.out = out - def header(self): + def header(self) -> None: """Print a table header.""" print() print(" Time | Node | Event | Detail", file=self.out) - def log(self, now, ident, event, detail): + def log(self, now: Number, ident: int, event: Any, detail: str) -> None: """Print a log line.""" print(f"{now:5d} | {ident:4d} | {event:10} | {detail}", file=self.out) diff --git a/simtfl/message.py b/simtfl/message.py index 3755ef8..003919e 100644 --- a/simtfl/message.py +++ b/simtfl/message.py @@ -2,12 +2,21 @@ Base classes for messages. """ -from dataclasses import dataclass + +from __future__ import annotations from typing import Any +from dataclasses import dataclass + + +class Message: + """ + Base class for messages. + """ + pass @dataclass(frozen=True) -class PayloadMessage: +class PayloadMessage(Message): """ A message with an arbitrary payload. """ diff --git a/simtfl/network.py b/simtfl/network.py index 88f11d3..a0a9bc9 100644 --- a/simtfl/network.py +++ b/simtfl/network.py @@ -2,17 +2,28 @@ Framework for message passing in a network of nodes. """ -from .util import skip -from .logging import NullLogger +from __future__ import annotations +from typing import Any, Optional +from numbers import Number -class Network: +from simpy import Environment +from simpy.events import Timeout, Process + +from .message import Message +from .node import Node, AbstractNetwork +from .util import skip, ProcessEffect +from .logging import Logger + + +class Network(AbstractNetwork): """ Simulate the network layer. """ - def __init__(self, env, nodes=None, delay=1, logger=NullLogger()): + def __init__(self, env: Environment, nodes: Optional[list[Node]]=None, delay: Number=1, + logger: Logger=Logger()): """ - Constructs a Network with the given `simpy.Environment`, and optionally + Constructs a `Network` with the given `simpy.Environment`, and optionally a set of initial nodes, message propagation delay, and logger. """ self.env = env @@ -21,26 +32,26 @@ def __init__(self, env, nodes=None, delay=1, logger=NullLogger()): self._logger = logger logger.header() - def log(self, ident, event, detail): + def log(self, ident: int, event: Any, detail: str) -> None: """ Logs an event described by `event` and `detail` for the node with the given `ident`. """ self._logger.log(self.env.now, ident, event, detail) - def num_nodes(self): + def num_nodes(self) -> int: """ Returns the number of nodes. """ return len(self.nodes) - def node(self, ident): + def node(self, ident: int) -> Node: """ Returns the node with the given integer ident. """ return self.nodes[ident] - def add_node(self, node): + def add_node(self, node: Node) -> None: """ Adds a node with the next available ident. """ @@ -48,22 +59,22 @@ def add_node(self, node): self.nodes.append(node) node.initialize(ident, self.env, self) - def _start(self, node): + def _start(self, node: Node) -> None: """ Starts a process for the given node (which is assumed to have already been added to this `Network`). """ self.log(node.ident, "start", str(node)) - self.env.process(node.run()) + Process(self.env, node.run()) - def start_node(self, ident): + def start_node(self, ident: int) -> None: """ Starts a process for the node with the given ident. A given node should only be started once. """ self._start(self.nodes[ident]) - def start_all_nodes(self): + def start_all_nodes(self) -> None: """ Starts a process for each node. A given node should only be started once. @@ -71,7 +82,7 @@ def start_all_nodes(self): for node in self.nodes: self._start(node) - def run_all(self, *args, **kwargs): + def run_all(self, *args, **kwargs) -> None: """ Convenience method to start a process for each node, then start the simulation. Takes the same arguments as `simpy.Environment.run`. @@ -79,7 +90,7 @@ def run_all(self, *args, **kwargs): self.start_all_nodes() self.env.run(*args, **kwargs) - def send(self, sender, target, message, delay=None): + def send(self, sender: int, target: int, message: Message, delay: Optional[Number]=None) -> ProcessEffect: """ (process) Sends a message to the node with ident `target`, from the node with ident `sender`. The message propagation delay is normally given by @@ -90,13 +101,13 @@ def send(self, sender, target, message, delay=None): self.log(sender, "send", f"to {target:2d} with delay {delay:2d}: {message}") # Run `convey` in a new process without waiting. - self.env.process(self.convey(delay, sender, target, message)) + Process(self.env, self.convey(delay, sender, target, message)) # Sending is currently instantaneous. # TODO: make it take some time on the sending node. return skip() - def broadcast(self, sender, message, delay=None): + def broadcast(self, sender: int, message: Message, delay: Optional[Number]=None) -> ProcessEffect: """ (process) Broadcasts a message to every other node. The message propagation delay is normally given by `self.delay`, but can be @@ -109,13 +120,13 @@ def broadcast(self, sender, message, delay=None): # Run `convey` in a new process for each node. for target in range(self.num_nodes()): if target != sender: - self.env.process(self.convey(delay, sender, target, message)) + Process(self.env, self.convey(delay, sender, target, message)) # Broadcasting is currently instantaneous. # TODO: make it take some time on the sending node. return skip() - def convey(self, delay, sender, target, message): + def convey(self, delay: Number, sender: int, target: int, message: Message) -> ProcessEffect: """ (process) Conveys a message to the node with ident `target`, from the node with ident `sender`, after waiting for the given message propagation delay. @@ -123,6 +134,97 @@ def convey(self, delay, sender, target, message): after the message has been handled by the target node. The caller should not depend on when it completes. """ - yield self.env.timeout(delay) + yield Timeout(self.env, delay) self.log(target, "receive", f"from {sender:2d} with delay {delay:2d}: {message}") yield from self.nodes[target].receive(sender, message) + + +# These tests are here rather than in node.py to avoid a circular import. + +__all__ = ['Network'] + +import unittest +from collections import deque + +from .logging import PrintLogger +from .message import PayloadMessage +from .node import SequentialNode + + +class PassiveReceiverTestNode(Node): + def __init__(self): + super().__init__() + self.handled = deque() + + def handle(self, sender: int, message: Message) -> ProcessEffect: + # Record when each message is handled. + self.handled.append((sender, message, self.env.now)) + # The handler takes 3 time units. + yield Timeout(self.env, 3) + + +class SequentialReceiverTestNode(SequentialNode): + def __init__(self): + super().__init__() + self.handled = deque() + + def handle(self, sender: int, message: Message) -> ProcessEffect: + # Record when each message is handled. + self.handled.append((sender, message, self.env.now)) + # The handler takes 3 time units. + yield Timeout(self.env, 3) + + +class SenderTestNode(Node): + def run(self) -> ProcessEffect: + # We send messages at times 0, 1, 2. Since the message + # propagation delay is 1 (the default), they will be + # received at times 1, 2, 3. + for i in range(3): + yield from self.send(0, PayloadMessage(i)) + yield Timeout(self.env, 1) + + # Test overriding the propagation delay. This message + # is sent at time 3 and received at time 14. + yield from self.send(0, PayloadMessage(3), delay=11) + yield Timeout(self.env, 1) + + # This message is broadcast at time 4 and received at time 5. + yield from self.broadcast(PayloadMessage(4)) + + +class TestFramework(unittest.TestCase): + def _test_node(self, + receiver_node: Node, + expected: list[tuple[Number, Message, Number]]) -> None: + network = Network(Environment(), logger=PrintLogger()) + network.add_node(receiver_node) + network.add_node(SenderTestNode()) + network.run_all() + + self.assertEqual(list(network.node(0).handled), expected) + + def test_passive_node(self) -> None: + # A `Node` subclass does not block on handling of previous + # messages, so it handles each message immediately when it + # is received. + self._test_node(PassiveReceiverTestNode(), [ + (1, PayloadMessage(0), 1), + (1, PayloadMessage(1), 2), + (1, PayloadMessage(2), 3), + (1, PayloadMessage(4), 5), + (1, PayloadMessage(3), 14), + ]) + + def test_sequential_node(self) -> None: + # A `SequentialNode` subclass *does* block on handling of + # previous messages. It handles the messages as soon as + # possible after they are received subject to that blocking, + # so they will be handled at intervals of 3 time units. + self._test_node(SequentialReceiverTestNode(), [ + (1, PayloadMessage(0), 1), + (1, PayloadMessage(1), 4), + (1, PayloadMessage(2), 7), + (1, PayloadMessage(4), 10), + (1, PayloadMessage(3), 14), + ]) diff --git a/simtfl/node.py b/simtfl/node.py index 5587cae..042c187 100644 --- a/simtfl/node.py +++ b/simtfl/node.py @@ -2,12 +2,46 @@ Base classes for node implementations. """ + +from __future__ import annotations +from typing import Any, Optional +from numbers import Number + from collections import deque +from simpy import Environment +from simpy.events import Event -from .util import skip +from .message import Message +from .util import skip, ProcessEffect -class PassiveNode: +class AbstractNetwork: + """An abstract base class for a network.""" + def log(self, ident: int, event: Any, detail: str) -> None: + """ + Logs an event described by `event` and `detail` for the node with ident + `ident`. + """ + raise NotImplementedError + + def send(self, sender: int, target: int, message: Message, delay: Optional[Number]=None) -> ProcessEffect: + """ + (process) Sends a message to the node with ident `target`, from the node + with ident `sender`. The message propagation delay is normally given by + `self.delay`, but can be overridden by the `delay` parameter. + """ + raise NotImplementedError + + def broadcast(self, sender: int, message: Message, delay: Optional[Number]=None) -> ProcessEffect: + """ + (process) Broadcasts a message to every other node. The message + propagation delay is normally given by `self.delay`, but can be + overridden by the `delay` parameter. + """ + raise NotImplementedError + + +class Node: """ A node that processes messages concurrently. By default it sends no messages and does nothing with received messages. This class is @@ -20,7 +54,7 @@ class PassiveNode: Note that the simulation is deterministic regardless of which option is selected. """ - def initialize(self, ident, env, network): + def initialize(self, ident: int, env: Environment, network: AbstractNetwork): """ Initializes a `PassiveNode` with the given ident, `simpy.Environment`, and `Network`. Nodes are initialized when they are added to a `Network`. @@ -32,13 +66,13 @@ def initialize(self, ident, env, network): def __str__(self): return f"{self.__class__.__name__}" - def log(self, event, detail): + def log(self, event: Any, detail: str): """ Logs an event described by `event` and `detail` for this node. """ self.network.log(self.ident, event, detail) - def send(self, target, message, delay=None): + def send(self, target: int, message: Message, delay: Optional[Number]=None) -> ProcessEffect: """ (process) This method can be overridden to intercept messages being sent by this node. The implementation in this class calls `self.network.send` @@ -46,7 +80,7 @@ def send(self, target, message, delay=None): """ return self.network.send(self.ident, target, message, delay=delay) - def broadcast(self, message, delay=None): + def broadcast(self, message: Message, delay: Optional[Number]=None) -> ProcessEffect: """ (process) This method can be overridden to intercept messages being broadcast by this node. The implementation in this class calls `self.network.broadcast` @@ -54,14 +88,14 @@ def broadcast(self, message, delay=None): """ return self.network.broadcast(self.ident, message, delay=delay) - def receive(self, sender, message): + def receive(self, sender: int, message: Message) -> ProcessEffect: """ (process) This method can be overridden to intercept messages being received by this node. The implementation in this class calls `self.handle`. """ return self.handle(sender, message) - def handle(self, sender, message): + def handle(self, sender: int, message: Message) -> ProcessEffect: """ (process) Handles a message by doing nothing. Note that the handling of each message, and the `run` method, are in separate simpy processes. That @@ -70,28 +104,28 @@ def handle(self, sender, message): """ return skip() - def run(self): + def run(self) -> ProcessEffect: """ (process) Runs by doing nothing. """ return skip() -class SequentialNode(PassiveNode): +class SequentialNode(Node): """ A node that processes messages sequentially. By default it sends no messages and does nothing with received messages. This class is intended to be subclassed. """ - def initialize(self, ident, env, network): + def initialize(self, ident: int, env: Environment, network: AbstractNetwork): """ Initializes a `SequentialNode` with the given `simpy.Environment` and `Network`. """ super().initialize(ident, env, network) self._mailbox = deque() - self._wakeup = env.event() + self._wakeup = Event(self.env) - def receive(self, sender, message): + def receive(self, sender: int, message: Message) -> ProcessEffect: """ (process) Add incoming messages to the mailbox. """ @@ -102,17 +136,17 @@ def receive(self, sender, message): pass return skip() - def handle(self, sender, message): + def handle(self, sender: int, message: Message) -> ProcessEffect: """ (process) Handles a message by doing nothing. Messages are handled sequentially; that is, handling of the next message will be blocked on this process. """ - # This is the same implementation as `PassiveNode`, but the documentation + # This is the same implementation as `Node`, but the documentation # is different. return skip() - def run(self): + def run(self) -> ProcessEffect: """ (process) Repeatedly handle incoming messages. If a subclass needs to perform tasks in parallel with message handling, @@ -127,92 +161,5 @@ def run(self): # This naive implementation is fine because we have no actual # concurrency. - self._wakeup = self.env.event() + self._wakeup = Event(self.env) yield self._wakeup - - -__all__ = ['PassiveNode', 'SequentialNode'] - -from simpy import Environment -import unittest - -from .logging import PrintLogger -from .message import PayloadMessage -from .network import Network - - -class PassiveReceiverTestNode(PassiveNode): - def __init__(self): - super().__init__() - self.handled = deque() - - def handle(self, sender, message): - # Record when each message is handled. - self.handled.append((sender, message, self.env.now)) - # The handler takes 3 time units. - yield self.env.timeout(3) - - -class SequentialReceiverTestNode(SequentialNode): - def __init__(self): - super().__init__() - self.handled = deque() - - def handle(self, sender, message): - # Record when each message is handled. - self.handled.append((sender, message, self.env.now)) - # The handler takes 3 time units. - yield self.env.timeout(3) - - -class SenderTestNode(PassiveNode): - def run(self): - # We send messages at times 0, 1, 2. Since the message - # propagation delay is 1 (the default), they will be - # received at times 1, 2, 3. - for i in range(3): - yield from self.send(0, PayloadMessage(i)) - yield self.env.timeout(1) - - # Test overriding the propagation delay. This message - # is sent at time 3 and received at time 14. - yield from self.send(0, PayloadMessage(3), delay=11) - yield self.env.timeout(1) - - # This message is broadcast at time 4 and received at time 5. - yield from self.broadcast(PayloadMessage(4)) - - -class TestFramework(unittest.TestCase): - def _test_node(self, receiver_node, expected): - network = Network(Environment(), logger=PrintLogger()) - network.add_node(receiver_node) - network.add_node(SenderTestNode()) - network.run_all() - - self.assertEqual(list(network.node(0).handled), expected) - - def test_passive_node(self): - # A PassiveNode subclass does not block on handling of - # previous messages, so it handles each message immediately - # when it is received. - self._test_node(PassiveReceiverTestNode(), [ - (1, PayloadMessage(0), 1), - (1, PayloadMessage(1), 2), - (1, PayloadMessage(2), 3), - (1, PayloadMessage(4), 5), - (1, PayloadMessage(3), 14), - ]) - - def test_sequential_node(self): - # A SequentialNode subclass *does* block on handling of - # previous messages. It handles the messages as soon as - # possible after they are received subject to that blocking, - # so they will be handled at intervals of 3 time units. - self._test_node(SequentialReceiverTestNode(), [ - (1, PayloadMessage(0), 1), - (1, PayloadMessage(1), 4), - (1, PayloadMessage(2), 7), - (1, PayloadMessage(4), 10), - (1, PayloadMessage(3), 14), - ]) diff --git a/simtfl/util.py b/simtfl/util.py index 12ab24d..e30997f 100644 --- a/simtfl/util.py +++ b/simtfl/util.py @@ -3,7 +3,15 @@ """ -def skip(): +from __future__ import annotations +from typing import Generator + +from simpy import Event + + +ProcessEffect = Generator[Event, None, None] + +def skip() -> ProcessEffect: """ (process) Does nothing. """ @@ -18,7 +26,7 @@ class Unique: Instances of this class are hashable. When subclassing as a dataclass, use `@dataclass(eq=False)` to preserve hashability. """ - def __eq__(self, other): + def __eq__(self, other: Unique): return self == other def __hash__(self):