Merge pull request #25 from ussserrr/dev

v2.1.0

CHANGELOG.md

@@ -273,7 +273,7 @@
  - Changed: make `platformio_ini_is_patched` a property instead of function
  - Changed: improved in-code docs
 
-## ver 2.0.0
+## ver 2.0.0 (28.10.20)
  - New: introducing CI/CD via Azure Pipelines. I tried to make as "general" system as possible with isolated environments, reproducible builds, etc. Due to a number of such a different tools in use and scattered infrastructure around them this task is very complex and the current configuration is far from ideal though
  - New: new project structure. All packages (core, CLI, GUI) are consolidated under the common `stm32pio` Python _namespace_
  - New: app version is completely removed from the repo and is "computed" at build-time from the VCS (GIT) current tag (using [setuptools_scm](https://github.com/pypa/setuptools_scm)). At run-time the version obtaining process depends: for newer Python it can be retrieved from a package metadata, for older one there is an auto-generated `version.py` file with a value stored in it
@@ -294,3 +294,44 @@
  - Changed: remove board absence warning in `Stm32pio` constructor (this should be done outside)
  - Changed: take out to the `settings.py` a strings that we looked for to determine successful CubeMX code generation
  - Changed: use newer `platformio project init` command, use verbose versions of CLI arguments
+
+## ver 2.1.0
+ - New: "validate environment" API. Allows to quickly verify tools specified in the config (+ corresponding test). Currently, implemented only for the CLI version
+ - New: store the most recent exception in the `last_error` config file parameter (currently CLI-only) (+ corresponding test)
+ - New: add shorthands for all CLI options (single-letters, e.g. `-c/--with-build`)
+ - New: ignore list settings API: specify files/folders/patterns to ignore during the cleanup (hence, new `clean()` method, tests)
+ - New: alternatively, use `git clean` as a removal tool
+ - New: API to store the current project folder' content as ignore list in the config file
+ - New: CONTRIBUTING.md guide (more like developing notes actually)
+ - New: COMMANDS.md reference
+ - New: CONFIG.md reference
+ - New: project config reference (a little too many parameters now)
+ - New: Python 3.9 CI runner
+ - New: log when the config is merging with another one (DEBUG verbosity level)
+ - New: handle filenames with whitespaces (both tools/project files) (#21)
+ - Fixed: `clean()` method doesn't look for the determined `.ioc` file but does it by itself which can cause some unwanted behavior (potential data loss)
+ - Fixed: remove done/forgotten TODOs
+ - Fixed: update embedding example to match the current API
+ - Fixed: `setup.cfg`: specify supported PySide2 version
+ - Fixed: `setup.cfg`: PyPA parsing issues
+ - Fixed: revert default CubeMX paths that seems more widespread among users
+ - Fixed: GUI. Recursive layout warning in Settings window
+ - Fixed: GUI. Remove the `ProjectID` parameter for the `initialized` signal to get rid of the annoying Shiboken overflow error
+ - Changed: bump up CubeMX, packages, test `.ioc` file, PlatformIO versions (both for local and CI builds)
+ - Changed: a completely revised documentation/examples/TODOs structure
+ - Changed: spawn project-state-related code to the `state.py` module
+ - Changed: spawn logging-related code to the `logging.py` module
+ - Changed: separate config from the main class (new `config.py` module)
+ - Changed: rename `lib.py` -> `project.py` module
+ - Changed: move available config "None" options to the `settings` module (`none_options`)
+ - Changed: separate CubeMX invoking code (new project's private `_cubemx_execute_script()` method)
+ - Changed: edit output behavior in case of occurred error in the `generate_code()` function
+ - Changed: do not cast strings where we can use path-like objects
+ - Changed: pretty config printer (`__str__()` implementation, just `print(project.config)`, that's all)
+ - Changed: remove `util.configparser_to_dict()` function (`ConfigParser` is already conforms with mapping protocol)
+ - Changed: takeout valid user response options to `settings.py` (`yes_options`/`no_options`)
+ - Changed: group and move CI-related code in the `settings.py`
+ - Changed: remove `f"{STAGE_PATH.name}.ioc"` occurrences in tests
+ - Changed: move `test_clean()` to unit tests
+ - Changed: GUI. Remove `go_to_this` option for the `addListItem` method (instead invoke on the list model)
+ - Changed: GUI. Implicitly pass the parent to the project constructor in `addListItem` method

CI/lockfile.yml

@@ -5,9 +5,9 @@
 # to have them all in one place (especially because they are all make sense mostly for the testing/CI purposes)
 
 variables:
-  cubemx_version: 601  # 6.0.1
+  cubemx_version: 611  # 6.1.1
   # we should store nothing but strings for variables values so we encapsulate YAML into string
-  cubemx_packages: "f0: 1.11.1"
-  platformio_version: 5.0.1
+  cubemx_packages: "f0: 1.11.2"
+  platformio_version: 5.0.3
   test_cases: |
     - nucleo_f031k6

CI/setup_test_env.py

@@ -20,7 +20,7 @@ def install_cubemx_mcu_packages(query):
         cubemx_script_content = '\n'.join([f"swmgr install stm32cube_{series}_{version} accept"
                                            for series, version in query.items()]) + "\nexit"
         cubemx_script.write(cubemx_script_content.encode())  # encode since mode='w+b'
-        subprocess.run(['java', '-jar', str(Path(os.getenv('STM32PIO_CUBEMX_CACHE_FOLDER')) / 'STM32CubeMX.exe'), '-q',
+        subprocess.run(['java', '-jar', Path(os.getenv('STM32PIO_CUBEMX_CACHE_FOLDER')) / 'STM32CubeMX.exe', '-q',
                         cubemx_script_name, '-s'])
 
 

CONTRIBUTING.md

@@ -0,0 +1,44 @@
+# Developing & contributing
+Find actual tasks at [TODO.md list](/TODO.md) / [GitHub issues](https://github.com/ussserrr/stm32pio/issues). This file focuses on some relevant topics regarding build/test/CI processes. The code is well commented in-place and that can be considered as a developer documentation really. Some parts also can be found in the [docs](/docs) folder.
+
+
+## Build
+Staring from v2.0.0, the PEP517-compatible build process is supported. This process, yet described and standardized in several PEPs, is still a pretty early one and not fully adopted by official Python tools (such as pip, twine), though (at the time of publication at least). So the current way to pack is a little messy and relies on different instruments. Better use the latest Python and build packages versions.
+
+For the dependencies list see [pyproject.toml](/pyproject.toml) file:
+```shell script
+$ pip install wheel setuptools setuptools_scm
+```
+
+To build a Python _wheel_ `setup.py` is not even required:
+```shell script
+$ pip wheel . --wheel-dir dist
+```
+but for the assembling of the source distribution tarball it is still necessary:
+```shell script
+$ python setup.py sdist
+```
+
+
+## Test
+Testing (code is located at the [`tests`](/tests) directory) is done via the `unittest` module from the Python standard library. It's compatible with the `pytest` runner, too. Single test stage is a CubeMX project (`.ioc` file). Several such targets can be placed inside the `fixtures` folder to test against. Then start testing specifying concrete fixture as an environment variable:
+```shell script
+stm32pio-repo/ $   STM32PIO_TEST_CASE=nucleo_f031k6 python -m unittest -b -v
+```
+Every run automatically instantiates a temporary directory (using `tempfile` module) where all the actions are performed so no repository file will be "disturbed". To run the specific group of tests or a particular test function you can use:
+```shell script
+stm32pio-repo/ $   python -m unittest tests.test_integration.TestIntegration
+stm32pio-repo/ $   python -m unittest tests.test_cli.TestCLI.test_verbosity
+```
+`.ioc` files and installed tools' versions should match otherwise the CubeMX will complain about their incompatibility.
+
+
+## CI/CD
+Azure Pipelines is used to automate test, build, and publish tasks (see [azure-pipelines.yml](/azure-pipelines.yml), [CI](/CI) for more information). The repo is tested against the matrix of all 3 major OSes and latest Python interpreters. Also, for the Linux runs the test percentage and coverage are calculated. Therefore, for these purposes some additional external dependencies are required:
+  - pytest
+  - coverage
+  - yaml
+
+There are some elements of the "reproducible builds" approach using several "lockfiles", isolated test fixtures and caching. Optional `platformio.ini.lockfile`, "freezing" the PlatformIO packages' versions needed for a successful build, can be placed inside an every fixture folder. This config is an ordinary .INI-style file which will be merged into the `platformio.ini` during testing.
+
+Overall, due to a number of such a diverse tools in use, the full-fledged "canonical" CI seems challenging to implement, and the current set up is far from ideal in that regard.