[Milestone] Quality Assurance processes are in place

[jm-clius]

2023 Key Milestones: https://notes.status.im/s/iylE6wdli#

Milestone: https://github.com/waku-org/pm/milestone/3

Priority Tracks: Secure Scalability, Production Readiness, Growth, Generating Revenue (Sustainability & Longevity).
Due date:

Summary

QA processes are defined for releasing of the 3 core Waku clients (go-waku, nwaku and js-waku), The processes includes:

• non-regression functional e2e testing
• non-regression performance testing

This incudes but is not limited to:

• black box testing of Waku protocol implementations as per RFC definition
• Any client-specific implementation (e.g. optimal usage of store protocol)
• Interoperability tests between the various clients (go-waku<>js-waku<>nwaku)
• Testing of Status specific Waku protocol usage (e.g. go-waku as client to nwaku service nodes)
• release-to-release performance analysis: Bandwidth usage, heaptrack, race condition, etc

Justification

Status Communities

Waku delivered major enhancements in 2023H1 to enable Status Community to securely scale using Waku. It is now critical that the software delivered remains stable, performant and bug-free.

Waku Network Gen 0

The aim of the Waku Network Gen 0 milestone is to deliver an autoscalable, bandwidth-capped, network to attract mature projects that need these guarantees to adopt Waku.
The delivery of these features should not be impeded by regression in release.

The two points above highlight the important to setting up QA processes now so that the delivery of Waku milestones are not impeded by software regressions.

Epics

• [Epic] End-to-end testing #34
• [Epic] Automated Release processes #86

[fryorcraken]

@jm-clius What are the main deliverable from your discussions with @fbarbu15 and @kaiserd ?

Form that, we can extract some high level epics.

• nwaku functional non-regression testing is part of release process
• nwaku <> go-waku function non-regression interop testing is part of release process etc.

[jm-clius]

@kaiserd, for some reason I can't find the milestone document for the testing team.
In any case, the roadmap is:

1. Writing test cases for the various service protocols
2. Implementing these first as unit tests within the individual clients (js-waku, nwaku and soon go-waku)
3. Setup a separate integrated testing environment where these test cases (and others) can be run against a "blackbox" network, with configurable nwaku, go-waku, js-waku clients for regular regression testing.

[fryorcraken]

Also thinking tracking "testing improvements" in a new epic for this milestone for tasks such as:

• Cover Several Environments As Part of Testing js-waku#52
• Improve and keep maintainability score of A go-waku#314

etc.

Something along the line:

Test coverage across all Waku clients is coherent:

Review test coverage across Waku clients and:

• set target if necessary/relevant
• review test strategy and ensure it's aligned with best practice
• Ensure that critical items have adequate test coverage
• Code quality tools? (ie codeclimate)

All items above are subjective/relative. However, this is value for each time to check that how testing is done within their software is adequate. Also welcome to consult @fbarbu15 for help.

[fryorcraken]

@kaiserd, for some reason I can't find the milestone document for the testing team. In any case, the roadmap is:

1. Writing test cases for the various service protocols

2. Implementing these first as unit tests within the individual clients (js-waku, nwaku and soon go-waku)

3. Setup a separate integrated testing environment where these test cases (and others) can be run against a "blackbox" network, with configurable nwaku, go-waku, js-waku clients for regular regression testing.

Thanks, I captured it in this issue: #34 cc @fbarbu15

Also, did we agree on some completion date or timeline @kaiserd @jm-clius ?

[chair28980]

An epic of orphans and testing items too important to ignore, but too small to be an epic on its own feels like a good move.

[weboko]

I think this is relevant and after this we can try to enrich the testing approach and do something like:

• add more complex testing - spawning nwaku nodes, connecting from some example, doing send - receive test;

It's of course is not silver bullet and just covers UI-waku network relationship so it won't have much impact as unit tests to waku network would have right now.

[fryorcraken]

Weekly Update

This work is tracked with vac:dst:software-testingwaku

[chair28980]

Closing milestone in favor of new PM process which includes a QA Epic for each defined MIlestone.