chore(web): restore testing against legacy devices via BrowserStack #11845
+13,076
−8,190
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
keymanapp-test-bot
bot
added
the
user-test-missing
User Test ResultsTest specification and instructions ERROR: user tests have not yet been defined Test Artifacts
|
jahorton
commented
Jun 21, 2024
| @@ -9,6 +9,7 @@ | |||
| "@typescript-eslint/eslint-plugin": "^5.59.1", | |||
| "@web/dev-server-esbuild": "^1.0.2", | |||
| "@web/dev-server-import-maps": "^0.2.0", | |||
| "@web/dev-server-legacy": "^2.1.1", | |||
There was a problem hiding this comment.
Seems small on the surface, but comes with a very significant dependency cost - just look at the "lines changed" count for package-lock.json!
jahorton
commented
Jun 21, 2024
| // you can get the browser capabilities from the browserstack website | ||
| browserstackLauncher({ | ||
| capabilities: { | ||
| // Requires the dev-server-legacy plugin to operate properly. |
There was a problem hiding this comment.
To be clear, for this older version of Chrome. Supposedly, version 63 or 64 would allow us to drop it, but I'd want to check that experimentally first.
…standard test-runner config
Note that they aren't yet linked to any build.sh actions or commands yet, though.
jahorton
force-pushed
the
chore/web/restore-legacy-auto-testing
branch
from
79fd953
to
45b1f73
Compare
|
Test (Keyman - iOS) build failure cause: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This working PR seeks to address #11501.
My progress has been impeded somewhat due to issues I documented in modernweb-dev/web#2756. There are some limitations I'd rather not have, but at least things are far enough along to unblock progress.
Some neat things: we actually get more data about the test runs within the BrowserStack dashboard now, along with greater control of test-run labeling. A rough glimpse of that:
As our test runs do have separate test sets for specific modules, we can label which module each run is for while keeping them all associated within the same overall build version label.
How about some neat network logs? This way, we can see what sort of load times different parts of the test suites are costing to have better data for setting timeouts and/or determining sources of related instability.
Even the "Text Logs" provide us a little more information than they used to.
Console logs? Sure, why not?
Messages were temporarily added while I was analyzing the issues that led to [@web/test-runner-browserstack] + [@web/dev-server-legacy]: issues when using BrowserStack launcher for targeting certain older devices modernweb-dev/web#2756.
Note that a new package had to be added in order to support the older version of Chrome used here as a test target -
@web/dev-server-legacy. It seems simple on the surface, but its dependencies are the source of the monstrous set of changes to package-lock.json.