The code in this repository is experimental and has been provided for reference purposes only. Community feedback is welcome but this project may not be supported in the same way that repositories in the official Apollo GraphQL GitHub organization are. If you need help you can file an issue on this repository, contact Apollo to talk to an expert, or create a ticket directly in Apollo Studio.
Note: The Apollo Router is made available under the Elastic License v2.0 (ELv2). Read our licensing page for more details.
This repository is a simple way to test the overhead of the three customization points of the Apollo Router:
- Coprocessors
- Rhai
- Configuration options
The current tests are:
- Setting a static header to subgraphs (Config, Rhai, Coprocessor)
- Setting 10 GUID headers on response to clients (Rhai, Coprocessor)
- JWT-based client awareness (Coprocessor)
The coprocessors are currently written in:
For the below tables, each section corresponds to the related test name. Each type relates to either the baseline (meaning no Router configuration), or the extensibility option. Languages imply a coprocessor.
The tests were run at 100 requests per second for 60 seconds against an Apollo Router version 1.19.0 on a Windows machine using WSL2 with Ubuntu.
To help with consistency, there are resource limits for both the router and the coprocessors when using Docker--currently 1 CPU core and 1GB of RAM.
This tests the overhead of setting 10 GUID headers on the response to the client using the RouterResponse stage. This is only available via Rhai or a coprocessor.
| Type | Min (ms) | Mean (ms) | p50 (ms) | p90 (ms) | p95 (ms) | p99 (ms) | Max (ms) |
|---|---|---|---|---|---|---|---|
| baseline | 0.48 | 0.68 | 0.65 | 0.88 | 0.93 | 1.10 | 4.68 |
| csharp | 0.69 (+0.21) |
0.90 (+0.22) |
0.87 (+0.22) |
1.09 (+0.21) |
1.15 (+0.22) |
1.27 (+0.17) |
6.10 (+1.42) |
| go | 0.66 (+0.18) |
0.83 (+0.15) |
0.79 (+0.14) |
1.02 (+0.14) |
1.09 (+0.16) |
1.21 (+0.11) |
4.98 (+0.30) |
| java | 0.72 (+0.24) |
0.99 (+0.31) |
0.94 (+0.29) |
1.24 (+0.36) |
1.36 (+0.43) |
1.64 (+0.54) |
7.11 (+2.43) |
| node | 0.74 (+0.26) |
0.95 (+0.27) |
0.91 (+0.26) |
1.14 (+0.26) |
1.21 (+0.28) |
1.33 (+0.23) |
5.82 (+1.14) |
| python | 0.81 (+0.33) |
1.00 (+0.32) |
0.96 (+0.31) |
1.20 (+0.32) |
1.27 (+0.34) |
1.37 (+0.27) |
5.78 (+1.10) |
| rhai | 0.53 (+0.05) |
0.75 (+0.07) |
0.72 (+0.07) |
0.96 (+0.08) |
1.03 (+0.10) |
1.19 (+0.09) |
5.02 (+0.34) |
This tests the overhead of validating a JWT, and using the JWT body to set the apollographql-client-name and apollographql-client-version headers. Those headers are then used for client identification within Apollo Studio.
This is only available via a coprocessor.
| Type | Min (ms) | Mean (ms) | p50 (ms) | p90 (ms) | p95 (ms) | p99 (ms) | Max (ms) |
|---|---|---|---|---|---|---|---|
| baseline | 0.49 | 0.67 | 0.64 | 0.87 | 0.93 | 1.11 | 4.90 |
| csharp | 0.76 (+0.27) |
1.02 (+0.35) |
0.98 (+0.34) |
1.17 (+0.30) |
1.25 (+0.32) |
1.42 (+0.31) |
32.42 (+27.52) |
| go | 0.68 (+0.19) |
0.84 (+0.17) |
0.81 (+0.17) |
1.03 (+0.16) |
1.10 (+0.17) |
1.20 (+0.09) |
5.06 (+0.16) |
| java | 0.82 (+0.33) |
1.13 (+0.46) |
1.07 (+0.43) |
1.42 (+0.55) |
1.62 (+0.69) |
2.04 (+0.93) |
7.92 (+3.02) |
| node | 1.03 (+0.54) |
1.28 (+0.61) |
1.25 (+0.61) |
1.50 (+0.63) |
1.57 (+0.64) |
1.72 (+0.61) |
6.19 (+1.29) |
| python | 0.84 (+0.35) |
1.03 (+0.36) |
1.00 (+0.36) |
1.19 (+0.32) |
1.27 (+0.34) |
1.38 (+0.27) |
5.77 (+0.87) |
This tests the overhead of setting a static header to each subgraph request. The header is named source with a value matching the extensibility option. This is available via all three extensibility options.
| Type | Min (ms) | Mean (ms) | p50 (ms) | p90 (ms) | p95 (ms) | p99 (ms) | Max (ms) |
|---|---|---|---|---|---|---|---|
| baseline | 0.48 | 0.69 | 0.65 | 0.88 | 0.94 | 1.12 | 4.72 |
| config | 0.49 (+0.01) |
0.68 (-0.01) |
0.65 (0.00) |
0.88 (0.00) |
0.93 (-0.01) |
1.09 (-0.03) |
4.92 (+0.20) |
| csharp | 0.80 (+0.32) |
1.06 (+0.37) |
1.04 (+0.39) |
1.23 (+0.35) |
1.29 (+0.35) |
1.39 (+0.27) |
5.77 (+1.05) |
| go | 0.74 (+0.26) |
0.97 (+0.28) |
0.95 (+0.30) |
1.12 (+0.24) |
1.18 (+0.24) |
1.28 (+0.16) |
5.22 (+0.50) |
| java | 0.81 (+0.33) |
1.14 (+0.45) |
1.08 (+0.43) |
1.41 (+0.53) |
1.58 (+0.64) |
2.09 (+0.97) |
7.58 (+2.86) |
| node | 0.92 (+0.44) |
1.13 (+0.44) |
1.09 (+0.44) |
1.32 (+0.44) |
1.40 (+0.46) |
1.61 (+0.49) |
6.60 (+1.88) |
| python | 0.60 (+0.12) |
0.73 (+0.04) |
0.69 (+0.04) |
0.86 (-0.02) |
0.90 (-0.04) |
1.01 (-0.11) |
5.08 (+0.36) |
| rhai | 0.53 (+0.05) |
0.72 (+0.03) |
0.68 (+0.03) |
0.92 (+0.04) |
0.98 (+0.04) |
1.16 (+0.04) |
5.11 (+0.39) |
You will need to have installed:
Note: go-task can be installed via brew.
Next, you'll also need an Apollo Graph Reference and Apollo Key. For the testing, we are using a local supergraph (located at ./router/supergraph.graphql), but the Coprocessor feature is restricted to enterprise customers only.
Once you have the necessary requirements:
- Copy the
.sample_envfile to.envand fill in the fields - Run
task test-allto run the available tests within the project.
During the development of this project, it was discovered that running these tests on MacOS may result in inconsistent results. We strongly recommend running these tests on a Windows machine if possible to ensure the results are consistent from run to run.
To add new coprocessors, you will need to:
- Add a new folder to the coprocessors
- Write the coprocessor to use the three static endpoints. Refer to the Go implementation for more details:
/static-subgraph/guid-response/client-awareness
- Add a Dockerfile to build and host the image
- Update the Taskfile.Test.yml to run the new coprocessor and report on it
- Add coprocessor to test tasks in Taskfile.yml (i.e. under
tasks.static.cmds)
To create new tests:
- Determine what you would like to benchmark against (Rhai, Config, and/or Coprocessors)
- Implement the test within all coprocessors and related extension points
- Following the format of the
static-subgraphfolder, create a new folder for the test and associated Router configurations - Create a new test setup under
includesin Taskfile.yml follow the pattern ofincludes.static - Create a new test task in Taskfile.yml follow the pattern of
tasks.static
See current tests for reference.
Source code in this repository is covered by the Elastic License 2.0. The default throughout the repository is a license under the Elastic License 2.0, unless a file header or a license file in a subdirectory specifies another license. See the LICENSE for the full license text.