feat(perf): add nim-libp2p v1.1 #262

lchenut · 2023-08-16T10:15:12Z

I'm a bit rusty with docker though, so I'm not 100% sure if what I did is correct.

mxinden · 2023-08-16T17:21:39Z

Triggered a CI run: https://github.com/libp2p/test-plans/actions/runs/5881897412/job/15951280528

mxinden · 2023-08-16T19:06:38Z

@lchenut missing step (2) and (3).

For a new implementation, in impl/Makefile include your implementation in the all target.

For a new version, reference version in runner/src/versions.ts.

https://github.com/libp2p/test-plans/tree/master/perf#adding-a-new-implementation-or-a-new-version

Excited to see the results.

mxinden · 2023-08-17T08:50:46Z

Perf run: https://github.com/libp2p/test-plans/actions/runs/5888783646/job/15970676824

On success you can see the results on: https://observablehq.com/@libp2p-workspace/performance-dashboard?branch=perf-nim

mxinden · 2023-08-17T09:42:23Z

perf/impl/nim-libp2p/v1.0/Makefile

+perf: perf.nim nim-libp2p
+	docker run --rm --user "$(shell id -u):$(shell id -g)" -v "$(shell pwd)":/usr/src/myapp -w /usr/src/myapp nimlang/nim:1.6.10 sh -c "cd nim-libp2p && nimble install_pinned && cd - && nim c --NimblePath:nim-libp2p/nimbledeps/pkgs -p:nim-libp2p -d:chronicles_log_level=WARN --threads:off -d:release perf.nim"


CI has failed:

bash: line 1: ./impl/nim-libp2p/v1.0/perf: No such file or directory

Is this actually producing a binary under the name perf?

That's what I was afraid of. It's a Docker problem, it creates an ELF executable, no matter what, where it should create a host-compatible executable.

But it seems that there's no nim image on docker hub with this option.

After a bit of research, the problem doesn't come from the executable itself, but come from the linking of the .so...

$ file perf perf: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=895de8bd9fdf766ac9448fa05056cb4370bb6f54, for GNU/Linux 3.2.0, not stripped

/lib64/ld-linux-x86-64.so.2 isn't found

And I have no idea what I should use instead.

mxinden · 2023-08-31T15:57:23Z

Friendly ping @lchenut. Any progress on the above linking issue?

lchenut · 2023-09-05T13:07:35Z

Hum... I was so sure about this one. After a bit of digging and not finding a clear way to fix this (non-existant). It seems that I forgot to include nim-libp2p/v1.0/. in a .PHONY. So nothing to do with linking.

mxinden · 2023-09-06T15:05:02Z

Cool. Mind resolving the merge conflicts @lchenut? I can then trigger another run.

perf/runner/src/versions.ts

lchenut · 2023-09-07T12:46:12Z

@mxinden done

Menduist · 2023-09-14T15:30:00Z

@mxinden friendly ping to trigger the CI :)

mxinden · 2023-09-14T16:56:09Z

Sorry for the delay. I was out sick with COVID for a week. Resolved conflicts with #299 and triggered CI:

https://github.com/libp2p/test-plans/actions/runs/6188558497/job/16800823070

mxinden · 2023-09-14T18:20:14Z

@lchenut @Menduist CI run failed:

Unable to find image 'nimlang/nim:1.6.10' locally
1.6.10: Pulling from nimlang/nim
675920708c8b: Pulling fs layer
15302ca1b35f: Pulling fs layer
aa23efb4e495: Pulling fs layer
ed29ac11846c: Pulling fs layer
d589f97156d3: Pulling fs layer
ed29ac11846c: Waiting
d589f97156d3: Waiting
aa23efb4e495: Verifying Checksum
aa23efb4e495: Download complete
675920708c8b: Verifying Checksum
675920708c8b: Download complete
675920708c8b: Pull complete
d589f97156d3: Verifying Checksum
d589f97156d3: Download complete
ed29ac11846c: Verifying Checksum
ed29ac11846c: Download complete
15302ca1b35f: Verifying Checksum
15302ca1b35f: Download complete
15302ca1b35f: Pull complete
aa23efb4e495: Pull complete
ed29ac11846c: Pull complete
d589f97156d3: Pull complete
Digest: sha256:e3a842e22e9f7317b83908ee0470f4d983dee2f587840f161e91bc6dab9fa6b2
Status: Downloaded newer image for nimlang/nim:1.6.10
oserr.nim(95)            raiseOSError
Error: unhandled exception: Permission denied
Additional info: /.nimble [OSError]
make[1]: *** [Makefile:6: perf] Error 1
make: *** [Makefile:18: nim-libp2p/v1.0/.] Error 2
Command failed: ssh -o StrictHostKeyChecking=no [email protected] 'cd impl && make'
Error: Process completed with exit code 1.

lchenut · 2023-09-29T14:31:57Z

@mxinden sorry for the delay... it should be good now.

mxinden · 2023-10-10T19:14:03Z

Triggered a new run:

https://github.com/libp2p/test-plans/actions/runs/6473803273/job/17577296660

diegomrsantos · 2023-10-17T12:31:55Z

Hi @mxinden, we often find ourselves needing CI runs for our PRs but lack the necessary permissions. Is there a way to streamline this process to minimize manual coordination? Thanks.

mxinden · 2023-10-26T12:32:23Z

Is there a way to streamline this process to minimize manual coordination? Thanks.

In case you want to run the perf tests off of your laptop against your own AWS account:

https://github.com/libp2p/test-plans/blob/master/perf/README.md#running-manually

In case you want to run the perf tests off of the nim-libp2p CI, you would need to run against your own AWS account. For that, the ./terraform/modules/long_lived module provides additional tooling, namely an AWS lambda that cleans up left over resources, just in case. Once that is in place, you can trigger the perf workflow off of the nim-libp2p CI with your AWS credentials.

Either way, I can't give you access to the Protocol Labs AWS account.

mxinden · 2023-10-26T12:32:59Z

Triggered a new run:

https://github.com/libp2p/test-plans/actions/runs/6473803273/job/17577296660

In case you have not seen the failures:


=== Starting client nim-libp2p/v1.1/tcp
bash: line 1: ./impl/nim-libp2p/v1.1/perf: No such file or directory
bash: line 1: ./impl/nim-libp2p/v1.1/perf: No such file or directory
bash: line 1: ./impl/nim-libp2p/v1.1/perf: No such file or directory
bash: line 1: ./impl/nim-libp2p/v1.1/perf: No such file or directory
bash: line 1: ./impl/nim-libp2p/v1.1/perf: No such file or directory
bash: line 1: ./impl/nim-libp2p/v1.1/perf: No such file or directory
bash: line 1: ./impl/nim-libp2p/v1.1/perf: No such file or directory
bash: line 1: ./impl/nim-libp2p/v1.1/perf: No such file or directory
bash: line 1: ./impl/nim-libp2p/v1.1/perf: No such file or directory
bash: line 1: ./impl/nim-libp2p/v1.1/perf: No such file or directory

lchenut · 2023-10-27T09:50:08Z

@mxinden It's my bad, I should have pinged you when I pushed last week. Like the last three times, it should work now.

mxinden

I am sorry that this is dragging along.

Unfortunately the output format has changed in the meantime. I don't expect this requires a large change on your end, though you will need to continuously emit the current throughput every 1s.

You can find the larger rational in #261. I expect this to be especially interesting for you, given that you looked into congestion controller slow start for your upcoming Gossipsub optimizations.

mxinden · 2023-11-20T19:09:18Z

perf/impl/nim-libp2p/v1.1/perf.nim

+  dur = dur + (await PerfClient.perf(conn, f.uploadBytes, f.downloadBytes))
+  let ns = dur.nanos
+  let s = Second.nanos
+  echo "{\"latency\": ", fmt"{ns div s}.{ns mod s:09}", "}"


Note that the output format has changed with #276.

The pull request updated the documentation. You can find the new output format here:

https://github.com/libp2p/test-plans/blob/master/perf/README.md#adding-a-new-implementation-or-a-new-version

lchenut added 3 commits August 14, 2023 12:08

Added nim perf implementation

b049b3d

fix arg parser

e0fa824

clean the makefile

f8d26bf

perf: update benchmark results

fe07117

add makefile & version.ts informations

0578c83

mxinden reviewed Aug 17, 2023

View reviewed changes

update .PHONY

757ec30

Menduist reviewed Sep 6, 2023

View reviewed changes

perf/runner/src/versions.ts Outdated Show resolved Hide resolved

Menduist reviewed Sep 6, 2023

View reviewed changes

perf/runner/src/versions.ts Outdated Show resolved Hide resolved

lchenut added 2 commits September 7, 2023 12:53

Fix version.ts

922a2fe

Merge remote-tracking branch 'origin/master' into perf-nim

2758f09

Merge branch 'master' into perf-nim

7b1f409

lchenut added 2 commits September 29, 2023 16:03

fix(perf): nim-libp2p - remove the docker's user option

5cfd56d

Merge remote-tracking branch 'origin/perf-nim' into perf-nim

473e013

Rename directory to match the nim-libp2p version

b16f05d

lchenut changed the title ~~feat(perf): add nim-libp2p v1.0~~ feat(perf): add nim-libp2p v1.1 Oct 20, 2023

mxinden reviewed Nov 20, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(perf): add nim-libp2p v1.1 #262

feat(perf): add nim-libp2p v1.1 #262

lchenut commented Aug 16, 2023

mxinden commented Aug 16, 2023

mxinden commented Aug 16, 2023

mxinden commented Aug 17, 2023

mxinden Aug 17, 2023

lchenut Aug 17, 2023

lchenut Aug 17, 2023

mxinden commented Aug 31, 2023

lchenut commented Sep 5, 2023

mxinden commented Sep 6, 2023

lchenut commented Sep 7, 2023

Menduist commented Sep 14, 2023 •

edited

Loading

mxinden commented Sep 14, 2023

mxinden commented Sep 14, 2023

lchenut commented Sep 29, 2023

mxinden commented Oct 10, 2023

diegomrsantos commented Oct 17, 2023

mxinden commented Oct 26, 2023

mxinden commented Oct 26, 2023

lchenut commented Oct 27, 2023

mxinden left a comment

mxinden Nov 20, 2023

		perf: perf.nim nim-libp2p
		docker run --rm --user "$(shell id -u):$(shell id -g)" -v "$(shell pwd)":/usr/src/myapp -w /usr/src/myapp nimlang/nim:1.6.10 sh -c "cd nim-libp2p && nimble install_pinned && cd - && nim c --NimblePath:nim-libp2p/nimbledeps/pkgs -p:nim-libp2p -d:chronicles_log_level=WARN --threads:off -d:release perf.nim"

feat(perf): add nim-libp2p v1.1 #262

Are you sure you want to change the base?

feat(perf): add nim-libp2p v1.1 #262

Conversation

lchenut commented Aug 16, 2023

mxinden commented Aug 16, 2023

mxinden commented Aug 16, 2023

mxinden commented Aug 17, 2023

mxinden Aug 17, 2023

Choose a reason for hiding this comment

lchenut Aug 17, 2023

Choose a reason for hiding this comment

lchenut Aug 17, 2023

Choose a reason for hiding this comment

mxinden commented Aug 31, 2023

lchenut commented Sep 5, 2023

mxinden commented Sep 6, 2023

lchenut commented Sep 7, 2023

Menduist commented Sep 14, 2023 • edited Loading

mxinden commented Sep 14, 2023

mxinden commented Sep 14, 2023

lchenut commented Sep 29, 2023

mxinden commented Oct 10, 2023

diegomrsantos commented Oct 17, 2023

mxinden commented Oct 26, 2023

mxinden commented Oct 26, 2023

lchenut commented Oct 27, 2023

mxinden left a comment

Choose a reason for hiding this comment

mxinden Nov 20, 2023

Choose a reason for hiding this comment

Menduist commented Sep 14, 2023 •

edited

Loading