-
Notifications
You must be signed in to change notification settings - Fork 23
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Now we can pick and choose what part of the deck to present. * Added a dedicated "Cargo" section. * split the deck into 4: * cargo commands and "features" * dependency resolution, different kind of dependencies * cargo workspaces * rust project build time Added some extra info: * Speaker notes for Dependencies deck has a list of different private registry vendors and projects * Workspaces deck has a shell function adapted from the one I use for trainings to set up a new workspace project
- Loading branch information
1 parent
8fa6d6b
commit 2b45476
Showing
5 changed files
with
362 additions
and
303 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,102 @@ | ||
# Cargo Workspaces | ||
|
||
## Cargo Workspaces | ||
|
||
Allow you to split your project into several packages | ||
|
||
* further encourages modularity | ||
* develop multiple applications and libraries in a single tree | ||
* synchronized dependency management, release process, etc. | ||
* a way to parallelize compilation and speed up builds | ||
* **your internal projects should likely be workspaces** even if you don't use monorepos | ||
|
||
## Anatomy of Rust Workspace | ||
|
||
```text | ||
my-app/ | ||
├── Cargo.toml # a special workspace file | ||
├── Cargo.lock # notice that Cargo produces a common lockfile for all packages | ||
├── packages/ # can use any directory structure | ||
│ ├── main-app/ | ||
│ │ ├── Cargo.toml | ||
│ │ └── src/ | ||
│ │ └── main.rs | ||
│ ├── admin-app/ | ||
│ │ └── ... | ||
│ ├── common-data-model/ | ||
│ │ ├── Cargo.toml | ||
│ │ └── src/ | ||
│ │ └── lib.rs | ||
│ ├── useful-macros | ||
│ ├── service-a | ||
│ ├── service-b | ||
│ └── ... | ||
└── tools/ # packages don't have to be in the same directory | ||
├── release-bot/ | ||
│ ├── Cargo.toml | ||
│ └── src/ | ||
│ └── main.rs | ||
├── data-migration-scripts/ | ||
│ ├── Cargo.toml | ||
│ └── src/ | ||
│ └── main.rs | ||
└── ... | ||
``` | ||
|
||
## Workspace Cargo.toml | ||
|
||
```toml | ||
[workspace] | ||
members = ["packages/*", "tools/*"] | ||
|
||
[dependencies] | ||
thiserror = "1.0.39" | ||
... | ||
``` | ||
|
||
using wildcards for members is very handy when you want to add new member packages, split packages, etc. | ||
|
||
## Cargo.toml for a workspace member | ||
|
||
```toml | ||
[package] | ||
name = "main-app" | ||
|
||
[dependencies] | ||
thiserror = { workspace = true } | ||
service-a = { path = "../service-a" } | ||
... | ||
``` | ||
|
||
## Cargo commands for workspaces | ||
|
||
* `cargo run --bin main-app` | ||
* `cargo test -p service-a` | ||
|
||
## Creating a workspace | ||
|
||
```sh | ||
#!/usr/bin/env bash | ||
function nw() { | ||
local name="$1" | ||
local work_dir="$PWD" | ||
mkdir -p "$work_dir/$name/packages" | ||
git init -q "$work_dir/$name" | ||
cat > "$work_dir/$name/Cargo.toml" << EOF | ||
[workspace] | ||
members = ["packages/*"] | ||
[workspace.dependencies] | ||
EOF | ||
cat > "$work_dir/$name/.gitignore" << EOF | ||
target | ||
EOF | ||
code "$work_dir/$name" | ||
} | ||
``` | ||
|
||
Example: | ||
```bash | ||
nw spaceship | ||
cargo new --lib spaceship/packages/fuel-control | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,130 @@ | ||
# Dependency Management with Cargo | ||
|
||
## Cargo.toml - A manifest file | ||
|
||
```toml | ||
[package] | ||
name = "tcp-mailbox" | ||
version = "0.1.0" | ||
|
||
[dependencies] | ||
async-std = "1" # would also choose 1.5 | ||
clap = "2.2" # would also choose 2.3 | ||
``` | ||
|
||
## Cargo.lock - A lock file | ||
|
||
* contains a list of all project dependencies, de-facto versions and hashes of downloaded dependencies | ||
* when a version is *yanked* from `Crates.io` but you have the correct hash for it in a lock file Cargo will still let you download it and use it | ||
* still gives you warning about that version being problematic | ||
* should be committed to your repository for applications | ||
|
||
## Dependency resolution | ||
|
||
* uses "Zero-aware" SemVer for versioning | ||
* `1.3.5` is compatible with versions `>= 1.3.5` and `< 2.0.0` | ||
* `0.3.5` is compatible with versions `>= 0.3.5` and `< 0.4.0` | ||
* `0.0.3` only allows `0.0.3` | ||
* allows version-incompatible transitive dependencies | ||
* except C/C++ dependencies | ||
* combines dependencies with compatible requirements as much as possible | ||
* allows path, git, and custom registry dependencies | ||
|
||
## How a dependency version is selected | ||
|
||
* for every requirement Cargo selects acceptable version intervals | ||
* `[1.1.0; 1.6.0)`, `[1.3.5, 2.0.0)`, `[2.0.0; 3.0.0)` | ||
* Cargo checks for interval intersections to reduce the number of unique intervals | ||
* `[1.3.5; 1.6.0)`, `[2.0.0; 3.0.0)` | ||
* for every unique interval it selects the most recent available version | ||
* `=1.5.18`, `=2.7.11` | ||
* selected versions and corresponding package hashes are written into `Cargo.lock` | ||
|
||
## Dependency resolution: Example | ||
|
||
```text | ||
└── my-app May install: | ||
├── A = "1" | ||
│ ├── X = "1" A = "1.0.17" | ||
│ └── Y = "1.3" => B = "1.5.0" | ||
└── B = "1" X = "2.0.3" | ||
├── X = "2" X = "1.2.14" | ||
└── Y = "1.5" Y = "1.8.5" | ||
``` | ||
|
||
## Where do dependencies come from? | ||
|
||
* Crates.io | ||
* Private registries (open-source, self-hosted, or hosted) | ||
* Git and Path dependencies | ||
* dependencies can be *vendored* | ||
|
||
Notes: | ||
|
||
* private registries | ||
* hosted: **Shipyard**, JFrog, CloudSmith | ||
* self-hosted: **Kellnr** | ||
* open-source: [Ktra](https://github.com/moriturus/ktra) - pronounced `['KO-to-ra]`, [Meuse](https://github.com/mcorbin/meuse) - `[Møs]` | ||
|
||
*Shipyard and Kellnr will also generate API docs for you* | ||
|
||
## Crates.io | ||
|
||
* default package registry | ||
* 100k crates and counting | ||
* **every Rust Beta release is tested against all of them every week** | ||
* packages aren't deleted, but *yanked* | ||
* if you have a correct hash for a yanked version in your `Cargo.lock` your build won't break (you still get a warning) | ||
|
||
## Docs.rs | ||
|
||
* **complete API documentation for the whole Rust ecosystem** | ||
* automatically publishes API documentation for every version of every crate on Crates.io | ||
* documentation for old versions stays up, too. Easy to switch between versions. | ||
* links across crates just work | ||
|
||
## Other kinds of dependencies | ||
|
||
* git dependencies | ||
* both `git+https` and `git+ssh` are allowed | ||
* can specify branch, tag, commit hash | ||
* when downloaded by Cargo exact commit hash used is written into `Cargo.lock` | ||
* path dependencies | ||
* both relative and absolute paths are allowed | ||
* common in workspaces | ||
|
||
## C Libraries as dependencies | ||
|
||
* Rust can call functions from C libraries using `unsafe` code | ||
* integrate with operating system APIs, frameworks, SDKs, etc. | ||
* talk to custom hardware | ||
* reuse existing code (SQLite, OpenSSL, libgit2, etc.) | ||
* building a crate that relies on C libraries often requires customization | ||
* done using `build.rs` file | ||
|
||
## `build.rs` file | ||
|
||
* compiled and executed before the rest of the package | ||
* can manipulate files, execute external programs, etc. | ||
* download / install custom SDKs | ||
* call `cc`, `cmake`, etc. to build C++ dependencies | ||
* execute `bindgen` to generate Rust bindings to C libraries | ||
* output can be used to set Cargo options dynamically | ||
```rust ignore | ||
println!("cargo:rustc-link-lib=gizmo"); | ||
println!("cargo:rustc-link-search=native={}/gizmo/", library_path); | ||
``` | ||
|
||
## `-sys` crates | ||
|
||
* often Rust libraries that integrate with C are split into a pair of crates: | ||
* `library-name-sys` | ||
* thin wrapper around C functions | ||
* often all code is autogenerated by `bindgen` | ||
* `library-name` | ||
* depends on `library-name-sys` | ||
* exposes convenient and idiomatic Rust API to users | ||
* examples: | ||
* `openssl` and `openssl-sys` | ||
* `zstd` and `zstd-sys` | ||
* `rusqlite` and `libsqlite3-sys` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,108 @@ | ||
# Rust Projects Build Time | ||
|
||
## Understanding Rust projects build time | ||
|
||
* Cargo keeps track of changes you make and only rebuilds what is necessary | ||
* when building a crate `rustc` can do most of work in parallel, but some steps still require synchronization | ||
* depending on the type of build, times spent in different build phases may be vastly different. | ||
* debug vs release | ||
* various flags for `rustc` and LLVM | ||
* a build from scratch vs an incremental build | ||
|
||
## Producing a build timings report | ||
|
||
`rm -rf target/debug && cargo build --timings` | ||
|
||
```text | ||
. | ||
└── target/ | ||
├── cargo-timings/ | ||
│ ├── cargo-timings.html | ||
│ └── cargo-timings-<timestamp>.html | ||
├── debug/ | ||
└── ... | ||
``` | ||
|
||
## Timings Report | ||
|
||
![Cargo Build Report for Rust Analyzer](./images/rust-analyzer-cargo-build-timings.png) | ||
|
||
## Reading the report | ||
|
||
* Cargo can't start building a crate until all its dependencies have been built. | ||
* Cargo only waits for `rustc` to produce an LLVM IR, further compilation by LLVM can run in background (purple) | ||
* a crate can't start building until its `build.rs` is built and finishes running (yellow) | ||
* if multiple crates depend on a single crate they often can start building in parallel | ||
* if a package is both a binary and a library then the binary is built after a library | ||
* integration tests, examples, benchmarks, and documentation tests all produce binaries and thus take extra time to build. | ||
|
||
## Actions you can take | ||
|
||
## Keep your crates independent of each other | ||
|
||
* Bad dependency graph: | ||
```text | ||
D -> C -> B -> A -> App | ||
``` | ||
* Good dependency graph (A, B, and C can be built in parallel): | ||
```text | ||
/-> A \ | ||
D -> B -> App | ||
\-> C / | ||
``` | ||
|
||
## Turn off unused features | ||
|
||
* Before: | ||
```toml | ||
[dependencies] | ||
tokio = { version = "1", features = ["full"] } # build all of Tokio . | ||
``` | ||
* After: | ||
```toml | ||
[dependencies] | ||
tokio = { version = "1", features = ["net", "io-util", "rt-multi-thread"] } | ||
``` | ||
|
||
## Prefer pure-Rust dependencies | ||
|
||
* crate cannot be build before `build.rs` is compiled and executed | ||
* crates using C-dependencies have to rely on `build.rs` | ||
* `build.rs` might trigger C/C++ compilation which in turn is often slow | ||
|
||
* e.g.: `rustls` instead of `openssl` | ||
|
||
## Use multi-module integration tests: | ||
|
||
* Before (3 binaries) | ||
```text | ||
├── src/ | ||
│ └── ... | ||
└── tests/ | ||
├── account-management.rs | ||
├── billing.rs | ||
└── reporting.rs | ||
``` | ||
* After (a single binary) | ||
```text | ||
├── src/ | ||
│ └── ... | ||
└── tests/ | ||
└── my-app-tests/ | ||
├── main.rs # includes the rest as modules . | ||
├── account-management.rs | ||
├── billing.rs | ||
└── reporting.rs | ||
``` | ||
* Also benchmark and examples | ||
|
||
## Other tips | ||
|
||
* split your large package into a few smaller ones to improve build parallelization | ||
* extract your binaries into separate packages | ||
* remove unused dependencies | ||
|
||
## Tools | ||
|
||
* `cargo-chef` to speed up your docker builds | ||
* `sccache` for caching intermediary build artifacts across multiple projects and developers |
Oops, something went wrong.