10.0.0 (#314)

* feat: refactor routing task for perf improvements
* feat: use max_command_queue_len for backpressure
* feat: remove auto_pipeline config
* feat: add i-hexpire feature flag
* feat: rename types
* fix: enable-rustls-ring in process_stream
* feat: add scan_page interface
* feat: make on_* event functions async
* feat: withscore arg for ZRANK and ZREVRANK
* feat: cancel scan streams
* feat: support replicas with options, expire options
* feat: make specialize-into-bytes ff default

---------

Co-authored-by: 埃拉 <[email protected]>
Co-authored-by: ArtemIsmagilov <[email protected]>

CHANGELOG.md

@@ -1,3 +1,51 @@
+## 10.0.0
+
+* Reduced memory footprint and significant write throughput improvements
+* Rename common interfaces to remove `Redis` prefixes
+* Add `WITHSCORES` to `ZRANK` and `ZREVRANK`
+* Add `GT|LT|NX|XX` options to `EXPIRE` and `EXPIREAT`
+* Add `scan_page` interface
+* Add optional message to `PING`
+* Remove deprecated or redundant config options
+* Refactor public types into submodules
+* Add `i-hexpire` feature flag
+* Support async blocks in `on_*` event handler functions
+* Add an interface to cancel scanning functions
+* Update `rustls-native-certs` to 0.8
+* Support `valkey://` scheme in `Config::from_url`.
+* Support combining `Options` and `Replicas` clients
+
+### Upgrading from 9.x
+
+This update contains some significant performance improvements in the form of reduced Tokio scheduling overhead and
+reduced memory usage during the frame encoding process. It also contains several cosmetic changes designed to support
+future scenarios where Valkey and Redis start to diverge from one another.
+
+### Notable Breaking Changes
+
+The compiler should guide callers through most of these changes.
+
+* The `auto_pipeline` config option was removed. All clients now automatically pipeline commands across Tokio tasks.
+* The `BackpressureConfig` struct was removed. Callers should use `max_command_buffer_len` instead.
+* The `HEXPIRE`, `HTTL`, etc, commands are now gated by an `i-hexpire` feature flag. Note that this requires Redis >
+  =7.2.5.
+* Some potentially redundant `ReplicaConfig` fields were removed. The client now uses the associated `ReconnectPolicy`
+  fields instead, where applicable.
+* The `types` module was becoming too large and needed refactoring. Many types were moved to submodules, which likely
+  requires changes to certain import statements.
+* Many of the common public types were renamed to remove the `Redis` prefix, such as `RedisConfig`, `RedisClient`,
+  `RedisPool`, etc.
+* `rustls-native-certs` was upgraded to 8.x.
+* The `specialize-into-bytes` feature flag was removed. This is now the default behavior.
+* The `on_error` and `error_rx` event handler interface now contains an optional server identifier.
+
+### Behavior Changes
+
+* In the past `fred` spawned a separate task per connection in order to read from all sockets concurrently. In 10.0.0
+  each client reads and writes to all connections in a single task.
+* Write throughput is improved by a factor of 3-5x depending on the use case.
+* All transactions are now pipelined automatically.
+
 ## 9.4.0
 
 * Change scanning functions to automatically continue when the current page is dropped

CONTRIBUTING.md

@@ -5,8 +5,8 @@ This document gives some background on how the library is structured and how to
 ## General
 
 * Run rustfmt and clippy before submitting any changes.
-  * Formatting should adhere to [rustfmt.toml](./rustfmt.toml), i.e. by running `cargo +nightly fmt --all`
-  * VS Code users should use the checked-in settings in the [.vscode](./.vscode) directory
+    * Formatting should adhere to [rustfmt.toml](./rustfmt.toml), i.e. by running `cargo +nightly fmt --all`
+    * VS Code users should use the checked-in settings in the [.vscode](./.vscode) directory
 * Please use [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/#summary).
 
 ## File Structure
@@ -43,7 +43,7 @@ The code has the following structure:
 
 This example shows how to add `MGET` to the commands.
 
-1. Add the new variant to the `RedisCommandKind` enum, if needed.
+1. Add the new variant to the `CommandKind` enum, if needed.
 
     ```rust
     pub enum RedisCommandKind {
@@ -80,7 +80,7 @@ This example shows how to add `MGET` to the commands.
 2. Create the private function implementing the command in [src/commands/impls/keys.rs](src/commands/impls/keys.rs).
 
     ```rust
-    pub async fn mget<C: ClientLike>(client: &C, keys: MultipleKeys) -> Result<RedisValue, RedisError> {
+    pub async fn mget<C: ClientLike>(client: &C, keys: MultipleKeys) -> Result<Value, RedisError> {
       // maybe do some kind of validation 
       utils::check_empty_keys(&keys)?;
 
@@ -94,10 +94,10 @@ This example shows how to add `MGET` to the commands.
     }
     ```
 
-    Or use one of the shorthand helper functions or macros.
+   Or use one of the shorthand helper functions or macros.
 
     ```rust
-    pub async fn mget<C: ClientLike>(client: &C, keys: MultipleKeys) -> Result<RedisValue, RedisError> {
+    pub async fn mget<C: ClientLike>(client: &C, keys: MultipleKeys) -> Result<Value, RedisError> {
       utils::check_empty_keys(&keys)?;
       args_values_cmd(client, keys.into_values()).await
     }
@@ -133,13 +133,13 @@ This example shows how to add `MGET` to the commands.
 
 4. Implement the interface on the client structs, if needed.
 
-    In the [RedisClient](src/clients/redis.rs) file.
+   In the [Client](src/clients/redis.rs) file.
 
     ```rust
-    impl KeysInterface for RedisClient {}
+    impl KeysInterface for Client {}
     ```
 
-    In the [transaction](src/clients/transaction.rs) file.
+   In the [transaction](src/clients/transaction.rs) file.
 
     ```rust
     impl KeysInterface for Transaction {}
@@ -155,8 +155,8 @@ Using `MGET` as an example again:
 1. Write tests in the [keys](tests/integration/keys/mod.rs) file.
 
     ```rust
-    pub async fn should_mget_values(client: RedisClient, _: RedisConfig) -> Result<(), RedisError> {
-      let expected: Vec<(&str, RedisValue)> = vec![("a{1}", 1.into()), ("b{1}", 2.into()), ("c{1}", 3.into())];
+    pub async fn should_mget_values(client: Client, _: Config) -> Result<(), RedisError> {
+      let expected: Vec<(&str, Value)> = vec![("a{1}", 1.into()), ("b{1}", 2.into()), ("c{1}", 3.into())];
       for (key, value) in expected.iter() {
         let _: () = client.set(key, value.clone(), None, None, false).await?;
       }
@@ -186,8 +186,7 @@ Using `MGET` as an example again:
     }
     ```
 
-These macros will generate test wrapper functions to call your test 8 times based on the following options:
+These macros will generate test wrapper functions to call your test 4 times based on the following options:
 
 * Clustered vs centralized deployments
-* Pipelined vs non-pipelined clients
 * RESP2 vs RESP3 protocol modes

Cargo.toml

@@ -11,13 +11,14 @@ name = "fred"
 readme = "README.md"
 repository = "https://github.com/aembke/fred.rs"
 rust-version = "1.75"
-version = "9.4.0"
+version = "10.0.0"
 
 [package.metadata.docs.rs]
 # do not show the glommio version of the docs
 features = [
     "i-all",
     "i-redis-stack",
+    "i-hexpire",
     "transactions",
     "blocking-encoding",
     "dns",
@@ -34,7 +35,6 @@ features = [
     "enable-native-tls",
     "full-tracing",
     "credential-provider",
-    "specialize-into-bytes"
 ]
 rustdoc-args = ["--cfg", "docsrs"]
 
@@ -46,7 +46,6 @@ test = true
 [features]
 default = ["transactions", "i-std"]
 
-specialize-into-bytes = []
 blocking-encoding = ["tokio/rt-multi-thread"]
 custom-reconnect-errors = []
 default-nil-types = []
@@ -156,6 +155,7 @@ i-redis-json = ["serde-json"]
 i-redis-stack = ["i-redis-json", "i-time-series", "i-redisearch"]
 i-redisearch = ["i-sorted-sets", "i-geo", "i-hashes"]
 i-time-series = []
+i-hexpire = []
 
 # Full and partial tracing
 full-tracing = ["partial-tracing"]
@@ -170,17 +170,16 @@ arc-swap = "1.7"
 async-trait = { version = "0.1" }
 bytes = "1.6"
 bytes-utils = "0.1.3"
-crossbeam-queue = "0.3"
-float-cmp = "0.9"
+float-cmp = "0.10"
 futures = { version = "0.3", features = ["std"] }
 log = "0.4"
 native-tls = { version = "0.2", optional = true }
 nom = { version = "7.1", optional = true }
 parking_lot = "0.12"
 rand = "0.8"
-redis-protocol = { version = "5.0.1", features = ["resp2", "resp3", "bytes"] }
+redis-protocol = { version = "6.0.0", features = ["resp2", "resp3", "bytes"] }
 rustls = { version = "0.23", optional = true, default-features = false }
-rustls-native-certs = { version = "0.7", optional = true }
+rustls-native-certs = { version = "0.8", optional = true }
 semver = "1.0"
 serde_json = { version = "1", optional = true }
 sha-1 = { version = "0.10", optional = true }
@@ -220,11 +219,9 @@ local-sync = { version = "0.1.1", optional = true }
 
 [dev-dependencies]
 axum = { version = "0.7", features = ["macros"] }
-base64 = "0.22.0"
 maplit = "1.0"
 pretty_env_logger = "0.5"
 serde = { version = "1.0", features = ["derive"] }
-subprocess = "0.2"
 tokio-stream = { version = "0.1", features = ["sync"] }
 
 [[example]]