Merge branch 'dev' into ImproveCpuMiner

.github/workflows/release-libs.yaml

@@ -10,14 +10,26 @@
 
 name: Release Libs
 
-on: 
-  # Manually run by going to "Actions/Release" in Github and running the workflow
-  workflow_dispatch:
+on:
+  pull_request:
+    branches:
+      - main
 
 jobs:
   libs_publish:
     runs-on: ubuntu-latest
     steps:
+      - name: Checkout code
+        uses: actions/checkout@v3
+
+      - name: Run check-versioning-lib-release.sh
+        run: |
+          ./check-versioning-lib-release.sh
+          if [ $? -eq 1 ]; then
+            echo "Script returned exit code 1, halting the workflow"
+            exit 1
+          fi
+
       - uses: actions/checkout@v3
       - uses: actions-rs/toolchain@v1
         with:
@@ -138,20 +150,25 @@ jobs:
       - name: Publish crate jd_client
         continue-on-error: true
         run: |
-          cargo publish --manifest-path=roles/jd-client/Cargo.toml
+          cd roles/jd-client
+          cargo publish
       - name: Publish crate jd_server
         continue-on-error: true
         run: |
-          cargo publish --manifest-path=roles/jd-server/Cargo.toml
+          cd roles/jd-server
+          cargo publish
       - name: Publish crate mining_proxy_sv2
         continue-on-error: true
         run: |
-          cargo publish --manifest-path=roles/mining-proxy/Cargo.toml
+          cd roles/mining-proxy
+          cargo publish
       - name: Publish crate pool_sv2
         continue-on-error: true
         run: |
-          cargo publish --manifest-path=roles/pool/Cargo.toml
+          cd roles/pool
+          cargo publish
       - name: Publish crate translator_sv2
         continue-on-error: true
         run: |
-          cargo publish --manifest-path=roles/translator/Cargo.toml
+          cd roles/translator
+          cargo publish

check-versioning-lib-release.sh

@@ -0,0 +1,53 @@
+#!/bin/bash
+
+git fetch origin main
+git fetch origin dev
+
+crates=(
+"utils/buffer"
+"protocols/v2/binary-sv2/no-serde-sv2/derive_codec"
+"protocols/v2/binary-sv2/no-serde-sv2/codec"
+"protocols/v2/binary-sv2/serde-sv2"
+"protocols/v2/binary-sv2/binary-sv2"
+"protocols/v2/const-sv2"
+"protocols/v2/framing-sv2"
+"protocols/v2/noise-sv2"
+"protocols/v2/codec-sv2"
+"protocols/v2/subprotocols/common-messages"
+"protocols/v2/subprotocols/job-declaration"
+"protocols/v2/subprotocols/mining"
+"protocols/v2/subprotocols/template-distribution"
+"protocols/v2/sv2-ffi"
+"protocols/v2/roles-logic-sv2"
+"protocols/v1"
+"utils/bip32-key-derivation"
+"utils/error-handling"
+"utils/key-utils"
+"roles/roles-utils/network-helpers"
+"roles/roles-utils/rpc"
+"roles/jd-client"
+"roles/jd-server"
+"roles/mining-proxy"
+"roles/pool"
+"roles/translator"
+)
+
+# Loop through each crate
+for crate in "${crates[@]}"; do
+  cd "$crate"
+
+  # Check if there were any changes between dev and main
+  git diff --quiet "origin/dev" "origin/main" -- .
+  if [ $? -ne 0 ]; then
+
+      # Check if crate versions on dev and main are identical
+      version_dev=$(git show origin/dev:./Cargo.toml | awk -F' = ' '$1 == "version" {gsub(/[ "]+/, "", $2); print $2}')
+      version_main=$(git show origin/main:./Cargo.toml | awk -F' = ' '$1 == "version" {gsub(/[ "]+/, "", $2); print $2}')
+      if [ "$version_dev" = "$version_main" ]; then
+         echo "Changes detected in crate $crate between dev and main branches! Versions on dev and main branches are identical ($version_dev), so you should bump the crate version on dev before merging into main."
+         exit 1
+      fi
+  fi
+
+  cd - >/dev/null
+done

roles/jd-client/README.md

@@ -1,38 +1,34 @@
 # JD Client
-The JD Client is a Sv2 proxy that support one extended channel upstream and one extended channel
-dowstream, and do job declaration. On start it will:
+
 * connect to the jd-server
 * connect to the template-provider
-* listen for and `OpenExtendedChannel` from downstream
+The JD Client receives custom block templates from a Template Provider and declares use of the template with the pool using the Job Declaration Protocol. Further distributes the jobs to Mining Proxy (or Proxies) using the Job Distribution Protocol. ```
 * transparently relay the `OpenExtendedChannel` to upstream 
-After the setup phase it will start to negotiate jobs with upstream and send the negotiated job
-downstream, so that everything that is dowstream do not need to know that job is negotiated.
 
 ## Setup
+
 ### Configuration File
-The `proxy-config-example.toml` is a configuration example.
 
 The configuration file contains the following information:
 
-1. The Upstream connection information which includes the SV2 Pool authority public key 
-   (`upstream_authority_pubkey`) and the SV2 Pool connection address (`upstream_address`) and port
-   (`upstream_port`).
-2. The maximum and minimum SV2 versions (`max_supported_version` and `min_supported_version`)
-3. The Job Declarator information which includes the Pool JD connection address (`jd_address`) and the Template Provider connection address to which to connect (`tp_address`).
-4. Optionally, you may want to verify that your TP connection is authentic. You may get `tp_authority_public_key` from the logs of your TP, for example:
-```
+1. The downstream socket information, which includes the listening IP address (`downstream_address`) and port (`downstream_port`).
+2. The maximum and minimum SRI versions (`max_supported_version` and `min_supported_version`) with size as (`min_extranonce2_size`)
+3. The authentication keys for the downstream connection (`authority_public_key`, `authority_secret_key`)
+4. A `retry` parameter which tells JDC the number of times to reinitialize itself after a failure.
+6. The Template Provider address (`tp_address`).
+7. Optionally, you may want to verify that your TP connection is authentic. You may get `tp_authority_public_key` from the logs of your TP, for example:
+
 # 2024-02-13T14:59:24Z Template Provider authority key: EguTM8URcZDQVeEBsM4B5vg9weqEUnufA8pm85fG4bZd
-```
 
 ### Run
-1. Copy the `jdc-config-example.toml` into `conf/` directory.
-2. Edit it with custom desired configuration and rename it `jdc-config.toml`
-3. Point the SV1 Downstream Mining Device(s) to the Translator Proxy IP address and port.
-4. Run the Translator Proxy:
-
-   ```
-   cd roles/translator
-   ```
-   ```
-   cargo run -p translator_sv2 -- -c conf/jdc-config.toml
-   ```
+
+Run the Job Declarator Client (JDC):
+There are two files when you cd into roles/jd-client/config-examples/
+
+1. `jdc-config-hosted-example.toml` connects to the community-hosted roles.
+2. `jdc-config-local-example.toml` connects to self-hosted Job Declarator Client (JDC) and Translator Proxy
+
+``` bash
+cd roles/jd-client/config-examples/
+cargo run -- -c jdc-config-hosted-example.toml
+```

roles/pool/README.md

@@ -1,4 +1,6 @@
+
 # SRI Pool
+
 SRI Pool is designed to communicate with Downstream role (most typically a Translator Proxy or a Mining Proxy) running SV2 protocol to exploit features introduced by its sub-protocols.
 
 The most typical high level configuration is:
@@ -18,47 +20,35 @@ The most typical high level configuration is:
 ```
 
 ## Setup
+
 ### Configuration File
-The `pool-config-example.toml` is a configuration example which can be copy/paste into `/conf` directory by the party that is running the SV2 Pool (most
-typically the pool service provider) to address the most preferred customization.
+
+`pool-config-hosted-tp-example.toml` and `pool-config-local-tp-example.toml` are examples of configuration files.
 
 The configuration file contains the following information:
 
-1. The SRI Pool information which includes the SRI Pool authority public key 
-   (`authority_pubkey`), the SRI Pool authority secret key (`authority_secret_key`), along with its certificate validity (`cert_validity_sec`). In addition to this, it contains the address which it will use to listen to new connection from downstream roles (`listen_address`) and the list of uncompressed pubkeys for coinbase payout (`coinbase_outputs`).
-2. The SRI Pool Job Negatiator information which includes the Template Provider address (`tp_address`) and the address it uses to listen new request from the downstream JDs (`jd_address`).
-3. Optionally, you may want to verify that your TP connection is authentic. You may get `tp_authority_public_key` from the logs of your TP, for example:
+1. The SRI Pool information which includes the SRI Pool authority public key
+   (`authority_public_key`), the SRI Pool authority secret key (`authority_secret_key`).
+2. The address which it will use to listen to new connection from downstream roles (`listen_address`)
+3. The list of uncompressed pubkeys for coinbase payout (`coinbase_outputs`)
+4. A string that serves as signature on the coinbase tx (`pool_signature`).
+5. The Template Provider address (`tp_address`).
+6. Optionally, you may want to verify that your TP connection is authentic. You may get `tp_authority_public_key` from the logs of your TP, for example:
+
 ```
 # 2024-02-13T14:59:24Z Template Provider authority key: EguTM8URcZDQVeEBsM4B5vg9weqEUnufA8pm85fG4bZd
 ```
 
 ### Run
-1. Copy the `pool-config-example.toml` into `conf/` directory.
-2. Edit it with custom desired configuration and rename it `pool-config.toml`
-   > <ins>**Warning**</ins><br>
-   > If you want to mine spendable bitcoin on regtest, you can do it with bitcoin-cli:
-   > 1. Get a legacy Bitcoin address:
-   >   ```
-   >   bitcoin-cli -regtest -rpcwallet="<PUT YOUR WALLET NAME HERE>" getnewaddress "test" "legacy"
-   >   ```
-   > 2. Retrieve its corresponding public key:
-   >   ```
-   >   bitcoin-cli -regtest getaddressinfo <PUT THE ADDRESS GENERATED HERE>
-   >   ```
-   > 3. Copy the pubkey showed in the output
-   > 4. Paste it in the `coinbase_outputs` of `pool-config.toml`, after deleting the one which is already present
-   > 5. Mine a block
-   > 6. Generate 100 blocks 
-   >   ```
-   >   bitcoin-cli -regtest generatetoaddress 100 bcrt1qc5xss0cma0zldxfzzdpjxsayut7yy86e2lr6km
-   >   ```
-   > Now the mined bitcoin are spendable!
-   
-3. Run the Pool:
-
-   ```
-   cd roles/v2/pool
-   ```
-   ```
-   cargo run -p pool_sv2 -- -c conf/pool-config.toml
-   ```
+
+There are two files found in `roles/pool/config-examples`
+
+1. `pool-config-hosted-tp-example.toml` runs on our community hosted server.
+2. `pool-config-example-tp-example.toml` runs with your local config.
+
+Run the Pool:
+
+```bash
+cd roles/pool/config-examples
+cargo run -- -c pool-config-hosted-tp-example.toml
+``` 

roles/translator/README.md

@@ -1,4 +1,6 @@
+
 # SV1 to SV2 Translator Proxy
+
 This proxy is designed to sit in between a SV1 Downstream role (most typically Mining Device(s) 
 running SV1 firmware) and a SV2 Upstream role (most typically a SV2 Pool Server with Extended
 Channel support).
@@ -20,36 +22,37 @@ The most typical high level configuration is:
 ```
 
 ## Setup
+
 ### Configuration File
-The `proxy-config-example.toml` is a configuration example which can be copy/paste into `/conf` directory by the party that is running the Translator Proxy (most
-typically the mining farm/miner hobbyist) to address the most preferred customization.
+
+`tproxy-config-local-jdc-example.toml` and `tproxy-config-local-pool-example.toml` are examples of configuration files for the Translator Proxy.
 
 The configuration file contains the following information:
 
 1. The SV2 Upstream connection information which includes the SV2 Pool authority public key 
    (`upstream_authority_pubkey`) and the SV2 Pool connection address (`upstream_address`) and port
    (`upstream_port`).
-1. The SV1 Downstream connection information which includes the SV1 Mining Device address
+2. The SV1 Downstream socket information which includes the listening IP address
    (`downstream_address`) and port (`downstream_port`).
-1. The maximum and minimum SV2 versions (`max_supported_version` and `min_supported_version`) that
+3. The maximum and minimum SRI versions (`max_supported_version` and `min_supported_version`) that
    the Translator Proxy implementer wants to support. Currently the only available version is `2`.
-1. The desired minimum `extranonce2` size that the Translator Proxy implementer wants to use
+4. The desired minimum `extranonce2` size that the Translator Proxy implementer wants to use
    (`min_extranonce2_size`). The `extranonce2` size is ultimately decided by the SV2 Upstream role,
    but if the specified size meets the SV2 Upstream role's requirements, the size specified in this
    configuration file should be favored.
-1. The Job Declarator information which includes the Pool JD connection address (`jd_address`) and the Template Provider connection address to which to connect (`tp_address`).
-1. The difficulty params such as the hashrate (hashes/s) of the weakest Mining Device that will be connecting to the Translator Proxy (`min_individual_miner_hashrate`), the number of shares needed before a mining.set_difficulty update (`miner_num_submits_before_update`) and the number of shares per minute that Mining Devices should be sending to the Translator Proxy (`shares_per_minute`). Ultimately, the estimated aggregate hashrate of all SV1 Downstream roles (Mining
-   Devices) (`channel_nominal_hashrate`), which is communicated to the SV2 Upstream to help it decide a proper difficulty target.
+5. The downstream difficulty params such as:
+- the hashrate (hashes/s) of the weakest Mining Device that will be connecting to the Translator Proxy (`min_individual_miner_hashrate`)
+- the number of shares per minute that Mining Devices should be sending to the Translator Proxy (`shares_per_minute`). 
+6. The upstream difficulty params such as:
+- the interval in seconds to elapse before updating channel hashrate with the pool (`channel_diff_update_interval`)
+- the estimated aggregate hashrate of all SV1 Downstream roles (`channel_nominal_hashrate`)
 
 ### Run
-1. Copy the `proxy-config-example.toml` into `conf/` directory.
-2. Edit it with custom desired configuration and rename it `proxy-config.toml`
-3. Point the SV1 Downstream Mining Device(s) to the Translator Proxy IP address and port.
-4. Run the Translator Proxy:
-
-   ```
-   cd roles/translator
-   ```
-   ```
-   cargo run -p translator_sv2 -- -c conf/proxy-config.toml
-   ```
+
+There are two files in `roles/translator/config-examples`:
+- `tproxy-config-local-jdc-example.toml` which assumes the Job Declaration protocol is used and a JD Client is deployed locally
+- `tproxy-config-local-pool-example.toml` which assumes Job Declaration protocol is NOT used, and a Pool is deployed locally
+
+```bash
+cd roles/translator/config-examples/
+cargo run -- -c tproxy-config-local-jdc-example.toml