Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hydra as a source #823

Open
wants to merge 62 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
0d51179
Scaffold new hydra source
Anviking Oct 1, 2024
2228417
testing skeleton and test scenario vectors added
paweljakubas Oct 8, 2024
3f69d86
Make sure to log unrecognized hydra messages
Anviking Oct 4, 2024
5835339
Polish and use `point = (seq, head_id)`
Anviking Oct 8, 2024
bc48a28
Drop `HeadIsOpen` message and make tests compile
Anviking Oct 9, 2024
6348e2d
failing unit test TxValid
paweljakubas Oct 11, 2024
6720a24
Make sure to decode the hex of the expected values in the tests
Anviking Oct 11, 2024
91f9005
Add basic intersect logic to hydra source
Anviking Oct 15, 2024
93d31ac
Drop unused `chain` value in hydra config
Anviking Oct 16, 2024
62743b0
Drop unused `breadcrumbs` value in hydra config
Anviking Oct 16, 2024
4a1d0ca
Make "skipping message" debug log clearer
Anviking Oct 16, 2024
92cf03d
fixup: whitespace and minor log message tweak
Anviking Oct 16, 2024
696d9fa
Merge pull request #4 from Anviking/anviking/hydra-intersection
Anviking Oct 16, 2024
662c528
Forward all `raw_json` of hydra messages
Anviking Oct 15, 2024
98a50d7
fixing deserializer and tests
paweljakubas Oct 17, 2024
5b7b885
make payload value optional and handle PeerConnected
paweljakubas Oct 17, 2024
4a929ad
handle Idle
paweljakubas Oct 17, 2024
b02b58d
add other events
paweljakubas Oct 17, 2024
48fbfd8
add test for Committed
paweljakubas Oct 17, 2024
723a21f
Merge pull request #5 from Anviking/anviking/raw_json
paweljakubas Oct 17, 2024
8583de3
Drop unneeded `Optional` from `payload` field
Anviking Oct 18, 2024
fba0423
Drop unneeded constructors of `HydraMessagePayload`
Anviking Oct 18, 2024
55f51b4
Merge pull request #6 from Anviking/anviking/simplify-message
paweljakubas Oct 18, 2024
8144d57
introduce HydraMessages
paweljakubas Oct 22, 2024
97e1da1
add two valid events test
paweljakubas Oct 24, 2024
10ea7b4
add three valid events test
paweljakubas Oct 24, 2024
ec44863
handle incorrect events in tests
paweljakubas Oct 24, 2024
f200d33
add test_scenario
paweljakubas Oct 24, 2024
35f91dd
add scenario_1 test
paweljakubas Oct 24, 2024
0c70206
add scenario_2 test
paweljakubas Oct 24, 2024
bf2deb2
Merge pull request #7 from Anviking/paweljakubas/fill-in-scenario-tests
paweljakubas Oct 25, 2024
f237af2
add tokio runtime skeleton in integration testing
paweljakubas Oct 29, 2024
e4e7656
websocket server-client scaffolding
paweljakubas Oct 29, 2024
18501fb
towards hydra emulation test
paweljakubas Oct 29, 2024
4e41225
finish hydra emulation testing
paweljakubas Oct 29, 2024
f157adc
try running binary
paweljakubas Oct 30, 2024
01fc6f1
add oura integration test working
paweljakubas Oct 31, 2024
3a68004
add scenario 2
paweljakubas Oct 31, 2024
ab725ec
add golden
paweljakubas Nov 6, 2024
e96758a
factor out `run_daemon`
Anviking Nov 7, 2024
365210b
factor out `mock_hydra_node`
Anviking Nov 7, 2024
f32dded
Tweak integration tests
Anviking Nov 7, 2024
ca0cf6f
Merge pull request #10 from Anviking/anviking/integration-test-tweaks
paweljakubas Nov 12, 2024
bbfc0c3
Merge pull request #8 from Anviking/paweljakubas/integration-tests
paweljakubas Nov 12, 2024
f53c10c
Make intersection logic more robust
Anviking Oct 30, 2024
bf14271
let `mock_hydra_node` take contents instead of file
Anviking Nov 12, 2024
a0dbc4d
Add basic tests for hydra source intersection
Anviking Nov 13, 2024
a9ac87c
Factor out `oura_output_from_mock_chain`
Anviking Nov 13, 2024
d1ddde3
Move `read_to_string` inside `hydra_stdout_test` again
Anviking Nov 13, 2024
3633457
Refactor `should_process` into a wrapping `process`
Anviking Nov 14, 2024
414f918
Minor style changes
Anviking Nov 14, 2024
3340f8a
Add additional intersection tests
Anviking Nov 18, 2024
dd047a4
Add parenthesis to clarify debug log message
Anviking Nov 18, 2024
8cc2233
Merge pull request #9 from Anviking/anviking/better-intersection
Anviking Nov 18, 2024
e2466af
Drop unused hydra `rollback_count`
Anviking Nov 14, 2024
de834ac
Drop hydra `chain_tip` with questionable implementation
Anviking Nov 14, 2024
ad53357
Correct dummy hash length from 32 to 28 bytes
Anviking Nov 18, 2024
de1c170
Merge pull request #11 from Anviking/anviking/drop-unused-metrics
Anviking Nov 18, 2024
5f6950b
Drop unneeded addition to .gitignore
Anviking Nov 18, 2024
f5b87dc
Rename `hydra_socket_url` -> `hydra_socket_path`
Anviking Nov 18, 2024
371ed76
Drop unused `Snapshot` struct
Anviking Nov 18, 2024
16bd38f
Merge pull request #12 from Anviking/anviking/additional-polish
paweljakubas Nov 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixing deserializer and tests
  • Loading branch information
paweljakubas committed Oct 17, 2024
commit 98a50d7cc0f088dbbf3217000d62bdad43a8360b
34 changes: 24 additions & 10 deletions src/sources/hydra.rs
Original file line number Diff line number Diff line change
@@ -21,8 +21,7 @@ pub struct HydraMessage {
pub seq: u64,
pub head_id: Option<Vec<u8>>,
pub payload: HydraMessagePayload,
pub raw_json: Value

pub raw_json: Value,
}

impl HydraMessage {
@@ -55,19 +54,30 @@ impl<'de> Deserialize<'de> for HydraMessage {
.ok_or_else(|| de::Error::custom("seq should be a u64"))?;

let head_id_str: Option<&str> = map
.get("head_id")
.map(|v| v.as_str().ok_or_else(|| de::Error::custom("head_id should be a string")))
.get("headId")
.map(|v| {
v.as_str()
.ok_or_else(|| de::Error::custom("head_id should be a string"))
})
.transpose()?;

let head_id = head_id_str
.map(|s| hex::decode(s).map_err(|_e| serde::de::Error::custom(format!("Expected hex-encoded headId"))))
.map(|s| {
hex::decode(s)
.map_err(|_e| serde::de::Error::custom(format!("Expected hex-encoded headId")))
})
.transpose()?;

let payload = HydraMessagePayload::deserialize(&map).map_err(de::Error::custom)?;

let raw_json = map;

Ok(HydraMessage { seq, payload, raw_json, head_id })
Ok(HydraMessage {
seq,
payload,
raw_json,
head_id,
})
}
}

@@ -195,7 +205,7 @@ impl gasket::framework::Worker<Stage> for Worker {
let (socket, _) = connect_async(url).await.expect("Can't connect");
let worker = Self {
socket,
intersect: intersect_from_config(&stage.intersect)
intersect: intersect_from_config(&stage.intersect),
};

Ok(worker)
@@ -219,12 +229,16 @@ impl gasket::framework::Worker<Stage> for Worker {
// TODO: search for the intersection point to ensure
// we're on the same chain.
match &self.intersect {
Point::Specific(slot,_hash) if &hydra_message.seq <= slot => {
debug!("Skipping message {} before or at intersection {}", hydra_message.seq, slot);
Point::Specific(slot, _hash) if &hydra_message.seq <= slot => {
debug!(
"Skipping message {} before or at intersection {}",
hydra_message.seq, slot
);
Ok(())
}
Point::Origin | Point::Specific(_, _) =>
Point::Origin | Point::Specific(_, _) => {
self.process_next(stage, hydra_message).await
}
}
}
Err(err) => {
21 changes: 19 additions & 2 deletions tests/hydra.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
use std::fs;

use oura::sources::hydra::{HydraMessage, HydraMessagePayload};
use serde_json::json;

type TestResult = Result<(), Box<dyn std::error::Error>>;

@@ -19,7 +20,9 @@ fn test_event_deserialization(expected: HydraMessage, input: &str) -> TestResult
fn hydra_scenario_1() -> TestResult {
let msgs = [HydraMessage {
seq: 0,
head_id: Some(vec![0u8; 32]),
payload: HydraMessagePayload::Other,
raw_json: "".into(),
}];
run_scenario(&msgs, "tests/hydra/scenario_1.txt")
}
@@ -28,7 +31,9 @@ fn hydra_scenario_1() -> TestResult {
fn hydra_scenario_2() -> TestResult {
let msgs = [HydraMessage {
seq: 0,
head_id: Some(vec![0u8; 32]),
payload: HydraMessagePayload::Other,
raw_json: "".into(),
}];
run_scenario(&msgs, "tests/hydra/scenario_2.txt")
}
@@ -37,13 +42,25 @@ fn hydra_scenario_2() -> TestResult {
fn tx_valid_evt() -> TestResult {
let evt = HydraMessage {
seq: 15,
head_id: Some(hex::decode("84e657e3dd5241caac75b749195f78684023583736cc08b2896290ab").unwrap()
.to_vec()),
payload: HydraMessagePayload::TxValid {
tx: hex::decode("84a300d9010281825820635ffa4d3f8b5ccd60a89918866a5bb0776966572324da9a86870f79dcce4aad01018282581d605e4e214a6addd337126b3a61faad5dfe1e4f14f637a8969e3a05eefd1a0098968082581d6069830961c6af9095b0f2648dff31fa9545d8f0b6623db865eb78fde81a039387000200a100d9010281825820f953b2d6b6f319faa9f8462257eb52ad73e33199c650f0755e279e21882399c05840c1f23b630cf3d0ffe4186436225906c81bcddb0a27a632696035d4bb2d32e646c81759789c35c940b9695a87a0978a0408cff550c8d8f9ab4ac6d6d29b82a109f5f6")
.unwrap()
.to_vec(),
head_id: hex::decode("84e657e3dd5241caac75b749195f78684023583736cc08b2896290ab").unwrap()
.to_vec(),
},
raw_json: json!(
{ "headId": "84e657e3dd5241caac75b749195f78684023583736cc08b2896290ab"
, "seq": 15
, "tag": "TxValid"
, "timestamp": "2024-10-03T11:38:45.449663464Z"
, "transaction":
{ "cborHex": "84a300d9010281825820635ffa4d3f8b5ccd60a89918866a5bb0776966572324da9a86870f79dcce4aad01018282581d605e4e214a6addd337126b3a61faad5dfe1e4f14f637a8969e3a05eefd1a0098968082581d6069830961c6af9095b0f2648dff31fa9545d8f0b6623db865eb78fde81a039387000200a100d9010281825820f953b2d6b6f319faa9f8462257eb52ad73e33199c650f0755e279e21882399c05840c1f23b630cf3d0ffe4186436225906c81bcddb0a27a632696035d4bb2d32e646c81759789c35c940b9695a87a0978a0408cff550c8d8f9ab4ac6d6d29b82a109f5f6"
, "description": "Ledger Cddl Format"
, "txId": "08bb77374329ca28cd3023cace2948d0fc23e2812e8998c966db8b457e6390fe"
, "type": "Witnessed Tx ConwayEra"
}
}),
};

let raw_str = r#"