Replacing Tokio's TcpStream with async-std TcpStream

[b-yap]

Somehow in K8s, Tokio's TcpStream is stuck at "write ready" and is never "read ready".
Even after connecting using Rust's Tcpstream and then converting it to Tokio's, the stream never changes to "read ready".

That's when I decided to use Rust's Tcpstream completely. async-std library.

In terms of code changes, these are the difference you will see:

Tokio	Async-Std	Rust Std
Can split the stream to read/write halves, with the write half owned by a struct (Connector)	provided an example but Connector now owns the TcpStream, so I found no need to split	Can only try to clone
Everything is async	Async	Doesn't need async
Need to add timeout logic	potentially will need it	Can set it in the stream

This also implies removing some tokio features that were added primarily for TCP reasons: net and io-util.

---

How to begin the review:

1. clients/stellar-relay-lib/src/connection/connector/connector.rs
   • As mentioned above, the whole TcpStream is owned by the Connector:

          pub struct Connector {
              ...
              - write_stream_overlay: OwnedWriteHalf,
              + tcp_stream: TcpStream,
          }

   • renamed fn new(...) to fn start(...) where it also starts connecting to Stellar Node
2. clients/stellar-relay-lib/src/overlay.rs
   • StellarOverlayConnection's fn connect(...) will directly use Connector's fn start(...) . No more fn create_stream(...)` calls.
   • renamed fn disconnect() to fn stop() to make it similar to other structs (Agent, Connector).
3. clients/stellar-relay-lib/src/connection/helper.rs
   • removal of fn create_stream(...)
4. clients/stellar-relay-lib/src/connection/connector/message_reader.rs
   • all r_stream: &mut tcp::OwnedReadHalf is replaced with mut stream: TcpStream, removing async in these methods.
   • in fn read_message_from_stellar(...):
     • instead of "peeking" first, I changed it to actually "reading" the 1st 4 bytes, to avoid double operations. On the previous logic, it usually 1. peeks first, 2. and then read. Doing only 1 operation saves time.
     • to pass the stream around, it has to be cloned; hence the call totry_clone()
     • a reference of the Connector is being passed around. This eliminates cloning.
5. clients/vault/src/oracle/testing_utils.rs
   • created a helper function: fn specific_stellar_relay_config(...) to specifically connect to certain config. And there are only 3 choices, based on the index of its list:
     • testnet have: sdftest1, sdftest2, sdftest3
     • mainnet have: Iowa, Frankfurt, Singapore

[TorstenStueber]

@b-yap as this could be a bug in Tokios TCP implementation, would it make sense to report a bug ticket to Tokio?

[gianfra-t]

Very nice you were able to solve this issue @b-yap !! Super strange why the stream gets stuck and is not able to read only in Kubernetes.

I just left some comments that in general are about using the non async version of the TcpStream and if they may block the thread in a meaningful way, when reading particularly.

[bogdanS98]

Everything looks fine except I'm not really sure if synchronous execution is going to achieve the same performance as async. Is there any particular reason why std::net::TcpStream was used instead of async_std::net::TcpStream?

[b-yap]

@bogdanS98 @gianfra-t
After Gian's comment about using spawn_blocking(...), I am also finding std as NOT the best solution.

I just checked async_std's repo and the last update was about 9 months ago?
I'm running through their issues and found:

• TcpStream read task does not return although data is available async-rs/async-std#1063

[gianfra-t]

All looks good to me @b-yap! Amazing that you found out how to solve this weird problem.

[gianfra-t]

Just curious, why do we want to get always the first node choice here? and then on the other test always the second.

[b-yap]

ah, sometimes an error already connected peer happens. I tried connecting to different nodes just to see if it works 🤞. The next best thing is to actually connect with different stellar accounts. Currently we only have 1 test account to play with.

[ebma]

Looks good to me too, great job 🎉 I think these changes even simplify the connectivity logic we had because we don't have to pass the two halfs of the stream around, which is even better.

Did you already test with Zoltan if this implementation works in Kubernetes @b-yap?

[bogdanS98]

Great! Looks good to me 👍🏼

[b-yap]

Couldn't have done it without you guys, @bogdanS98 @gianfra-t .

@ebma yes I did