Re-use downloaded pieces in subspace-gateway

[shamil-gadelshin]

Tiny objects will request the same piece multiple times from subspace-gateway. Each request is a relatively expensive operation because it causes multiple requests to DSN.

Alternatives

The obvious optimization would be to introduce a small in-memory piece cache, but this only works if the cache is big enough.

Another possible optimization is to take a batch of mappings, sort them in piece index order, and re-use downloaded pieces until they are no longer needed.

[nazar-pc]

If it is actually small, what is the probability of cache hit vs cache miss? Or in other words, would this actually help or just result in higher CPU/RAM usage?

[shamil-gadelshin]

AFAIK we are talking about sequential requests from the same client rather than optimizing requests in general.

@clostao will likely have more information about the actual case.

[nazar-pc]

Then a better approach might be to send a batch request and get batch response back with multiple requests to the same piece(s) being de-duplicated. This seems much more efficient and precise than just slapping a cache of an arbitrary size on top.

[clostao]

The purpose of the cache is to avoid re-fetching a piece from the DSN for every requested object within that piece. While grouping requests by piece would optimise retrieval and reduce redundant fetches, there is a very common scenario where a request needs to be duplicated:

Files in the DSN are stored in IPLD format, meaning a file consists of multiple chunks published on-chain, along with a head node that represents an array of hashes mapping the chunks. With the grouping approach, we would need to fetch the piece containing the head node twice. This is because, until we retrieve the object mapping content from the head node, we cannot determine which chunks make up the file.

That said, I agree that batching requests is a cleaner and worthwhile improvement. However, the cache could optimise file retrieval, reducing DSN fetches by one in most cases. Regarding the requirements of this cache, the TTL requirements of this cache would be minimal (in the order of seconds).

[teor2345]

Note that the gateway RPC server currently accepts multiple mappings per request, but doesn’t do anything to de-duplicate piece requests:

subspace/crates/subspace-gateway-rpc/src/lib.rs

Line 140 in 3d5483e

async fn fetch_object(&self, mappings: GlobalObjectMapping) -> Result<Vec<HexData>, Error> {

One possible implementation is:

• add an API to the object fetcher which accepts multiple mappings, sorts them, then fetches pieces as needed, only dropping them when a mapping with the next piece is reached (this would be a lot simpler after Optimize object mappings and object fetcher for subspace-gateway.  #3318)
• call that new API from the RPC server and HTTP server
• make the HTTP server take multiple mappings

[nazar-pc]

With the grouping approach, we would need to fetch the piece containing the head node twice. This is because, until we retrieve the object mapping content from the head node, we cannot determine which chunks make up the file.

I do not see why would it need to be downloaded twice. If you downloaded the piece already, keep it around (within limits) for the duration of the request. You can think of this as a local "cache" that is per-request, but precise and targeted rather than global, possibly limited to a single piece only.

However, the cache could optimise file retrieval, reducing DSN fetches by one in most cases. Regarding the requirements of this cache, the TTL requirements of this cache would be minimal (in the order of seconds).

It depends on the number of requests and latency. With any serious usage of the gateway the piece you have retrieved at the beginning will be evicted long before you retrieve the head node and have a chance to reuse it. Assuming the cache is actually small.

[clostao]

The download is needed twice because the service that extracts the links from a IPLD node is not the subspace gateway.

The workflow would be: Auto-Files Gateway asks for an object mapping hash, subspace gateway downloads piece containing this object and returns the object mapping content. The Auto-Files gateway parses that IPLD node and get that it's needed to fetch X amount of links. Some of these links are very likely to be placed in the piece already downloaded, though this would suppose another request to the subspace-gateway.

[clostao]

Regarding cache size requirements, I see that would depend on the number of requests per unit of time but how would latency affect? If the cache.set is performed when the piece is retrieved from the DSN, the latency of DSN retrieval wouldn't affect the cache size.

If we had a TTL-based cache it's size would be maxCacheSize = pieceSize * Vr * TTL, right?. Being Vr the number of request per unit of time.

[clostao]

Note that the gateway RPC server currently accepts multiple mappings per request, but doesn’t do anything to de-duplicate piece requests:

subspace/crates/subspace-gateway-rpc/src/lib.rs

Line 140 in 3d5483e

async fn fetch_object(&self, mappings: GlobalObjectMapping) -> Result<Vec<HexData>, Error> {

One possible implementation is:

• add an API to the object fetcher which accepts multiple mappings, sorts them, then fetches pieces as needed, only dropping them when a mapping with the next piece is reached (this would be a lot simpler after Optimize object mappings and object fetcher for subspace-gateway.  #3318)
• call that new API from the RPC server and HTTP server
• make the HTTP server take multiple mappings

Having deduplication in the batched request it's the biggest optimisation we could do right now. For giving an example of how these two optimisations (same-piece object mappings batching vs caching pieces) compare:

A file is composed by N object mappings.

• No optimisation: There would be needed a DSN fetch to get file head plus (N-1) requests. Total N
• Batched requests: There would be needed a DSN fetch to get file head plus (N-1)/OpP requests. Being OpP = Object mappings per piece. Total (N-1)/OpP + 1
• Piece Cache: There would be needed a DSN fetch to get file head plus (N-1)/OpP - 1 requests, because the first one would be a cache hit. Being OpP = Object mappings per piece. Total (N-1)/OpP

[nazar-pc]

The download is needed twice because the service that extracts the links from a IPLD node is not the subspace gateway.

Well, in-memory cache will not help with this either way.

What is the rationale of splitting this logic between two applications? If it is expected that users will request files with IPLD format as an entrypoint, then it may make sense to integrate that into the gateway itself, WDYT? Maybe gateway should not even be in monorepo in that case, we already expose low-level libraries from which gateway can be built.

Regarding cache size requirements, I see that would depend on the number of requests per unit of time but how would latency affect?

It depends on what data is being requested, if you have multiple requests for random pieces then the cache will be mostly useless due to small size and large amounts of cache misses.

[clostao]

What is the rationale of splitting this logic between two applications?

I don't know exactly what @shamil-gadelshin thinks about this, but for me the separation of these logics makes sense because I see two different concerns.

The re-construction of object mapping ideally should be responsibility of this basecode since the entities of segment, pieces and object mappings are created & managed within this repo. This fact in addition to the premise that the object mappings should not be coupled to a specific format (in this case IPLD) for me takes to the conclusion of preferring the separation in two different layers.

It depends on what data is being requested, if you have multiple requests for random pieces then the cache will be mostly useless due to small size and large amounts of cache misses.

Okey, I see how it'd affect.

[nazar-pc]

The re-construction of object mapping ideally should be responsibility of this basecode since the entities of segment, pieces and object mappings are created & managed within this repo

Isn't what what https://github.com/autonomys/subspace/tree/main/shared/subspace-data-retrieval is for?

[clostao]

I see that this crate implements the piece fetch and object construction though other parts like DSN connection handling is not implemented. Anyways, generally what I'm understanding that you're suggesting is that instead of having the subspace-gateway as a service to have a crate that manages similarly so the auto-files gateway can use it. This makes sense to me because I see some flaws currently like having a call to object mapping indexer in this repo is some sort of circular dependency node -> indexer -> gateway.

The current approach would be faster to implement since wouldn't require us to implement some IPLD-related tools that we've already built in TypeScript though would have some restrictions (or having to use some workarounds) in the optimisations we can perform. Since the current main objective is to figure out where the bottlenecks are going to be I'd prefer to continue with current version even though it means to not implement the optimisation of this issue.

[teor2345]

I see that this crate implements the piece fetch and object construction though other parts like DSN connection handling is not implemented.

We could eventually split the gateway into a library and (tiny) binary, or split the HTTP server and DSN setup out into separate crates (like the subspace-gateway-rpc crate).

Then if we wanted to implement a generic piece cache on top of the piece provider, it would go in the DSN setup crate. And if we wanted to cache pieces within object reconstruction, we’d add a batch interface to subspace-data-retrieval, and update the other crates to use it.

I think while we’re changing interfaces like this, it will be easiest to have them all in the same monorepo. We can split things out later if we settle on a different design.

[shamil-gadelshin]

What is the rationale of splitting this logic between two applications? If it is expected that users will request files with IPLD format as an entrypoint, then it may make sense to integrate that into the gateway itself, WDYT? Maybe gateway should not even be in monorepo in that case, we already expose low-level libraries from which gateway can be built.

The rationale for splitting the logic was to move forward with PoC and discover bottlenecks, issues (like this one), and other concerns.

We could eventually split the gateway into a library and (tiny) binary, or split the HTTP server and DSN setup out into separate crates (like the subspace-gateway-rpc crate).

Yes, the exact components/services composition will likely change after testing/benchmarking.