High latency local writes

[niclar]

Hi we're evaluating tiledb for persistence of some of our high frequency data.

We're using the c++ library (clang18, libc++, ubuntu22, low latency kernel etc), writing to local disk we're seeing latencies in the 8 - 16ms range per write query.
I'd expect it to be on a sub ms level given that the disks have an avg write latency at ~.4ms.

The tiledb runs on a dedicated core, with default setting except;

      tiledb::Config config;
      config["sm.compute_concurrency_level"] = "1";
      config["sm.io_concurrency_level"] = "1";

(I still see tiledb creating 26 threads however, all idle but 4 with small cpu util)

the write query is created as;

tiledb::Query query(ctx, array, tiledb_query_type_t::TILEDB_WRITE);
query.set_buffer("A", a, 16);
query.set_buffer("B", b, 16);
query.set_buffer("C", c, 16);
query.set_buffer("D", d, 16);
query.set_buffer("E", e, 16);

query.submit();
query.finalize();

the schema is created as;

auto d1 = tiledb::Dimension::create<int64_t>(ctx, "A", { {0, static_cast<int64_t>(std::pow(2, 62) - 1)} }, 1000000);
auto d2 = tiledb::Dimension::create<int8_t>(ctx, "B", { {0, static_cast<int8_t>(2)} }, 3);

tiledb::Domain domain(ctx);
domain.add_dimensions(d1, d2);

auto cAttribute = tiledb::Attribute::create<float64_t>(ctx, "C");
auto dAttribute = tiledb::Attribute::create<float64_t>(ctx, "D");

tiledb::ArraySchema arraySchema(ctx, tiledb_array_type_t::TILEDB_SPARSE);
arraySchema.set_domain(domain);
arraySchema.add_attributes(cAttribute, dAttribute);

Further there's a few hundred tiledb::Context(s) in flight catering to writes to different tiles.

Please advice on how to get the per query latencies down to acceptable levels (sub ms).

[niclar]

Can we get some attention to this issue, or some color to the lib design decisions affecting the performance if nothing else as this determines the path forward..
(a natural workaround that might work I presume if one is not to constrained, is to offset the fixed costs by batching up to a page size)

Best,