Readiness endpoint available very late

[darkobas2]

Context

bee-2.1.0

Summary

Batchstore sync is taking unusual long time to complete. I have measured times of 25 - 60 minutes with several rpc endpoints

Expected behavior

usually sync was completed in 30 seconds or so.

Steps to reproduce

start a new bee and wait for /readiness endpoint to return 200

#!/bin/bash

function measure_time_to_ok() {
  start_time=$(date +%s.%N)
  attempts=0
  max_attempts=6000000
  sleep_interval=5

  while [[ $attempts -lt $max_attempts ]]; do
    response=$(curl -s -o /dev/null -w %{http_code} localhost:1635/readiness)
    if [[ $response -eq 200 ]]; then
      end_time=$(date +%s.%N)
      elapsed_time=$(echo "$end_time - $start_time" | bc)
      echo "Curl returned 200 OK after $elapsed_time seconds"
      return 0
    fi
    attempts=$((attempts + 1))
    sleep $sleep_interval
  done

  echo "Curl failed to return 200 OK after $max_attempts attempts"
  return 1
}

measure_time_to_ok

[ldeffenb]

I have never observed the stamp batch sync to be completed in 30 seconds on a new bee instance. I question your initial premise, not your current observations which align better with my accumulated experience with new nodes (or nuked nodes).

[darkobas2]

@ldeffenb i am talking about batchstore

 "level"="debug" "logger"="node/batchstore" 

and not pull sync, state store or how its called. this starts AFTER bathstore sync is complete

[darkobas2]

@ldeffenb it could be that you are correct, i am not putting my hand on fire to blame batchstore... my issue could be also with readiness endpoint comming alive much later than before... which could have been introduced with restricted mode deprecation or something..... so probably duplicate of: #4734

but i wouldnt care about the api in general just the readiness endpoint which influences the current behavious of kubernetes deployments

[ldeffenb]

@ldeffenb i am talking about batchstore

Yes, I understood that you were talking about the stamp batch syncing and not the full reserve sync via pullsync. I cannot speak to the readiness endpoint as monBee doesn't use it, but I do see many of the APIs not being available until the stamp batches have been fully synced from the RPC provider in a new node. Not to mention the fact that peers are also not connected until after the stamp batches have been synced (at least, I believe that's been my observations as well).