gnb crashes on SDP answer - Cannot find QFI mapping for DRB4.

[NUCLEAR-WAR]

Hi,

I have problem with srsGNB, it crashes whenever I get 18x/200OK with SDP during my N5 QoS tests:

/srsRAN_Project/lib/sdap/sdap_entity_impl.h:55: virtual srsran::srs_cu_up::sdap_rx_pdu_handler& srsran::srs_cu_up::sdap_entity_impl::get_sdap_rx_pdu_handler(srsran::drb_id_t):
Assertion `rx_it != rx_map.end()' failed - Cannot find QFI mapping for DRB4.
/mnt/srsran/srsran_init.sh: line 55:    40 Aborted                 (core dumped) gnb -c /etc/srsran/gnb.yml -c /etc/srsran/qos.yml

2024-08-08T20:03:20.526163 [FAPI    ] [W] [   266.0] Real-time failure in FAPI: Received late UL_TTI.request from slot 265.19
2024-08-08T20:03:42.086532 [RLC     ] [W] du=1 ue=0 DRB2 DL: Reached maximum number of RETX. sn=34 retx_count=4
2024-08-08T20:03:44.759495 [RLC     ] [W] du=1 ue=0 DRB1 DL: Reached maximum number of RETX. sn=18 retx_count=4
2024-08-08T20:05:42.551506 [CU-UP   ] [E] ue=0: psi=2 could not remove ul_teid at session termination. ul_teid=0x000002
2024-08-08T20:05:42.551513 [CU-F1-U ] [W] Could not find UL GTP Tunnel={Addr=192.168.199.168 TEID=0x000002} at CU to remove.
2024-08-08T20:06:14.494113 [SDAP    ] [E] ue=0 psi=1: Cannot overwrite existing DL mapping for QFI=2

gnb.zip

gnb_crashes.zip

[NUCLEAR-WAR]

here is a coredump in case needed :

#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140408968447552) at ./nptl/pthread_kill.c:44
        tid = <optimized out>
        ret = 0
        pd = 0x7fb382b00640
        old_mask = {__val = {8097864050867857772, 7954869253094798127, 8101247163489872244, 2322227060592356972, 2336349463791167862, 4195787367660089971, 8457607694888694387,
            6877103715052567152, 7520858976974502002, 2316664838264483425, 4195787367660089971, 8457607694888694387, 6877103715052567152, 7592921037662350949,
            8387223380833366125, 8679104368171316063}}
        ret = <optimized out>
        pd = <optimized out>
        old_mask = <optimized out>
        ret = <optimized out>
        tid = <optimized out>
        ret = <optimized out>
        resultvar = <optimized out>
        resultvar = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        __futex = <optimized out>
        resultvar = <optimized out>
        __arg3 = <optimized out>
        __arg2 = <optimized out>
        __arg1 = <optimized out>
        _a3 = <optimized out>
        _a2 = <optimized out>
        _a1 = <optimized out>
        __futex = <optimized out>
        __private = <optimized out>
        __oldval = <optimized out>
        result = <optimized out>
#1  __pthread_kill_internal (signo=6, threadid=140408968447552) at ./nptl/pthread_kill.c:78
No locals.
#2  __GI___pthread_kill (threadid=140408968447552, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
No locals.
#3  0x00007fb3a512f476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
        ret = <optimized out>
#4  0x00007fb3a51157f3 in __GI_abort () at ./stdlib/abort.c:79
        save_stage = 1
        act = {__sigaction_handler = {sa_handler = 0x0, sa_sigaction = 0x0}, sa_mask = {__val = {0, 140408968408608, 0, 140407782526368, 0, 140408968408544, 0, 94163284857088,
              140408968408720, 140408968408672, 0, 0, 0, 0, 7969129189619049984, 0}}, sa_flags = -2102431488, sa_restorer = 0x55a41932f134}
        sigs = {__val = {32, 8102082850960524881, 2338053640979705449, 11193527980612, 140407782999456, 140408968408720, 94163285058305, 4294967296, 0, 4294967295, 0, 0,
            268435456, 0, 0, 0}}
#5  0x000055a4180e2606 in srsran::detail::print_and_abort(char const*, int, char const*, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) ()
No symbol table info available.
#6  0x000055a4187ce778 in srsran::srs_cu_up::sdap_entity_impl::get_sdap_rx_pdu_handler(srsran::drb_id_t) ()
No symbol table info available.
#7  0x000055a4181f5672 in srsran::srs_cu_up::pdu_session_manager_impl::handle_drb_to_setup_item(srsran::srs_cu_up::pdu_session&, srsran::e1ap_drb_to_setup_item_ng_ran const&) ()
No symbol table info available.
#8  0x000055a4181f5d88 in srsran::srs_cu_up::pdu_session_manager_impl::modify_pdu_session(srsran::e1ap_pdu_session_res_to_modify_item const&, bool) ()
No symbol table info available.
#9  0x000055a4181e76ec in srsran::srs_cu_up::cu_up::handle_bearer_context_modification_request(srsran::srs_cu_up::e1ap_bearer_context_modification_request const&) ()
No symbol table info available.
#10 0x000055a4181f0cb2 in srsran::srs_cu_up::e1ap_cu_up_adapter::on_bearer_context_modification_request_received(srsran::srs_cu_up::e1ap_bearer_context_modification_request const&) ()
No symbol table info available.
--Type <RET> for more, q to quit, c to continue without paging--
#11 0x000055a418808725 in srsran::srs_cu_up::e1ap_cu_up_impl::handle_bearer_context_modification_request(asn1::elementary_procedure_option<asn1::e1ap::bearer_context_mod_request_ies_container> const&) ()
No symbol table info available.
#12 0x000055a4186f49b2 in srsran::task_details::smallbuffer_table_t<srsran::priority_task_worker<(srsran::concurrent_queue_policy)1, (srsran::concurrent_queue_policy)1, (srsran::concurrent_queue_policy)0>::priority_task_worker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::array<unsigned int, 3ul> const&, std::chrono::duration<long, std::ratio<1l, 1000000l> >, srsran::os_thread_realtime_priority, srsran::os_sched_affinity_bitmask const&)::{lambda()#1}, void>::call(void*) const ()
No symbol table info available.
#13 0x000055a418626a44 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<srsran::unique_thread::make_thread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, srsran::unique_function<void (), 32ul, false>, srsran::os_thread_realtime_priority, srsran::os_sched_affinity_bitmask const&)::{lambda()#1}> > >::_M_run() ()
No symbol table info available.
#14 0x00007fb3a54f9253 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
No symbol table info available.
#15 0x00007fb3a5181ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
        ret = <optimized out>
        pd = <optimized out>
        out = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140725324715328, 1917391027270731333, 140408968447552, 2, 140409545693136, 140725324715680, -1874353259636963771,
                -1874417963004836283}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}}
        not_first_call = <optimized out>
#16 0x00007fb3a5213850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
No locals.

[herlesupreeth]

hmm, could be an issue with srsRAN.. did you try using the latest git commit in the srsRAN gNB image??

[NUCLEAR-WAR]

I downloaded it today and used it util now its stable, I will let it run few times and see.

[NUCLEAR-WAR]

I see sometimes underflow, at one point it was flooding the console, but now its ok, I only see then warning at startup:
"The PRACH detector will not meet the performance requirements with the configuration {Format B4, ZCZ 0, SCS 30kHz, Rx ports 1}."

I'm using the same config from the docker srsRAN, the device are attaching ok and I did not notice any performance issues though.

[NUCLEAR-WAR]

here is an update : it looks like the crushes with the old version due to the QoS modify requests, the new version act weird after getting PDUSessionModify a few times at some point he just stop responding and need to be stopped and restarted, but happend rare from the whole night yesterday I encountered it only once.
Lets wait for srsRAN Team to add the support for the session modify issue_768