Disappearing funds from xbridge(reward for fixing in BTC - more details on discord)

[nnmfnwl7]

Most easy way explained, wallets has UTXO funds sitting on addresses, but Xbridge itself who is the only responsible for reserving/releasing UTXOs for orders, thinks some/all UTXOs are reserved but is not able to say what order UTXOs reserved for.

Bug impact/what is causing:

xbridge things UTXO is reserved/locked/used whatever you we name it, but is not able to provide order id
but CLI of wallet ie DOGE/LTC sees UTXO as available/unlocked
utxos are disappearing from xbridge availability, blocknet daemon/wallet/bots need to be restarted

(like which coins ? Is is only DOGE ? Are you creating exact or partial orders? etc. - anything you think might help)

exact orders+ DOGE LTC pair twice, with both coins.
but saw other pairs also

summary:

Running xbridge scripts ie dxmakerbot which is using dxmakeorder and dxcancelorder on pair ie DOGE vs LTC causing that call DOGE/LTC wallets cli listunspent minconfirm=1 amount does not match xbridge dxgetUtxos coin=DOGE/LTC include_used=false . Xbridge think UTXOs are mark as used even if they have no order ID to be provided when callingdxgetutxos DOGE true or dxgetutxos LTC true.

ideally also provide the xbridge log from a day when it occurred

I was looking at logs, but as i mentioned before, i think we need debug/messages.
As i mentioned before idea was to update xpcxbridge.cpp code i was pointing on, especially that part of cpp code where locked and unlocked utxos are processing to get results which utxos are locked and which are unlocked.

What behavior did you expect?

Xbridge and wallet CLI locked/available amounts must match.

How reliably can you reproduce the issue, what are the steps to do so?

setup ie Blocknet + LTC + DOGE+ DxBot on LTC/DOGE, just let Bot create and cancel orders... will hit bug soon/later/6h.
bug seems like hit also when no orders are accepted... just creating and cancelling orders...

What version of Bitcoin Core are you using, where did you get it (website, self-compiled, etc)?

4.3.2 also 4.3.3 also self compiled also pre-build also gui also daemon.

What type of machine are you observing the error on (OS/CPU and disk type)?

Debian GNU/Llinux, lots of free space, lots of free ram, so not a missing resources bug.

Any extra information that might be useful in the debugging process.

Just for inspiration by brainlesswojak 😁 >> Maybe adding some debug logs to rpcxbridge.cpp will be enough >>
dxMakeOrder() >> any error or cleanup we missed to handle ?
dxCancelOrder() >> any error or cleanup we missed to handle ?
dxGetUtxos() >> UTXO which is reserved for order should also have order id, but this RPC call in situation when bug is already hit, order is ""(empty string) >>

std::set<xbridge::wallet::UtxoEntry> excluded = xapp.getAllLockedUtxos(token);
    std::vector<xbridge::wallet::UtxoEntry> unspent;
    if (!conn->getUnspent(unspent, !includeUsed ? excluded : std::set<xbridge::wallet::UtxoEntry>{}))
        return uret(xbridge::makeError(xbridge::BAD_REQUEST, __FUNCTION__, "failed to get unspent transaction outputs"));

    UniValue r(UniValue::VARR);
    for (const auto & utxo : unspent) {
        UniValue o(UniValue::VOBJ);
        o.pushKV("txid", utxo.txId);
        o.pushKV("vout", static_cast<int>(utxo.vout));
        o.pushKV("amount", xbridge::xBridgeStringValueFromPrice(utxo.amount, conn->COIN));
        o.pushKV("address", utxo.address);
        o.pushKV("scriptPubKey", utxo.scriptPubKey);
        o.pushKV("confirmations", static_cast<int>(utxo.confirmations));
        o.pushKV("orderid", "");
        if (excluded.count(utxo) > 0) {
            auto orderid = xapp.orderWithUtxo(utxo);
            o.pushKV("orderid", orderid.IsNull() ? "" : orderid.GetHex());
        }
        r.push_back(o);
    }

maybe also some debug in case when order is accepted... maybe we hit some corner case .... order been accepted but error happen and i missed that error in logs so some specific debug for this would be also good.

[nnmfnwl7]

reproduce hint steps

compile blocknet wallet with debug symbols

coin=blocknet
mkdir -p ~/Downloads/ccwallets/${coin}/git.src
cd ~/Downloads/ccwallets/${coin}/git.src

export CC=clang
export CXX=clang++

git clone https://github.com/blocknetdx/blocknet.git ./
git checkout origin/4.3.3

sh autogen.sh

proxychains make -j4 -C depends bdb boost
cd "depends/built/" && cc_archdir=`ls` && cd ../../
tar xvzf depends/built/${cc_archdir}/bdb/*.tar.gz -C ./depends/${cc_archdir}/
tar xvzf depends/built/${cc_archdir}/boost/*.tar.gz -C ./depends/${cc_archdir}/

./configure --quiet LDFLAGS="-L`pwd`/depends/${cc_archdir}/lib/" CPPFLAGS="-I`pwd`/depends/${cc_archdir}/include/" CXXFLAGS="-O3 -march=native" --with-boost-libdir=`pwd`/depends/${cc_archdir}/lib/ --disable-bench --disable-gui-tests --disable-tests --enable-reduce-exports --without-miniupnpc --without-zmq --with-gui=auto --enable-debug

make -j4

download litecoin wallet

https://github.com/litecoin-project/litecoin/releases/tag/v0.18.1

download dogecoin wallet

https://github.com/dogecoin/dogecoin/releases/tag/v1.14.5

update blocknet config ~/.blocknet/blocknet.conf with

listen=1
server=1
rpcbind=127.0.0.1
rpcallowip=127.0.0.1
port=41412
rpcport=41414
rpcuser=BlockDXBlocknet
rpcpassword=<custom blocknet password here>
dxnowallets=1
rpcworkqueue=256
txindex=1

update litecoin config ~/.litecoin/litecoin.conf with

server=1
listen=1
rpcbind=127.0.0.1
rpcallowip=127.0.0.1
port=9333
rpcport=9332
txindex=1
addresstype=legacy
changetype=legacy
rpcuser=BlockDXLitecoin
rpcpassword=<custom litecoin password here>
deprecatedrpc=signrawtransaction

update dogecoin config ~/.dogecoin/dogecoin.conf with

listen=1
server=1
rpcallowip=127.0.0.1
port=22556
rpcport=22555
rpcuser=BlockDXDogecoin
rpcpassword=<custom doge password here>
txindex=1

update xbridge configuration ~/.blocknet/xbridge.conf with

[Main]
ExchangeWallets=BLOCK,DOGE,LTC
FullLog=true
ShowAllOrders=true
[BLOCK]
Title=Blocknet
Ip=127.0.0.1
Port=41414
AddressPrefix=26
ScriptPrefix=28
SecretPrefix=154
COIN=100000000
MinimumAmount=0
TxVersion=1
DustAmount=0
CreateTxMethod=BTC
GetNewKeySupported=true
ImportWithNoScanSupported=true
MinTxFee=10000
BlockTime=60
FeePerByte=20
Confirmations=0
Username=BlockDXBlocknet
Password=<custom same blocknet pass here>
Address=
TxWithTimeField=false
LockCoinsSupported=false
JSONVersion=
ContentType=
CashAddrPrefix=
[DOGE]
Title=Dogecoin
Ip=127.0.0.1
Port=22555
AddressPrefix=30
ScriptPrefix=22
SecretPrefix=158
COIN=100000000
MinimumAmount=0
TxVersion=1
DustAmount=0
CreateTxMethod=BTC
GetNewKeySupported=false
ImportWithNoScanSupported=true
MinTxFee=100000000
BlockTime=60
FeePerByte=200000
Confirmations=0
Username=BlockDXDogecoin
Password=<custom same dogecoin pass here>
Address=
TxWithTimeField=false
LockCoinsSupported=false
JSONVersion=
ContentType=
CashAddrPrefix=
[LTC]
Title=Litecoin
Ip=127.0.0.1
Port=9332
AddressPrefix=48
ScriptPrefix=50
SecretPrefix=176
COIN=100000000
MinimumAmount=0
DustAmount=0
CreateTxMethod=BTC
GetNewKeySupported=true
ImportWithNoScanSupported=true
FeePerByte=10
MinTxFee=5000
TxVersion=2
BlockTime=150
Confirmations=0
Username=BlockDXLitecoin
Password=<custom same litecoin pass here>
Address=
TxWithTimeField=false
LockCoinsSupported=false
JSONVersion=
ContentType=
CashAddrPrefix=

setup dxmakerbot for more easy to reproduce bug

coin=dxbot
mkdir -p ~/Downloads/ccwallets/${coin}/git.src
cd ~/Downloads/ccwallets/${coin}/git.src

git clone https://github.com/nnmfnwl7/dxmakerbot.git ./
git checkout origin/fazer_dxmakerbot_latest_alfa

# rpc password copy
echo "tradingaddress = {}" >> utils/dxsettings.py
echo "rpcport = 41414" >> utils/dxsettings.py
tempusrpass=`cat ~/.blocknet/blocknet.conf | grep rpcuser | cut -d "=" -f2`
echo "rpcuser = '$tempusrpass'" >> utils/dxsettings.py
tempusrpass=""
tempusrpass=`cat ~/.blocknet/blocknet.conf | grep rpcpassword | cut -d "=" -f2`
echo "rpcpassword = '$tempusrpass'" >> utils/dxsettings.py
tempusrpass=""

# install DXBOT dependencies
pip3 install -r requirements.txt

# copy example dxmakerbot v2 configurations
cp howto/examples/bot_v2_sell_block_buy_ltc.py ./bot_v2_sell_doge_buy_ltc.py
cp howto/examples/bot_v2_sell_ltc_buy_block.py ./bot_v2_sell_ltc_buy_doge.py

# manually edit config files by your needs, but at least --makeraddress and --takeraddress is needed to edit
edit bot_v2_sell_doge_buy_ltc.py
edit bot_v2_sell_ltc_buy_doge.py

# run bot 1 on separated terminal window
python3 dxmakerbot_v2_run.py --config bot_v2_sell_doge_buy_ltc

# run bot 2 on separated terminal window
python3 dxmakerbot_v2_run.py --config bot_v2_sell_ltc_buy_doge

# to cancel opened orders CTRL + C
python3 dxmakerbot_v2_run.py --config bot_v2_sell_doge_buy_ltc --cancelmarket
python3 dxmakerbot_v2_run.py --config bot_v2_sell_ltc_buy_doge --cancelmarket

to check if bug occurred you need to compare values

./litecoin-cli getwalletinfo | grep balance
./dogecoin-cli getwalletinfo | grep balance

./blocknet-cli dxgetUtxos LTC false
./blocknet-cli dxgetUtxos DOGE false

./blocknet-cli dxgetUtxos LTC true
./blocknet-cli dxgetUtxos DOGE true

When bug hit and there are no orders are created,
command getwalletinfo returns correct value,
but dxgetUtxos is not able to return all UTXOs with include_used=false which should return all UTXOs because all should be free because no orders are created.

[walkjivefly]

Encountered this while testing SYS v4.3. Got inaccessible UTXOS for TBLOCK:

blockops@ops2:~/.syscoin$ btcli getwalletinfo
{
  "walletname": "",
  "walletversion": 169900,
  "balance": 233.29950035,
  "unconfirmed_balance": 0.00000000,
  "immature_balance": 3190.95924580,
  "txcount": 2619,
  "keypoololdest": 1637311989,
  "keypoolsize": 1000,
  "keypoolsize_hd_internal": 1000,
  "unlocked_until": 1738625539,
  "paytxfee": 0.00000000,
  "hdseedid": "3c9b490c660ec231a5346674b0a1005698d3f7b7",
  "private_keys_enabled": true
}
blockops@ops2:~/.syscoin$ btcli listunspent
[
  {
    "txid": "ad7d20d1d84c23837b42602b868918480215165f4db0d195f083d0ce3855710a",
    "vout": 1,
    "address": "y7zoKVGCEAQRGGphypACF6GHxVrwd2Q6i4",
    "label": "DX",
    "scriptPubKey": "21024e597aae9fd2023097b1f1e661b4fab20c47d05cbfc38efdbc1ae07c5f1c9e63ac",
    "amount": 187.99668580,
    "confirmations": 83,
    "spendable": true,
    "solvable": true,
    "desc": "pk([49dafd96/0'/0'/0']024e597aae9fd2023097b1f1e661b4fab20c47d05cbfc38efdbc1ae07c5f1c9e63)#tu3zw047",
    "safe": true
  },
  {
    "txid": "edbfd730a38ca39718d6903d19988ee70b111a0c848c5fb3175a52c47f7ba51c",
    "vout": 1,
    "address": "yAVRPxF555TSC8pgF7x55h31YCHGayRb8X",
    "scriptPubKey": "21023824f98c41165206fcd78605d8165845c7bb5c6ea21d6baad8a39a13e5919cbbac",
    "amount": 40.98921150,
    "confirmations": 856,
    "spendable": true,
    "solvable": true,
    "desc": "pk([49dafd96/0'/1'/1']023824f98c41165206fcd78605d8165845c7bb5c6ea21d6baad8a39a13e5919cbb)#03nlayf4",
    "safe": true
  },
  {
    "txid": "9c672387b1b518c5c90f0fbb7b2b710c4771777e308ae205831e3e31b0bf764f",
    "vout": 0,
    "address": "y7zoKVGCEAQRGGphypACF6GHxVrwd2Q6i4",
    "label": "DX",
    "scriptPubKey": "76a91478d7acd85d7e6aae0bd5758dd82b98f0e9a9324488ac",
    "amount": 0.47552500,
    "confirmations": 22,
    "spendable": true,
    "solvable": true,
    "desc": "pkh([49dafd96/0'/0'/0']024e597aae9fd2023097b1f1e661b4fab20c47d05cbfc38efdbc1ae07c5f1c9e63)#k2uwrvta",
    "safe": true
  },
  {
    "txid": "37224abd63f85c78cc164843cd5bc3c8c424b135aa4efda1869dcc12280e74d4",
    "vout": 1,
    "address": "y3aKvhgNP4FWyLyfvpUafhznyJwTyCHDyK",
    "scriptPubKey": "21031537e828def65f0a4ee58778990a376473c20fb726dc9db573cf0630cd482953ac",
    "amount": 3.83807805,
    "confirmations": 9934,
    "spendable": true,
    "solvable": true,
    "desc": "pk([49dafd96/0'/1'/5']031537e828def65f0a4ee58778990a376473c20fb726dc9db573cf0630cd482953)#cher4xqt",
    "safe": true
  }
]
blockops@ops2:~/.syscoin$ btcli dxgetutxos TBLOCK
[
  {
    "txid": "9c672387b1b518c5c90f0fbb7b2b710c4771777e308ae205831e3e31b0bf764f",
    "vout": 0,
    "amount": "0.47552500",
    "address": "y7zoKVGCEAQRGGphypACF6GHxVrwd2Q6i4",
    "scriptPubKey": "76a91478d7acd85d7e6aae0bd5758dd82b98f0e9a9324488ac",
    "confirmations": 22,
    "orderid": ""
  }
]

[nnmfnwl7]

btcli dxgetutxos TBLOCK

And comparing btcli dxgetutxos TBLOCK true and btcli dxgetutxos TBLOCK false results, has them same "orderid": "" 'empty' string but different number of utxos?