E mined block failed verification #9496
Comments
|
When was the last time you saw this message? |
|
They are still happening. Started a few days ago. 2024-10-01 14:10:23.869 E Transaction not found in pool |
|
I restarted the container, immediately the same error started showing. |
|
Even a restricted RPC allows users to submit a mined block. Can you try if you get this message with no external RPC at all? |
|
Oh, I didn't know that. possibly just someone trying to submit bad blocks over and over? |
|
Possible, yes. But not sure why. |
|
Do you run with |
|
I see the same issue on my node starting at 2014-09-30 at 20:00 |
|
I do run with public-node. |
|
is there a way to see the IP of the sender which spams my node and add it to a block list? |
|
It might show up with log level 2 |
|
with log-level=2 I see this errors: and this: showing up regularly |
|
Doesn't seem related to me based on the error message. |
You should see (on loglevel 1) the ip which is contacting rpc endpoints |
|
this is more detail: looks like somebody tried to send 2 tx which are not accepted |
|
again more detail about the other error: |
|
I dont see the same mined block failed verification log. your issue seems unrelated @thilool |
|
it does not show up in the level 2 log. 024-10-04 22:12:46.650 I HTTP [194.233.152.128] POST /json_rpc |
|
here another one: btw I am running p2pool as well may this has something to do with it? |
|
these error are always in between: 2024-10-04 22:15:05.178 I Including transaction <96583190dda605d5b45597fd445a283c3f1c284bc5cf7bfc38a98cc2054970ac> maybe this is some hint what is going on there. |
|
I stopped p2pool to check but the errors still show up: The IP 162.218.65.219:18080 seems to always show up right before the error |
|
If youre not using --public-node, youve likely posted your node's address publicly on a website like monero.fail. you have a 194* ip using your RPC port and submitting bad blocks. You also have malicious node peers. restart you nlde and add |
|
enable-dns-blocklist is always on on my node so this did not prevent the issue. |
|
162.218.65.219:18080 this node should have been blocked by the dns blocklist |
|
monero | 2024-10-06 05:01:05.741 E Transaction not found in pool |
I have DNS blocklist set in my conf @nahuhh I guess restart monerod will force new DNS query for blocklist Shouldn't a peer be banned after submitting x bad blocks? Just realised as they are pushing to RPC they are not a peer, I think DNS blocklist is going to ban P2P peers and this might be why it don't work in this case |
Yah they are trying to DOS us by forcing constant block and TX verification I guess? |
The above ip is if a spy node peer. They arent the ones submitting the blocks dns blocklist only works for p2p peers. The bad blocks are being submit via RPC.
Those arent peers submitting bad blocks. do you have please see my previous comment - disable public-node and stop providing a remote node for strangers. |
They should spy through Tor lol that is dumb of them |
Probably they want this lol, I have the resources to absorb what they are doing so meh I can keep public RPC up |
|
I have been trying to think about what these nodes are trying to do, although just a general DOS attack would be a valid reason I find it weird how an entity like linking lion would want to do that. Obviously I am not 100% certain but they could be trying to exploit a timing difference when handling a block to see if a node has a stem pool transaction. When we receive a block from RPC, we try to build the full block, including all the txs using our tx-pool: monero/src/cryptonote_core/cryptonote_core.cpp Lines 1526 to 1538 in 9866a0e As you can see we are using I did some rough tests submitting blocks to my own node with and without some unknown txs, when an unknown tx is included the response is around 50-100x faster. I think an easy way to fix this would be to not look at the whole pool for restricted RPC. FWIW this doesn't explain the other P2P messages/RPC requests that were happening around the same time. It would be good to get more log data if anyone has any. |
Perhaps in a situation like mining where block propagation time matters, the intention is to try DOS other nodes such that their blocks that are mined have a greater chance of being propagated first, that said, I'm not sure of the stats for orphaned blocks in XMR but I would reckon it would be quite low yea? I think to find the originator of a dandelion++ TX all a malicious node has to do is not relay that stem TX and then it will force the originator to fluff it anyway wouldn't it? It would seem like a better way to do this would be to sybil the fk out of nodes get a really good connection footprint across the entire P2P network and then withhold stem TX waiting for maker to fluff it. I am assuming that each node that receives a stem TX has a fixed amount of time it waits for fluff and so if it is 10m then obviously the 10m will expire for the maker first then it fluffs. This could be fixed by making the time all nodes wait significantly random, then we get plausible deniability on who is the maker again. But this might be how it works now unsure, I been wanting to ask someone who knows. |
If this was the case they would just send empty blocks to make your node compute the PoW hash each time, this isn't what they are doing though, in your logs: Each of those
Yeah, we do already have randomised timers for this |
^ perps proxys |
|
@thisIsNotTheFoxUrLookingFor did it stop after banning all these IPs? |
Yup, I am running my node (RPC) through cloudflare and I poped in a WAF rule and it has stopped them dead... for now till they roll their proxies and get new IPs.
|
|
In just under 19 hours I have blocked > 75k requests from them haha |
|
I keep playing whackamole with their proxies. Perhaps if we could impose some restrictions on computationally expensive RPC calls in a restricted node this would assist to resolve the issue? Also they are grabbing a tx list of tx in the mempool of every node, it looks like they are trying to observe tx as they propagate through public nodes. They have a lot of IP addresses across a massive surface area of providers, could be a nation state actor or someone using a botnet of hacked devices. I might just whack a password on my node for now, if people are using cake wallet they have a range of public nodes to use anyway by default or choosable in the wallet, probably no merit in allowing these actors to continually scrape our mempools (/gettransactions & /get_transaction_pool) to observe what tx we see. |
|
@thisIsNotTheFoxUrLookingFor dont just add a password, make sure you disable You might want to set firewall rules and perhaps change the port |
Always it has been disabled. They are scraping
My node is behind cloudflare so I have some defence against them by utilising cf's WAF. I will play with them I don't mind it, waste their resources haha. In the last 24hrs alone I have blocked 129k of their RPC requests |
|
Hi all, As per discussion once I enabled log-level=1 I was able to see a number of peers submitting bad transactions as well as submitting blocks that failed verification. Example of bad block submitted:
Example of bad transaction:
My questions are: Would it be safe to assume that the peer using RPC requests to submit a mined block that is failing verification is malicious? Thanks. Happy to provide more logs and examples if needed. |
I'm yet to ban any IPs edit: banned 2 IPs, (the ones in above post), and following this rabbit hole and have noticed I am receiving communications from the banned IPs in @Boog900's list (eg: 128.140.85.154) |
Ran this public node for years, has the latest version, restricted-rpc is enabled and CPU use doesn't show monerod mining at all. ..yet logs are flooded with these:
2024-09-23 17:51:00.906 E mined block failed verification
First i've seen it, I appear to be on the latest block and handling connections ok anyway.
The text was updated successfully, but these errors were encountered: