MQTT-topics at the same time? undecoded and decoded messages?

[ozett]

Is your feature request related to a problem? Please describe.
is it possible to get the "undecoded"-topic together with the decoded messages?
i activated the command for "undecoded" data. but decoding stops as expected.
https://docs.openmqttgateway.com/use/ble.html#advanced-setting-up-an-external-decoder

Describe the solution you'd like
would love too have both topics sending in parallel

Describe alternatives you've considered
fiddling in the source myself. but not really.

Additional context (historic data-snapshot, in realworld only one shows up)

[DigiH]

Frohes Neues @ozett

I think you got a bit confused between undecoded and the additional raw manufacturerdata of decoded messages.

Have a look at our conversation

#1372

and my comment and link about known manufacturerdata. My suggestion to read through the whole BLE usage page still stands ;)

[ozett]

dito, Frohes Neues @DigiH (and the World outside) 😄 🚀

yeah, you are right about a little confusion.
and i will try to read again the whole BLE usage-Page.

but, maybe my demand was only a little bit unclear?
i hoped to get always the aditional /undecoded/ topic.

1. /undecoded with the manu-/service-data as string
2. normal behaviour MAC-IDss-Topics with some meaningful JSON nodes of information

i will study the BLE-usage-Page if my confusion comes from some inattention and overlooking, that this is already the case after activating the externel-decoder option

i was also wondering, why espresence shows up the name of some devices (like appe-pencil and the toothbrush XMI), but on first sight they are missing in OMG. maybe the name-request is missing in OMG? (only wild guesses, but i think a developer can better decide than me, if thats a misunderstanding of some functionality or maybe a missing feature

[DigiH]

i hoped to get always the aditional /undecoded/ topic.

1. `/undecoded` with the manu-/service-data as string

The topic is always the MAC address, but to always get the additional raw manufacturer/service-data included in the decoded messages it is the two known*data options which need to be set to true.

The undecoded option really is only for not decoding any data on an ESP32 at all, but to send undecoded messages to a central gateway for decoding there.

[ozett]

Great, thanks. i understand that
to get ONE fixed topic and not to filter on the many MAC-topics i have to use the undecoded option.
to have always the BLE data-strings i acivate the known-data option(s), but each string is coming within the spefiic MAC topic for each device. ok, 🤔

[DigiH]

Bad copy/paste job, but this is what you will then get - decoded info in the message along with the raw manufacturerdata, available for further processing in node-red

[ozett]

no problem to set one filter over all the MQTT-topics starting with the MACs.

trying some OMG-options came from my hunt for some devices missing in OMG, but showing up in ESPRSENCE.
any hint to make them also visible in OMG ?
for example i dont see broadcasts from Microsoft-IDs (or the SMI-X3 toothbrush) in OMG.
or again my lack of understanding some details about the BT broadcast mechanics?

[DigiH]

From your animated gif it looks as if espresense has continuous scanning, whereas your OMG gateway likely has the default 10 seconds scan window every 55 seconds. Which might lead to some devices only being recognised after a while, depending on their broadcast interval.

You could try OMG with the build flags

'-DTimeBtwRead=0'
'-DScan_duration=1000'

for it to also have continuous scanning - re:

https://docs.openmqttgateway.com/use/ble.html#setting-the-time-between-ble-scans-and-force-a-scan

Assuming that both gateways are in the same location and at the distance to all the devices, it could also be that there is a difference in the MinimumRSSI setting

https://docs.openmqttgateway.com/use/ble.html#setting-the-minimum-rssi-accepted-to-publish-device-data

Best to then compare the MAC addresses to see if and which devices might be different or just being reported differently - e.g. where is the OMG recognised Stratos MAXO in the espresense list?

While the approach and usage scenarios for both gateways are quite different, on my side I seem get all my devices in both.

[DigiH]

For me the interesting question with your animation above is:

How many of the seven devices listed under the OMG gateway which are not Apple Continuity braodcasts, i.e. which do not have manufacturerdata starting with "4c00…", do NOT have decoded info along with their manufacturer/service-data, and are therefore unknown devices to Decoder and could possibly be added with decoders for them?

[ozett]

two settings do tweak on OMG:

1.1) 🔴 not done. -> is there a mqtt-command to set the "Scan_duration" on runtime, or only configurable as build-flag?

1. 🟢 done. -> send command to OMG for "128" ms BLE cycle like documented on OMG. using the espresence default value (!?)
   https://github.com/ESPresense/ESPresense/search?q=BLE_SCAN_INTERVAL

2. 🟢 done -> Min-rssi. The documentation could be improved with stating what impact to exprect from smaller/bigger values
   i am no expert what value to use to make it similar to espresence (rssi@1m?, rssi-tx/rx?))
   https://github.com/ESPresense/ESPresense/search?q=rssi

so i followed the OMG-documentation for "all devices":

accept all the devices by the following command:
mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{"minrssi":-200}'

an Acknowledgement for command/changes on MQTT would be nice.
to check all settings maybe a complete configuration/changes report on the topic: omg/LOLIN32LITE_BLE/SYStoMQTT ?

edit: fount it on another topic:

now i will oberve the effects an report later

---

[ozett]

Graphing the RSSI on MQTTexplorer is helpful, but no filter found to eliminate apple devices. i will try harder...

---

[ozett]

@DigiH

How many of the seven devices listed under the OMG gateway
#1375 (comment)

Answer attached:

1. filtered for non-apple and re-send on new topic
2. Question persists: does espresence discover (other than OMG) also devices which send no manu or service-data, but broadcast only their name on BLE level?

---

[DigiH]

not done. -> is there a mqtt-command to set the "Scan_duration" on runtime, or only configurable as build-flag?

Currently this is only settable as a build flag.

1. done. -> send command to OMG for "128" ms BLE cycle like documented on OMG. using the espresence default value (!?)

I'm afraid you got confused here between the interval between two scans (TimeBtwRead) and the BLE Scan Interval, which is named as such on both platforms. To be honest though, your wrongly assumed 128 ms doesn't make that much difference to the suggested 1 ms. There still was some intention behind giving you the values for the default continuous scan environment. ;)

2. Question persists: does espresence discover (other than OMG) also devices which send no manu or service-data, but broadcast only their name on BLE level?

Not quite sure if I understand this question fully, and to whom it is directed. For OMG we can see "VS9-EU-…" with only the name but no apparent manufacturer- or service-data.

But I'm also lost now as to what the remaining issue here is. The original issue and its heading about raw (undecoded) manufacturer- or service-data being displayed along with the decoded info should have been addressed with the known data flags.

So through OMG you get decoded messages for your GAEN broadcasting devices and two SwitchBot Meters.
The rest of the devices show their undecoded data, with or without a name broadcast, as they are currently not known to Theengs Decoder.

If possible inclusion and decoding of these devices is the goal, could you create separate issues for each of them in the Theengs Decoder repo, as the reverse engineering of the encoded data is very individual.

[ozett]

long thread, should end sometimes, youre right.

last things:

to the suggested 1 ms.

so i understand that is only changeable by buildflag '-DScan_duration=1000' .
i also had problems with build-settings for my wemos-d1 32, so.. -> i make another thread

For OMG we can see "VS9-EU-…" with only the name but no apparent manufacturer- or service-data.

True! That proofs that OMG is showing devices with name only.
still wondering how espresence manages to shows up my apple-pencil and the SMI-X3 toothbrush as i think OMG does not. (have to have a closer look) -> i make another thread

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 7 days since being marked as stale.