JSON message is too long (> 256 bytes)

[bam80]

My MQTT client app seems have a restriction of 256 bytes for the message payload length:
https://github.com/RadioShuttle/MQTTPushClient_Android

So I can't parse JSON's in the MQTT messages if they are bigger.

Could we have a setting to exclude some data from the JSON, as temperature in Fahrenheit and such?

Originally reported here: pvvx/ATC_MiThermometer#444

[koenvervloesem]

We could add a setting which lets you choose from publishing:

• only tempc
• only tempf
• both tempc and tempf

But do you think that will solve your issues for all Theengs Gateway's messages? In your case dropping tempf will only shave off 17 bytes. How much above 256 bytes is that message payload?

Is there no way to lift the restriction in your MQTT client app?

[bam80]

In my case, only last 2 bytes of the JSON do not fit: the tailing "}, so even just excluding tempf would help :)

It also rather cumbersome to have both tempc and tempf in the output, as one would usually need either one or another.

[1technophile]

Outside of tempc and tempf there are sensors, sending a lot of keys(for example the ones with accelerometers or the pollution ones), 256 is quite low for the json size. I would encourage to contact the app developer to increase this limit.

[bam80]

Is there no way to lift the restriction in your MQTT client app?

That would be the right thing to do,
but considering I already reported some issues there and didn't get any feedback, it's very well possible I won't get it in the future, thus changing the app might be quite a big thing.

On the other hand, keeping only one temp in the JSON might be useful anyway, so I would appreciate it if the problem is addressed here, too.

[1technophile]

What is your use case with the app?

[bam80]

Thermostat

[1technophile]

Do you control a valve with the app?

[bam80]

I will control the thermostat itself, but for now I need proper temperature sensor's graph first.
(The thermostat is for regulating room temperature, it's based on Tasmota smart plug)

[1technophile]

Theengs app does not answer this need?

[bam80]

I didn't look at Theengs app yet, thanks.

But I need proper MQTT dashboard, with widgets, push notifications, etc.
I find https://github.com/RadioShuttle/MQTTPushClient_Android quite attractive in that regard, considering lack of other open source options.

[bam80]

We could add a setting which lets you choose from publishing:

* only tempc

* only tempf

* both tempc and tempf

Could we add that option?
Having both temps has no sense most of the time.

[DigiH]

Hi @bam80,

My MQTT client app seems have a restriction of 256 bytes for the message payload length

While I understand the problem you are having with your Android client, might it not be a better idea to contact the developer of the app to see if the message length handling could not be improved there?

I'm only saying this because the messages you are seeing for your LYWSD03MMC/MJWSD05MMC_PVVX decoded messages are by no means the longest which would be produced by many other devices compatible with Theengs Decoder, the library which Theengs Gateway uses for decoding and which does the tempc to tempf conversion already in the library.

So even if we would exclude the Fahrenheit temperature here, just for your case - looking at the whole message though, even cutting the very long "model_id": "LYWSD03MMC/MJWSD05MMC_PVVX" would save more than double the space than the much shorter "tempf": 71.114,;) - it would still mean that a lot of other even longer messages produced by other Theengs Gateway compatible devices would still cause issues with that app. Then the next user would want the battery level, but not voltage, the next only voltage …

The on board library unit conversion has been implemented to allow for global compatibility, as not every user is using the produced MQTT messages with a controller allowing for automatic unit conversions.

The only really satisfying solution I can see is that the mentioned app does extend its MQTT message length compatibility, as longer messages are not that uncommon, and AFAIK

the official MQTT protocol allows messages with a maximum size of 268435455 bytes

although i think that might be pushing it a bit for real world usage :)

[DigiH]

Another solution I could think of would be an intermediate app/script which subscribes to the topic Theengs Gateway sends out, then does some cutting down of any unwanted properties for your case, to then re-publish the shorter messages on a different topic to which the app subscribes to.

[bam80]

I filed an issue to the app's github:
RadioShuttle/MQTTPushClient_Android#2
As I said, I doubt it will be addressed.

Then the next user would want the battery level, but not voltage, the next only voltage …

Wouldn't it be a right thing if the user could control the data passed over the MQTT, anyway?
That messages can be consumed by MCUs through WiFi, where every byte matters if the device is powered by battery..

[bam80]

Theengs app does not answer this need?

I wanted to give it a try but I see it costs money.
Is it possible to receive some "trial" build to estimate if it suits my needs?

[bam80]

To speed up the process, I purchased the app from Google Play Market, but unfortunately it didn't install after the purchase, so I had to refund.
Any way I could debug it or run it in the Android Virtual Machine first (without purchasing)?

[bam80]

Outside of tempc and tempf there are sensors, sending a lot of keys(for example the ones with accelerometers or the pollution ones), 256 is quite low for the json size. I would encourage to contact the app developer to increase this limit.

Hi, I reached out the app author. So far, the response is the following:

First of all 100 MQTT queued/persistent push messages up to 255 bytes are more than sufficient for most use cases.

RadioShuttle/MQTTPushClient_Android#4 (comment)

Also here:

This is all correct and done by purpose.

RadioShuttle/MQTTPushClient_Android#2

And the verdict:

Dear Andrey Butirsky, we have a couple of thousand accounts, and this is how it works and how it is designed, and all users are fine with this. We don't destroy our design for your requests.
You need to look for another solution or develop your own Push Server and App development.
Regards Helmut

RadioShuttle/MQTTPushClient_Android#4 (comment)

Thoughts?