Problem with invalid JSON

[iz1kga]

Hi,

making some experiments wit 1.2.x version I found problems with interface.nodes dict. Sometimes appear a raw key that contains IMHO invalid json and so I can’t dump it.

{‘num’: 2988737488, ‘user’: {‘id’: ‘!b22483d0’, ‘longName’: ‘VillarFocchiardo_83d0’, ‘shortName’: ‘Vil’, ‘macaddr’: ‘rGeyJIPQ’, ‘hwModel’: ‘TBEAM’, ‘raw’: id: “!b22483d0”
long_name: “VillarFocchiardo_83d0”
short_name: “Vil”
macaddr: “\254g\262$\203\320”
hw_model: TBEAM
}, ‘position’: {‘latitudeI’: 451074598, ‘longitudeI’: 72341957, ‘altitude’: 478, ‘latitude’: 45.1074598, ‘longitude’: 7.2341957}}

looking at ‘raw’: there is something strange? Do you agree?

Regards

[geeksville]

Hmm. ‘raw’ isn’t really json (or a python dictionary), it is the protobuf object itself. Possibly our tostring function isn’t handling that well?

[iz1kga]

I think something similar, if i’m not wrong maybe quote is missing.

I’ll investigate more to give more info.

thanks

[iz1kga]

Normally when I print

receivedNodes = copy.deepcopy(interface.nodes)
print(“getNodes() --------->”)
print(receivedNodes)
print("------------------------------")
json.dumps(receivedNodes)

I get

getNodes() --------->
{’!c4fc6a04’: {‘num’: 3304876548, ‘user’: {‘id’: ‘!c4fc6a04’, ‘longName’: ‘KGA1.2_6a04’, ‘shortName’: ‘K12’, ‘macaddr’: ‘JArE/GoE’, ‘hwModel’: ‘HELTEC’}, ‘position’: {‘latitudeI’: 455000000, ‘longitudeI’: 75000000, ‘altitude’: 430, ‘latitude’: 45.5, ‘longitude’: 7.5}, ‘lastHeard’: 1617028537}, ‘!b2248518’: {‘num’: 2988737816, ‘user’: {‘id’: ‘!b2248518’, ‘longName’: ‘TestKga8518’, ‘shortName’: ‘Tst’, ‘macaddr’: ‘rGeyJIUY’, ‘hwModel’: ‘TBEAM’}, ‘position’: {‘latitudeI’: 451073811, ‘longitudeI’: 72335574, ‘altitude’: 422, ‘batteryLevel’: 100, ‘time’: 1617028493, ‘latitude’: 45.1073811, ‘longitude’: 7.2335574}, ‘lastHeard’: 1617028448, ‘snr’: 11.0}, ‘!b22483d0’: {‘num’: 2988737488, ‘user’: {‘id’: ‘!b22483d0’, ‘longName’: ‘VillarFocchiardo_83d0’, ‘shortName’: ‘Vil’, ‘macaddr’: ‘rGeyJIPQ’, ‘hwModel’: ‘TBEAM’}, ‘position’: {‘latitudeI’: 451073729, ‘longitudeI’: 72335785, ‘altitude’: 431, ‘time’: 1617028193, ‘latitude’: 45.1073729, ‘longitude’: 7.233578499999999}, ‘lastHeard’: 1617028165, ‘snr’: 9.25}, ‘!c4fc6d68’: {‘num’: 3304877416, ‘user’: {‘id’: ‘!c4fc6d68’, ‘longName’: ‘TOGEU01_6d68’, ‘shortName’: ‘TG0’, ‘macaddr’: ‘JArE/G1o’, ‘hwModel’: ‘HELTEC’}, ‘lastHeard’: 1617003008, ‘snr’: 10.25}}

Sometimes appears the raw

getNodes() --------->
{’!c4fc6a04’: {‘num’: 3304876548, ‘user’: {‘id’: ‘!c4fc6a04’, ‘longName’: ‘KGA1.2_6a04’, ‘shortName’: ‘K12’, ‘macaddr’: ‘JArE/GoE’, ‘hwModel’: ‘HELTEC’, ‘raw’: id: “!c4fc6a04”
long_name: “KGA1.2_6a04”
short_name: “K12”
macaddr: “\n\304\374j\004" hw_model: HELTEC }, 'position': {'latitudeI': 455000000, 'longitudeI': 75000000, 'altitude': 430, 'raw': latitude_i: 455000000 longitude_i: 75000000 altitude: 430 , 'latitude': 45.5, 'longitude': 7.5}, 'lastHeard': 1617028435}, '!b2248518': {'num': 2988737816, 'user': {'id': '!b2248518', 'longName': 'TestKga8518', 'shortName': 'Tst', 'macaddr': 'rGeyJIUY', 'hwModel': 'TBEAM', 'raw': id: "!b2248518" long_name: "TestKga8518" short_name: "Tst" macaddr: "\254g\262\205\030”
hw_model: TBEAM
}, ‘position’: {‘latitudeI’: 451073991, ‘longitudeI’: 72334075, ‘altitude’: 421, ‘batteryLevel’: 100, ‘time’: 1617028092, ‘raw’: latitude_i: 451073991
longitude_i: 72334075
altitude: 421
battery_level: 100
time: 1617028092
, ‘latitude’: 45.107399099999995, ‘longitude’: 7.233407499999999}, ‘lastHeard’: 1617028424, ‘snr’: 0.75}, ‘!b22483d0’: {‘num’: 2988737488, ‘user’: {‘id’: ‘!b22483d0’, ‘longName’: ‘VillarFocchiardo_83d0’, ‘shortName’: ‘Vil’, ‘macaddr’: ‘rGeyJIPQ’, ‘hwModel’: ‘TBEAM’, ‘raw’: id: “!b22483d0”
long_name: “VillarFocchiardo_83d0”
short_name: “Vil”
macaddr: “\254g\262$\203\320”
hw_model: TBEAM
}, ‘position’: {‘latitudeI’: 451073729, ‘longitudeI’: 72335785, ‘altitude’: 431, ‘time’: 1617028193, ‘raw’: latitude_i: 451073729
longitude_i: 72335785
altitude: 431
time: 1617028193
, ‘latitude’: 45.1073729, ‘longitude’: 7.233578499999999}, ‘lastHeard’: 1617028165, ‘snr’: 9.25}, ‘!c4fc6d68’: {‘num’: 3304877416, ‘user’: {‘id’: ‘!c4fc6d68’, ‘longName’: ‘TOGEU01_6d68’, ‘shortName’: ‘TG0’, ‘macaddr’: ‘JArE/G1o’, ‘hwModel’: ‘HELTEC’}, ‘lastHeard’: 1617003008, ‘snr’: 10.25}}

and then i get: TypeError: Object of type User is not JSON serializable

IMHO the problem is the missing quote in the raw value.

Regards

[iz1kga]

@geeksville Should I open an issue on GIT?

Regards

[sachaw]

Feel free to open an issue if you have a reproducible bit of code, as it may well be a bug

[iz1kga]

Thanks.

after some time of execution json.dumps fails.

I’ll Open the Bug.

Regards

import meshtastic
from pubsub import pub
import json

def update(interface, packet):
    print("--- --- --- interface.nodes: --- --- ---")
    print(interface.nodes)
    print("--- --- --- JSON DUMPS --- --- ---")
    print(json.dumps(interface.nodes))

if __name__ == '__main__':
    interface = meshtastic.SerialInterface()
    pub.subscribe(update, "meshtastic.receive")

[iz1kga]

I Had time to look into the code, for what I can understand raw protobuf object cannot be direcly serialized with json.dumps().

So how to serialize meshtastic.SerialInterface().nodes easily?

[geeksville]

Hmm. Strip out the raw property before serializing?

[iz1kga]

I Think so too, as I see the Raw can appear in “User” ad in “Position”. I’ll to search the raw key around the dict or I can assume it’ll appear only in theese positions?

Regards