C# and sending protobuf messages out via MQTT

Hi All, I am still a relative new comer in terms of meshtastic and the interacting with it. MQTT I/we use extensively for other products and factory systems.

For background my topology at present is three (MT) nodes, All are T-LoRa 2.1-1.6,

  1. WiFi/MQTT to my local broker
  2. External Application processor connected by serial
  3. the same as (2).
    All nodes are running FW 2.2.24. I have two secondary channels (on all nodes, numbered and named the same).
    On the Gateway node ALL channels have the MQTT UP/DN link set enabled. MQTT is enabled and JSON is enabled (although messages are published on both PB and JSON topics). No encryption or TLS. Proxy to client not enabled (I don’t think this is necessary as the node is connected to network via WiFi).
    I have got the latest proto files (well they are the 16/3/2024 pull) and compiled these to .cs files with protoc (external to C#, but I don’t think that is the issue)

I am happily sending text messages through the mesh (channel 0 for the moment) and the MQTT (GW) node is publishing them to the mqtt broker (topic: msh/2/c/LongFast/!. I have an application, written in C#, it is an MQTT client and subscribed to the channel topic (msh/2/c/LongFast/#). I am getting the (text/telemetry/private) messages in and deserialising and displaying them to a text box. Kewl, Tick.

I mention Private_App messages as I have node (2) sending data as Private_App type (button press) which I am receiving by node (3) (output to some LEDs) and also is being published on the ./c/LongFast topic which I am getting in the C# app, deserialising and then mimicking the LEDs. This is all working as I expect. I am using the Protobuf topic (msh/2/c/LongFast/#) as the private message doesn’t seem to be getting published to the json topic (I figured it didn’t know how to deal with it, so it doesn’t. Rightly or Wrongly). I am looking to utilise the Private_APP message type (or other “private” types) for my application, and across different MT node types (esp32, nRF).

Now the issue I have is sending messages OUT of the C# App. Just trying the Text_Message_App first.
I set up a MeshPacket (myMeshPacket) and the ServiceEnvelope (outSE). Code as follows.

        MeshPacket myMeshPacket = new MeshPacket();             //Create the Mesh Packet container
        myMeshPacket.Decoded = new Meshtastic.Protobufs.Data(); //initialise inner container

        //Package into a meshPacket
        String payloadBytes = TB_Publish.Text;
        myMeshPacket.Decoded.Payload = ByteString.CopyFrom(payloadBytes, Encoding.UTF8);
        myMeshPacket.Decoded.Portnum = PortNum.TextMessageApp;
        myMeshPacket.Decoded.WantResponse=false;    //Get Ack
        myMeshPacket.Id = (uint)rnd.Next(0,0xFFFFFF);   //Random number
        myMeshPacket.From = 0x254D544E; // Gateway node MAC/ID
        myMeshPacket.To = 0xFFFFFFFF;   //Broadcast
        myMeshPacket.Channel = 0;        //Channel 0
        //          myMeshPacket.ViaMqtt = true;

        //Package into an MQTT transport packet
        ServiceEnvelope outSE = new ServiceEnvelope(); //Create the service Envelope
        outSE.ChannelId = "LongFast";   //Channel Name
        outSE.GatewayId = "!254D544E";  //Gateway name
        outSE.Packet = myMeshPacket;     //Packet
        
        //Serialise
        byte[] outMSG = new byte[236];
        //MemoryStream outMSG2 = new MemoryStream();
        CodedOutputStream outStream = new CodedOutputStream(outMSG);
        outSE.WriteTo(outStream);

        //At this point the array is too long, so shorten it somehow.
        byte[] outMSG2 = new byte[outStream.Position];  //Create an array of length of the stream
        Array.Copy(outMSG, outMSG2, outStream.Position);    //Copy to the new array

        //Publish to the correct topic
        Task.Run(() =>
        {
            if (mqttClient != null && mqttClient.IsConnected)
            {
                mqttClient.Publish("msh/2/c/LongFast/!254D544E", outMSG2);
            }
        });

Note: I have substituted the real gw nodeid for “254D544E”, same case as well.

Now this works, to the point that it is published to the broker AND my app (via topic subscription) receives the packet back again (and deserialises) but nothing seems to be getting on to the mesh (and I’m not sure how to sniff/log it either).

So I am wondering a few things

  1. does the MQTT App need to initialise a client relationship with the gateway node (similar to serial application processor needing to issue an mt_request_node_report() prior to receiving data). My expectation is that this should not be necessary as the MT gateway node should have subscribed to the topic and thus be listening for it.
  2. is the topic (msh/2/c/LongFast/!254D544E correct? Should this be publishing on a different topic (eg msh/2/c/LongFast/Broker)
  3. is there something in the ServiceEnvelope or the MeshPacket that is either not present, not required, incorrect and this is what is causing it to not work.
  4. is there something else that I have completely overlooked?
  5. is there a better way to achieve this?

Unless the C# code is at fault please don’t criticise the code, for me it is just a means to an end, if it works and achieves the goal it is fine, there are most probably more elegant and efficient ways of doing it. Having said that I am open to learning how to do things better in C#.

So, that is it. If more information is required I will endeavour to provide it.
Thanks.

try adding
myMeshPacket.RxTime = (uint)DateTimeOffset.UtcNow.ToUnixTimeSeconds();

be sure your connected node is allowing mqqt downlink on channel configuration

Hi!

Sorry for late reply but I was trying to do something similar and stumbled upon this thread.
In my case it was the same, protobuf on MQTT was published and nothing then happened on the mesh.

Turns out when you set GatewayId and From to the same node ID as your Router, it will just discard the MQTT messages assuming they originated from it (you can see that in the logs). I set both GatewayId and From parameters to be different than my Router and it sent out the message to the mesh just fine.
Here’s a snippet of code in Python that uses paho MQTT library to publish protobuf.

from meshtastic import mqtt_pb2
import time

...
MESH_TOPIC = "msh/EU_868/2/e/serial/!da544760"
...

env2 = mqtt_pb2.ServiceEnvelope()
env2.channel_id = 'serial'
env2.gateway_id = '!da544761'
setattr(env2.packet, 'from', 3662956385)
setattr(env2.packet, 'id', int(time.time()*1000)%420)
env2.packet.to = (1<<32)-1
env2.packet.channel = 1
env2.packet.decoded.portnum = 1
env2.packet.decoded.payload = b'Lorem Ipsum '*3
env2.packet.decoded.want_response = False
env2.packet.rx_time = int(time.time())
env2.packet.hop_start = 0
#env2.packet.via_mqtt = True
env2.packet.hop_limit = 3
env2.packet.want_ack = False
print("DEBUG")
print(env2.SerializeToString())
print(env2)

client.publish(MESH_TOPIC, env2.SerializeToString())

olokelo. That is exactly the solution to the issue. I made the change to my code it is just works now. Now to get the rest of the project updated.

Best Regards.