Feature Request: Supporting mesh packets on UDP over IP would allow Meshtastic nodes to create a mesh network over IP, when connected to the same subnet, in the very same way they create it with LoRa.
Furthermore, this would allow testing of the mesh protocol, for developing a network interface agnostic protocol, without LoRa connections, or without any type of other IP network infrastructure (as the Meshtastic nodes would manage the mesh on UDP).
This could also be a easy way to create mesh connections between repeaters over WLAN/LAN.
This feature might be achievable with moderate effort, as the mesh protocol is already designed by @geeksville , and the WLAN-client feature has already been designed by @mc-hamster (as I recall).
MQTT is on the roadmap, as @mc-hamster noted, but does rely on reliable connections, such as TCP over IP, and other external devices, such as MQTT servers.
This feature has been shortly discussed here: iOS Experiments
Good point. Iāve been reluctant to add UDP support but the cases you list are compelling. Also for aviation applications (or any application where at least one node has great line of sight) a very fast spreadfactor should work fine.
So meshtastic could provide a nice long range zeroconf IP link in that case.
I think it would not be too hard to have the meshtastic python app implement a user space TAP/TUN network interface. So that a zeroconf IP link for any node on the mesh would appear on a āmesh0ā interface. Iāll do the preliminary investigation/doc reading this week.
How about āsimplifyingā things (the most simple things tend to be the most complex ones), at least for the beginning?
Packets on UDP could be carried within the UDP payload.
Therefore, the mesh protocol would handle all the basic things on both LoRa and UDP the same way. However, UDP links should be assigned shorter tx delays.
The IP link can listen for packets, and broadcast packets, on UDP on a fixed IP and port, regardless of the assigned client IP. All mesh packets would be parsed from the UDP payloads. This should mitigate the process, as changing IP addresses, according to the node number, wouldnāt be necessary.
Creating the UDP link between two nodes:
Add listening and broadcast features for UDP over IP on the node trough WLAN client (this is mostly done?)
Connect nodes to the same net (UDP pass through / no NAT)
Add listening for UDP packets on 223.6.5.43:6543:udp
Add broadcast mesh packets as UDP payload on 223.6.5.43:6543:udp
Send mesh packets trough both LoRa and UDP (the whole ordinary mesh packet within UDP payload), if no known route exists. Send regarding routing table, on LoRa or UDP link, if a valid route exists
Listen and receive mesh packets also from UDP payloads on 223.6.5.43:6543:udp
Parse the UDP payload for designated node addresses
Record node routes from both LoRa and UDP
Receive the message if intended to us, change hop counter & retransmit if not (do not retransmit from UDP to UDP)
Send ACK if required
UDP transport would be transparent for the Python user, as the nodes mesh protocol would handle the logical routing between LoRa and UDP as necessary.
This approach does not allow direct UDP connections between python host and the node, but a direct UDP mesh connection between two nodes.
I donāt have a keyboard until Monday so I canāt write much now. But I was surprised at how easy the TUN api was, so I think just implementing TUN will be easier than what you describe (and it will work for any IP protocol/port number).
This is now implemented and approximately works. Tested with ICMP pings and UDP echos (Iāve even done SSH over the meshtastic mesh).
Probably not ready for others yet because this testing turned out to be a great stresstest and I found some cases where we are dropping packets. The drops were hidden by the ack/retry mechanism we use most of the time for mesh packets. For IP I donāt use that feature (because IP handles its own retransmissions if needed). So Iāll be debugging and fixing that root cause sometime in the next few weeks.
But if anyone is super brave, this code is in the latest python tool: (It works on linux, might work on OS x, I doubt it works on windows)
meshtastic --tunnel
Connected to radio
INFO:root:Starting IP to mesh tunnel (you must be root for this *pre-alpha* feature to work). Mesh members:
INFO:root:Node !2462abf84098 has IP address 10.115.64.152
INFO:root:Node !2462abdddf38 has IP address 10.115.223.56
INFO:root:Node !fd1004ec5d56 has IP address 10.115.93.86
INFO:root:Node !246f28979058 has IP address 10.115.144.88
btw - if using the default āvery long range but very slowā channel settings, this really conveys how slow LORA is and not a good fit for many IP applications
Meshtastic-python$ ping -i 30 -W 30 10.115.64.152
PING 10.115.64.152 (10.115.64.152) 56(84) bytes of data.
64 bytes from 10.115.64.152: icmp_seq=1 ttl=64 time=17785 ms
64 bytes from 10.115.64.152: icmp_seq=2 ttl=64 time=17099 ms
64 bytes from 10.115.64.152: icmp_seq=3 ttl=64 time=18498 ms
64 bytes from 10.115.64.152: icmp_seq=4 ttl=64 time=17495 ms
64 bytes from 10.115.64.152: icmp_seq=5 ttl=64 time=17306 ms
btw: we currently send the entire IP packet for every packet. An easy optimization would be to strip out much of the 20 byte header (because it is mostly redundant with our mesh info), which would probably cut ping times by 30%.
@geeksville Correct me if Iām wrong here: The original feature request was clearly to implement using UDP connections to extend the mesh. But as far as I understand it you now implemented IP connection on top of the mesh.
I want to point out that in Europe (and probably other countries as well) the 863 to 870 MHz band has limitations with either a transmission duty cycle of 0.1%, 1% or 10% (link 12) depending on the exact band. Limiting the duty cicle is important to stay within the legal regulations.
Does Meshtastic obey those duty cycles?
I think using a IP via LoRa bridge is not really an option for the European version.
Good point! OMG you are right. When I read @CycloMies original request my brain just immediately convolved it with (almost the opposite) common request of āIād like to be able to pump IP over these radios.ā Which I had never been excited about (and still not) because the data rate is so low.
I think for cyclomiesā true request the best solution will be the (eventual) mqtt gateway and (optional) helper web service.
wrt the 1% duty cycle limit in EU: The original ā1.0 meshtasticā of a āradio for hikers that occasionally sends small position messages and a few text messagesā seemed fine in my rough spreadsheet. But for all this post 1.0 stuff - I have no idea and you are right to be concerned. There are no limits in the code that stop API clients from talking too much. Perhaps we could solve that eventually with using the existing ack/nak mechanism to return a fault code of ārate limited, packet droppedā to the API consumer. Alas it will be a while (after June 2021?) before I have time to focus on that (though Iād eagerly help anyone interested who wants to add this).
I think both, MQTT and UDP have their up and downsides, but in terms of building a mesh network UDP would be the winner and focus for me. As CycloMies pointed out UDP connections could form an actual mesh, while the MQTT solution would form more of a star topology which is more likely to fail and requires additional hard and software. The simplicity of UDP would allow to connect multiple radios (e.g. with directional antennas in an isolated location) via WLAN by putting one in access point mode and the others in client mode. This would not require additional hardware running a MQTT server and could help to bridge longer distances and build the global mesh as well. It would also be a good and fast approach to get the community more active. Node operators would need to get into contact with other node operators to form the mesh.
Personally I think having a global mesh with multiple public and private channels as early as possible would really help to get more people involved. Beeing able to chat with people all over the world right after setting up a node would make it way more interesting to contribute. Without like-minded people to talk to on a global mesh the experience when first setting up a node is just not very satisfying and there is no point of leaving a stationary node running if yourāre the only user. I would really go for pitching the story āBecome part of a global mesh networkā.
IP and SSH over LoRa - incredible job @geeksville! This feature might be more valuable, in the future, than we might think todayā¦
Thank you @Noki for describing the idea behind meshing over UDP.
UDP could be understood, in this case, similarly to wireless connections: direct connections to all within the reach (within the same LAN, WLAN, VPNā¦). Therefore, nodes could handle both LoRa and UDP interfaces similar, by building a mesh net from nodes they āhearā/reach to, and by maintaining a list of all nodes (both LoRa and UDP neighbors). Mesh packets could be sent the same way on both LoRa and UDP, as the interfaces are responsible for the lowest levels of networking.
UDP is handy, as you can send packets always to a predefined IP and port, regardless of the IP addresses interfaces are currently using. To do UDP meshing, the IP and port to listen to on UDP, for incoming mesh packets, has to be predefined.
The nicest features of UDP meshing would be the a) speed, and b) the ability to seamlessly link the mesh on both LoRa and IP nets. Think about a Meshtastic repeater on a hill, with a 4G/5G WLAN modem nearby with VPN client capability. You could then connect the node to the modem with WLAN, and the modem to the rest of your VPN network. With this setup, users could send ordinary mesh packets from LoRa trough VPN to anywhere on the mesh (limited by the hop count). This configuration would help the maintenance of repeater nodes, as one could do it trough the VPN with help of predefined mesh commands.
Any comments are always appreciated!
PS. @mc-hamster Iām also a big fan of a automatic air time calculation and restricting system. The app could print a value to inform users about the bandwidth left usable on the mesh (answering to the questions: āWhen will my packet be sent? and How much air time will my message consume of the allowed air time?ā)
I think Iām done with tracking air time usage for transmitted packets, itās just untested right now. Because of the nature of this, actually testing will take a while. It keeps the history of up to 48 hours ā each hour is reportable for historical analysis and graphing. I think itād look cool to have a line graph of channel utilization and @geeksville suggested historical data may be used to make mesh routing smarter.
The framework is built to support reporting on the airtime of the current device, channel utilization of all devices on the mesh network (as defined by the channel configuration) and all lora devices on the current frequency, including those that are not part of our network. Those last two are just not connected to any hooks. Itās for future use.
I also donāt know how exactly to use this information but Iāll trigger and expose the report on the web interface for now (@crossan007 - if this interests you, iāll put it in a json response, otherwise Iāll just dump it into a text file).
Moreover, UDP meshing would let users easily link nodes on different MHz to the same mesh, by connecting both nodes to the same WLAN. UDP meshing would handle the rest of the routing.
while testing mesh-tunnel functionality on TTGO T-Beam V1.1 433MHz i noticed that sending side will crash after a while. error also was present in 1.1.31.
reproduce:
1.reset device to defaults and then meshtastic --setch-shortfast and start mesh-tunnel on both sides
2.ifconfig mesh0 mtu 200 both sides too
3.start listening on recive side: nc -l -p 608 -v|pv|dd of=/dev/null
4.start send from sender: dd if=/dev/urandom |pv|nc revicer_ip_address 608
after ~225KiB i get
Trigger powerFSM 9
Trigger powerFSM 3
Trigger powerFSM 9
assertion āisReceivingā failed: file āsrc/mesh/RadioLibInterface.cppā, line 212, function: void RadioLibInterface::handleReceiveInterrupt()
abort() was called at PC 0x4010cec7 on core 1
It would be REALLY great to include wireshark into this endeavor. Then you could have node to multi-node WAN connections with the same security as the private LoRa network. it would be a great seamless way to have transparent bridging without compromising security
This feature might be more valuable, in the future, than we might think todayā¦
