Meshtastic <--> Secure Scuttlebutt

Meshtastic integration with Secure Scuttlebutt

Secure Scuttlebutt (SSB) is a decentralized data platform, specifically made for offline-first social networks. There are many similar decentralized platforms out there, famous ones this days seem to be Mastodon and Matrix. But for my use-cases, communities that have no connectivity, they both lack true p2p, relying on servers, which disable device-as-infraestructure.

There are spaces where federated and p2p have been converging, Mastodon example and Matrix example. And as convergences happen they start facing similar issues, like identity spread around multiple devices and device storage for example.

There’s lots that makes Scuttlebutt special, specially the community and end-goals behind the protocol. But technically, it’s been growing for the offline/off-grid first perspective. So it’s advanced a lot on how to enable security and at the same time discovery for completely p2p gossip networks that don’t rely on DHTs.

The reasoning behind integrating SSB and Meshtastic is that they both provide the cheapest transports possible. First SSB turns every device into a node which communicates within Bluetooth or Wi-Fi range and can carry messages sneakernet-style. LoRa is the most accessible long range transport out there, so it’s naturally the next option for extending a network, say to another community for example.

Next steps would include a Wi-Fi long range link, HF, satellite or cable. But all of those are in another order of magnitude of accessibility (service-provider availability, licensing and cost) and usually open up to the Internet, which is not desired by every community.

Challenges

SSB relies heavily on cryptography to be secure. Hashes are long pieces of data that are terrible to carry over a low-bandwidth transport such as LoRa. A simple “like” in SSB looks like this:

{
 "key": "%a8acTq/Bss5G1j7JsggyHibLJ+PIM/1XvhifOLEXqUA=.sha256",
 "value": {
 "previous": "%z0IQ/vul09vRhnk+nvpVGrzHE6K7UD99h9NdL2gr+8s=.sha256",
 "author": "@2RNGJafZtMHzd76gyvqH6EJiK7kBnXeak5mBWzoO/iU=.ed25519",
 "sequence": 377,
 "timestamp": 1518257352380,
 "hash": "sha256",
 "content": {
 "type": "vote",
 "channel": null,
 "vote": {
 "link": "%ehhSqb2l/GbqhdWhvrHs0LkfTjA2LB4krK/kdY/eCqg=.sha256",
 "value": 1,
 "expression": "Like"
 }
 },
 "signature": "yrXKc7C5riUVXB6ckaLZ7zzvn63G6AdzIEPM1esuu3EDS2whDr5DL2N8ILPGoa4pzp9cMd4O6kJEJEkr1/IPB==.sig.ed25519"
 },
 "timestamp": 1580754824704
}

That’s 543 bytes for a *Like message×, about 400 bytes just for the envelope. But a message can easily be well over 1Kb. All this in JSON format of course.

Is this even possible?

We obviously don’t want to replicate the whole SSB network over LoRa, that’s insane. But it makes sense to replicate messages of key types. For example, we’d want to replicate identities (public-key, identity-fusion, name, location) over Lora, and maybe public and private messages of a type “Emergency”.

The networks we’re thinking about are composed of neighboring communities. So we can expect and play with certain amounts of trust (less security concerns).

ssb.proto would look something like this:

message SsbMessage {
 message Value {
 string previous = 1;
 string author = 2;
 uint32 sequence = 3;
 uint64 timestamp = 4;
 string hash = 5;
 string content = 6;
 string signature = 7;
 }

 string key = 1;
 Value = 2;
 uint64 timestamp = 3;
}

We could use distributed hash tables over LoRa and a shared secret (maybe the channel name) to hash the hashes into smaller-sized hashes that are re-constructed after each send/receive. As our chances of collision will be very small in a such a small network. But don’t even know if it’s worth it.

We could also automatically break the messages into smaller chunks, sending first headers then broken up chucks of content.

But then the biggest issue is on SSB’s side. It keeps an append-only log for each feed (key pair), where every message refers to the last one, so that logs are temper-proof. Receiving a special message from LoRa without having previous messages to check will break its security. But it’s probably a worth-while trade-off, as ssb-partial-replication has done (?).

Conclusion

The end experience we’re aiming at is for complex, multi-user, multi-threaded interactions that SSB offers, while using LoRa as transport in remote locations. I believe this and voice-over-LoRa are key features that would perfectly fit within the ecosystem of applications that can satisfy needs of networks of offline communities.

Thoughts?

7 Likes

Are you proposing a scuttlebutt-light client on-device or a gateway at the edge that forwards to and from lora clients?

If there are people regularly moving from village to village a wifi device may do reasonably well for syncing local network segments.

The LoRa bandwidth isn’t too bad if you can use the fast settings and are in an environment where you don’t need to limit the duty cycle. (And limiting to text payloads)

Do you feel like a true mesh network is needed for this backhaul connection? A p2p link could be much more efficient.

In theory you could have nodes running multiple radios and do relay instead of repeat, and / or bond radios and channels in to a higher thru put link. But I think that is far beyond meshtastic and would be better as a new project.

We’re thinking on one Meshtastic device per village here. So we can use a lot of strategies at the client’s end by limiting characters in messages and limiting amounts of message sent per time cycle.

@Spor7biker villages can be really far, and movement isn’t as regular. P2p will be wonderful for whole feed syncs when using WiFi (specially to the Internet), but we can benefit from close to real-time LoRa transport, as long as we don’t completely spam the network.

The LoRa bandwidth isn’t too bad if you can use the fast settings and are in an environment where you don’t need to limit the duty cycle. (And limiting to text payloads)

I still don’t fully understand duty cycles. In theory I could use 100% duty cycles per day on a channel in a remote place where no collisions with other broadcasts would happen, correct? Would the only downsides be more power and illegality?

Do you feel like a true mesh network is needed for this backhaul connection?

In last year’s experiment there was no way to make point-to-point links, a true mesh is really desirable.

And a mesh fits really well with Scuttlebutt’s nature, as I believe many of the SSB messages will be posts that village leaders will exchange publically to organize events (Krahô life is about partying). So we want the posts to reach everywhere. Another use case is a private message to reach someone thru the Internet. We want everyone to have the private message, so that the first person who visits the Internet will send the private message.

The challenge might come in educating about the use of these “emergency” messages and/or creating crypto-barriers for malicious nodes or bad actors.

In theory you could have nodes running multiple radios and do relay instead of repeat, and / or bond radios and channels in to a higher thru put link. But I think that is far beyond meshtastic and would be better as a new project.

That’s really interesting, I find the use of multiple radios a good modular solution to scale up as the network needs it. Is there a thread or project for that? The use of one 433Mhz for mesh and one 446Mhz for PTT Voice really excites me.

1 Like

@ghettodev still slowly trying to understand how to tackle the challenge, but I’d say we’re going for a way to compress SSB feeds thru hashtables, and send only really compressed messages thru Meshtastic, containing essential (sequence and content payload for example), and decompressing them at the other end. This method would require an initial handshake to spread these hashtables around using LoRa.

But yes, a SSB web client that does compression/decompression and communicates with Meshtastic.js.

1 Like

It’s mostlly about fair use of a shared resource. Ideally devices are designed to listen before broadcast. The cool thing about LoRa is in theory more than one device can be transmit in a given area due to different power levels and spreading factors.

I fully get what you are trying to do, and the appeal of LoRa, and the unique situation where there are few if any other radio operators in the area. But at a certain point I start to wonder if a traditional solution ends up having a better cost to value ratio. Especially when you factor in the true cost of time and money for the long term operation.

Cheep, power efficient devices pushing the limits of range are likely to be bandwidth limited. Running a repeat mesh cuts that bandwidth in half with just a single node required as a middle connection for two end points.

If I thought LoRa was the only route forward I would start to focus on point to multi point where the “repeaters” are actually multi radio relays and use the 1 watt devices as directly attached to directional antennas as possible.

Village Alpha transmits to mountain node on freqA and mountain node talks to village Beta on freqB. Add additional radios and frequency use for other villages in range of the mountain node. The mountain noad can then act as a router, possibly running it’s own scuttlebutt instance.

Running a repeat mesh cuts that bandwidth in half with just a single node required as a middle connection for two end points

We’ve learned that with traditional WiFi mesh, that’s why the LibreRouter has 3 radios, 2 MiMo 5ghz for mesh and one 2.4Ghz for access point. But it’s very expensive because it’s so good.

If I thought LoRa was the only route forward I would start to focus on point to multi point where the “repeaters” are actually multi radio relays and use the 1 watt devices as directly attached to directional antennas as possible.

That sounds ideal. But we want to let the network grow as small as possible. When it reaches a certain bandwidth trashhold it should combine user efforts to evolve. That evolution could be in many ways: LoRa, HF, Sattelite, WiFi, TVWS, who knows, maybe even sound or smoke.

I think what we’re trying to achieve is the ballance between application usability and easy-to-setup and cheap devices. Ideally we should be able to start with nodes using one 0.1W radio and see how far can we push the mesh with a really simple social network. It obviously depends a lot on how well setup the nodes are. I’d like to better understand all these LoRa concepts to be able to calculate realistic bandwidth between X numbers of nodes.

Here’s a realistic use case:

Use case

  • LB (local offline SSB user)
  • LM (local SSB user)
  • LP (local SSB pub)
  • IB (Internet SSB user)
  • IP (Internet SSB pub)

LB sends a public identification message to the LoRa mesh

LB --> name, full_pub_key --> SSB (about) --> compression --> LoRa mesh -->

All nodes eventually receive the identification message. Clients pull the message from the LoRa node and reconstructs the about message (using trusted or already known secrets, such as Meshtastic channel), adds it to their db, and replicates it further using any means available (Bluetooth, Intranet, Internet).

LB sends a private message to IB

Since we don’t know where the message receiver is at any given time, LB sends a private message to the whole LoRa mesh:

LB ----> boxed_msg ---> SSB ----> compression --> LoRa mesh ----> allNodes

       LB-11, LB-12 <--- SSB <------ decompression <----- LoRa node1 <-----

        LB-21 <--- SSB <------ decompression <----- LoRa node2 <-----

(Internet) <--- LM-31 <-- LP-31 <-- SSB <-- decompression <-- LoRa node3 <--
   |
   |
   |---> IP ---> IB ---> unbox

All LoRa nodes eventually receives the message and stores it.

LP-31 pulls messages from node3 decompresses the post message, adds to it’s ssb-db, and replicates it.

LM-31 replicates with LP-31 through the local network.

LM-31 goes to town and replicates with IP

IB connects to IP and eventually receives the private message.

The big issue I see is as soon as the use of the link approaches full capacity it will take longer and longer for the data to propagate and sync. People may start trying to resend messages and make the situation worse.

With protocol overhead a 1kbs link gets streached real thin for multiple users. If it is ok for the majority of messages to show up the next day that may work for you. But at that point forget about using it for emergency coms. Emergency messages may get through if using meshtastic sub channeles and the different payloads are using the appropriate priority settings.

In a mesh, using the long range options it could take over a minute for a 1k payload to propagate. This is actually really hard to pin down, because you can’t ensure the transfer is uninterrupted, doesn’t require repeat of packets that didn’t make it, or other devices start sending higher priority payloads. And probably a lot more I haven’t thought of.

Another way of thinking about this is how many messages a day should long range support to be worth the cost and effort. Is it worth it if the nodes have to be a fixed location due to the long transmission times?

1 Like

how many messages a day should long range support to be worth the cost and effort.

Exactly! Is there a way to estimate that? 100, 500, 1000 messages a day? I really think there won’t be that much activity in this 140 character written message social network. But might be enough to organize events and stay in touch with the outside.

When we introduce voice messaging, that’ll probably take a whole new design.

Is it worth it if the nodes have to be a fixed location due to the long transmission times?

For our use case yes. There might not even be mobile nodes.

Using this as a baseline

http://www.techplayon.com/wp-content/uploads/2017/12/Lora1.png

Without getting into overhead, using the longest range setting and having a theoretical 293 bps a 1kb payload would take around 6 seconds to transmit.

Double that for overhead (12 seconds), double again for a single repeat for a simple three node mesh with 1 hop (retransmit) limit and we get 24 seconds.

The tricky part in estimating is what is our retransmission rate (for packets that didn’t get an ack). Let’s assume 50%, so 1.5 x 24 seconds gives us 36 seconds.

24 hours x 60 min gives us 1440 min in a day x 60 seconds is 86,400 a day.

Lets hedge our bets and say the mesh is busy with other stuff half the time leaving us 43,200 seconds a day.

43,200 / 36 = 1,200 1kb payloads a day.

Could be better, I think I’ve been fairly conservative. Would be great if others can update that basic math and reasoning with additional factors.

Keep in mind that that would be an even distribution of payloads throughout the day. You may want to adjust those numbers down to half a day for practical reasons since few messages would be sent at night.

1 Like

Thank you very much @Spor7biker :pray: That’s actually much better then I was expecting.

Here’s a bit of math a very active member of SSB posted:

The last 500 messages, that I got last night, at 390000 bytes equates to around 780 bytes per message average.

For our case, 500 messages is a lot, and they won’t be that large. We’re thinking on a way to compress-decompress messages to make them smaller. The mesh we’re imagining is fairly large, let’s say 8-12 nodes, which could also enable us to use faster settings.

But I think it might just be possible.

Hopefully a larger mesh with faster transmission times will help. But it could easily make the numbers worse.

The current, rather simple flood method meshtastic uses could be problematic.

I’ve been a long time lurker on these forums, but haven’t been quite sure if meshtastic fits my use-case.

I’m primarily interested in a system with fixed solar hubs for radio communication between family and friends if the internet and phone systems go down. I live in the US pacific northwest where we just experienced terrible fires this summer(that burnt down some family members homes) and an ice-storm this winter that knocked power out for several days. We have several groups of family and friends that all prep together and we’d like to stay connected in the case of another disaster. I would also like to be able to improve perimeter security with extremely low power LoRa connected motion sensors. I’ve been really enjoying your posts because your use-case in Brazil is extremely interesting and aligns very closely with ours.

I bought 3 T-beams to get more familiar with the project, but haven’t gone much further than range testing and firing up platform IO / compiling the code because I do not care for the ESP32 and don’t want to sink development time into learning it’s architecture. I do really like the nRF52 devices though and have a little bit of limited experience with them through work. I’ll be ordering the Wisblock starter kit soon.

At this point, I’ve spent most of my time focusing on hardware, as that is what I’m most comfortable with. My prototype pcbs and parts for a RAK4270 based motion sensor have just arrived, but now I’m thinking I might wait for the arrival of the footprint compatible RAK3172 before putting in programming time. I’ll also be ordering more 5.8dBi antennas from rak wireless after comparing them against a number of antennas off digikey and finding them to have the best SWR. The motion sensor project uses a pcb antenna to enable me to use a off the shelf waterproof enclosure, but the external RAK antenna will work perfect for the receiving solar hub. The RAK3172 uses an ST processor and will not run meshtastic firmware itself, but if I do go with meshtastic for the communication network, I’d like to write firmware that’s compatible enough to send a status message.

I had never heard of SSB, but had been planning a public and synced ledger that everybody could read but not necessarily interpret without the correct keys. SSB not only covers and implements that, but has all kinds of other features I had never even considered. After reading this post and being introduced to SSB, I’ve installed many-verse and am loving it! I’m honestly so impressed with the SSB project.

I think an increasing number of meshtastic users are coming up with use-cases that don’t fit the hiker or paragliding model, I just recently saw a poll on this forum and over 65% of people checked disaster communication and connecting communities.
httpss://meshtastic.discourse.group/t/use-case-popularity/3210

I would love it if the SSB model could be used to merge the different use cases, with hikers and paragliders as regular users and repeaters acting similar to SSB’s Pubs.

5 Likes

After 2 weeks, the RAK3172 still says it’ll be available in 2 weeks… I decided to put my motion sensor board together and get testing with the RAK4270. The low iq TPS63900 switching converter, the low current motion sensor circuit, and the debug interface all seem to work! I’m going to port Semtech’s SX1262 demo code and then start looking into what needs to be done to interface with the meshtastic protocol. I also need to get the power consumption down, my goal is to get the battery life to 5 years on the CR123 with infrequent motion events. I measured the SWR of the antenna and it’s about 2.5, but I’ll hopefully get it lower when my rf capacitor and inductor kits arrive and I can do some impedance matching.


I did a little further reading into SSB. It looks like messages are append only and cannot be skipped, but pictures and files are treated differently as blobs? Maybe with compression of
the jsonn before transport, the original SSB could be made to work with LoRa if it ignores blobs and only handles messages? At least for higher data-rate links?

3 Likes

There are projects that allow out of order message processing for SSB. They where mentioned above.

Yes, but it seems the append only structure and it’s benefits might be able to be maintained without modification and without the potential trade-offs of ssb-partial-replication.

I also like the idea of doing the least amount of modifications possible. It would be awesome if compatibility with existing default SSB networks could be preserved and JSON could just be compressed/decompressed at the LoRa transport level. I haven’t dug into it yet, but many-verse supports Bluetooth syncing, potentially a good reference for extending to LoRa syncing.

Latency might be slightly higher, but I also like the way SSB accomplishes something close to mesh networking but through p2p. If my understanding is correct, I believe default SSB would route a message through multiple repeaters as they sync so long as the entire network were all at least friends of friends.

1 Like

Using long range settings bandwidth is very limited compared to Bluetooth and latency significantly higher, especially when multiple hops are required.

Yes, I’m aware. Semtec has a nice application for calculating LoRa data rates and estimated link budgets for both their SX126X devices as well as the 2.4GHz SX1280 at different settings.

Latency wouldn’t be improved, but total network data rates could be improved by having multiple nodes syncing p2p at the same time but on either separate channels or spreading factors.

It would also be interesting to explore a 1 to many syncing system where any nodes in range syncs when a message is broadcasted rather than requiring a p2p sync between each node.

At BW 500 and SF7, 20kbps is possible with an estimated 139dB link budget, which I think would be more than enough for many use cases, especially with good antennas. Duty cycle and bandwidth limitations could complicate things depending on the local regulations, but I think it’s still well worth pursuing SSB over LoRa.

The RAK3172, based on the STMWL LoRa enabled processor is finally available! They lowered the price to $6 too!

2 Likes

but I think it’s still well worth pursuing SSB over LoRa.

Have you considered trying just gzip compressing the SSB json payloads and using the python API sendData to send it as an array of bytes?

gzip should smush the json down at least 75% - so the 780byte per message average mentioned upthread might only need 200bytes or less over the mesh. With the non-real-time assumption behind SSB even a slow/longrange Meshtastic mesh might get 4 ot 5 time Spor7biker’s 1,200 messages per day estimate for 1kb messages?

Would seem to be a fairly simple thing to do with some small low powered (think RaspberryPi style) linux device running some python and hanging off a t-beam’s usb port (full disclosure, I haven’t actually tried doing anything like than yet…)

3 Likes