Code Refactoring

Development
1

Hi everyone,
I have been extensively discussing with Meshtastic author @geeksville about something I really believe should be done now that the project is in an embryonal phase: code refactoring.

Considering the level of participation and ideas, a modular structure would be best suited, with an abstraction layer to work on in order to support multiple embedded and not embedded platforms (posix systems, RPI).

I am taking inspiration from the great ArduPilot software I contributed to in the past, you can see my current development status on this GitHub repo.

Such huge effort will surely pay off as new developers will be free to work with easy abstractions without having to handle hardware-specific problems like the implementation of the micros() function on the ESP or on the nRF.

If anyone wants to help out, please reply to this thread.

5 Likes
2

Brief TODO:

  • abstract threading and semaphores for both multicore (ESP32) and non-multicore (nRF) platforms
  • complete FS, I2C, SPI and GPIO interfaces for the aforementioned board families
  • get rid of Arduino.h, Wire.h and other AVR-bound libraries and port them to HAL
2 Likes
3

I know its reliant on the Arduino core but might be worth taking some inspiration from the abstraction Adafruit has done https://github.com/adafruit/Adafruit_BusIO

4

Back here with some interesting findings. Talking over reddit to FreeRTOS devs, it comes out that such OS is perfectly capable of supporting our needs supporting all the boards we plan to use as well as (experimental) Posix.
Despite newer than FreeRTOS, the Zephyrproject is worth mentioning as an industry-backed (Nordic semiconductors and Intel) RTOS focused on reliability and security.
I’d advocate for Zephyr as it is much more stable, backed by the producers of the boards we are going to use and supports deep sleep, BLE and LCD (which FreeRTOS doesn’t).

GPS and LoRa boards still require custom implementation.

1 Like
5

I support the idea of improving, or refactoring a code base to position Meshtastic for the future. I also want to keep focus on the MVP that current users are able to use with this product. An experimental FORK of the codebase that begins a migration to FreeRTOS or Zephyr certainly could be in motion. This will require some duplication of effort where bug fixes in the existing product are ported to the FORKed codebase. Use of CI/CD for features is also key. I ordered my boards last night so don’t have hardware currently.

6

Currently, some part of Meshtastic code uses FreeRTOS combined with custom hal-sort of for GPS and LoRa and a sprinkle of Arduino.h somewhere.

In the future, Meshtastic will just pick a RTOS and stick to it, extending the main code to include unsupported hardware like GPS or LoRa radio boards. Zephyr seems to have experimental support for GPS and BLE while LoRa should be pretty trivial thanks to the SPI abstraction.

7

In the future, Meshtastic will just pick a RTOS and stick to it, extending the main code to include unsupported hardware like GPS or LoRa radio boards. Zephyr seems to have experimental support for GPS and BLE while LoRa should be pretty trivial thanks to the SPI abstraction.

That’s one possible future. I’m somewhat skeptical :wink:

8

re: dependency on Arduino core

I think over time we will abstract that away (in fact, real soon after 1.0 I want to have it possible to build and run on posix for a variety of reasons which I can elaborate on if useful). Currently we wrap the freertos thread and message queue apis with little wrapper classes. Eventually we will make non vxworks classes that implement the same interfaces.

9

Well that’s my hope, the other possibility is the code becoming a huge mess full of random #ifdef, unportable classes and complicated workarounds to fix platform-specific implementations. Ultimately scaring new developers and hindering collaboration.

I’ve seen many nice projects going down this route, I hope Meshtastic will not.

1 Like
10

yes, but it is quite possible to have a clean app/library/platform that can work with multiple OSes with appropriate abstractions.

I think porting meshtastic to run on only one particular RTOS only would be a mistake, though if you’d like to try it you’re welcome to do so.

11

You’ve seen my progress about abstractions, it’s a lot of work that can be saved by using a ready-made RTOS.

After all, that’s why they exist, to provide portable code for multiple platforms and let developers concentrate on the high-level part.

12

We currently use a readymade OS (FreeRTOS) on our current two supported architectures. The “lot of work” in your current attempt reflects the code needed to work with FreeRTOS.

Moving away from FreeRTOS to Zephyr would bring enormous new work, because virtually none of the numerous (great) libraries we use would be available to you ‘off the shelf’. Because these libraries (including RadioLib) assume arduino as their OS abstraction. If you swap out one RTOS (FreeRTOS) for another you are also saying goodbye to the arduino ecosystem. i.e. you would be signing up to write a bunch of new libraries (possibly including BLE and wifi).

(As an aside, not all “arduino” platforms use FreeRTOS at the bottom, the Cubecell uses something different for instance)

This is part of why when I started I didn’t use a different RTOS (because FreeRTOS is kinda crufty - it is just ubiquitous). I wanted to build from the existing work of others.

Therefore, I think the correct OS abstraction is to provide a small set of adapter classes (for Thread, MessageQueue, and possibly one or two others). Those interfaces (or HAL if you prefer ;-)) will have an FreeRTOS implementation and soonish a Posix implementation (for running on linux like hardware).

RadioLib when going to Linux will most likely be done as a “ArduinoOnLinux” library. That library would need to provide an implementation of the (small) arduino Spi and GPIO APIs.

So that’s my verbose way of saying I think porting to one RTOS is not a great approach. Much better to slightly abstract that OS dependency away. But if you want to try it, more power to you, and I hope it works out great.

13

While I agree on the FreeRTOS choice (Zephyr looks very interesting but the documentation is awful), I’m strongly in favour of getting rid of Arduino dependencies otherwise they’ll come back to haunt us in the future.

We can use FreeRTOS SPI and GPIO API and port RadioLib, it’s just a matter of writing the configuration file and pin mapping.

14

Though FreeRTOS doesn’t provide the SPI and GPIO apis (that comes from the arduino clone api). And it is more than just pin mapping, because particular platforms, when they implement SPI do their own DMA based streaming (which is chipset/OS specific).

That’s part of why I think to eventually have the option of a non arduino-arch at the bottom an “ArduinoOnLinux” library is in order. Which would be mostly posix, but would need to use the linux ioctls for SPI and GPIO control, and some notion of user space ‘interrupt handlers’.