Options for dealing with badly placed and configured nodes on a large mesh

[mikecarper]

Platform

Cross-Platform

Description

Add another bit to the hop count only with the faster transmit times of the quickest 2 short presets. 10x+ quicker transmissions in comparison to LongFast so the extra hops should allow for better coverage with badly placed and configured nodes.

[caveman99]

we don't have another bit we can use.

[Talie5in]

@caveman what about extending into hex? 0-9A-F
That should give "15" in the same space.

[mikecarper]

That's why it was a 3.x request

[caveman99]

@caveman what about extending into hex? 0-9A-F That should give "15" in the same space.

0-7 is 3 bits. 0-F is 4 bits, no amout of changing between decimal and hexadecimal will change that.

[caveman99]

That's why it was a 3.x request

the 6 and 7 hop counts are a waste already. 15 will not change a thing in that. I don't see this even on 3.0. We have looked at this extensively with Meshtaticator.

[mikecarper]

@caveman99 in the Seattle area we repeatability get contacts at 6-7 hops out

[Dokument]

@mikecarper , if you have time could you give some examples on a map or some prose talking about the geography between the longest reliable hops? Are there multiple "clusters" with limited (but reliable) links between them, by any chance?

I'm thinking about maybe, rather than increasing the general hop limit, whether the concept of "regions" might be useful.

Can you say more about "Regions", or point to any documentation. This is the first I am hearing about it for Meshtastic. I can speak more to the topology of the Puget Sound and the challenges we are working with, but I want to understand regions first.

[fifieldt]

Purely theoretical at the moment and only in my head. Just an idea from the old days of community wireless networks. For large meshes, separate the mesh into "interior" and "exterior" routing. "Interior" routing works just like right now. "Exterior" routing would be used to join disparate clusters of "interior" routed mesh that are separated by geography. I'll have to get out a diagram and more words than I can type on a cellphone to explain properly. Haven't even thought through the concept fully yet :)

[Dokument]

I'd like to hear more when you do get a chance to capture your thoughts.

I'm not sure how that would look in the Seattle/Puget Sound region. We use a private broker server which shows the most nodes in our area (of course, only those that provide locations). https://mqtt.davekeogh.com/

We have a lot of shifting terrain with many north-south running hills and mountains. So it is much easier to get messages north and south, but harder to go east and west. This is of course very general, there is much more nuance to it.

[Dokument]

I was really hoping to use neighbor info to generate data that could be used for simulation, unfortunately this option was removed which broke this possibility. I understand I can still collect neighbor info other ways, but that won't yield the data I would like to have.

[fifieldt]

Thanks for your response! Interesting that you're hailing from Seattle. Going back 20 years, the community wireless network there had these "regions": WestSeattle, CapitolHill North, CapitolHill South, Belltown, Ballard, NorthSeattle, Magnolia, University, SouthSeattle, IntlDist, BeaconHill, Plateau, QueenAnne, Tacoma, Everett, Bothell, Auburn, SnowNet, Anacortes, Federal Way, Bellingham, Shoreline, Renton, Somerset, NorthBend .

So, the theory is that within each region you have a regular mesh, then there's some "magic" to get between regions. Regions in a lora network are probably larger than the old 2.4G wireless days :)

[mikecarper]

Here's some examples on long fast with mqtt exit points.

min terms of a map mesh sense is running on a couple other nodes

[GUVWAF]

It sounds like your mesh could benefit from a (couple) strategically placed ROUTER/REPEATER node(s). Even when it occasionally takes 7 hops to reach a node, it doesn't mean it needed it. The protocol tries its best to use the least amount of hops necessary, but it's certainly not guaranteed this happens. Also PR #5534 might help here; sometimes, especially if the mesh is busy, packets with a lower hop limit arrive earlier due to queuing delays. Currently even if the same packet with higher hop limit arrives afterwards, the one with lower hop will be rebroadcasted.

15 hops might be something to consider for v3.x, but I doubt it will eventually lead to a better experience. More hops will increase channel utilization, reduce reliability and increase delays. If there's a way around it by proper placement of nodes, that would certainly be better.

[Dokument]

It sounds like your mesh could benefit from a (couple) strategically placed ROUTER/REPEATER node(s). Even when it occasionally takes 7 hops to reach a node, it doesn't mean it needed it.

There are currently about 80 ROUTER mode nodes in the region (Western Washington essentially). Unfortunately there aren't many levers for us to manage this as a community. Meshtastic nodes are run by anyone and everyone, not everyone is reachable, and there is really no way for us as a community to deal with the large number of ROUTER mode nodes in our area other than by adding more ROUTER mode nodes in better positions to improve propagation. More levers here would be really helpful.

Given the common growth challenges of Meshtastic, I suspect this situation is encountered by other groups as well.

[GUVWAF]

I see, didn’t know that, then my advice should actually be the opposite: turn ROUTERs that are eating up hops into CLIENTs, but I understand that’s hard to realize. To be clear: you’re currently still using the default LongFast? Have you considered going to a faster preset with a group of dedicated people that’s easier to manage?

I think we have to be realistic and accept that there will be limits to what a LoRa mesh can achieve and that may mean you have to be closer to the source to be able to receive their packets. Sure, if you increase the hop limit from 5 to 7 and let MeshSense perform a scattershot of traceroutes, you will get responses from some nodes you previously couldn’t. However, that’s survival bias. It doesn’t mean the overall performance of the mesh has increased. Maybe other packets are now failing because of this; it’s hard to evaluate in real-life.

However, maybe indeed the hop limit can be higher for ShortFast/ShortTurbo – I simply don’t know. If you want, you could experiment with this now by modifying the firmware on some nodes to not decrement the hop limit, for example.

That said, in 2.6 we'll use next-hop routing for direct messages. Currently this still adheres to the hop limit, but after a while -when the majority of nodes have updated- I think we can consider skipping the hop limit check for direct messages as the additional overhead would be limited.

[Dokument]

Have you considered going to a faster preset with a group of dedicated people that’s easier to manage?
...
I think we have to be realistic and accept that there will be limits to what a LoRa mesh can achieve and that may mean you have to be closer to the source to be able to receive their packets. Sure, if you increase the hop limit from 5 to 7 and let MeshSense perform a scattershot of traceroutes, you will get responses from some nodes you previously couldn’t. However, that’s survival bias. It doesn’t mean the overall performance of the mesh has increased. Maybe other packets are now failing because of this; it’s hard to evaluate in real-life.

However, maybe indeed the hop limit can be higher for ShortFast/ShortTurbo – I simply don’t know. If you want, you could experiment with this now by modifying the firmware on some nodes to not decrement the hop limit, for example.

Absolutely agree. Ideally everyone uses the lowest hop limit to achieve their contact. I do like your idea of different limits for different modem presents. That seems like a happy medium. Overall, I do think consistency (with a minimum bar of functionality) is the higher priority.

Regarding changing modem_presets. Absolutely, we are currently building out 3 LoRa meshes with the current focus being on Long_fast due to it being the default. It feels like kicking the can to move to another slot or preset as I expect these same issues to pop up there too.

[garthvh]

The puget mesh there are two many routers so we must add more routers philosophy has led to a big increase in channel utilization in the area. Most of their routers should be client.

[Dokument]

See, when you shutdown a response to a question you ask, it lets me know that you are not interested in the answer.

Have a nice day.

[mikecarper]

What data is showing that a router makes any sense in an urban environment with many nearby nodes? This is just not the case, adding more routers is not helping.

Stop using router in the city, period, how can I be more clear?

Can you make a ignore all but these repeaters mode? We have some "red team" nodes out there.

[garthvh]

See, when you shutdown a response to a question you ask, it lets me know that you are not interested in the answer.

Have a nice day.

You have taken zero of the advice you have been given, you are hurting the mesh.

[powersjcb]

Fork the mesh to fix regions problems.

The technical fix here is probably not about hop limits, but some clever dynamic routing. (And it won't matter anyways since your new routing system won't support old firmware - you'd be stuck forking anyways)

SLC switched to LF/51 5 months ago and we have rock solid links between our mountain routers since then.

If we could get better noise filtering on our routers, there's some certainty we can also probably move to MF or SF. (Our summit routers have trouble RXing)

Side node, I support the sentiment of longer hop limits. Tho I want it to be able to setup super long range weather station links spanning 300 miles and 5 mountains. (While also extending the local mesh)

[mikecarper]

SLC has it fairly easy terrain wise if your trying to talk to the main population centers North/South. Just a few hills to deal with unless you're trying to go East/West. I would need 6+ hops with the best placed nodes to go 30 miles due to the terrain in Seattle. This would be all within 2 miles of major 4 lane highways so not out in the sticks.

Moving off LongFast and getting more hops is what we need.

[mikecarper]

Being able to flag rebroadcasts from badly configured nodes might be another solution. Make the ignore list a CLI only option so it's not easy to use. If there's a red team node in the basement set to router/repeater, other nodes could not decrease the hop count and then do it's own rebroadcast of the message even if the message was only received by the red teams node.

[mikecarper]

Could we add more router options? We have router and router_late; in theory that helps with a every other type of long hop. could we have router_late_1, router_late_2, and router_late_3 instead of router_late? Things that could wait several minutes to rebroadcast if the net is busy and not drop key data packets. Top priority would be text and position packets; the rest could be dropped in router_late_2 and router_late_3 if channel usage is too high. Tough terrain means we have a lot of picket fence situations with some nodes.

Another mode called rooftop that will always repeat zero hop client_mute neighbors is another idea.