Fix queue sizes for Heltec_v2 and LilyGo_TLora_V2_1_1_6 (fixes issues with #1334)

[olanwe]

This addresses #1334: Building the firmware for Heltec v2 and LilyGo T-Lora V2.1_1.6 fails due to insufficient RAM. The WiFi companion appears to require more RAM than the BLE version (which also uses a queue length of 256).
The queue size was reduced to 64 for these two devices (128 would also work, but leaves less free memory).
Sorry for the inconvenience caused by this.

[olanwe]

Small ping on this one - WiFi builds for Heltec v2 and LilyGo T-Lora V2.1_1.6 are still failing on current dev.

[MikesAllotment]

I don't think the is the correct way to fix this issue. The Heltec V2 works perfectly well with Queue Size 256. The way to fix it is to reduce MAX_CONTACTS. At the moment MAX_CONTACTS is currently set to 160 in platformio.ini ... this is way too many. If you reduce MAX_CONTACTS to 100 then you can keep QueueSize as 256.

[olanwe]

I think so — I introduced this issue in #1334. 🫣

Originally, the queue size was set to 16. I increased it to 256 (like the other companions), which has caused this issue.

The question is: what should have higher priority — storing more messages offline or supporting more contacts?

350 contacts is the standard for other companions, so 160 contacts is already relatively low. (My own contact list is already full with 350 rotating companions.)

[MikesAllotment]

I've been using Heltec V2 for over a year - ever since the first release of MC ... queue size has always been 256 in every version I've used. I've noticed that the builds failed recently - but that's because someone increased MAX_CONTACTS to 160. Reducing MAX_CONTACTS back down to a sensible value fixes the problem.

In reality, people shouldn't need 100s of contacts in their Companion... people disable AutoAdd and use the Discover List to decide which Contacts to add to the Companion's Contact list. AndyK posted a video a while back suggesting this is a better way to manage Contacts - use the Discovered List (which is stored in the App, not the Companion) and only add Contacts to the Companion that you actually interact with on a daily basis.

Companion WiFi uses more SRAM as it stores the WiFi credentials - but again - dropping the MAX_CONTACTS back down to a sensible value fixes the issue.

I would much rather have a larger offline message queue, than wasting storage space for contacts that aren't needed.

[olanwe]

ever since the first release of MC ... queue size has always been 256 in every version I've used.

For the WiFi build, this was definitely not the case. Liam also mentioned that the WiFi companions were missed when the queue size was increased (see comment in #1331).

If you look at my changes in #1334, you’ll see that I just added OFFLINE_QUEUE_SIZE=256 to the WiFi companion section. Immediately after that change, the WiFi build started to fail.
I don’t own a v2 myself, but since I maintain the WiFi builds for the patcher, I noticed right away that this introduced a build issue.

I don’t have a strong preference for a specific solution, but it would be good to get this resolved at some point. For now, I’m applying this PR manually each time I build a new batch of firmware files.

[MikesAllotment]

You are correct re WiFi build - that was added fairly recently - but the OFFLINE_QUEUE_SIZE for Companion BLE has always been 256. As I say - the problem is we've always been sailing pretty close to the wind with memory size in the V2, and MAX_CONTACTS has always been set lower than other variants precisely for this reason - there simply isn't enough memory available in the V2 for the same number of Contacts as some of the other variants.

I noticed that the WiFi build failed for the V2 and suggested in Discord to Scott & Recof that we should reduce the MAX_CONTACTS for the WiFi variant to allow for the extra space required for the WiFi credentials.

I will always take offline message storage over Contacts any day of the week - especially when Contacts can be more effectively managed using the Discovered List.

Reducing offline queue size to 64 is a bad idea i.m.h.o @recrof @ripplebiz

[olanwe]

especially when Contacts can be more effectively managed using the Discovered List.

I agree - we could probably even omit repeaters from the contact list entirely if the tools (especially path and trace) also considered repeaters from the discovery list. But that’s a separate topic.

[MikesAllotment]

FYI I'm using a Heltec V2 WiFi Companion with offline queue size 256 and max contacts 100 - this combo fits in the available space.

Also using Heltec V2 Companion BLE with offline queue size 256 and max contacts 140. So there's definitely no need to reduce the offline queue size as this can be managed through setting the MAX_CONTACTS correctly for the different flavours.

[weebl2000]

@MikesAllotment @olanwe I would love to see if you can flash this PR #1601 and see how much it cleans up your storage.

You can build it here: https://mcimages.weebl.me/?commitId=fix/onetime-blob-cleanup

[liamcottle]

Is this still required? Looks like the PR only changes the WiFi variant.

[MikesAllotment]

Yes - it's required - only the WiFi build was broken due to the additional space required to store the WiFi credentials.
It's a balance between offline queue size and max contacts... Queue size 256 and max contacts 100 fits in the limited space available.

[MikesAllotment]

@MikesAllotment @olanwe I would love to see if you can flash this PR #1601 and see how much it cleans up your storage.

You can build it here: https://mcimages.weebl.me/?commitId=fix/onetime-blob-cleanup

I just tried building this for the Heltec V2 Companion Wifi and got page after page of errors - but basically ": DRAM segment data does not fit." ... so the problem remains with the MAX_CONTACTS being too high for the WiFi variant.

@liamcottle - yes the MAX_CONTACTS needs to be reduced for the WiFi variant specifically.

Specifically OFFLINE_QUEUE_SIZE=256 and MAX_CONTACTS=100 for the V2 WiFi Companion only

[MikesAllotment]

@MikesAllotment @olanwe I would love to see if you can flash this PR #1601 and see how much it cleans up your storage.

You can build it here: https://mcimages.weebl.me/?commitId=fix/onetime-blob-cleanup

Also @weebl2000 - using your tool to try to build the WiFi variant - it obviously doesn't know my WiFi credentials which need to be set in the Platformio.ini as part of the build - so unable to test whether this works unless I have access to the source pls.

[liamcottle]

Yes - it's required - only the WiFi build was broken due to the additional space required to store the WiFi credentials. It's a balance between offline queue size and max contacts... Queue size 256 and max contacts 100 fits in the limited space available.

@MikesAllotment Ok, looks like this PR doesn't do that.

It adjusts OFFLINE_QUEUE_SIZE from 256 to 64, and before the PR linked in previous comments, it was 16 for the WiFi variant.

Since you're not using the WiFi companion, I guess it could be merged as-is, as the downgrade in queue size wouldn't affect you, and no one else would have had the queue size upgrade on the WiFi variant anyway, since it never compiled.

I don't know if it's safe to decrease the MAX_CONTACTS from 160 to 100, as those slots may be in use already by other users.

Can merge if you're fine with this.

[MikesAllotment]

I AM using the WiFi companion - that's the whole point. For the WiFi companion specifically the OFFLINE_QUEUESIZE can be 256 with MAX_CONTACTS 100.

For the BLE variant the MAX_CONTACTS can stay at 160. I'm pretty sure no-one else uses the Heltec V2 WiFi Companion apart from me anyway ;-) LOL

[liamcottle]

Right, so this PR is not required then. I'll close this and make the adjustments you have requested instead.

[liamcottle]

@MikesAllotment adjusted to MAX_CONTACTS=100 in 7d8d66c

[olanwe]

@MikesAllotment @olanwe I would love to see if you can flash this PR #1601 and see how much it cleans up your storage.
You can build it here: https://mcimages.weebl.me/?commitId=fix/onetime-blob-cleanup

Also @weebl2000 - using your tool to try to build the WiFi variant - it obviously doesn't know my WiFi credentials which need to be set in the Platformio.ini as part of the build - so unable to test whether this works unless I have access to the source pls.

@weebl2000 – If you replace the default WiFi credentials with "myssid_patchable" and "mypwd_patchable" (and extend both fields to the maximum allowed length by padding the remaining space), it would be possible to use my WiFi Companion Patcher to inject custom credentials and flash your firmware to a WiFi companion.