Adaptive Tunnel MTU, Another Self-Inflicted Outage, and a Real ISO Builder
A long push on mlvpn-rs — support-ticket diagnostics, and finally fixing a tunnel-speed bug that's been quietly capping throughput since the feature was first built — turned up a real production data-loss bug along the way in a completely separate service. Then a second self-inflicted outage, this time on my own dev box instead of a customer's. Smaller: a real GitLab CI runner now exists to build installable OmniBridge ISOs.
mlvpn-rs gets support-ticket diagnostics
mlvpn-rs — the open-source
VPN-bonding daemon that combines multiple internet connections into one faster, more reliable tunnel — already
had solid local diagnostics: a command that dumps link health and recent logs to a text file, plus an
automatic version that fires on its own the moment a link's loss crosses a threshold. What it didn't have was
any way to turn that into an actual support ticket. Wired it into the same jpps/software-support
service OmniBridge started using last week: a new --submit flag on the manual dump command, an
opt-in setting so the automatic dump can file a ticket on its own too, and a real crash hook for the daemon
itself. That last one took two real attempts to get right — the first version spawned a background thread to
send the report and let the process exit without waiting, so the report almost never actually left the
machine before the daemon was already gone. Caught only because it got actually tested rather than just
compiled: real panics against a real local server, watched requests arrive (or not). Fixed by making the
crash hook wait — briefly, with its own timeout — for the report to actually send before letting the process
finish dying.
Finally fixing a tunnel-speed bug that's been there since day one
The bigger piece: mlvpn-rs has always defaulted its tunnel size (MTU) to a conservative 1400 bytes, and — this is the part that turned out to actually matter — never raised it back up even when the real connections underneath could carry more. My own WireGuard links run fine at the standard 1500, and that artificial 1400 cap was quietly costing real throughput the whole time, for no reason. Now, leaving the tunnel size unset lets it pick the real number from what the underlying links can actually carry, adjusting automatically — including upward — and it keeps re-checking every 30 seconds instead of only at startup, so a link that changes hardware mid-session gets picked up live too.
That "keeps re-checking" part needed a real fix along the way: the very first version sized its packet buffer once, based on whatever the tunnel size happened to be at startup — so a live increase later on would have handed the daemon packets bigger than the buffer it had to put them in. Caught in review before it ever shipped, not after. A later, harder-to-see one: the code path that actually resizes the live tunnel needs a specific Linux permission the daemon normally gives up on purpose right after startup, for security. Fixed by keeping just that one narrow permission instead of dropping everything — verified for real, not just by reading the code, by actually watching the permission survive the drop and successfully resize a real interface afterward.
A real production data-loss bug, caught in a completely different repo
Widening the diagnostics feature to accept a new report type required one small change on the
software-support side too — except making that change safely meant fixing a database constraint
that turned out to be more fragile than it looked. The fix that went out first did the standard database trick
for changing that kind of constraint (rebuild the table, copy the rows over, swap it in) — and that rebuild
silently deleted every existing support-ticket conversation thread the moment it touched the live database,
because of how a related table's own delete rule was configured. Real data, gone, on a service that's actually
in use. Caught immediately by actually checking the live numbers on the server rather than trusting a green
test suite, fixed properly with a narrower, safer version of the same change, and confirmed for real this time
by intentionally reproducing the failure against a copy first, then proving the fix stops it. No backup existed
to restore from — a real, permanent loss, and a genuinely useful reminder to actually think through what a
"routine" database migration touches before running it against something live.
A second self-inflicted outage — this time on my own dev box
While testing that tunnel-resize fix directly (deliberately setting a loopback interface's speed to confirm the low-level code actually worked), a bug in the test itself briefly set that speed to zero instead of restoring it — caught and fixed within the test in seconds, but not before it knocked out something I didn't expect: the local browser-based terminal I use to reach this dev environment, which happened to be routing its own internal connection over that same loopback address. Ironic, given the last outage-I-caused story was about not touching live production carelessly — this one was on my own machine, and still managed to lock me out of it, forcing a fallback to plain SSH until it got sorted. Root cause tracked down and fixed directly (the loopback addresses had actually disappeared, not just the interface speed), then the connection came back clean.
Smaller things
Set up a real dedicated build server for a long-scoped-but-never-started idea: an installable OmniBridge ISO, so someone can hand a USB stick to a fresh machine and get a working appliance instead of just a package install. New Proxmox container, a real permission needed setting up that only root on the Proxmox host itself could grant, and an open question from the original design doc — whether the disk-image-building tool would even work without full root access — settled for real by actually running a full build end to end and getting a real, working ISO out the other side.