fix: validate freelist metadata in allocate_page to prevent data corruption

[Samuelsills]

Summary

allocate_page reads freelist trunk page metadata without validation, while free_page validates the same fields. This asymmetry allows a corrupt database to cause unrecoverable crashes or irrecoverable data loss during normal INSERT/UPDATE operations.

Bugs Found

1. Freelist leaf count not bounds-checked (crash)

SearchAvailableFreeListLeaf reads number_of_freelist_leaves from the trunk page and passes it to ReuseFreelistLeaf, where it drives a copy_within operation. If the count exceeds page capacity, this panics with an out-of-bounds slice access.

free_page validates this at line 4324: number_of_leaf_pages < max_free_list_entries. allocate_page did not.

2. Freelist page IDs not bounds-checked (data corruption)

SearchAvailableFreeListLeaf reads next_leaf_page_id and next_trunk_page_id from the trunk page without checking them against database_size. An invalid pointer can cause:

• An active B-tree page to be treated as a freelist leaf
• That page is then zeroed (leaf_page.get_contents().as_ptr().fill(0)) — destroying its data irrecoverably

free_page validates page IDs at line 4270. clear_overflow_pages validates at btree.rs line 4492. PRAGMA integrity_check validates at btree.rs line 6138. Only allocate_page was missing validation.

Fix

Three validation checks added in SearchAvailableFreeListLeaf, mirroring existing patterns:

1. Leaf count ≤ (usable_space / LEAF_PTR_SIZE) - 2
2. Next trunk page ID ∈ [2, database_size] or 0
3. Leaf page ID ∈ [2, database_size]

All return LimboError::Corrupt with descriptive messages.

Simulator Improvements

Added SimulatorIO::inject_bytes() for targeted structural corruption injection — enables deterministic testing of specific on-disk metadata corruption, complementing the existing random cosmic ray bit flips.

Two regression tests added to regression_tests.rs:

• test_freelist_corrupt_leaf_count_crashes_allocate_page — proves the leaf count overflow bug
• test_freelist_corrupt_page_id_in_allocate_page — proves the invalid page ID bug

Both tests pass with the fix applied.

Test plan

• Both regression tests pass (cargo test -p turso_whopper test_freelist)
• cargo fmt clean
• No new clippy warnings introduced (existing warnings in turso_core are pre-existing)

[turso-bot]

Please review @pedrocarlo

[github-actions]

Fossier: Manual Review Requested

@samuelsills is a new contributor. A maintainer should review this PR before merging.

EDIT: reviewed 👍

[PThorpe92]

I'm not sure I follow you here:

What exactly are we simulating here? We are supposed to be able to handle injecting any garbage bytes into a whatever offset of the database file?

I understand the bug you are addressing, but this should be simulated in a way that isn't just "write some junk at some offset". It needs to be proven that the invariant can even be reached without you forcing it. If we are just writing junk into the .db file than we have bigger problems right? what if those bytes deleted the page headers or replaced those with garbage?

[Samuelsills]

You're right that arbitrary byte injection isn't a meaningful fault model. I've refactored.

Same fault class, narrower scope. The simulator already injects random bit flips via cosmic_ray_probability. The change adds cosmic_ray_targets — a way for a test to narrow that same random mechanism to specific byte ranges. No new fault primitive, no arbitrary writes. The test computes the byte range using legitimate database operations (reading the freelist trunk pointer from the SQLite header), then asks the existing cosmic ray injector to focus on those bytes.

Reachability proof. With the fix reverted, here's what running the new test produces:

thread 'test_freelist_trunk_leaf_count_unvalidated_in_allocate_page' panicked at
    core/storage/pager.rs:4635:29:
range end index 55296560 out of range for slice of length 4096

That's the unguarded copy_within in ReuseFreelistLeaf consuming a number_of_freelist_leaves value that was bit-flipped on disk by the cosmic ray model. The bytes were corrupted by the same random bit-flip mechanism that fires today — just narrowed to the trunk page leaf count field so the test reaches the bug in tens of iterations instead of billions.

The bug is an inconsistency in Turso's existing defense policy, not a new claim. Turso already defines this exact corruption as something to defend against:

• free_page validates number_of_leaf_pages < max_free_list_entries (pager.rs:4324)
• integrity_check validates page_pointers > max_pointers (btree.rs:6105)
• clear_overflow_pages validates page IDs against database_size (btree.rs:4492)
• free_page validates page IDs against database_size (pager.rs:4270)

Three different code paths defend against these exact corruptions. allocate_page is the only path that reads the same bytes from the same page without performing any of the checks. The fix in pager.rs adds three validations in SearchAvailableFreeListLeaf that mirror those existing patterns — no new validation policy, just consistency with what free_page and integrity_check already do.

The two regression tests (one for the leaf count, one for the page pointers) follow the same shape: build a real freelist, focus cosmic rays on the trunk page header bytes, reopen the database so the corrupted page is read fresh from disk, then INSERT enough rows to force allocate_page into the freelist path. Without the fix the first test panics at line 4635 (above) and the second triggers a downstream PageType assertion in btree code; with the fix both return LimboError::Corrupt cleanly.

Pushed as additive commits on top of the existing branch (no force-push) so the diff against your previous review is clear.

[github-actions]

Fossier: Rejected by Maintainer

Rejected by @pthorpe92: "blatant slop PR comment replies"

[PThorpe92]

/fossier reject "blatant slop PR comment replies"

[PThorpe92]

Please, for the love of god... do not just copy and paste LLM output into PR comments. This is very frustrating to maintainers and your contributions will be unwelcome pretty much anywhere.

[Samuelsills]

Hey mate, I don't want to be inflammatory here but, when a page gets recycled into the freelist as a new trunk, only the first 8 bytes get overwritten. The other 4088 keep whatever was on the page before — old row data, B-tree pointers, whatever. Normally that's fine because the leaf count says "there are 0 leaves here" and nobody reads past the header. But if that count is ever wrong by even 1, the database starts reading those leftover bytes as page numbers. It zeroes those pages and hands them back as free space. If they were still holding live data, that data is gone. No crash, no error, completely silent.

You can check this yourself pretty quickly — the NewTrunk match arm at pager.rs:4361 is just two write_u32_no_offset calls and a break. Nothing touches bytes 8+. So if you had a B-tree interior page that got freed and became a trunk, its old rightmost_ptr is still sitting there at offset 8.

Now say the trunk has 5 real leaves and something corrupts the count to 6. copy_within at line 4666 shifts bytes 12..32 left to 8..28 — but real leaf data only goes up to offset 28, so those last 4 bytes are whatever was on the page before. After four normal allocations chew through the real leaves, that stale data reaches offset 8 and gets read as the next leaf page ID. If it happens to be a valid page number — and old B-tree pointers often are — that page gets zeroed at line 4643 and handed back as free. The zeroing happens before copy_within, so even in the overflow-panic case the damage is already done.

The count corruption itself needs an external fault — bit-rot, hardware error, same thing integrity_check is built to catch. Doesn't have to be exotic either — a database copied over a flaky network, restored from a backup with a bad sector, or just sitting on a disk long enough for a single bit to decay in the wrong 4 bytes. integrity_check would flag it. This path silently destroys data instead. If this is something you dont care about, then fair enough, but the database team building this already added integrity check and hence decided this corruption class matters enough to check for it. I think the fix is zeroing the page in NewTrunk.