fix(puffin): implement LZ4 footer compression

[1fanwang]

Which issue does this PR close?

• Closes BUG: failed to write compressed puffin footer #2419

Supersedes #2438, which the stale bot closed after it sat in the review queue. Same change, rebased onto current main, with @xanderbailey's review feedback from that PR already folded in. GitHub refuses to reopen the old PR, hence the new one.

What changes are included in this PR?

The Puffin spec allows a footer payload to be uncompressed or LZ4-compressed. PuffinWriter already exposes a compress_footer flag that records LZ4 as the footer codec, and validate_puffin_compression already accepts Lz4 as a valid Puffin codec — but the codec itself was never implemented, so close() with compress_footer = true failed:

FeatureUnsupported => LZ4 compression is not supported currently

The reader had the symmetric gap: a Puffin file carrying an LZ4-compressed footer could not be read back.

This implements the codec with lz4_flex in the frame format — a single frame with content size present, as the spec requires. lz4_flex was already in the lock file as a transitive parquet dependency, so no new crate enters the tree.

Are these changes tested?

Yes. Unit tests cover the codec and the Puffin writer/reader:

• Codec round-trips (empty, highly compressible, less compressible) across LZ4, Zstd and Gzip, plus an LZ4 frame magic-number assertion.
• The issue reproducer — close() with compress_footer = true — now succeeds, and footer metadata plus blobs round-trip through an LZ4 footer.
• The reader honors the FooterPayloadCompressed flag and decodes an LZ4-compressed footer.

The tests drive the real write/read path, not the codec in isolation: reverting the two Lz4 arms in compression.rs makes all four fail at close() with the error from the issue.

Before the fix — Lz4 arms reverted

running 4 tests
test compression::tests::test_lz4_frame_magic_number ... FAILED
test puffin::metadata::tests::test_lz4_compressed_footer_is_decoded ... FAILED
test puffin::writer::tests::test_compress_footer_lz4_round_trips ... FAILED
test puffin::writer::tests::test_compress_empty_footer_lz4_succeeds ... FAILED

---- puffin::writer::tests::test_compress_empty_footer_lz4_succeeds stdout ----
thread 'puffin::writer::tests::test_compress_empty_footer_lz4_succeeds' panicked at crates/iceberg/src/puffin/writer.rs:327:30:
called `Result::unwrap()` on an `Err` value: FeatureUnsupported => LZ4 compression is not supported currently

test result: FAILED. 0 passed; 4 failed

After the fix — cargo test -p iceberg --lib -- compression:: puffin::

running 47 tests
test compression::tests::test_compression_codec_compress ... ok
test compression::tests::test_lz4_frame_magic_number ... ok
test compression::tests::test_snappy_compression_is_unsupported ... ok
test puffin::metadata::tests::test_lz4_compressed_footer_is_decoded ... ok
test puffin::writer::tests::test_compress_empty_footer_lz4_succeeds ... ok
test puffin::writer::tests::test_compress_footer_lz4_round_trips ... ok
test puffin::writer::tests::test_write_lz4_compressed_metric_data ... ok
test puffin::writer::tests::test_uncompressed_empty_puffin_file_is_bit_identical_to_java_generated_file ... ok
test puffin::writer::tests::test_zstd_compressed_metric_data_is_bit_identical_to_java_generated_file ... ok
...
test result: ok. 47 passed; 0 failed; 0 ignored; 0 measured; 1426 filtered out

cargo clippy -p iceberg --all-targets --all-features -- -D warnings, cargo fmt --check and cargo machete are clean.