​Apache Iceberg V4 Adaptive Metadata Tree

Meetup: Seattle (October 27, 2025) Speaker: Amogh Jahagirdar LinkedIn: https://www.linkedin.com/in/amogh-jahagirdar-aab204b6/ Slides: https://docs.google.com/presentation/d/1_FGPHORBTDL1US8caOvVuLEPsL48_oOP/edit?usp=sharing&ouid=108557621002966898161&rtpof=true&sd=true ​The current Apache Iceberg content metadata tree has a manifest list representing the root of each snapshot pointing to data/delete manifests, which in-turn point to different data/delete files on disk; this provides effective pruning and tracking but introduces latency for small commits. ​This talk introduces the V4 adaptive metadata tree, a new proposed metadata tree structure that enables single-file commits for small writes while maintaining scalable organization for large tables. By embedding column statistics at all levels in the tree, it enables even more efficient pruning, reduces planning overhead, and further optimizes data/delete file planning. The new structure also makes it possible to identify file additions and removals for change detection in each snapshot without comparing to the previous snapshot. Apache Iceberg, Apache, Iceberg, the Iceberg logo, and the Apache feather logo are either registered trademarks or trademarks of the Apache Software Foundation. All other products or name brands are trademarks of their respective holders, including the Apache Software Foundation.