Trino on Ice: Using Iceberg To Replace the Hive Table Format

Brian Olsen - Percona Live 2021 - Altinity Community Track

Trino (formerly PrestoSQL) is a ludicrously fast query engine that evolved from the need to replace the slow query turnaround speeds of the Hive engine. Trino grew in popularity under the label of Presto for years as an interactive query engine that lives over your data lake. While this operation was certainly a step away from the initial big data days of waiting hours to days for queries to complete, there were still many tedious rules engineers had to follow in order to correctly create, manage, and use their data in the datalake due to the Hive table format.

Apache Iceberg, a table format created at Netix, aims to address many of these issues. Iceberg simplifies the life of the engineer by decoupling the logical view of the data from the physical layout of the data using techniques like hidden partitioning and allows for in-place schema migration of your tables. Iceberg also increases the speed at which you can query your system by tracking files at the le level versus the partition level and so much more. 2 of 3 Speakers

With the marrying of Trino and Iceberg, companies can take advantage of a full replacement of the big data days of old and move into the next generation of datalakes that simplify the mental load of their data engineers and focus on building out the business logic and other tasks. In this talk I will cover some of the examples of the issues Iceberg solves from the lens of Trino. ∎

Video

Link: https://youtu.be/5-Q74rCX2Z8