The art of structuring real-time data streams into actionable insights

Speaker

About this talk

For updates and more, join our community 👉 https://www.linkedin.com/company/devoxx-united-kingdom Real-time data can be messy, unpredictable, and hard to manage. To unlock its full potential, you need a way to turn raw streams into clean, structured data. In this talk, we’ll show you how to use Apache Kafka, Apache Flink, and Apache Iceberg to organize real-time data streams efficiently and prepare them for advanced use cases, including AI applications. We’ll start by explaining how Kafka handles high-speed data streams and how Flink processes these streams in real time. You’ll learn how to use Flink to transform raw data into structured formats, ensuring it’s ready for storage and analysis. Then, we’ll dive into Iceberg, demonstrating how it stores and organizes structured data for easy querying, versioning, and integration with machine learning pipelines. Through clear examples, we’ll walk you through building a practical pipeline that turns chaotic data streams into organized schemas. By the end of the session, you’ll know how to manage real-time data effectively and set the stage for downstream AI and analytics. Whether you’re a beginner or an experienced developer, this talk will give you the tools to simplify and enhance your data pipelines!