Highlights from Flink Forward Berlin 2024!

Hey Data Pros,

Wow, Flink Forward Berlin 2024 was jam-packed with insights and innovation! I was thrilled with my experience at the conference and wanted to share some of the standout announcements and takeaways with you. The energy at the event was incredible, with so many developments for the data streaming community and major news from Ververica!

Here are my key highlights:

1. Bring Your Own Cloud (BYOC):
   Ververica unveiled a powerful new way to deploy Ververica’s Streaming Data Platform, bringing together the security and control of enterprises' cloud infrastructure.
   Key features that stood out to me were:

   • Full control of the enterprises' cloud infrastructure: configure, monitor, and scale, while enabling complete data security with Ververica’s zero-trust architecture.

   • Seamless Integration: You can deploy Ververica’s Streaming Data Platform on your existing AWS accounts, (and shortly also GCP and Azure) and integrate with your existing DevOps, CI/CD pipelines, and other critical tools.

   • More cost-effective: Customers can use their cloud provider's preferential pricing and spend commitments to buy the Ververica Streaming Data Platform via marketplaces like AWS. Ververica’s usage-based pricing means you pay only for what you use, with streaming infrastructure scaling to your business needs.

2. Fluss – Unified Streaming Storage For Next-Generation Data Analytics
   ​The new Fluss project demonstrated a scalable, integrated storage solution for real-time data processing with Flink. I was surprised how much it cuts down on resource costs, providing a purpose-built storage layer that allows organizations to unify real-time and long-term data processing in a single architecture—a much-needed innovation in today’s data-intensive world. By supporting both batch and streaming data, Fluss enables efficient historical and live data processing, making it an ideal solution for AI, ML, and analytics workloads with optimized infrastructure.

3. Flink 2.0 sneak peek
   ​This update was super exciting! We got a preview into Flink 2.0: reducing memory and disk space requirements, while keeping latency low. Materialized Tables (introduced in 1.20 as an MVP feature), get the enhancements needed for production-ready use. Plus, it introduces a unified engine that automatically adapts between batch and stream processing, perfect for evolving workloads.

4. Customer Highlights: Booking.com and Uniper ​
   Both companies took the stage to talk about the impact that Ververica has made to their operations for scaling efficiently. I also got to chat to Sidhartha Choudhury, Senior Product Manager at Booking.com, in more detail about this!

5. Streaming Data Meets AI:
   ​Mike Gualtieri, VP Principal Analyst, Forrester: Mike’s keynote was a showstopper! He spoke on the three emerging AI categories: Predictive AI, GenAI, and Agentic AI. Agentic AI is the “last mile” where real-time and historical data meet, allowing AI agents to react instantly to changes in data. It’s all about empowering companies to harness their data fully. It’s great to see the focus Forrester has on this topic, reinforced in his talk: "Enterprises must invest in streaming data platforms to create a real-time intelligence network".

​Flink Forward was so insightful! I would recommend it for anyone in the data space; from finance industries building fraud detection applications to e-commerce retailers personalizing customer recommendations based on live behavior data. Flink is a powerhouse, and when combined with other tools (like Apache Paimon, making up Fluss!), it’s a recipe for real-time value.

Let me know if you’d like to dive deeper into any of these topics!

Best,
Ravit

Founder & Host of The Ravit Show
​