Julien Tournay – Data Processing @Spotify using Scio (45 minutes)

Two years ago, Spotify introduced Scio, an open-source Scala framework to develop data pipelines and deploy them on Google Dataflow. In this talk, we will discuss the evolution of Scio, and share the highlights of running Scio in production for two years. We will showcase several interesting data processing workflows ran at Spotify, what we learned from running them in production, and how we leverage that knowledge to make Scio faster, and safer and easier to use.


Leave a Reply

Your email address will not be published. Required fields are marked *