NerdOut@Spotify Spotify R&D

Register for Spotify’s roadmap webinar on April 30, 2024 — and see what’s coming next from Spotify for Backstage, the open source platform for building internal developer portals. We’ll show you our latest developer tools, including a sneak peek at new Spotify Plugins for Backstage and a first-look at Spotify Portal for Backstage — a full-featured developer portal that is quick and easy for any engineering org to adopt. See demos from Spotify’s team and learn how to apply for the private beta — work with us to build the next great developer portal: yours!



Host and principal engineer Dave Zolotusky has a quick chat with Helen Greul, head of engineering for Backstage at Spotify, about the event. They talk about the CNCF’s recent BackstageCon in Paris, the growing popularity of the Backstage platform, and why the roadmap webinar on April 30 isn’t one to miss for fans of developer experience and wizardry.




Register for the April 30 roadmap webinar
Learn more about Spotify and Backstage
Listen to Ep. 01: What Is Backstage?
Watch the 2-minute Backstage explainer video



Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng, LinkedIn, and YouTube!

How did we learn to do event delivery at scale at Spotify? It’s been a journey. When you do something like tap the play button in the Spotify app, that’s an event. And getting that event data is fundamental to the Spotify experience. Without it, we wouldn’t be able to make music recommendations, pay artists fairly, or track down pesky, hard-to-find bugs. At the most basic level, this seems like a straightforward process: record an event, send that event data to a server somewhere, do something useful with it. Easy, right? But now, multiply that process by 50 million events per second. So, how do we make sure all that important data is delivered reliably, from our client apps to the cloud? 

Host and principal engineer Dave Zolotusky talks with 9-year Spotify veteran Riccardo Petrocco about our journey building a event delivery system that can reliably handle a trillion events around the world, moving from Kafka to the cloud, building systems that are simple enough so that nobody tries to find a way around them and encourages “doing the right thing”, the definition of “quality data”, the value of moving up the stack and focusing less on the data pipes and more on what’s in them, and how Backstage makes it easier for our developers to discover, consume, produce, and manage data. 

Learn more about Spotify’s data journey:


NerdOut@Spotify, Ep.09: It’s All About the Data
Data stories on the Spotify Engineering Blog

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng, LinkedIn, and YouTube!

We’ve seen generative AI and large language models do some amazing things in the past year — but how do you go from a tech demo to a real shipping product? In this Release Notes episode of the NerdOut@Spotify podcast, we’ll hear about what it took to ship our Voice Translation pilot, which takes podcasts recorded in English and uses AI to generate the original podcaster’s voice speaking in Spanish (with German and French coming next).

Host Dave Zolotusky talks with senior machine learning engineering manager Sandeep Ghael about how we brought expertise from across the company in order to go from a weekend prototype to releasing fully translated episodes of Lex Fridman, Armchair Expert, and other podcasts — in just six weeks.

Read more about Voice Translation for podcasts:

Spotify’s AI Voice Translation Pilot Means Your Favorite Podcasters Might Be Heard in Your Native Language


Hear the results on Spotify:

Lex Fridman Podcast — “Interview with Yuval Noah Harari”
Armchair Expert — “Kristen Bell, by the grace of god, returns”
The Diary of a CEO with Steven Bartlett — “Interview with Dr. Mindy Pelz”
More episodes on the Voice Translation Hub


Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

Over the summer, Spotify helped Tesla engineers ship a major update to their built-in media player. In this Release Notes episode, host Dave Zolotusky talks with Spotify engineering manager Geetika Arora and senior product designer JC Chhim about collaborating with Tesla to improve the in-car listening experience, the value of having a familiar user experience across devices, and how there’s more to a great collaboration than just picking the right SDK for the job.

Introducing Release Notes — a new series of mini episodes on the NerdOut@Spotify podcast. There are hundreds of teams at Spotify working on so many different things — from playlists that change throughout the day, to realistic voice translations, to a smarter way to shuffle songs. In each episode of Release Notes, we focus on one thing we shipped and what went into building it. You’ll see these mini episodes from time to time in the main podcast feed right alongside our regular episodes.

Learn more about our SDKs on the Spotify for Developers site: developer.spotify.com

Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com

You should follow us on Twitter @SpotifyEng and on LinkedIn!

How do you get a machine to find a song that’s similar to another song? What properties of the song should it look for? And then does it just compare each track to every other track, one by one, until it finds the closest match? When you have a catalog of 100 million different music tracks, like we do at Spotify, that would take a long time. So, for these kinds of problems, we use a technique known as nearest neighbor search (NNS). This past summer at Spotify, we built a new library for nearest neighbor search: It’s called Voyager — and we open sourced it.

Host and principal engineer Dave Zolotusky talks with Peter Sobot and Mark Koh, two of the machine learning engineers who developed Voyager. They discuss using nearest neighbor search for recommendations and personalization, how to go from searching for vectors in a 2D space to searching for them in a space with thousands of dimensions, the relative funkiness and danceability of Mozart and Bach, how to find a place on a map when you don’t have the exact coordinates, tricky acronyms (Annoy: “Approximate Nearest Neighbor Oh Yeah”) and initialisms (HNSW: “Hierarchical Navigable Small World”), why we stopped using our old NNS library, why we open sourced the new one, how it works for use cases beyond music (like LLMs), and looking for ducks in grass.

Learn more about Spotify Voyager:

About Voyager

Voyager on GitHub

Voyager documentation for Python

Voyager documentation for Java



Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.comYou should follow us on Twitter @SpotifyEng and on LinkedIn!

In the very old days, if you needed more storage for your database, you had to walk into the data center and install another server. Now you can just log into your cloud provider’s console and click a few buttons. Voilá, more storage. So easy! But what if you’re replicating that storage configuration for hundreds of databases at once? Suddenly, that’s a lot more clicking. Not so easy! (Plus, very tedious and very error prone.) So instead of living with this “ClickOps” approach, we developed a declarative infrastructure model — our very own “infrastructure as code” solution for managing cloud resources at Spotify scale. Instead of manually configuring each resource, developers just describe the state they want. And once we adopted declarative infra, we unlocked ways to improve not just how we manage resources, but also how we update policies, manage dependencies, and make other changes to code across our entire fleet of repos — quickly, safely, easily. In other words, programmatically.


Host Dave Zolotusky talks with David Flemström — who went from pushing the limits of Spotify’s infrastructure as a feature developer to working on the platform team in order to improve infrastructure for all of our developers. The two Daves discuss what declarative infrastructure means at Spotify, our journey to adopting it (going from Puppet to cloud consoles, to something better than both) and why we did it, how our model works (Kubernetes!), how it changed the relationship between our feature teams and our platform teams, how this shift helped enable Fleet Management at Spotify, and where we’re going next with abstracting infrastructure so that it helps our engineers do more, more easily.


Learn more about declarative infrastructure and Fleet Management: NerdOut, Episode 12: Fleet First
Fleet Management at Spotify (Part 1): Spotify’s Shift to a Fleet-First Mindset
Fleet Management at Spotify (Part 2): The Path to Declarative Infrastructure
Fleet Management at Spotify (Part 3): Fleet-wide Refactoring


Read what else we’re nerding out about on the Spotify Engineering Blog: engineering.atspotify.com


You should follow us on Twitter @SpotifyEng and on LinkedIn!