Haskell in Production: Chordify

In this edition of our “Haskell in Production” series we interview Jeroen Bransen from Chordify, an online platform, which turns any music or song into chords. Jeroen has been working at Chordify since 2016. Prior to joining it, he obtained his PhD in Software Technology from Utrecht University, where he became acquainted with two of the company’s founders. Initially hired as a Haskell developer, Bransen continues to write Haskell code. Over the years, his responsibilities expanded to include leading the Haskell team and maintaining servers.

We discussed how Chordify ensures the correctness of Haskell code and scalability of its codebase, which libraries they use in their work, and more.

Interview with Chordify

Where in your stack do you use Haskell?

Our backend is completely Haskell-powered, so the website, the Android app and the iOS app all connect to our Haskell backend API. And next to that we also have many offline data-processing jobs written in Haskell.

What specific qualities of Haskell make it especially valuable for your purposes?

Many things, but the first that come to mind are the advanced type system and the efficiency! We use quite a lot of type-level trickery, which at first glance can be daunting, but in the end really pays off in writing correct code. I’d say, the joke of “it compiles, ship it!” is true; from time to time, I do release code that I have not actually run (for example because reproducing certain behavior is hard in a development setup). In general, changing existing code or refactorings of some kind don’t usually break anything in our codebase.

Furthermore, the lightweight threading model of Haskell is great for web-server purposes: we handle thousands of requests per second on quite low-end hardware. Interestingly, our CI server that does compilation needs a lot more RAM than our production servers, which is the price you pay for having strong compile-time guarantees, and thereby the efficient runtime.

Can you describe a complex problem you solved using Haskell? How did the language’s features assist in finding a solution?

One that I really like is in-memory caching using Redis. Redis is a common tool, for which some standard Haskell libraries exist, but because data needs to be serialized and deserialized, it’s relatively easy to make runtime errors with it (e.g. you can store an Int and try to retrieve it as String, and this will only fail at runtime). We made a Haskell-style interface around this, which we also open sourced, which brings stronger typing. With this library, Redis keys are linked to a Haskell type, so as long as you have proper types (e.g. newtypes everywhere) around your data, you can’t easily make such mistakes anymore. And then it extends to doing more complex Redis transactions.

The nice thing about this is that creating such a library is quite tricky, and it took effort to get all types right, but now that we have it, the compiler helps us tremendously with all Redis interaction, and as far as I am aware, we have never experienced Redis-related bugs.

What is your approach to testing in Haskell? How do you ensure the reliability and correctness of your Haskell code?

If you have a good type-system, you don’t need testing! 😁 Of course, that’s not true, but I do think the type system plays a big role in reliability and correctness. Next to that, we have both property-based tests (e.g. with QuickCheck), as well as unit tests (on individual functions, but also on the API level).

How do you ensure that a large Haskell codebase, like the one at Chordify, remains maintainable and scalable over time?

This is a tricky question, because I think Haskell is not much different from any other language in this respect. What helps, as one of my coworkers always says, is that “there are no bad Haskell developers.” The learning curve for Haskell is high, so if you make it as far as getting a job as a Haskell developer, you probably also pick up good habits in general development relatively easily, and that helps in keeping the codebase in good shape (half of our backend team has a PhD). But still, there are certainly parts of the codebase that most people don’t know much about. 🙂

There’s a complex AI technology behind the chords at Chordify. Could you share some insights into how it works, and the role Haskell plays in this process?

The algorithm is mainly based on deep neural networks, and we’ve written a nice blog article about it. While our Music Information Retrieval researchers use mostly Python to develop this algorithm and to train these networks, we run all of it using Haskell in production. Or rather, we glue everything together with Haskell, we use FFI bindings to the TensorFlow runtime so that we can send data from Haskell to it and get results. But, for example, for beat tracking there is quite some interesting post-processing going on, which we’ve implemented in Haskell.

Are there any tools or libraries in Haskell that you find indispensable for your work at Chordify?

There are so many great libraries we take for granted (e.g. network), but I can mention a few specific ones that we use with great pleasure:

• servant, to specify the whole API (though we actually use OpenAPI declarative specifications, and generate servant types from that);
• warp, the amazingly fast web server;
• persistent with esqueleto, for writing typed MySQL queries;
• amazonka, for calling the Amazon API.

And finally, HLS is a great tool for developers. Of course, I can mention many more of them here, but these were the first to come to mind.

You have worked as a team lead at Chordify since 2016. What are the takeaways you have learned from your experience there?

Just do it! Coming from academia, it’s quite tempting to spend a lot of time thinking about the best solution to a problem and trying to do it 100% correctly. However, in the real world (and especially the internet), data is not always as nice as in the scientific context, and even with a lot of effort, getting code to work right 100% of the time is unlikely.

Instead, having a high development speed and releasing often gives us the opportunity to have millions of users worldwide test our code, which is usually much more informative than anything you could test in a development setup. And while we, of course, strive for offering our users the best experience possible, we’re not dealing with human lives, so I (now) think it’s better to briefly break things on production every now and then, than having long development cycles. And as said before, Haskell gives us quite a lot of compile-time guarantees, so it only happens rarely that we really break something (and we have proper monitoring, so we then rollback quickly).

Can you offer any tips for developers transitioning from imperative to functional programming languages like Haskell?

The compiler is your friend, not your enemy! I think that one of the challenges of making this transition is the fight with the compiler; the more static guarantees the compiler can give you, the more annoying it can be for developers. Haskell is typically a language with quite a high barrier, e.g. it’s hard to get a working program. But in the end, this pays off, because once the compiler is happy, there is little room for error, and refactoring (which you never plan, but it always happens at some point) is also much less painful.

How do you foresee the role of Haskell evolving in software development, particularly in the music technology sector?

I actually think most of the influence has already happened. iOS development moved from Objective C to Swift, Android development from Java to Kotlin, TypeScript is used on top of JavaScript, and in many other places in software development, aspects of functional programming have found their way. Of course, Haskell is on the far end of that spectrum with its pure functions and strong type system, but I would say, the broader adoption of its core features (many static compiler guarantees) has been happening. And the music technology sector is not much different from other sectors in that regard.

For more interviews with companies that use Haskell to solve real-world problems, check out our Haskell in Production series. Also, be sure to follow us on Twitter or subscribe to our mailing list (via the form below) to receive new Serokell articles via email.