Functional Futures: Live-Coding Rust with Tim McNamara

In this month’s episode of Functional Futures, our guest is Tim McNamara – a Rust evangelist, the author of Rust in Action, and a prolific YouTube content creator.

During the episode, we live-code in Rust, talk about Rust programming patterns, and discuss Tim’s book – Rust in Action.

You can check out the full episode on our YouTube channel (recommended) or listen to the audio version on our podcast page.

Below are some highlights from the conversation we had with Tim.

Interview with Tim McNamara

Programming patterns

Jonn: One thing that I noticed that we talked about while you were implementing breadth-first search is how there are some tricks, you could even say patterns. For example, the stuff that we talked about: when you don’t know who will own a node, we introduced a kind of, as you called it, manager entity that would own everything in the structure. In general, when I was just learning programming, programming patterns were the hot thing, everyone was drawing UML diagrams and this other stuff, and, as I moved to functional programming, all of this stuff reduced, no pun intended.

What is your take on programming patterns with Rust (and in general)?

Tim: I think that there is a place for patterns insofar as saying that we as a programming profession, let’s say, or as people – because we work in teams, we need to be able to write code that others in three months or three years time are able to understand. It’s very difficult to write code that is readable by others, and so the benefit of patterns is to be able to, in some sense, create a common vocabulary in the programming community about things like iterators, for example. It’s so commonplace to have types that actually know how to iterate over some collection, that many languages no longer actually require you to implement an iterator. And so that’s useful.

But like everything there, it’s detrimental in some other senses, because if your problem does not fit the pattern, you find that actually you’ve sort of painted yourself into a corner of a room. Or, actually, if you’re mopping the floor – that’s a better analogy. If you clean and then you get to the corner, and it’s like: “well, I’m now stuck because I’m trying to implement a pattern,” that’s when it becomes a problem.

One issue that you will encounter as you become more proficient in Rust or in a language that respects or has a robust type system, you will have a tendency to over abstract or to create very elegant abstractions, which are in some sense very nice, but actually they obscure your intent. At least, that’s my experience. I think when you begin programming, everything is all concrete types, it’s all kind of simple. You then learn some more skills and you pick up bigger problems, but then your tendency is to always reach for these very, very complex systems, and then advanced programmers will sort of swing back to simpler methods. I think patterns are very useful for teams and for people who are learning how to program, but it’s also important to remember that they are not rules. The intent is not to restrict you, but actually to provide you with more flexibility.

Rust in Action

Jonn: You’ve written two books on Rust by now, right?

Tim: No, I’ve got Rust in Action, but that’s the only full book that I’ve written. There is actually another one, but it’s actually not public yet, but there’s one coming, by the end of the year there’ll be something else as long as – my family, I’ve got little kids, I’ve got a career, I need to be able to, you know, have a social life as well – so we’ll see.

Jonn: Yeah, one of our biggest interviews was with Saša Jurić, and, actually, we were hanging out a lot as he was writing the book, and he was saying that for his first book, he underestimated the amount of time it will take him by an order of magnitude, basically.

Tim: Yeah, for me it will be close to an order of magnitude. It took me nearly five years, because for various reasons it was about a year before the project officially got started. Originally, I was brought in by Manning as a second author. They had a Rust in Action project in 2015, I think, but the first author actually pulled out. We were working together for a couple of months, and then their first author kind of pulled away, and then the publisher said: “do you want to carry on?” I said: “no, we should write a better book, the manuscript we’ve got so far is kind of rubbish”.

I had this idea that I wanted to write the best programming book that I had ever seen. I wanted to convince myself that it’s possible to write an excellent book. I had sort of noticed that the trend in technical publishing had been declining over a few years. I think things that were published in the late 90s were probably a lot better than things that were published in say 2012 or 2013, and I wanted to be able to write a book that I would want to buy. That’s really difficult. So, the reason why book quality has been declining and it’s getting faster and faster, is that it’s simply really, really hard. So I spent, I would estimate, between mostly 10 to 15 hours a week. I tried to do 2-4 nights for 2 hours a night, plus a period in one of the weekend days, which was about a 3-4 hour period. So, then let’s estimate roughly 10 hours a week. For four years, that’s 2 000 hours. And like it doesn’t stop, it turns out that once you’ve written the book, you then need to advertise it, otherwise they won’t buy it, and then it’s useless.

However, Rust in Action has done extremely well. Most technical books sell fewer than 5 000 copies over their entire lives. I’ve sold, I think, about 20 000 copies in its first six months. That’s because Rust is growing, Rust in Action is a good book, it’s different from every other book, but yeah, it’s actually really, really hard.

Jonn: First of all, it’s really great to hear that you’re making a return to the glorious books of the 90s and drawing inspiration from back then, because I remember, for example, the Linux reference manual or C++ reference. Those are references, but those are narratively well written books as well. It’s just mind-blowing how good they were.

How is it with Rust in Action – do you add to the explanation of what things are the explanation of how to go about actually working with these things?

Tim: I want to answer your question in three ways, I think.

So, first of all, I was starting the book when the official book by Steve Klabnik and Carol Nichols was nearly finished. I wanted something that was different from that book. There’s no reason to spend money – mine is not a free web book, so it needs something extra. And I thought: Rust is being promoted as a systems programming language, and so I want to teach people who have only used JavaScript or only used Python, or only used, let’s say, Java or C#, or anything else, and they are looking at Rust and saying: “I’m scared of systems programming, I’ve heard that systems programming is hard,” and what I wanted to do was teach people enough of that so that they don’t get intimidated by the jargon. And so that’s one of the ways that my book distinguishes itself.

The other thing is that it does indeed provide strategies. I think, in chapter four (it’s devoted to ownership and borrowing) I go through three different methods for resolving issues around ownership and borrowing for beginners.

I think one of the things that also makes my book a little bit different, it’s kind of the third way, is that I wrote it for myself. I was a Python programmer before I was a Rust programmer, I did natural language processing. I wasn’t even a real software engineer, I was a data scientist. I wrote Python to be able to do text analysis, and it was too slow and used too much memory, and so I decided to start writing Python extensions. But if you read any material on writing native extensions for Python, in C extensions for Python, there will often be a paragraph which is “this is very dangerous, you might crash your program, you might introduce memory leaks, you might do this, you might do that, you know, effectively you’ll crash your computer, you’ll introduce security holes” and so Rust provided me as like an everyday non-programmer. I’m not a professional computer scientist, I have a humanities degree, I don’t have a computer science degree from university.

I wanted a book for myself. There are better Rust programmers in the world than I am. My book was written for people who are still learning, and I definitely still remember learning, and I hope that I did not have kind of this expert blindness problem. I find books that are written by professionals or people who are extremely good are often very difficult to read because they have forgotten what it is like to learn. There’s so much there – learning Rust is more than learning a programming language, it involves learning a lot of jargon, it involves understanding some new concepts like ownership and borrowing, for example, and so there’s almost multiple layers of learning that are required.

And my big message to people who are interested in this new high-performance programming language is that they should take it slow, and just learn things piece by piece. And also that if they don’t end up using Rust they actually haven’t lost out because you can apply a lot of the techniques, in particular, functional programming, to other domains or other other programming languages as well.

Tips for Rust beginners

Jonn: If I could humbly add something to your message when you said take it slow, I would also suggest beginners to also not be scared of cloning your data or copying your data. I had this huge problem of thinking: “oh, okay, I’m writing Rust, it’s kind of an imperative language, and I must squeeze out every single bite, I must not even prematurely optimize but, like, everything that can be mutable must be mutable, within the boundaries of a module, let’s say."

But it’s actually false. You can clone so much and your Rust programs will still be like 10, 100 times faster than your PHP, Python, Java programs because no matter how much you will tell Rust to clone your stuff, you will not manage to clone your stuff as much as, let’s say, a virtual machine, where you know you just copy everything on change, pretty much. So don’t be afraid of taking it slow in practice and don’t be afraid of making it slow by cloning a bunch of things.

Tim: Absolutely, there’s this idea that cloning is expensive or cloning is slow. It takes 20 nanoseconds. You can fit hundreds of them inside a millisecond. It is slow for the cpu and it is slow compared to just kind of like copying data, but it’s not slow from the point of view of your users or you as a programmer. And there’s also plenty of time to learn more techniques that don’t require clone, but for now absolutely clone or to_string, or to_owned – these methods are there for a reason, and the reason is often that they’re useful. So use them.

Jonn: Yup, and in general, sometimes people are scared of Rust’s ergonomics, but actually if you take time to appreciate how many options you have, as Tim mentioned, when you’re writing Rust programs, to write meaningful program reasonably quickly, you will see that that ergonomics are there. And in my opinion, for example, immutability by default is like just an ergonomic tool, it kind of nudges you in the way of “like, okay, don’t worry about it, just create a bunch of stuff, use it, and the memory will be freed by [inaudible] design." I think those are good messages to wrap it up and go to questions from the audience.

Questions from the live chat

[…]

Jonn: Another question is actually very interesting: is there no Constraint kind in Rust? So I don’t know if, Tim, you know what a Constraint kind is?

Tim: I have not used higher-kinded types myself, I’m not a Haskell programmer.

Jonn: I’ll just provide some context: so, basically, when you write out something like a functor or graph, or something, you would have a type variable, and you would then insert this type variable in that constructor, and kind of this functor or something that takes some type variable would be * -> * -> Constraint.

So I guess the question is, is there a similar way to reason about kinds in Rust?

Not to the same extent. Rust’s type system is not as sophisticated as Haskell’s, there have been discussions, there have been proposals made to be able to do something similar in Rust.

The shortest answer would be no, but you can do things that are similar with enums. So, we have sum types in Rust which just happen to, I think, use the wrong keyword enum, and you can provide, in some sense, a form of restriction via some of those variants as well. We haven’t demonstrated that in the code that I’ve shown before, but that would be the closest I think just thinking about it, that you can emulate higher-kinded types, but you can’t get to the same extent, it’s not quite as natural as Haskell.

Thanks for the insightful information about the fact that they are trying to add them, and also maybe it’s a topic to attract the Haskell crowd to your YouTube channel.

Another question. So there’s a very popular approach of learning by building, so the person asks what’s the best way to go about learning Rust and what projects could they try to build to learn?

So I don’t want to promote – this is a lovely book shows Rust in Action and laughs.

One of the things that distinguishes Rust in Action is that you build a big example in every chapter of the book, so we build a database, we simulate a CPU emulator, we implement HTTP multiple times, and every time we remove one layer of abstraction, and by the end we’re actually sending tcp requests ourselves. And so if you’re the kind of person who likes projects, Rust in Action is your book.

There are free resources. One that I like is a tutorial for creating a text editor in Rust. And there’s also a really lovely book related to game programming called Hands-on Rust, it implements a 2d platform game and it’s quite fun.

Final question: as a functional programmer, are there ways I could leverage my previous experience to kick-off learning Rust?

Yes, very much so.

Rust prefers higher-order programming and a functional style over almost all of its collections, and so you may think of Rust as an imperative language because it is normally associated with C++. However, Rust was originally an ML, it was implemented first in OCaml, and so a lot of your functional learning will flow through into Rust quite naturally.

Jonn: As a Rust beginner, I found out that it’s similar to the way we write functional Python. I know, it’s kind of weird, but you have kind of functional wrappers around like kind of imperative, sometimes imperative-like loops when it’s just convenient or effective to modify some dictionary, and have like a functional API or a functional interface to it. In my beginner experience, this is what I do in Rust a lot to a pretty good success, which means that then you will compose those functional wrappers together pretty much as if you’re writing Haskell or something like that.

We would like to thank Tim for talking with us and participating in a live-coding session! If you want to see more of his content, be sure to follow him on Twitter and subscribe to his channel on YouTube.

If you would like to read, watch, or listen to interviews with software developers and creators of programming languages, you are welcome to follow us on Twitter – we’ll make sure to keep you updated.

Our team of Rust specialists is dedicated to delivering high-quality software solutions that meet the specific requirements of today’s high-tech businesses. Let us assist you in solving your problem!