Elixir in Production: Glific

In this installment of Elixir in Production, I interview Donald Lobo, the founder and lead architect of Glific, an open-source communication platform.

We talk about their product, how Elixir and its awesome open-source ecosystem has helped them build it, what problems they ran into, and what you should do if you want to start your own Elixir project.

Could you tell us more about Glific and your role there?

My name is Donald Lobo. I’m the founder and lead architect of Glific. Glific is an open source two way communication platform built on WhatsApp Business API that is focussed on helping non-profit and social sector organizations communicate with thousands (and potentially millions) of their beneficiaries in an automated and scalable manner. Most NGOs do not have any technical person on staff so having really good UI/UX and a simple, easy-to-use system was super important.

We are a team of 1 product designer, 1 support and documentation person, 3 frontend developers and 3 backend developers.

Glific is currently used by more than 20 NGOs in India, and we are on-boarding 2-3 new NGOs on a monthly basis. Some of the cool use cases using Glific include linking a community of volunteers, Covid Patients and hospital beds during the pandemic in India. Other NGOs have used it to deliver arts and creative content on a weekly basis, mental health wellness videos and more to their beneficiaries.

Where do you use Elixir in your stack? How big of a part of the Glific codebase is written in Elixir?

Glific is basically split into the frontend component and the backend component. The frontend component is a React Application that talks to the backend via GraphQL.

The backend is 100% Elixir. The current application has approx 250 Elixir modules. You can find our source code on GitHub. We use Gigalixir for our deployment, which really helps us significantly from an operational perspective, i.e. we don’t really think about it too much!

How did you decide to choose Elixir? Are there any specific requirements that made you choose it?

Glific is a message passing application, with minimal processing of those messages. We wanted an environment which facilitated this in an easy manner. We also knew that WhatsApp had used Erlang quite effectively in its early days. Finally, we wanted to expand our skill set and learn something new while we were doing it, and hence we stumbled on and chose Elixir.

We spent the first two months learning Elixir and functional programming, and writing small prototypes and throw-away code just to explore the ecosystem and build up our internal knowledge base. This was definitely an exercise worth doing as a group, and all of us struggled, but learned a lot with this experience. It was quite good to have multiple people walk down the same path of learning at the same time.

Are there any Elixir libraries or frameworks that you found useful and would like to feature?

I would say the Elixir community and folks in the ecosystem are super helpful and friendly. We’ve been stuck quite a few times, and just chatting on the slack channel, has helped us clarify our thoughts and get moving again. This probably has been the best part of the experience. We try to support other folks also and try to pay it forward in our own small way.

Some of the packages that we use a lot include:

• Absinthe/GraphQL – Ben Wilson is an amazing super helpful individual on multiple levels.
• Oban Pro – Sorentwo has always been generous with his time.
• PoW – Customizing this library has been so easy. Dan has documented it so well.
• Cachex – Easy to use, and well optimized library.
• Dialyzer – We try to spec every single function. Catches a lot of errors that way.
• AppSignal – Again, great support, super nice people.
• Gigalixir – Can’t say enough good things about Jes.

We use the rest of the stack: Phoenix, Ecto, ExDoc, ExUnit etc. I see these being part of the framework rather than libraries.

Did Elixir help you accomplish something that would be hard to accomplish with any other programming language?

We’ve really liked the concurrency aspect of Elixir and the ability to process things in parallel. So blasting a bunch of messages turned out to be really simple and easy (using Tasks.async_stream). I’ve also realized that adding more functionality is a lot faster in functional programming, since the data dependencies are minimal.

Are there any places where Elixir has fallen short (ecosystem support, dynamic typing, etc.)?

We integrate a fair bit with the Google suite of products (BigQuery, Google Cloud, DialogFlow). There are no native client libraries for this, which makes things a wee bit harder.

In our first few months, trying to figure out the dreaded out-of-memory errors, process killed caused us a few sleepless nights. Those were a bit hard to debug initially, but now we have a pretty good process of tracking them and fixing them quickly. It also helps that we now have a relatively stable codebase.

Which companies or projects should check out Glific for their own use?

I think companies working with or for social sector organizations should definitely check out Glific. Companies that are interested in messaging with a large group of users are another good set. Folks interested in contributing to open source for good projects are welcome, since we’d love to increase our developer ecosystem.

What are the future plans of Glific?

In the short term, we will be releasing a mobile app, so NGO staff can be alerted on their phone wherever they are. I also suspect we will need to do some re-architecting as we scale to delivering thousands of messages per minute. We also need to improve on incorporating better data analysis and visualization tools into the system. Google DataStudio is great, but I suspect as things get more complex, we will need something more sophisticated.

In the long term as message volume grows, we will incorporate machine learning and natural language processing so NGOs can respond in a better manner and also triage important issues so they can be dealt with quickly. I’m quite excited about that part.

And finally in the areas where our NGOs work, literacy levels are quite low. There is a theory that smartphones can bypass the literacy gap with the use of emojis, audio and video. So, I suspect we will be doing more work in this area, with audio transcription etc.

Any tips you would give to those wanting to start their own Elixir project?

A couple of things:

• Treat it as a marathon, not a sprint.
• Invest time in learning and improving. Reading the weekly elixir newsletters and the articles within it is important, so you are aware of different ways of doing things.
• Ensure you have good tests and test coverage. As your Elixir knowledge grows, this allows you to rewrite parts of your codebase in a better manner.
• Join the community. Ask questions, help others, improve the documentation and more. Be part of the larger ecosystem.

I would like to thank Donald for the informative interview! If you want to check out or contribute to Glific, you can visit their homepage or GitHub.

For more articles like this, I recommend you to check out our Elixir in production tag – we have quite a few stories there already. And if you have an Elixir project of your own that you would like to talk about, I would love to interview you – don’t hesitate to send me an email. 😉