Est. Reading time:
If you just want to see my solutions to the challenge, here is my repo : Solutions
Overview
I recently solved a series of challenges called Gossip Glomers. This is a series of challenges put forth by fly.io that to me felt like a tiny intro to distributed systems. I did not find these challenges too hard...but the right amount of fun :). For those who don't know fly.io :
We’re Fly.io. We run apps for our users on hardware we host around the world. This post isn’t about our platform. Rather, it’s an elaborate plot to get you to write some code just for the hell of it.
quoted from : fly.io blog, an incredible blog, you should read it!
built with the help of Jespen (A tool made by Kyle Kingsbury to make distributed systems safer.) These challenges use something called Maelstrom that is built on top of Jespen
Jepsen pushes vendors to make accurate claims and test their software rigorously, helps users choose databases and queues that fit their needs, and teaches engineers how to evaluate distributed systems' correctness for themselves.
About Maelstrom :
Maelstrom is in my opinion an incredible platform, Some of its notable features :
- Gives a very configurable network, can induce faults, partitions delays and whatnot
- Gives very comprehensive reporting, from a results disect to timing diagrams of all messages, everything one needs to evaluate their system.
- Has various workloads like KV, grow counters, unique number gen
- Provides services on top of a network to run more complex workloads, like :
- Linear KV
- Serial KV
- lww-kv
- lin-tso
Furthermore, it's beautifully documented, you can find it here : https://github.com/jepsen-io/maelstrom
About the challenges :
Each challenge is composed of several parts:
- The workload acts as a set of clients to your distributed systems. These clients send different types of messages as defined by the challenge and expect certain constraints to be met. These workloads can vary between a simple distributed counter to multi-operation, transactional database systems.
- The simulated network injects network partitions or slows messages between nodes.
- The verification system uses Jepsen to check consistency and availability constraints required by the challenge.
- And finally, the binary for the node which is written by you!
The challenges start rather easily from Echo to building a Multi-Node distributed KV store that satisfies read committed in a total availability guarantees. Towards the end of these challenges, you start worrying more about transaction guarantees and tradeoffs!
Conclusion
If you are even the slightest bit interested in systems, this is a must-try list of challenges. But in my opinion, they barely scratch this wide world of distributed systems. That's it for this blog!