Learning from Giants #15
Uber's 3 generations of API gateways, User research as you grow, Clean code considered harmful, Roaring bitmaps, and mentorship best practices.
👋 Hi, this is Mathias with your weekly drop of the 1% best, most actionable, and timeless resources to grow as an engineering or product leader. Handpicked from the best authors and companies.
Did a friend send this to you? Subscribe to get these weekly drops directly in your inbox. Read the archive for even more great content. Also: I share these articles daily on LinkedIn.
The story and architecture of Uber's API gateways
When scaling fast, the number of APIs exposed to the outside grows exponentially. At the same time, audit, security, and reliability concerns become increasingly important. That's when most companies outgrow out-of-the-box reverse proxies like Nginx. Then there are two possible paths: heavy customization of existing tooling or building from scratch.
"An API gateway provides a single point of entry for all our apps and provides an interface to access data, logic, or functionality from back-end microservices. It also provides a centralized place to implement many high-level responsibilities, including routing, protocol conversion, rate limiting, load shedding, [...]."
With dozens of teams exposing thousands of endpoints, your API gateway can quickly become a painful bottleneck. Deploys get slower and slower, as well as riskier; test suites are slow, load keeps growing. It takes a mission-driven, product-minded team to build and scale an API lifecycle platform.
📗 Uber's 3-part API Gateway series tells the story of a multi-year effort to scale and evolve Uber's solution to that problem as the company grew immensely. The first article tells the story of Uber's first, second, and third-generation gateways. The second and third detail the latest solution and the challenges the team solved to keep up with the company's scale.
How user research can fuel the right decisions at every stage of growth
“Far too many products still fail because there’s simply no demand for them.”
“How does that happen? By overlooking user research.”
“Early stage: Research at this stage doesn’t need to be perfect, but it needs to provide a general signal on which direction to go.”
When scaling out of early-stage, it’s easy to miss the research train again. Most companies will have a function that talks to customers and gets feedback, but user research can be much more. It needs to evolve into a standalone function that drives both short-term tactical research and longer-term strategic insight.
“At scale: companies operating at scale must optimize around the edges, and small changes matter much more. Research is conducted systematically across the entire product lifecycle.”
📗 Ryan Glasgow’s Research Twice, Build Once: How to Know Your Users as You Grow explains how user research is an impact multiplier at every growth stage. Research can help make the right decisions in all phases of product development, from discovery to concept testing, usability testing, and to live product and satisfaction feedback. When done right, research can give more insight than an AB test in less time.
Clean code arguments considered harmful
“Obsessing with “clean code” and removing duplication is a phase many of us go through.”
“Clean code” is often reduced to a 0/1 situation based on the amount of duplicated code in a code change. 10% duplicated code is "dirty", 0% is "clean".
Why are you doing this 2000 LoC refactor? “Because it’s cleaner”, you answer. That sentence can justify anything. But is zero duplication the end goal?
Rarely. Writing software qualifies as engineering because it can’t be reduced to “clean” vs. “dirty”. It’s a world of trade-offs.
“I suggest to think deeply about what you mean when you say “clean” or “dirty”. Do you get a feeling of revolt? Righteousness? Beauty? Elegance? How sure are you that you can name the concrete engineering outcomes corresponding to those qualities?”
📗 Dan Abramov’s Goodbye Clean Code reflects on his multiple years of writing software, from an early mistake of doing a significant refactoring for “cleanliness” to realizing why it was harmful years later. Hopefully, it’ll make you reconsider invoking “clean code” next time!
“Don’t be a clean code zealot. Clean code is not a goal. It’s an attempt to make some sense out of the immense complexity of systems we’re dealing with.”
A primer on Roaring Bitmaps
"Various operations exist in search and database indexes that boil down to having two sets of integers and needing to intersect or union them quickly."
Such operations are usually achieved with bitmaps, and fast bitwise AND and OR CPU instructions. But CPU is not the bottleneck here, it is memory.
"Unfortunately, bitmaps suffer from awful compression in common cases involving very large sets of integers."
That's particularly true in extreme cases like very sparse, or very dense bitmaps.
Introducing Roaring Bitmaps: optimized bitmaps that offer better compressions while conserving extremely fast set operations.
"Roaring bitmaps use a kitchen sink of algorithms and techniques to achieve better compression and faster performance than other bitmap implementations."
📗 Vikram Oberoi's A primer on Roaring bitmaps introduces this data structure that is now embedded in almost all the most popular analytics and search databases: Lucene, ClickHouse, Hive, and many others. It's interesting to see none of the heuristics are particularly complex, but it's their combination and optimization together that makes Roaring bitmaps such a popular structure.
Developers mentoring other developers
Imagine two people's cooking skills. One has been relentlessly given feedback on meals, and the other has cooked for themselves with occasional friends saying it was great. After a few years, you have both of them cook something for you. There will be a world of difference.
That’s mentorship.
“Mentorship is a learning relationship between an experienced person and someone who wants to grow."
The professional world and your professional skills are similar. Whether it is formal or not, mentorship is always a great accelerator.
"Mentorship doesn't need to be formal for a less experienced person to learn from a more experienced one. And for the most part, it isn't. [...] Code reviews are frequent examples of informal mentorship."
"Having a more "formal" mentoring relationship can help you grow faster and in a more focused way."
📗 Gergely Orosz's Developers mentoring other developers: practices I've seen work well is not specific to software and applies to most professional environments. It covers a lot of angles to make mentorship a success for mentors, mentees, and companies, with clear, actionable advice.
"I'm a firm believer that as a mentee, you need to invest time and effort in mentorship, to get value out of it. Set expectations clear from the start. Come prepared to the catch-ups with your mentor, making good use of their time."