Learning from Giants #28
Learning from Giants #28
The definition of Impact, Prisma's database guide, "Pre-mortems" by Shreyas Doshi, Cold metadata storage at Dropbox, and CSS Layout algorithms demystified.
👋 Hi, this is Mathias with your weekly drop of the 1% best, most actionable, and timeless resources to grow as an engineering or product leader. Handpicked from the best authors and companies.
Did a friend send this to you? Subscribe to get these weekly drops directly in your inbox. Read the archive for even more great content. Also: I share these articles daily on LinkedIn.
What is Impact?
With a little self-reflection, working on your impact is one of the most apparent career accelerators. And that's true whether your company explicitly evaluates impact or not. Most companies assess based on some form of impact.
"Dropbox measures the success of its engineers largely on business impact. But what does impact mean? The definition is by necessity a bit vague."
Last tip: as a manager, you should explicitly convey that impact is not a one-time thing.
"Here are three guiding principles on how to think about impact as it relates to career progression. Consistency, Velocity, Accountability."
📗 Dropbox's What is Impact introduces the company's open-source Engineering Career Framework. And while it's written for Dropbox engineers, it's one of the best definitions I've found of what impact means for engineers at a tech company.
As a bonus, the article introduces one of the best open engineering career frameworks.
Prisma’s Data Guide
I've been sharing a lot of excellent database papers lately, but I agree they can be daunting, especially if you're no software engineer.
So today, we're going wider. A guide that introduces databases, the different types of databases, and how to use them.
Getting into tech and don't know where to start? Want to understand your friends' database discussions? Are you looking for resources for your junior engineers?
📗 Prisma's Data Guide is a collection of hundreds of articles the company has written lately about databases. It's a very actionable, use-case-oriented content gold mine. After reading the intros, pick a database and start playing with it. I'd recommend SQLite since it's the simplest (and trendiest!).
Shreyas Doshi’s “Pre-mortems”
A lot of things can go wrong in Product Management. You're responsible for so many things, often without a direct management relationship with the different people executing your plan. The worse thing is that, in hindsight, these issues often look preventable.
But people don't get promoted for preventing problems.
Introducing pre-mortems:
"If you do a pre-mortem right, you won't need a tough post-mortem."
"My teams at Stripe have routinely run a modified version of this pre-mortem script, particularly for major product launches. And while we haven't been 100% mistake-free, pre-mortems have enabled "calm product launches" and have enhanced team productivity and morale."
📗 Shreyas Doshi's Pre-mortems article is an actionable guide to running pre-mortems his way. Beyond the meeting definition, the author showcases problem categories they created to offset politics and raise people's actual concerns: Tigers, Paper Tigers, and Elephants.
Here's an Elephant for you: you should run pre-mortem meetings 🙃.
Alki: cold metadata storage at Dropbox
If you read Notion's database sharding story I shared a few months ago, you might have wondered what comes next. Every system has limits, and sharding only gives breathing room for a few more years of hyper-scale.
That's what happened with Edgestore, Dropbox's sharded MySQL cluster.
"Capacity expansion in Edgestore works by increasing the number of database clusters and redistributing the shards. [...] Around fall of 2017, we split Edgestore from 256 clusters with 8 shards per cluster to 512 clusters with 4 shards per cluster."
"Our levers for capacity expansion were quickly closing and we were heading into a capacity crunch."
There are probably a few complex technical solutions that you can use to do 2x, maybe 10x. But that won't solve the problem for long. And at what cost?
Going back to first principles, the team started questioning if Edgestore being a one-size-fits-all database was still a good principle. So they looked at the use cases and discovered something:
"We noticed that the largest dataset stored in Edgestore, was from audit logs for our customers. [...] Edgestore, optimized for low latency and high throughput reads of single objects, was a poor fit for this use case due to those largely cold traffic patterns."
📗 Dropbox's Alki post describes how the team solved the problem of audit logs, and, more generally, cold append-only metadata storage by building a data store specifically for that purpose: Alki. It's a super detailed peek at how the team designed the system, the trade-offs, cloud primitives, and different subsystems that ended up serving hundreds of TB of cold data.
Understanding CSS layout algorithms
Every engineer I know has learned CSS by doing rather than reading. Including myself. And every time we hit a new problem, we Google (ChatGPT?) it, and Stackoverflow comes to the rescue.
We get better at it with time because of the thousands of similar problems we’ve solved. Yet even experienced engineers still hit Stackoverflow regularly when working with CSS.
It's like a football-field-size scratch card. You cannot make sense of what's behind until you've scratched a good 40% of the surface. That makes CSS unique: the underlying concepts are so complex and entangled that you could be scratching during your entire career.
"CSS is so much more than a collection of properties. It's a constellation of inter-connected layout algorithms."
TLDR: sometimes, even in software, it's worthwhile to read the docs.
"When I started digging into the layout algorithms, everything started to make more sense. Mysteries that had bothered me for years were solved."
📗 Josh Comeau's Understanding Layout Algorithms" is exactly what we all needed. A straightforward guide to everything you thought you knew but really didn't. It may topple your mental model that took years of scratching, but it's well worth it!