Learning from Giants #33

How Google Spanner does 99.999% write availability, Communicating with Extreme Clarity, and How to create your company's Career Path.

👋 Hi, this is Mathias with your weekly drop of the 1% best, most actionable, and timeless resources to grow as an engineering or product leader. Handpicked from the best authors and companies.

Did a friend send this to you? Subscribe to get these weekly drops directly in your inbox. Read the archive for even more great content. Also: I share these articles daily on LinkedIn.

---

Google Spanner's high-availability writes

Spanner is Google's distributed relational database with 99.999% availability. How it works is the topic of many articles. Let's focus on how it achieves such availability for the trickiest path of databases: writes, while keeping consistency.

"In relational databases, providing both high availability and high consistency in writes is a very hard problem."

Because high availability means you must have multiple servers capable of handling writes while staying coordinated for consistency. Engineers call this "write replication".

"[In Spanner], in order for a write to succeed, a leader needs to synchronously replicate the change to the other replicas."

Spanner’s multi-zone write replication.

The read/write leader is chosen amongst all nodes through a leader election mechanism. All writes go through the leader and synchronously replicate to other nodes. If the leader goes down, the leader election mechanism will immediately elect a new one that will take-over writes without downtime.

But synchronous replication of writes brings another set of problems: non-leader server availability. If one goes down, it can't synchronously record the leader's writes.

"This is where Spanner does something better. Spanner only requires a majority of the Paxos voters to successfully write. This allows writes to succeed even when a read- write replica goes down."

📗

Jaana Dogan

's Spanner's high-availability writes gives an overview of the challenges of building a high-available, consistent relational database, the choices the Spanner team made, and their consequences. An excellent article to apply the distributed systems theory understanding you've been building from the pieces I've shared in the past!

---

Communicating with Extreme Clarity

"If you're working in a growing org or company, it's a certitude that there'll be fewer people that are closer to your work tomorrow than there were yesterday. If we communicate in a way that requires the reader understand the details of your space, you risk creating gaps, overlaps and misunderstandings, and that's bad for everyone — especially your customers."

So what's extreme clarity, and how's it different?

"Extreme Clarity means communicating something in a way that leaves no ambiguity in the mind of the reader and that all readers ultimately leave with the same understanding."

And this, whoever your reader is—a new team member with no context, your boss, or your company's executive team.

You can learn extreme clarity by trying harder, taking a step back before hitting send, and following general rules. But one thing will stay true: you must be intentional about it. For every memo and every message, ask yourself whether it's clear enough for anyone to understand.

📗

Simon Cross

' Communicating with Extreme Clarity explains what "Extreme Clarity" means by giving an example of an internal message with and without it. The author then lists a few rules of thumb for extreme clarity, including Shorter is (generally) better than longer, Highlight changes in state, and Proof-read, or better: peer-read.

Communicating with Extreme Clarity

---

Product & engineering career paths

Start-up advice: figure out levels before titles.

At some point in a company's growth, your product and engineering evaluation and career progression will need some structure. This structure will help support people's growth and ambitions, unify expectations across teams, and set a fair compensation policy.

It'll materialize into a Career Path document detailing formal levels and the impact and behavior expectations of each.

But don't make the mistake of focusing on titles. They're just noise compared to clear level expectations.

So how do you create that Career Path (also called Ladder)?

1. Have a look around. Building an innovative company is hard enough; you don't have to create a radically different career path.

2. Start simple. Most career path failure stories have one thing in common: they went too specific, too fast.

3. Describe impact rather than a list of action items. Career paths can't and shouldn't be checklists; there will always be a personal appreciation element.

Here are three of my personal favorites to get you started:

• Monzo's Engineering Progression Framework v2.0,

• Figma's Product Design Ladder (2020),

• Buffer's two-dimensional Career Framework, with levels based on Radford's benchmarks.

Figma’s design ladder skill rubrics.