Learning from Giants #21
Learning from Giants #21
Microservice architectures the right way, The best product document templates, Introduction to character encodings, and the impact of flaky test on DX.
👋 Hi, this is Mathias with your weekly drop of the 1% best, most actionable, and timeless resources to grow as an engineering or product leader. Handpicked from the best authors and companies.
Did a friend send this to you? Subscribe to get these weekly drops directly in your inbox. Read the archive for even more great content. Also: I share these articles daily on LinkedIn.
Designing Microservice Architectures the Right Way
When setting up a project or a company's tech stack for the first time, there are many decisions to make. You have to pick your architecture and your principles. And because you value consistency, you may have to get on with them for years after.
Yet we only get to start a small number of large projects in our careers. The learning loop takes years to close.
That's where standing on the shoulders of giants matters. Learn from these people that have started and carried on with large projects.
📗 Hence today's post, a talk from Michael Bryzek on Designing Microservice Architectures the Right Way, specifically on how he did so at Flow. The way the Flow APIs are designed, with significant attention to the engineering workflows, is why this talk has hundreds of thousands of views and shares. It has been a foundational resource for many software engineers (including me!).
"We have a relatively unique approach to software delivery that is focused primarily on a single foundation: empowering our people to build quality software efficiently.
Lenny’s Favorite Product Management Templates
Last week, I shared an article about Writing Well, giving crucial advice on improving your writing skills. One of the points was structure, which was understated. A good structure will get you most of the way to a great document.
And while there is some excellent advice on that topic, like using numbered lists instead of bullets, sometimes we need a little starter pack. A template.
📗 Lenny Rachitsky's My favorite product management templates is a curated list of the best product templates by the famous product influencer. From 1-Pager product requirement documents to more structured workshops and frameworks, it has all you need to shine as a structured product person who Writes Well!
A character encoding primer
ASCII, ANSI, Latin-1, ISO 8859-1, Unicode, UTF-7, UTF-8… Character encodings are a mess. But one that our everyday tools abstract for us. So the rare encoding problems we encounter lead us to Stack Overflow patches instead of deep understanding.
"Encoding is simply the process of transforming information from one format into another. It's not magic, it's not hard, and having a good understanding of the topic will help you a lot in the field."
First: ASCII.
"ASCII characters are stored in a fixed number of bits in memory-all the time, every time."
A fixed number of 7 bytes to hold a finite set of different characters (128). The issue? It does not scale to the thousands of characters people use worldwide.
"The newer, better solution is to not map characters directly at all, and to instead map them to a very specific conceptual entity which can then be described by a computer in multiple ways."
That's Unicode. In short, it separates mapping all the characters in the world from encoding them into bytes. This gives more flexibility and room for optimization.
"On the mapping side, Unicode maps virtually every letter or symbol, in every language, to a unique code point, each of which is described by a U+ followed by four characters of hex, e.g. D is U+0044."
"For the encoding side, Unicode has numerous ways of converting these code points into actual bits and bytes on a computer; these are called Unicode Transformation Formats (UTFs)."
UTF as in UTF-8, UTF-16. These are Unicode encodings.
📗 Daniel Miessler's An Encoding Primer is a must-read to understand character encodings once-and-for-all. After many years in software, I'm surprised I learned so much about such basic concepts, and you will probably be too.
How flaky tests impact developer experience
First, a definition.
"A flaky test is a test case that can both pass and fail without any changes to the code under test."
If you've been in software for a while, you probably have seen many flaky tests in your career. And usually, we either ignore them and move on, or fix them and move on, but unless this becomes a giant DX pain, we rarely have a global strategy against flaky tests.
Should we, though?
Careful prioritization processes require quantifying their impact and better framing the problem. Some researchers have tried doing that for you.
📗 The Surveying the Developer Experience of Flaky Tests paper from Owain Parry, Gregory M. Kapfhammer, Michael Hilton, and Phil McMinn contributes a few answers to that overall topic. By conducting a scientific study through surveys and analysis of StackOverflow answers, as well as prior research, the paper analyzes the definition, impact, and causes of flaky tests.
Some conclusions from the paper:
"Consider beyond code." Many flaky tests are due to environment disparities between local and CI.
"Flaky tests aren't completely useless." While they may be due to a poor test design, the survey also shows they can as often indicate a flaw in the code under test.
Impact: "Flaky tests can become an obstacle to the effective deployment of CI". Particularly to the trust and usefulness of such CI processes if they become distrusted and bypassed by default.
"Repair promptly". Letting such tests rot is not the solution, according to the survey! That is probably one of the main takeaways, although quite obvious.