Many years ago, a friend and I joked about a cinematic universe concept: a world where everything was nuclear-powered. I mean everything. Cars, laptops, pencil sharpeners, even pencils.
The gag was about creating a Michael Bay-eque landscape of explosions, but it was an allegory for over-engineering that stuck with me. So, I’d like to coin a term: “nuclear powered pencil”.
A nuclear powered pencil is something that is drastically and fundamentally over-engineered, to solve a relatively simple problem. (And also, has unnecessary danger as a result.)
I will share one of my own: as an eager beginner, I was tasked with building a point-in-time backup solution for a specific system. I designed something fancy which time-indexed all backups in a database, deduplicated files, and thinned out backup granularity over time. It used dedicated nodes with no ingress, to pull the files in from application nodes, and prevent application nodes from being able to mess with the backups. Sounds great, right?
It worked pretty well. The big problem? It took a long time to build, and was too challenging for others to quickly debug. It ran a database and multiple dedicated machines.
In retrospect, the entire tracking database was unnecessary. Had I done the napkin math, I would have realized that storage was cheap, and that deduplication, backup-thinning, and strong compression were virtually useless. Furthermore, there were few enough files that I could just list the object store directly, and get reasonable performance.
Additionally, I substantially complicated the architecture in order to achieve append-only backups. This is a critical feature, but… did I need to add a wrapper around it myself? In retrospect, I’m not 100% sure that the chosen data store didn’t support append-only access back then. And even if it definitely didn’t, maybe I should have just chosen a less desirable data store that had all required features?
Nuclear powered pencils are easy to write. They often come from a good place: the desire to make something perfect. But, almost always, they come from a lack of understanding of the problem. If you think you might be designing one, or see someone else doing so, here are some good questions to ask:
Is there another service/tool that could implement (or avoid the need for) some features? If so, is there a compelling reason not to use it?
Is it solving the overall goal by solving the minimum subset of problems?
Is there a good reason for anything “clever” in this design?
Are you working too-far-removed from the problem? Are you interacting with the components/systems you actually care about?
Is ___ necessary at the projected scale? Have you done the rough, order-of-magnitude math?
