Welcome to our new blog series where we take you behind the scenes of Klotho and share some of the technical and product challenges we face while building our platform. In each article, we’ll explore a different topic and invite you to join the conversation with our engineers.
In this edition, we’ll dive into a discussion between two Klotho engineers – Yuval Shavit and Gordon Goetz. They’ll be discussing the best way to design and develop customer-facing software features: should they be large and comprehensive, or small and modular? Let’s find out what they have to say!
Yuval, there’s something I’ve noticed that is an undercurrent for a lot of our debates: your first instinct is usually to provide an all-in-one, “batteries included” experience, whereas I usually prefer giving people small, composable pieces.
On reflection, there’s a few reasons for my preference, but for the purposes of this discussion, let’s focus on just one of them: maintenance costs. Writing and maintaining more things costs more. There are some cases where the benefit outweighs this cost, but I believe the default is that the benefit doesn’t.
The cost problem is definitely real. Where I come from is that someone has to pay it: either us, or our users who take our components and put them together. The question for me then becomes whether it provides more value in aggregate to all our customers than each would be able to provide themselves.
It can be tricky, because I see the value in providing something of small value to a great audience is a net large effect; however, these small values may not register to an individual user to request or ask us about.
These little things can really add up. I’ve noticed a pattern through the years: people are often reluctant to bring in a dependency like Apache Commons or Guava, because the specific function you want it for right now is so small. But once you finally do bring it in, everyone uses it.
The same thing happened when Java added lambdas: their convenience opened up a whole world of use cases. Those use cases were always possible, but suddenly they were actually being done.
The question is still, when is it worth it? I believe the best tools for identifying these are dog-fooding and user shadowing. Another factor is the aggregation size: how many users would benefit from the improvement.
All told this whole process takes non-negligible effort on our part to justify the value, so I generally only approach it when I’ve got an intuition that it’ll pay off. That intuition may be where we differ in our bar for what we believe is worth looking into.
I definitely agree about dog-fooding being a good tool! And that’s actually where a lot of my “batteries-included” improvements come from. For example, when I created a GitHub action for our CI/CD, I wanted to do something that seems like it should be pretty simple: compile a typescript app, klotho-ify it using a config I’ve checked in, deploy it. It ended up being 50 lines of copypasta yaml. As a user (of my own script, in this case), that feels annoying and undifferentiated; and that’s exactly the dog-fooding signal I honed on.
I think this is another area where we might have different viewpoints. I think 50 lines of copy/paste is pretty simple. Similarly in the Apache-Commons or Guava packages, those small functions could easily be copied in where needed (or even in-lined) without adding huge dependencies. Even small dependencies have risks, like the left-pad incident.
That said, the point does still stand that there’s a class of work where small improvements to a wide audience are still worth undertaking. How do you feel about the opportunity cost and the cost of measuring value?
I don’t have an exact rubric for how I approach these things. But in the GitHub action example, it’s also very easy to generalize for anyone else who has that kind of a vanilla app. Half of the configuration points are GitHub Secrets that I’d need to parameterize anyway, and the others are trivial to put in as env vars. Sure, you then need to write a separate GH action to invoke the more general one, but that’s a minute or two of work. All told, I think the cost is on the order of five minutes. At that point, I lean towards doing it.
Well, five minutes now, and then whatever other maintenance work that comes from it. We’ve been focusing a lot on the benefits, but we started this discussion by also considering the costs. Perhaps where we differ here is that I would rather wait for users to come to us to show what value it would provide them than to try to derive the cost-benefit estimation internally.
For the state that Klotho (the company) is in, I think that calculus virtually always lands in favor of doing the batteries-included work. Either nobody will care about it, in which case I’ve only wasted five minutes; or somebody does care and wants this-or-that extra feature, in which case the five minutes gained us a valuable customer interaction and helped us learn how people want to use the product.
I think this is a good way to think about it that I hadn’t really considered. When the cost is that miniscule, the calculus does indeed favour doing the work. Where it might get interesting is when that cost is higher — where’s the line? How do you determine where that line is?
That’s a really good question. Truthfully, I enjoy these kinds of things both as a consumer and as an author — so I’ll be heavily biased towards doing them. So for me, the first step in finding that line is to recognize my bias and raise the question to the team, by filing it as a ticket that we can discuss in triage. Then, the second part is for me to really listen to the team’s reaction, and especially people who weigh things differently. From there, I think it’s mostly subjective.
Regarding the maintenance, I guess the only concern is a noisy user that requires more support than it’s worth — but that could be addressed on a case-by-case basis.
Yeah, and it’s important to stay vigilant on that. More broadly, and to your point about the cost-benefit, it’s vital to keep an eye on the overall product’s north star. I tend to go down rabbit holes on ergonomics, so it’s definitely good that you keep me honest and remind me to always make sure the cost-benefit is actually there!
Cool! Good Convo!
Same! have a great day!