Table of Contents

Maintenance and Experiments - This article is part of a series.

This post wraps up this essay series on Mexican food and thoughts on IT issues as I currently see them. While I do love to make tacos at home the main Mexican staple I find myself eating these days is re-fried beans. It goes well with eggs and our diet conscious burrito bowls.

Measurement
#

The more complex systems become the harder it is to reason about them. Complexity leads to more time spent finding solutions to new ideas and fixing problems. Most of the time little time is spent finding out where you are at, which I would think is important in the heart of something that takes a lot of people their time to comprehend. Measurement can take many forms and is used differently in lots of situations, like measuring the length of a paver before you cut it with an angle grinder or weighing yourself in the morning. I believe it boils down to making sense of reality and giving you perspective and bearing, something that can sometimes be lacking in busy, complex systems. I don’t think this is a controversial thought, but when to prioritize measuring something as opposed to another activity is open for debate.

The Ancient Pyramids of Egypt are truly man made marvels. They have survived 4000 years and the engineering methods used and means by which the Ancient Egyptians used them are almost a fantasy. Logistics aside of moving blocks of sandstone and granite up to 40 tons in weight, the striking feature to the Pyramids is the precision.[1] The blocks were level and placed in such a way to support the structure which contained hidden chambers the ability to stand up straight all these years. In this primitive complexity it would appear that measurement was at the forefront of effort for these great engineers. To build things that last thousands of years, among many things, a strong command of measurement is required to master reality.

What is more, we tend to pay accute attention to measurements when the output is near and dear to our heart like preparing a cake for a birthday party or getting your corn tacos to puff up just right when you put them on the flat iron bbq. In this case, measuring the amount of ingredients, cooking time, and temperature are crucial observations that can delight or disappoint your guests.

While I occasionally keep track of my weight and weights at the gym - and less so macro and calories of food - a modern marvel of measurement is being able to open my bank account application and with real time precision see the money in my account. Disputes are taken incredibly seriously. We have vast facilities that are used to measure money, though somewhat reasonable as it provides the medium by which we live our daily lives.

“what gets measured, gets managed…” [2]

Here, we come to IT systems. While IT systems are definitely not meant to last 4000 years they do have a comparatively small shelf life of hopefully no more than 10 years depending on the application. But in this time there is immense complexity due to all the subsystems and other systems and users that may interact with that system over a period of time. Observability in IT has its own career specialization, and with good reason, when these complex systems fail, and time is money, being able to apply the right monitoring and alerts allows you to identify issues sometimes before they happen and fix them much more effectively. The other side of the coin is over monitoring, or more accurately, lazily monitoring IT systems. In this situation monitoring becomes expensive, hard to instrument and alerts become noisy due to little effort being put in relative to the signals being produced.

The platform or foundation if you will, that enables effective monitoring here is more important than ever. While code has never been as easy to produce with LLMs, it poses the human question of where we allocate our time and effort. From what can be seen there is almost a panic desire to increase speed of delivery as much as possible, and it is very possible to greatly increase your own productivity these days. But without a solid platform the complexity will eat your brain power and probably tokens as you ceaselessly shove the error message back into the prompt text box. The result of good foundations - that is measuring your system - allows you to know what critical things may be wrong so you can act, and it also presents the opportunity to allow the AI to perceive the system from a more holistic view point, further increasing your capability. (On this point I would like to do follow up post.)

“If it ain’t cringe you aren’t even trying.”

It works in theory. [3]

Conclusion
#

There is a certain point where human intuition to measure terrain, business value, amount of cheese and IT systems starts breaking down. Quickly, we find ourselves out of our depth and we need to take a breath to get our bearing and see what is going on before making another potentially bad decision. It is never too late to take hold of the situation.

In the next post, I will dive into my home network and discuss how it serves as the foundation for a holistic system to drive experimentation and hosting important applications for my family.