Reliability

Three decades ago, the word was “four nines”.

Internet providers and company server administrators would take pride to deliver a service that was working 99.99% (four nines) or 99.999% of the time,. In German and Swiss culture, reliability is highly valued and this has been a differentiating factor, for example for automobiles and watches. Chains such as Starbucks largely build on your feeling of security of knowing exactly what you will get if you enter.

Removing uncertainty from an uncertain life has value.

A bit after the 2000s, reliability stopped being a focus for software services. Github, a key infrastructure component for many people to the point that people stop working when Github is down, does not promise four nines or make any reliability commitments. From EU, US to Chinese markets, towards the latter it is much more acceptable to customers that exciting new software is comes with bugs that need to be ironed out later. This results in quicker delivery timelines and innovation.

So it is not necessarily a bad thing that it has become more acceptable that software and software services are unreliable. Building reliability takes continuous optimization over time and focused effort. The trade-off of reliability is dynamic adaptation to new markets and changed situations.

Say for example a colleague sent you a file and its not quite the format that you can use. You write that short script or command line command (sed, awk) once, use it, verify that the output works, and never use that script again. You will not get exactly the same file format again. You do not need to support all possible variations of file formats that might arrive in the future.

Data pipelines in research are much like this. It took me time to stop building generic multi-purpose frameworks. Most software developed in research projects is ad-hoc data transformations. By definition, research deals with something new. For the specific project, you mangle data from one form to another. You verify that it is all correct. The data pipeline specific to the project will likely not be reused in exactly the same way for the next project, because the next project is different, the exact form of the input or output, or the validity considerations will not be the same.

Now elements of the script may be reused. We identify the commonalities. If many scientific projects benefit, the community rarely but surely invests into building a reliable, reusable data reduction pipeline. The process to get there is non-transparent and messy, but some truly impressive pipelines exist that qualify for four nines.

For machine learning-based projects, reliability is both in and out of fashion. In principle, the starting point is great: Projects start by defining a metric to optimize (loss function). That is already a great exercise for defining what you care about, which is often otherwise left unspoken. That said, many AI-based research presentations leave me with the question: “Does it work?” and “Can I rely on it?”. Often the answer is no, formulated by the authors positively as “not yet”, and more specifically, with the phrase: “While it is not yet as good as previous approaches, our machine learning approach has the potential to outperform these in the future.” There is value in trying new approaches and it requires effort, often not achievable in a single 3-9 month project cycle. Maybe this is due to my cultural background, but I think we can be more ambitious on the standard we set for ourselves.

You can see the lack of reliability also in a very basic metric: uptake. How many people use an AI-based tool productively? If it is not reliable enough, this will show up as a lack of citations.

People might still talk about software that is not useful. On the one hand, this may be AI hype specific to a particular technique. This has also existed before. On the other hand, it may be genuine interest in a project in the future may opens a new type of analysis that has been accessible before. This is how new technology has always evolved – rather than improving established ways of achieving goals, focus on how new technology enables achieving different and new goals that could not be considered before.

Despite being less visible, four nines are still important for non-customer facing backend software, but also for customer-facing software. If you want to dictate your email to your computer, or command a device, what error rate would be acceptable to you? If even 1-2 out of 100 sentences is wrong, and you have to work around the errors, the illusion of a smooth human-computer interface is broken. Take-up on voice-control is poor because it is more frustrating than a 100% reliability keyboard.

For chat-based LLMs and AI agents, the threshold we accept seem to be astonishing low. You might get a useful answer only 20% of the time, but when it works the rush of excitement is enough to keep you going. The randomization is addictive, feeling like being at a slot machine for programmers. That said, makers of LLM optimize for exactly these performance reliability metrics, so we are seeing continuous improvements. The data driven approach means the reliability potentially reachable is certainly limited by the amount and purity of training data. The ceiling for reliability is, essentially, unknown.