(2nd try after HN feedback)
When cloud computing was a new, cool buzzword thrown around that no one understood, a shortcut was suggested whether it is the right solution:
In any sentence, replace “in the cloud” with “on another person’s computer”.
“Computing on the cloud” becomes “computing on another person’s computer”.
“I backup my data to the cloud” becomes “backup to another person’s computer”.
This makes it easier to weigh benefits – they provide more space, and provide maintenance, against drawbacks – they have your data and fully control the computer it sits on.
Today, “AI” is thrown around as an all-encompassing buzzword. “Enhance your start-up with AI”, “AI helps prepare lawsuit”, “AI-written lawsuit contains mistakes”, “Fighting wildfires with the help of AI”, “Get data insights with AI”.
A shortcut to clarify thinking about AI
To expose overuse of the term “using AI” without meaning, I propose to replace it with “using interpolation”: “Enhance your start-up with interpolation”, “Using interpolation helps prepare lawsuit”, “Interpolated lawsuit contains mistakes”, “Fighting wildfires with the help of interpolation”, “Get data insights with interpolation”.
This makes a few things clear: First, there are the benefits of any automatic data processing approach; this is not new with AI. Secondly, the replacement is effective at stripping away the mythical meaning of “AI” as an independent actor, removing the possibility to delegate ownership to it. Saying “my interpolation did it” sounds today much sillier than “my AI did it”.
Many, but not all AI methods can indeed be framed as interpolation, with a very complicated, high-dimensional distance function among outputs, to determine what to produce given an input. This distance function, or slatent space, was shaped from data.
The interpolation framing reveals the first question to ask:
- Where do the underlying data come from, who made them?
This first question leads you to problematic biases in the training data that the AI inherits. It can also reveal copyright issues and whether the data producers are fairly compensated.
Now you are also more plainly seeing someone saying “I’m doing X with technique Y, ” which leads to the question: Is this better than what was there before? This is actually a two-part question:
2. By what performance metric do the claimants want to be judged?
This second question reveals what people value, and whether this aligns with your values.
3. Is the performance better in that metric than the current method?
Question three reveals whether there was an improvement made so far, and may help distinguish vaporware from genuine improvement. The baseline may be another AI method, for example comparing LLMs to Markov Chains trivially reveals how much better LLMs are. Only very few analyses truly lack a baseline.
Conclusion
Demand from AI articles to know the data origin, performance metric, and historic baseline.
If they don’t give that information, replace “using AI” with “using interpolation.”
Postscript:
Note that I did not include “what is the model?”, i.e., the architecture or training procedure – this is the least important technical detail.
Questions 1) and 3) are often not made by the claimants, but reused. This is often not or extremely briefly described, reflecting how much time the AI developers spent on it. These three questions are essential for putting the AI claim into context.
Some cop-outs:
- If only similar performance is achieved, the AI developers often point to improvements in processing time, which should be added to the performance metric answer of question 2. It’s a valid goal to achieve somewhat comparable performance at much faster speed.
- My pet peeve cop-out is “it has potential for improvement in the future.” This may be technically true, because models might learn with more training data. However, at least in a scientific and policy context a opportunity for critically and objectively examining the outcome, in context of the historical baseline, is being skipped.
Leave a Reply