When will the LLMs be “done”? And was Gemini a mistake or exactly what Google was going for?
The botshit obfuscation field is growing.
The problem with LLMs is not dissimilar to a tech demo, MVP or prototype problem:
The appearance of completeness or level of completeness is almost invariably overestimated.
Additionally, the presumption that something is say, 80% ready, means that the remaining gap to completion somehow will occur at a predictable pace is deeply flawed.
I’ve had several conversations over the last couple of weeks where it was explained to me that this tech was “god-like” and that ChatGPT was the “culmination of every inventor that had ever lived”.
Meanwhile, reports started flowing in about gibberish (quite humorous) was flowing out of OpenAI’s LLM.
It was later patched and according to the postmortem, the following was stated:
“LLMs generate responses by randomly sampling words based in part on probabilities. Their ‘language’ consists of numbers that map to tokens”
Of course this shouldn’t be a surprise to anyone with the slightest inclination of curiosity about these models, but still — it needs to be said repeatedly in the face of Sam Altmans marketing spin and campaign for GDP levels of funding.
It is very possible that the complexity and very nature of the current systems will never actually be fixable, and that what we see is more extensive (and expensive) post-result edits, guardrails, RAGs, human intervention.
On Hallucinations:
Perhaps you’ve heard about that little gem..ini from yesterday:
Examples of flawed and miscalculated RAGs?
Clumsy guardrails and RAG may be unwieldy and underdeveloped, the Gemini story was not an alignment issue.
Saying all of this, however, to somehow try and explain away the militarized social engineering broadside attempted by Google is actually perhaps their best PR strategy moving forward.
Heres some great information on why RAGs aren’t going to get us out of this mess:
Now, it’s been said that the image generation issue with Gemini is based on ‘misalignment’ and ‘overly noble goals for diversity’. Here is some more interesting cases of how the Gemini model has been corrupted:
