ProgrammerHumor
youAreAbsolutelyCorrectIMadeItUp
Discussion
This is one of the things that bug me the most. Is it really that difficult to instruct the LLM to say "I am sorry, I don't know about that particular topic"?!
"I don't know" is not part of the training data set. it's literally just an extrapolation machine, going "if a gives f(a) and b gives f(b), then surely c gives f(c)"
If they can't find any data about it, then respond with "I don't know", I am sure they can make that work somehow
Thats not how NN work. They will fit the curve no matter how badly it fits.
But also ...;
When you figure out how to unilaterally make people respond like that, let me know.
You kind of can get a "I don't know" - but not super reliably - by measuring the model perplexity.
Basically you look at the probability distribution of candidate tokens and if the variance is high (aka confidence is low) then you warn the user about that.
That said, it's a quite brittle strategy since that perplexity can be high for reasons different to the model not knowing
If it can be high despite it knowing then it’d give more false positives than false negatives on admittance of not knowing about a topic, right?
I.E. if it doesn’t know about a topic its very likely to say so, but if it does know stuff it might still say it doesn’t. Seems like a fine enough solution to me, especially compared to whatever we have now.
The whole technology is based off of inference. If it responded with "I don't know" for anything not directly in its training data, it would just be a big hashmap; the whole use of prior data to extrapolate onto new data is the whole point of machine learning and AI in the first place.
And that's exactly why this approach can never work reliably.
A stochastic parrot is a nice toy, and you can use it to produce convincing bullshit, but it's definitely not the path to real AI.
Yes, it is.
Yes it is
Smt like claude and private llms ale do that.
There is no function in the LLM for him to determine that he doesn't know about the topic that he is not trained about or your very specific question. That is a function that people are still trying to add to him but it is really hard to do it. It's easier to make a function to return a response but not to determine that he doesn't know enough for him to return a response that says in short that he really doesn't know.
AI When you ask about anything
FTFY
please don't hallucinate bro they got my family 😭
You’re so right! Here is the correct solution…
Good catch! The correct solution is…
Spot on! Here is the corrected solution…
I wonder that not more people get tired by that and just accept that it does not work, and actually never will with this "token guessing" approach.
True.. it happens many times when AI gets confused with prompts and hallucinates basically.
GPT o1 almost went full Jihad on me for telling it that Zig's @intCast only takes one argument, and not two like it insisted it did. It did not back off at all.
The bullshitting is strong in that one.
It's futile to try to "argue" with an LLM. It will always just continue the prompts, with whatever it "memorized" from training, or what is hardcoded in system prompts.
It can't learn from the prompts as it can't reason.
Source? My source... is that I made it the fuck up!
So everything. It makes everything up. The chance that the made up thing coincides with reality is increasing. And that's what we perceive as artificial "intelligence" becoming better. There is still no understanding.
But it really get better at convincing you that this bullshit is true, making AI more dangerous and their users more stupid.
Definitely does better than my boss at this point