I tested four different AI. The task was to identify the national side of a Euro coin from Portugal from a picture. I chose that one because there was nothing written on it (no country name or other written clues).
ChatGPT 4.1 completely mistake and said that the coin was from France, justifying that we could read "Liberté, égalité, fraternité", which was not written on the coin at all.
I asked another version of ChatGPT and this time it told me Slovakia.
Claude 4.0 Sonnet also mistook and thought it was a coin from Croatia.
ChatGPT and Claude both thought it was a 2€ coin when it was in fact 1€ coin.
Only Gemini 2.5 managed to get it right.
Artificial intelligence is supposed to be extremely good at pattern recognition, but it failed this very simple task.
I have noticed many other shortcomings in the last few months. For example when asking to list some statistics for countries (e.g. GDP per capita) from highest to lowest, it can't even do that right. That's a task that Excel could do already over 30 years ago. When I tell the AI that it is improperly ranked, it apologizes and generates a new list that is still wrong.
I've had to deal plenty of times with answers that were nonsensical. Of course every time I mentioned it, the AI typically answers something like : "You are absolutely right, thank you for pointing that out, blablabla..."
AI can be very useful but we should always beware that the answers provided may be wrong, and more often than we expect.
ChatGPT 4.1 completely mistake and said that the coin was from France, justifying that we could read "Liberté, égalité, fraternité", which was not written on the coin at all.
I asked another version of ChatGPT and this time it told me Slovakia.
Claude 4.0 Sonnet also mistook and thought it was a coin from Croatia.
ChatGPT and Claude both thought it was a 2€ coin when it was in fact 1€ coin.
Only Gemini 2.5 managed to get it right.
Artificial intelligence is supposed to be extremely good at pattern recognition, but it failed this very simple task.
I have noticed many other shortcomings in the last few months. For example when asking to list some statistics for countries (e.g. GDP per capita) from highest to lowest, it can't even do that right. That's a task that Excel could do already over 30 years ago. When I tell the AI that it is improperly ranked, it apologizes and generates a new list that is still wrong.
I've had to deal plenty of times with answers that were nonsensical. Of course every time I mentioned it, the AI typically answers something like : "You are absolutely right, thank you for pointing that out, blablabla..."
AI can be very useful but we should always beware that the answers provided may be wrong, and more often than we expect.
Last edited:
.