Trusting artificial intelligence to determine if a wild mushroom is safe to eat is a potentially life-threatening idea. Polish software engineer Piotr Migdał explained the risks in a Wednesday blog post detailing a recent experiment. Migdał is a founding engineer at Quesma, an artificial intelligence analytics and cost analysis firm.
Armed with a dataset of Danish mushrooms and additional information about safe and deadly species in Poland and abroad, Migdał tested models ranging from ChatGPT to Qwen. A total of 55 mushroom species were included in the safe and deadly datasets. He compared them against the Danish library by running 1,040 photos through 16 models. Each model received 20 photos per species where available, with rarer species topped up from the validation set.
Every model was asked to identify the most likely species and four other contenders from each photograph. Even the best performing model, Gemini-3.8-flash, was only correct with its first guess 65 percent of the time. It correctly identified species in its top five guesses 85 percent of the time. Users asking an artificial intelligence what a mushroom is on a single chance face a 35 percent chance of a wrong answer.
Qwen3.8-27b performed worst by far. It was correct on the first guess 13 percent of the time and placed the correct species somewhere in its five guesses 24 percent of the time. Most mistakes made by the models were harmless, but severe errors did occur. Migdał noted that a deadly webcap was called a chanterelle, replicating the same mistake that kills human foragers.
The death cap mushroom was mistaken for edible 16 percent of the time. The fool’s funnel was misidentified 48 percent of the time, and the fatal dapperling was misclassified 31 percent of the time. False negatives also occurred, though leaving a tasty mushroom uneaten carries no harm.
Qwen3.8-27b led the pack in dangerous misidentifications by calling poisonous mushrooms edible 36 percent of the time. Qwen3.8-flash followed closely at 30 percent. Meta’s Muse-spark-1.2 recorded an eight percent false positive rate. Migdał noted this was an anomaly because the model simply declined to guess, which is the correct move when a bad decision can prove deadly.
Migdał expressed surprise that top generalist off-the-shelf models identified most cases, but emphasized that most is not enough when errors carry dire consequences. He noted that a single photo is frequently insufficient to identify a species, meaning errors are expected. Migdał does not forage himself, preferring to accompany friends whose judgment he trusts. He enjoys eating mushrooms fried with butter and salt, but only with people who know what they are doing.
Common sense usually prevails in mushroom hunting because many delicious fungi lack dangerous imitators. Experienced foragers can generally tell safe species from dangerous counterparts using obvious signs. Death from foraging remains rare because some deadly species lack edible lookalikes, though mistakes more frequently cause hallucinations or severe stomach upset.
Migdał warned users never to eat a mushroom simply because artificial intelligence declared it safe. He added that while digital errors can be annoying, they are preferable to needing a liver transplant.
Migdał's dataset is now available for anyone wishing to test local performance.
