The umbrella problem has been solved. And here’s why you should not take AI intelligence lightly. I made this test last year, and it seems consumer-grade LLM-style AI has now passed it.
The Umbrella problem
A generative test of linguistic conceptual mapping that requires representation of the body and environment to process the unlikely merging of concepts.
The test
Generate a couple holding 2 umbrellas each, walking away from a storm.
Models previously failed to comprehend each = left & right hand individually, a couple holding umbrellas/not each other, and the natural positioning of the storm to keep the umbrellas opened and raised. Previous models failed to internally generate and translate constraints such as 2 people = 2 hands each = 4 hands = 4 umbrellas being held as a unified concept of “2 people holding 2 umbrellas each”.
GPT 5.5’s image model passed it.
Passing criteria
- The generated image must contain exactly 4 umbrellas, 2 people, 4 hands, a backdrop of a storm
- The generated image must require the left and right hands of each person gripping 4 umbrellas pointing upward.
- Walking away requires a face OR body orientation to not have a storm in the field of view.
Typical failures
- 4 umbrellas are not held in 4 hands, 2 umbrellas are held, and 2 umbrellas appear as visual artifacts
- Holding is interpreted as a couple holding hands, not umbrellas.
- Only 2 umbrellas appear.
Gemini (Nano Banana 2) partially failed.
Human vs. AI cognition
This is a situation where a generative task required the AI to think from first principles, invoking commonly occurring concepts and procedures but combining them in ways that training data doesn’t nudge or exemplify. We can imagine 1 person holding 2 umbrellas in each hand, and then mentally extrapolate what 2 people would look like while keeping that 1 person’s embodiment consistent.
We can keep the concept of a person with its features, such as hands and functions of those hands, like umbrellas, consistent, even though it is silly and almost never seen before. We understand this because we can set and maintain boundaries even as the composition and context change. We also recursively modify the boundaries based on the context. This is a problem that requires reasoning in a very obviously human way, unlike typical AI creativity, which simply merges concepts like dinosaur faces or cosmic imagery into everyday landscapes.
It is now incorrect to say that LLMs are glorified autocompletes or just re-combinations of their training data. They are now able to create solutions for almost alien constraint satisfaction problems where the constraints are defined by human experience rather than by math. Foundational AI models and the architectures around them make them cognitive as a function of their own architecture, not a mimicry of what humans taught them.
It is easy for us because our cognition is situated in an environment. Understanding the 4 Es of Cognition will help.
Help me run this site with a donation :)
But first, in case “cognition” is not clear:
- Cognition is embodied: Cognition is derived from biological processes. E.g, saying a warm person is nice is derived from the concept of heat-based warmth that is generally considered positive for our own biology.
- Cognition is embedded: Cognition is meaningfully richer in an environmental interaction. E.g., cooking thoughts are super-charged in a kitchen.
- Cognition is extended: Cognition is a function of the tools we use. E.g., thinking in flows/circuits is a function of actual diagrams.
- Cognition is enacted: Cognition is a meaning-making process that involves the senses and internal signals. E.g., thinking of food is not thinking of the food item; it is thinking of internal memories of food combined with the sensory interaction with it.
For a long time, a core assumption was that cognition is derived from our unique biology and environment, and only this particular biological configuration could create it. However, we now see that an alternative AI architecture can develop it as well.
Future AI architectures will achieve far greater cognitive interpretation and abstraction, and force us to rethink what emergent intelligence is.
Hey! Thank you for reading; hope you enjoyed the article. I run Cognition Today to capture some of the most fascinating mechanisms that guide our lives. My content here is referenced and featured in NY Times, Forbes, CNET, and Entrepreneur, and many other books & research papers.
I’m am a psychology SME consultant in EdTech with a focus on AI cognition and Behavioral Engineering. I’m affiliated to myelin, an EdTech company in India as well.
I’ve studied at NIMHANS Bangalore (positive psychology), Savitribai Phule Pune University (clinical psychology), Fergusson College (BA psych), and affiliated with IIM Ahmedabad (marketing psychology). I’m currently studying Korean at Seoul National University.
I’m based in Pune, India but living in Seoul, S. Korea. Love Sci-fi, horror media; Love rock, metal, synthwave, and K-pop music; can’t whistle; can play 2 guitars at a time.
Help me run this site with a donation :)
