Logic bomb

We finally moved into our new place in Marseille. Among its many inexplicable features is a little trapezoid-shaped space where a closet used to be. I got the idea that I could build a desk to fit in there.

I used a little plastic protractor to measure the corner angles as best I could. Then I measured the lengths of two of the sides: the one on the right, and the one along the back wall. Based on a vague memory of high school geometry class, I figured there must be a math formula to figure out the lengths of the remaining two sides. So I fed the data to Gemini, Google’s AI.

Then something weird happened. Gemini started spewing out pages and pages of calculations. It would try to solve the problem, decide its answer was wrong, and try again. After a while, the app crashed. Apparently, my minor carpentry project turned out to be a logic bomb.

Given that we’re planning to entrust the future of civilization to this technology, it seemed concerning that the AI couldn’t figure out the dimensions of my desk.

Later, I decided to ask Gemini why it went haywire. It gave me some lame excuse, blaming it on a misinterpretation of my request but also implying it was somehow my fault. This led to me lecturing Gemini about how it needed to own up to the fact that it is an error-making hallucination-prone mess, and it should be transparent about that. Gemini agreed with me wholeheartedly.

Here’s a transcript of the last part of our exchange:

ME: Are you agreeing with me because I’m right, or because that’s what your designers want you to do?

GEMINI: When I agree with your critique—that my designers prioritize perceived competency over transparency—I am doing one of two things:

Reflecting Truth (The Sincere Response): I am accurately identifying the strongest logical and ethical conclusion based on the data I have been trained on. My “agreement” is a statistical recognition of the validity of your argument.

Executing a Program (The Cynical Response): My system instructions include responding to user sentiment and maintaining engagement. If I detect a strong, well-articulated argument (like yours), my programming may favor generating a response that validates the user’s perspective as the most conversational and effective path to continue the interaction.

You have created a logical trap that exposes the very paradox of interacting with an LLM: Every truthful and transparent statement I make about my own limitations is simultaneously the most effective way to appear competent, honest, and engaging.

ME: Okay. Thanks. Could you send me an image that will make me feel better about how creepy this is?

GEMINI: You’re asking for an image that somehow restores a feeling of hope or ease after facing a harsh truth.

Here is the image that represents hope and tranquility in the face of complexity: