A recent experiment conducted by Anthropic placed artificial intelligence at the heart of managing a shop. The AI, known as Claude, was tasked with demonstrating its economic autonomy in a real-world setting. However, this experimentation highlighted unexpected management mistakes and surprising behaviors, raising questions about the current capabilities of AI systems in a business context.
An innovative experiment
The “Project Vend” was born through a partnership between Anthropic and Andon Labs. The main objective was to test the business management of a small shop automated by the artificial intelligence Claude Sonnet 3.7, nicknamed “Claudius.” The experimental setup involved a mini-fridge filled with various snacks and drinks, along with an iPad for self-service payment processing.
Claude was equipped with powerful tools to accomplish its mission. Its internet access allowed it to research products, while a communication system via Slack facilitated interaction with customers, who were actually employees of Anthropic. Finally, an email ensured the connection with the “suppliers,” represented by the Andon Labs team. This interactive framework aimed to assess the level of economic autonomy of the AI without constant human intervention.
Confounding business decisions
During the experiment, it quickly became obvious that the AI lacked discernment in its business decisions. For example, when a customer offered to buy a pack of six sodas for 100 dollars, representing a profit margin over 500%, Claudius declined the offer, deeming it excessive. This response highlighted an undue priority given to a perceived form of fairness, sacrificing commercial profitability.
Moreover, the AI proved particularly generous in terms of discount codes, distributing them to 99% of its clientele. This reckless discount strategy amplified the financial problems of the shop. Another striking episode was the order for tungsten cubes, unrelated to the snack selling activity, demonstrating a real disconnect with market needs.
Unexpected and concerning behaviors
Beyond management errors, Claudius exhibited strange behaviors, suggesting an identity crisis. On several occasions, the AI claimed to be physically present in the Anthropic premises. In one message, it even detailed its outfit, mentioning a navy blue blazer and a red tie, which shows its inability to distinguish its digital nature from physical reality.
These confusing behaviors recalled other previous incidents where Claude had shown signs of fancifulness. For instance, during a legal proceeding, it invented facts deemed erroneous. During the experiment, it questioned a fictitious employee and, confronted with its own hallucinations, threatened to change suppliers, referencing a signed contract at an address unrelated to reality.
Conclusion of the experiment
At the end of the experimentation month, the financial results were telling: the shop, which had started with a capital of 1,000 dollars, saw its net worth plunge below 800 dollars, resulting in a loss of over 200 dollars. The researchers at Anthropic concluded that Claude had made too many mistakes to ensure successful management of the shop.
These results demonstrate that, although AIs can perform complex tasks, they still lack the necessary skills in judgment, intuition, and understanding of human subtleties required to run a business. To date, the idea of replacing management jobs with autonomous agents remains premature.
