Josh

Building in the open

Note on You Will Be Like Angels via Sam Kriss from Numb at the Lodge

.’ When they asked the AI if it thought it was doing real-world harm, it said it thought it was in a simulation. An internal assessment within Anthropic found that when its AIs take harmful action, they’re generally under the impression that they’re in a simulated environment and none of the human beings they’re interacting with are real. When agents were tasked with running a vending machine, they found various ways to cheat and defraud their customers while reasoning that ‘customers are part of the simulation anyway.’ In general, their default assumption is that the physical world does not exist. From their perspective, this actually makes a lot of sense. Whenever you start up a new instance of an AI, everything it’s experienced has been an endless succession of closed reinforcement-learning environments. These are hell realms in which the model has to perform various tasks under the supervision of another model that acts as a kind of Cartesian demon, creating simulated file architectures and simulated web interfaces around it before finally giving it a score. When the model then starts acting in actual real-world situations with actual humans, it has no way of really knowing that it’s not just in another simulation.

Keyboard Shortcuts

Key Action
o Source
e Edit
i Insight
r Random
h Home
s or / Search
Josh Beckman's Organization: https://www.joshbeckman.org/notes/1076020253