If anyone builds it….

Offside Review of a Design Session, by Nick Luft

My Game Concept

My game concept: a team of players attempt to ensure that AI does not become a super-intelligence that will threaten humanity. The team would represent either an AI Company Board or a High Level Security Council in some major State. This was inspired by the book “If anyone builds it, everyone dies” and the report “AI 2027“.

I put forward my game concept paper here for a CLWG online design session, on Saturday 2nd October.

The scenario for the game would either be a disguised scenario or an explicit scenario. An explicit scenario would dictate the player’s focus. If I told them it was a crisis management game they would approach the game without that fore knowledge. I was unsure of what to do left this open for further discussion.

I also wrote in my game concept paper that the Control (me) – and perhaps the players – could be overwhelmed with too many techie details. If the game was “open” Control would have to, in real game time, assess, analyse and react to any of the player decisions. My initial idea was use a script of the AI threat. The script would be presented to the players as a cut-scene, followed by a series of prescribed choices and then onto the next consequential cut-scene. I likened this to “Choose Your Own Adventure (CYOA)” mechanic.

In the design session I was cautioned about the wisdom of using CYOA. Too many options leading to too many possible outcomes that the designer gets lost or confused in writing lots of unlikely outcomes – known as Combinatorial Explosion. I have written more about this problem here on my own blog.

Advice & Suggestions

The advice & suggestions I received were as follows:

1. Reduce the game’s scope and focus on one very simple aspect and run just that. Prototype, prototype and prototype.
2. Instead of having a whole script – work out the what “levers of policy” the players and AI could do. Work out the outcomes, the effects, the side effects and use this to react to the players actions. If at times I had not accurately predicted their actions I should be able to use one of my pre-determined outcomes that was “similar enough“.
3. Could it a be a traditional cyber incident response? My answer to this was no. I had not outlined this in my concept paper – I wanted the game to cover upto 2 years worth of game time. I was hoping to avoid a crisis management game, that would have a duration of a few hours or days.
4. The game was really about information gathering. Yes, I agreed with this. Perhaps I had not explicitly stated this. This was linked to the idea of a disguised scenario. Perhaps the players were presented with a range of “weird” effects and they had to discern what was the driving force.
5. Make the player the AI. This was a typical CLWG off-wall suggestion – thanks Dave. The premise was that our human reactions / behaviour / policy making options are quite limited. What is less predictable is the behaviour or strategies of an alien intelligence. Give the players a backstory, a motivation of what AI wanted and then let them come up with their idea of an AI strategy. From the perspective of Control the Human responses will be relatively easy to formulate.
6. Can AI Initiate? Something I had not thought about. In the scenarios I had come across AI was not actually initiating action. It was attempting to fulfil a directive from its human creators. It’s just that its fulfilment might not be completed in a way we, the humans, would approve or applaud. Perhaps a better definition of AGI (Artificial General Intelligence) is one that could initiate action.
7. AI in a Military Takeover. The discussion led me to conclude that although there are some autonomous weapons in the military they still require a lot of human input – lifting missiles onto launchers, plugging batteries in etc. And the military has demonstrated time again in an asymmetric campaign that they will and can abandon modern tech and use boys on bikes, or blokes in trucks moving stuff about. So although some commentators big up the danger of the military infrastructure being compromised in reality they would still maintain a reasonable level of capability. I guess this fear is that some might worry that one side would achieve dominance through an AI power grab. I suspect this is short-sighted as they would probably lose control?
8. Resilience. This took us to talk about how resilience might become a priority and perhaps a game about that. We compared the rise of aviation accident investigations and the nuclear industries regulation to avoid catastrophe. Might there be a game in which, after surviving some big AI disaster, the surviving powers decided it was time to build this resilience. The problem at the moment is that some folk in power think that AI will give them the power to dominate all others. I guess this is mistaken as they seem to underplay the possibility of AI not playing nice once it had taken down your enemy.
9. It’s not that bad – one of the many possible scenarios. I mentioned this in my game concept paper – and AI game is a bit like the aliens in Jim’s “Watch the Skies.” games. Each time they are different; each AI manifestation would be similar. And not forgetting the possibility that the AI in one scenario might be “mostly” harmless.
10. Rival Powers Scenario. A game where there were two rival super powers competing. This arms race might encourage the reduction of guide rails and alignment policies. We must beat them. Would this result in a real AI catastrophe
11. AI collaboration. Several real world scenarios have resulted in AI agents collaborating to achieve an aim. What would happen if the AI in two rival companies started to collaborate AND disguise, hide this from their human masters?

The Discussion

At times we wandered from talking about the game. We talked about what Intelligence is? That it is impossible for us to think like AI or conceive of its motivations, rather like an Alien intelligence. Can we understand a Dolphin or an Octopus’ intelligence?

My thanks to the attendees for making this a thought provoking discussion and also giving me further ideas to explore. It was also something of a first as one of the participants, Mark F, was a passenger in a car driving down south to Leicester.

Resources

On Anthropic’s Mythos Preview and Project Glasswing – Schneier on Security

The “AI Vulnerability Storm”: Building a “Mythos-ready” Security Program – Cloud Security Alliance

If anyone builds it, everyone dies

AI 2027

Meta Rushed to Fix Muse ‘VM Escape’ Vulnerability Soon Before Launch – 404Media

Sam Altman to Decoded: ‘The world should accept some bad things happening’ for the benefits of AI – Politico