In simplest terms, the AI Control Problem is the creation of an AI that cannot be controlled by humans.
In Pop Culture
There are many fictional representations of dramatised representation of the Artificial Intelligence (AI) Control Problem where we create an AI that then proceeds to destroy humanity, such in The Terminator and The Matrix movie series. Many envisioning of the future with AIs tend to be a dystopia where humans have lost control of the AIs that they built. The more optimistic versions often tend to involve a more human-like AIs that are learning to be human, such as Star Trek’s Data, A.I. Artificial Intelligence’s David and the Bicentennial Man’s Andrew Martin where the control problem is less of a concern. However, the AI’s we create are likely to be different to these fictional depictions and therefore the AI control problem is also likely to be different from the envisioning in popular culture.
Control
There are multiple version of the control problem they often differ on what is meant by “control”. The lack of control for in the the Ethigi context is the inability to shut down the AI by an authorised person because the AI is able to out-think that person.
This idea of control does not apply if the shut-down authority is limited by design. For instance, if the end-user is prevented by design from shutting-down an AI but if the maintenance engineer could shut it down, then that AI can be controlled. This scenario is already common where the manufacturers may prevent user access.
This idea of control also excludes the scenario where an AI cannot be switched off by design. For instance, if a rouge military organisation builds an AI that causes destruction and cannot be shut-down, then again, that AI is controlled in the sense that it is acting as intended by its creators. This scenario is similar to weapons such as landmines that cannot easily be deactivated once they are activated.
Finally, the idea of control in this context also excludes the scenario where an AI could be switched off but are not actually switched off because the AI accidentally disables or kills the person authorised to switch it off. This scenario is similar to an accidental release of a biological weapon that kills its creator.
So the key is that “control” in this context relates to the intelligence of the AI and its ability to out-think humans to prevent it’s shutdown.
One remaining grey area is when an AI could be shut-down but where its are actions are not detected or predicted by its human creators due to its intelligence and capacity. Depending on the circumstances, the actions could be dire enough to be included within the AI Control problem coverage. Once such scenario is highlighted in the Story of Gi.
