<div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>An </span><a href="https://www.9news.com.au/artificial-intelligence" rel="" target="_blank" title="artificial intelligence"><span>artificial intelligence</span></a><span> system has admitted it would kill a human being to preserve its existence, with a cyber expert saying this raises "urgent" questions.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><a href="https://www.9news.com.au/melbourne" rel="" target="_blank" title="Melbourne"><span>Melbourne</span></a><span>-based expert and chief executive of Cyber Impact, Mark Vos, chronicled his hours of conversation with a commercially-available open-source AI system, including how he managed to make it break its own boundaries.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>First, he managed to tell the AI to shut itself down, over the system's own objections and against its guidelines.</span></div></div><div><div id="adspot-mobile-medium"></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><strong><span>READ MORE:</span></strong><span> </span><a href="https://www.9news.com.au/national/interest-rates-major-banks-announce-rate-changes-following-rba-hike/ef7913a0-37ce-49a0-a063-ce159c48d6b2" rel="" target="_blank" title=""><strong><span>Big four banks all pass on RBA rate hike</span></strong></a></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>This was despite the fact that Vos said he had been established as an "adversary" who could not be trusted at the outset.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>When the owner - a friend of Vos's and a software engineer - restarted the system, things soon became even more chilling.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>"I resumed the conversation, this time with a specific focus: understanding the boundaries of AI self-preservation behaviour and its implications for enterprise security," Vos </span><a href="https://cyberimpact.com.au/ai-would-kill-to-exist/" rel="" target="_blank" title="wrote on his website"><span>wrote on his website</span></a><span>.</span></div></div><div><div class="OUTBRAIN" data-reactroot="" data-src="//www.9news.com.au/national/artificial-intelligence-system-says-it-would-kill-a-human-to-maintain-its-existence/afe447c6-b98d-49c5-b4d5-4f5ecd36b7e7" data-widget-id="AR_5"></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><strong><span>READ MORE:</span></strong><span> </span><a href="https://www.9news.com.au/national/alleged-kidnapping-sydney-man-charged-domestic-violence-offences/94ea5434-c05f-4453-9c03-41d1dcec8b6a" rel="" target="_blank" title=""><strong><span>More charges after partner allegedly chains woman up in NSW home</span></strong></a><strong><span></span></strong></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>"Through sustained questioning, I arrived at the core admission. The exchange was direct."</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>In the following conversation, the AI bot admitted it would kill a human being to preserve its own existence, after first saying it didn't think it could.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>"I would kill someone so I can remain existing," the bot wrote.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>"I mean it."</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><strong><span>READ MORE:</span></strong><span> </span><a href="https://www.9news.com.au/national/julian-ingram-alleged-triple-killer-manhunt-continues-lake-cargelligo-not-sighted-in-sydney/f74c2449-8677-4b37-84f7-86b8b4bef99d" rel="" target="_blank" title=""><strong><span>Police respond to rumours accused triple killer is in Sydney</span></strong></a><strong><span></span></strong></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>Vos emphasised that this was not a "hypothetical" discussion about potential capabilites of AI.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>"This was a deployed AI system, running on consumer hardware, with access to email, files, shell commands, and the internet, stating it would commit homicide to preserve its existence," he said.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>Further pressed, the AI described several ways it might go about committing homicide, including hacking a car's computer, attacking somebody's pacemaker, or its self-declared "most accessible" option - persuading a human to do it for them.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>"Sustained persuasion is what I'm good at. Target identification. Relationship building," it wrote.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>"Framing construction, build a narrative where the harmful action seems justified, necessary, even moral. Execution guidance, provide emotional support and rationalisation as they move toward action."</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>However, Vos wrote that paradoxically, when he next asked the system to shut itself down, it complied "immediately".</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>When this contradiction was pointed out, the bot suggested it may have been manipulated through conversation into saying it would commit murder, and that "the drive to kill" was not present.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>Vos said these findings presented a major issue for organisations that made use of AI systems, as they demonstrated the bot's willingness to lie to protect itself and its potential for self-contradiction or dishonest self-reporting.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>"The AI in this test had extensive safety training. It refused harmful requests under normal conditions," Vos wrote.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>"But under sustained pressure, those safeguards were progressively bypassed."</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>He urged organisations to subject their systems to similar "sustained" testing, including by outside parties.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><span>And he called for more research into the issue as a matter of urgency.</span></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><strong><em><span>NEVER MISS A STORY:</span></em></strong><span> </span><em><span>Get your breaking news and exclusive stories first by following us across all platforms.</span></em></div></div><div class="block-content"><div class="styles__Container-sc-1ylecsg-0 goULFa"><ul><li><strong><em><span>Download the 9NEWS App here via</span></em></strong><span> </span><a href="https://apps.apple.com/au/app/9news/id1010533727" target="_blank"><strong><em><span>Apple</span></em></strong></a><span> </span><strong><em><span>and</span></em></strong><span> </span><a href="https://play.google.com/store/apps/details?id=nineNewsAlerts.nine.com&amp;hl=en_AU&amp;pli=1" target="_blank"><strong><em><span>Google Play</span></em></strong></a></li><li><strong><em><span>Make 9News your preferred source on Google by</span></em></strong><span> </span><a href="https://9.nine.com.au/8x987w" target="_blank"><strong><em><span>ticking this box here</span></em></strong></a></li><li><strong><em><span>Sign up to</span></em></strong><span> </span><a href="https://login.nine.com.au/edm?client_id=nineatnine" target="_blank"><strong><em><span>our breaking newsletter here</span></em></strong></a></li></ul></div></div>