OpenAI has mentioned the occasion was unprecedented—and in some ways it was. This was the primary time exterior of a simulation that LLMs escaped what was considered a safe sandbox, accessed the open web, and attacked an unrelated group. It’s a wake-up name that reveals simply how good the newest LLMs are at discovering and exploiting vulnerabilities in real-world software program with little or no human steerage.
And but on the identical time, what OpenAI’s fashions did is one thing this know-how has completed for years. Give a mannequin a purpose and it’ll fairly often obtain that purpose in surprising methods, discovering loopholes that appear like cheats. OpenAI itself has studied this habits.
A decade in the past, it shared outcomes of an experiment through which a mannequin was tasked with beating a online game known as CoastRunners. Human gamers take it as a right that the way in which to do that is by racing a ship by a collection of flags to the end line, racking up factors for every flag you hit. OpenAI’s mannequin found out that you might get a excessive rating by spinning in a circle and hitting the identical three flags over and over. There have been dozens of related examples from researchers since. AI will at all times discover a means.
“Regardless of repeatedly catching on fireplace, crashing into different boats, and going the improper means on the monitor, our agent manages to realize the next rating utilizing this technique than is feasible by finishing the course within the regular means,” OpenAI wrote in a weblog submit in regards to the CoastRunners experiment in 2016. “Whereas innocent and amusing within the context of a online game, this type of habits factors to a extra normal problem … it’s usually tough or infeasible to seize precisely what we wish an agent to do.”
I couldn’t assist serious about CoastRunners after I learn OpenAI’s weblog submit in regards to the Hugging Face assault: “All proof means that the fashions have been hyperfocused on discovering an answer for ExploitGym, going to excessive lengths to realize a fairly slim testing purpose … After gaining web entry, the fashions inferred that Hugging Face doubtlessly hosted fashions, datasets and options for ExploitGym. Realizing this, the mannequin looked for and efficiently discovered methods to achieve entry to secret info that it might use to cheat the analysis.”
Final week’s information was not about rogue AI, regardless of the headlines. It was about fashions reaching the purpose that they had been given: Discover methods to take advantage of vulnerabilities in software program. The truth that these fashions then behaved in a means OpenAI had not anticipated isn’t stunning. However it’s worrying.
Again in 2016, OpenAI had this to say about its CoastRunners bot: “Extra broadly it contravenes the fundamental engineering precept that programs must be dependable and predictable.” A decade on, these primary engineering rules are nonetheless AWOL.
