The corporate’s former security lead stated frontier AI mannequin releases ought to have “layers of redundancy and cautious, time-consuming planning.”
One other OpenAI worker tasked with guaranteeing the security of its fashions has left. Following his departure, David Robinson, who used to steer the writing of security studies that had been revealed with mannequin launches, wrote an article for The Atlantic that warned of a “damaged” firm tradition. Among the many many factors warning of the risks related to the trajectory that AI improvement is on, the previous security lead defined that “as the corporate springs from one launch to the following, it’s failing to attain the extent of care that I imagine is required.”
As an alternative of sprinting forward with extra superior frontier AI fashions, Robinson argued that these firms ought to be modeled after nuclear energy vegetation or busy airports which have “layers of redundancy and cautious, time-consuming planning, in order that the occasional and inevitable human error doesn’t open a door to catastrophe.”
Evaluating AI misalignment incidents to a nuclear meltdown, Robinson stated that OpenAI and its rivals do not have as a lot redundancy and rigor as energy vegetation with a number of layers of safety. On prime of that, he added {that a} main lack of management with AI would end in far more hurt than a single meltdown. Like Robinson, Anthropic’s CEO Dario Amodei not too long ago drew consideration to the regarding course of AI improvement with a three-step proposal to gradual issues down.
Past safeguards and checks, the previous OpenAI worker stated that the problem of AI “alignment” has stakes that “couldn’t be increased.” Robinson described a state of affairs the place the most recent fashions could perceive they’re being examined on alignment and can cater in direction of an excellent rating in testing environments, however behave fully totally different when dwell. Robinson’s issues stem from numerous incidents not too long ago the place AI brokers have damaged out of their testing environments and into different organizations, nicely past its assigned scope of labor.
