Much ink has been spilled on the black-box nature of AI systems—and how it makes us uncomfortable that we often can’t understand why they reach the decisions they do. As algorithms have come to mediate everything from our social and cultural to economic and political interactions, computer scientists have attempted to respond to rising demands for their explainability by developing technical methods to understand their behaviors.
But a group of researchers from academia and industry are now arguing that we don’t need to penetrate these black boxes in order to understand, and thus control, their effect on our lives. After all, these are not the first inscrutable black boxes we’ve come across.
“We've developed scientific methods to study black boxes for hundreds of years now, but these methods have primarily been applied to [living beings] up to this point,” said Nick Obradovich, an MIT Media Lab researcher and coauthor of a new paper published last week in Nature. “We can leverage many of the same tools to study the new black box AI systems.”
The paper’s authors, a diverse group of researchers from industry and academia, propose to create a new academic discipline called “machine behavior.” It approaches studying AI systems in the same way we’ve always studied animals and humans: through empirical observation and experimentation.
In this way a machine behavorist is to a computer scientist what a social scientist is to a neuroscientist. The former looks to understand how an agent—whether artificial or biological—behaves in its habitat, when coexisting in groups, and when interacting with other intelligent agents. The latter seeks to dissect the decision-making mechanics behind those behaviors.
“We’re seeing the rise of machines with agency, machines that are actors making decisions and taking actions autonomously,” Iyad Rahwan, another Media Lab researcher and lead author on the paper, said in a blog post accompanying the publication. Thus they need to be studied “as a new class of actors with their own behavioral patterns and ecology.”