Pentagon Backed Research Employs AI to Detect 'Social Norm Violations' in Written Text

JJohn July 26, 2023 1:26 AM

A Pentagon-supported research indicates artificial intelligence's potential to scrutinize and evaluate extensive text to determine if its human authors have committed any transgressions. Pioneered by two Ben-Gurion University scientists, the study utilizes predictive models to examine messages for 'social norm violations' using GPT-3 and zero-shot text classification.

Using AI to spot 'Social Norm Violations'

The study utilizes an advanced language model, GPT-3, developed by OpenAI, and a unique data parsing method called zero-shot text classification. The combination allowed researchers to identify and categorize instances of 'social norm violations' throughout numerous text messages. The premise is based on the idea that, despite cultural differences, human reactions to norm violations may be similar.

Identifying and Teaching Social Norm Violations to AI

The project's primary objective is to automatically pinpoint social norms and the associated violations. However, this is a complex task because it first requires the identification of signals suggesting a social norm violation. For instance, turning up to work inebriated and unkempt would be a violation of a commonly accepted norm among most workforces. Yet, teaching a machine to recognize and flag such behavior as a norm violation is not straightforward.

The scientists' approach to identifying social norm violations is to concentrate on a narrow selection of 'social emotions' that indicate a violation, particularly guilt and shame. They strived to use AI to perceive when a mobile user might feel remorseful about their actions. To accomplish this, they produced their artificial 'synthetic data' using GPT-3 and employed zero-shot text classification to train predictive models that could recognize these 'social emotions' automatically.

Somewhat disconcertingly, the research received funding from the Pentagon's Defense Advanced Research Projects Agency (DARPA). Established in 1958, DARPA has been a leading force in U.S. military research and development for over six decades, often supporting the creation of crucial technological breakthroughs. The research aligns with DARPA's computational cultural understanding program, aiming to enhance cross-cultural language comprehension technologies.

AI's Role in Expanding Defense Community's Powers

In essence, the study seems to be an extended form of sentiment analysis, a popular area in the surveillance industry. Furthermore, it indicates that AI will likely play a crucial role in expanding the capabilities of the U.S. defense community, potentially leading to alarming results. The implications of this research highlight the ever-evolving and impactful role of AI in areas such as defense and surveillance.

More articles

Also read

Here are some interesting articles on other sites from our network.