UniversalNER: The New AI Model Surpassing ChatGPT's Named Entity Recognition

JJohn August 13, 2023 6:51 AM

Researchers from the University of Southern California (USC) and Microsoft Research devise UniversalNER, a new AI model trained using a method called 'targeted distillation'. This innovative model can recognize over 13k+ entity types and outperforms the Named Entity Recognition (NER) accuracy of ChatGPT by 9% F1 on 43 datasets.

The limitations and potential of LLMs

ChatGPT and its counterparts, known as large language models (LLMs), have wowed us with their generalization capabilities. However, let's not ignore that their training and inference costs can be a real downer. Luckily, 'instruction tuning' has swept to the rescue, becoming a favorite method for condensing these complex LLMs into more budget-friendly and transparent student models, like Alpaca and Vicuna. These student models, while impressive, still have a way to go before they match up to the ideal LLM, especially in targeted downstream applications.

The researchers decided to take a different route. Instead of going for a generic distillation, which could only create a superficial approximation of the original LLM across all applications, they decided to venture into 'targeted distillation'. In this process, student models are trained through mission-focused instruction adjustment for a diverse application class like open information extraction. Choosing named entity recognition (NER) for their study, they showed how targeted distillation can effectively emulate LLM’s capabilities for a particular application class.

The researchers from USC and Microsoft Research turned to ChatGPT to generate instruction-tuning data for NER from a wealth of unlabeled online text. With the help of LLaMA, they used this data to create the UniversalNER models, abbreviated as UniNER. This model stands as a testament to the potential of targeted distillation and the power of combining big data with intelligent AI models.

UniversalNER's groundbreaking performance

UniversalNER didn't just make a mark; it made a splash. In the largest and most varied NER benchmark to date, UniversalNER came out on top, crushing scores of other models. It touted state-of-the-art NER accuracy across tens of thousands of entity types. But what's even more impressive? It beat ChatGPT’s NER accuracy by a whopping 7-9 absolute points in average F1. Now that's what we call a game-changer.

UniversalNER surpasses multi-task instruction-tuned systems

UniversalNER didn't stop at outdoing ChatGPT; it also significantly outperformed state-of-the-art multi-task instruction-tuned systems like InstructUIE. To further evaluate the effects of different distillation components, the researchers rolled up their sleeves and conducted extensive ablation tests. The cherry on top? They're providing their distillation recipe, data, and the UniversalNER model to the AI community. It's an open invitation for further study and exploration into targeted distillation.

More articles

Also read

Here are some interesting articles on other sites from our network.