Microsoft announced Phi-3, a 3-billion-parameter language model that provides advanced inference capabilities similar to much larger models at a significantly lower cost.
Developed by Microsoft Research, the new model will be available on the company’s Azure AI platform, allowing businesses to use state-of-the-art natural language processing and inference for a variety of applications.
“We have created a very small model with capabilities that rival much larger models, including approaching the GPT-3.5 level. I don’t think anyone knew how much size would be needed to achieve capabilities approaching that of GPT-3.5.”
he stated Sébastien Bubeckvice president of Microsoft GenAI, told VentureBeat.
Phi-3 is Microsoft’s latest effort to push the limits of compact language models. Starting with the coding-oriented Phi-1 a year ago, Phi-1.5 and Through Phi-2 models the Phi series has shown impressive performance on coding, common sense, and general natural language benchmarks with up to 1-2 billion parameter models.
With Phi-3 a Microsoft developed a general purpose 3 billion parameter model that approaches the capabilities of industry leading models such as OpenAI’s GPT-3.5 model, but at a significantly lower cost and with the flexibility to run on mainstream hardware or even smartphones. The breakthrough in parametric efficiency enables transformative AI use cases for businesses that were not cost-effective before.
Technically, Phi-3 is NVIDIA It runs on GPU-optimized ONNX Runtime and can be deployed across multiple GPUs or machines to optimize throughput. The model architecture uses efficient attention mechanisms and optimized numerical accuracy to achieve high performance with relatively few parameters.
“The base layer is in a small model. We can bring in our data and fine-tune this generic model to achieve incredible performance in narrow verticals,”
Bubeck explained.
The introduction of Microsoft Phi-3 and its planned integration into the Azure AI platform represents a significant step forward in making large language modeling capabilities available and cost-effective for businesses of all sizes. As more companies aim to operationalize AI and unlock the value of unstructured data, targeted models like Phi-3 will be essential to realizing this vision.
(source)
