Apple discreetly released an open-source multimodal LLM in October.

Firstly, a large language model (LLM) is part of an artificial intelligence (AI) program that can recognize and generate text, among other tasks.

Apple and Columbia University researchers released an open-source multimodal language model (LLM) called Ferret in October 2023, which initially received little attention. However, with the recent release of open-source models from Mistral and Google’s Gemini model coming to the Pixel Pro and Android, there has been increased interest in the potential for local LLMs to power small devices. Apple announced two new research papers introducing new techniques for 3-D avatars and efficient language model inference, potentially enabling more immersive visual experiences and allowing complex AI systems to run on consumer devices like iPhones and iPads.

The AI community celebrated Apple’s unexpected entry into the open-source LLM since Apple has always been known as a “walled garden.” Bart de Witte, a European non-profit focused on open-source AI in medicine, posted on X that Apple joined the open-source AI community in October. Tristan Behrens, a German AI music artist and advisor, also weighed in, expressing excitement for the day when Local Large Language Models (LLLMs) run on the iPhone as an integrated service of a redesigned iOS.