Apple Unveils OpenELM: Open Source AI Models for On-Device Use

OpenELM (Open-source Efficient Language Models), Apple’s newest AI innovation, is a big step towards AI processing happening on the device itself. The Hugging Face Hub now has this set of models available. The Hugging Face Hub is a popular place for AI developers to share and work together on code. The release comes with a detailed white paper that talks about how these models were made and what they can do.

The OpenELM series has eight models. Half of them are already trained using Apple’s CoreNet library, and the other half have been fine-tuned with specific instructions to make them more useful.

One interesting thing about these models is that they use a layer-wise scaling strategy to make the best use of parameters across the transformer model’s layers. This method not only makes things faster and easier, but it also makes things a lot more accurate. For example, OpenELM gets a 2.36% improvement in accuracy while using only half as many pre-training tokens as older models like OLMo.

Instead of the usual practice in the industry, which is to only share model weights and inference code based on privately held datasets, Apple has chosen to share a lot of information. This includes the whole training framework, logs, multiple checkpoints, and configurations. All of these are based on datasets that are open to the public. The goal of this openness is to help natural language AI make faster progress and more reliable results.

By making these models open source, Apple is letting other researchers look into and build on their work, which could help fix problems with data and model design that are caused by biases or risks. This also gives developers and businesses the chance to change the models to fit different needs.

Additionally, Apple’s plan to share its new ideas in open source forums is a smart way to get top talent. The company isn’t being as secretive as it usually is because they want to create a place where researchers can work together and share their findings, which will improve AI in both the academic and professional worlds.

Apple devices don’t yet have these AI features built in, but iOS 18 is said to include a number of new AI features. There is also talk that these powerful models could be processed on the device to protect user privacy.