Apple has released a public demo of the 4M AI model it developed in collaboration with the Swiss Federal Institute of Technology Lausanne (EPFL). The technology debuts seven months after the model was first open-sourced, allowing informed observers the opportunity to interact with it and assess its capabilities. Apple says 4M was built by applying masked modeling to a single unified Transformer encoder-decoder “across a wide range of input/output modalities — including text, images, geometric and semantic modalities, as well as neural network feature maps.”

The result “leads to models that exhibit several key capabilities,” Apple researchers explain in the paper 4M: Massively Multimodal Masked Modeling:

• They can perform a diverse set of vision tasks out of the box.
• They excel when fine-tuned for unseen downstream tasks or new input modalities.
• They can function as a generative model that can be conditioned on arbitrary modalities, enabling a wide variety of expressive multimodal editing capabilities with remarkable flexibility.

Touted as “a framework for training any-to-any multimodal foundation models,” the 4M website claims it functions “across tens of modalities and tasks.”

VentureBeat reports the 4M launch “represents an important step in expanding access to sophisticated AI technology,” since it is available on Hugging Face. The system can be used to create images from text prompts and perform complex object detection. It can also accept natural language prompts to manipulate 3D scenes.

Notably, the open-source release “marks a significant departure from Apple’s traditionally secretive approach to research and development,” VentureBeat adds. “By making 4M publicly accessible on a popular open-source AI platform, the company is not only demonstrating its AI capabilities but also courting developer interest and fostering an ecosystem around its technology.”

Meanwhile, TechRadar and CNET are among those speculating whether and when Apple will launch a fee-based AI tier, like its competitors have done. When the company’s AI, called Apple Intelligence, was announced in June at WWDC 2024, the company touted its free availability to users with compliant iPhone, iPad and Mac devices.

In another post, CNET intriguingly writes that Apple Intelligence looks to be coming to the Apple Vision Pro mixed reality headset, “but not this year.”

Related:
Apple’s Devices Are Lasting Longer, Making AI Strategy Even More Critical, Bloomberg, 6/30/24
What Every Company Can Learn From Apple’s AI Strategy, Forbes, 7/1/24
How Apple Is Borrowing from Its 1980s Playbook When It Comes to AI, MarketWatch, 6/29/24