The University of Science and Technology of China (USTC) and Tencent YouTu Lab have released a research paper on a new framework called Woodpecker, designed to correct hallucinations in multimodal large language AI models. “Hallucination is a big shadow hanging over the rapidly evolving MLLMs,” writes the group, describing the phenomenon as when MLLMs “output descriptions that are inconsistent with the input image.” Solutions to date focus mainly on “instruction-tuning,” a form of retraining that is data and computation intensive. Woodpecker takes a training-free approach that purports to correct hallucinations from the basis of the generated text. Continue reading Woodpecker: Chinese Researchers Combat AI Hallucinations

OpenAI began previewing vision capabilities for GPT-4 in March, and the company is now starting to roll out the image input and output to users of its popular ChatGPT. The multimodal expansion also includes audio functionality, with OpenAI proclaiming late last month that “ChatGPT can now see, hear and speak.” The upgrade vaults GPT-4 into the multimodal category with what OpenAI is apparently calling GPT-4V (for “Vision,” though equally applicable to “Voice”). “We’re rolling out voice and images in ChatGPT to Plus and Enterprise users,” OpenAI announced. Continue reading ChatGPT Goes Multimodal: OpenAI Adds Vision, Voice Ability

Startup Reka AI is releasing in preview its first artificial intelligence assistant, Yasa-1. The multimodal AI is described as “a language assistant with visual and auditory sensors.” The year-old company says it “trained Yasa-1 from scratch,” including pretraining foundation models “from ground zero,” then aligning them and optimizing to its training and server infrastructures. “Yasa-1 is not just a text assistant, it also understands images, short videos and audio (yes, sounds too),” said Reka AI co-founder and Chief Scientist Yi Tay. Yasa-1 is available via Reka’s APIs and as docker containers for on-site or virtual private cloud deployment. Continue reading Yasa-1: Startup Reka Launches New AI Multimodal Assistant

Meta Platforms is amping up its AI play, with plans to launch a suite of personality-driven chatbots as soon as next month. The company has been developing the series of artificially intelligent character bots with a goal of using them to boost engagement with its social media brands by making them available to have “humanlike discussions” on platforms including Facebook, Instagram and WhatsApp. Internally dubbed “personas,” the chatbots simulate characters ranging from historical figures like Abraham Lincoln to a surfer dude that dispenses travel advice. Continue reading Meta Plans Personality-Driven Chatbots to Boost Engagement

Based on the premise that it is impractical to deploy an all-purpose LLM for specific use cases, a group of researchers from Google, Baidu, DeepMind and Meta founded Reka AI in July 2022. A year later the company has emerged from stealth mode with news of $58 million in Series A funding led by DST Global and Radical Ventures. Strategic partner Snowflake Ventures also participated, along with angel investor Nat Friedman, former CEO of GitHub. The Sunnyvale, California-based startup says it is building “enterprise-grade state-of-the-art AI assistants for everyone, regardless of language and culture.” Continue reading Reka AI Raises $58 Million to Customize LLMs for Enterprise

Meta Platforms has built and is open-sourcing ImageBind, an artificial intelligence that combines six modalities: audio, visual, text, thermal, movement and depth data. Currently a research project, it suggests a future in which AI models generate multisensory content. “ImageBind equips machines with a holistic understanding that connects objects in a photo with how they will sound, their 3D shape, how warm or cold they are, and how they move,” Meta says. In other words, ImageBind’s approach more closely approximates human thinking by training on the relationship between things rather than ingesting massive datasets so as absorb every possibility. Continue reading Meta’s Open-Source ImageBind Works Across Six Modalities

Microsoft’s AI-powered Bing search engine has been drawing in excess of 100 million daily active users and logged half a billion chats. With OpenAI’s GPT-4 and DALL-E 2 models driving the action, it has also created over 200 million images since debuting in limited preview in February. Seeking to build on that momentum, Microsoft is adding new features and integrating Bing more tightly with its Edge browser. The company is also ditching its waitlist in a move to open preview. “We’re underway with the transformation of search,” CVP and consumer CMO Yusuf Mehdi said at a preview event last week. Continue reading Microsoft’s Next Generation of Bing AI Interacts with Images

Microsoft researchers have unveiled Kosmos-1, a new AI model the company says analyzes images for content, performs visual text recognition, solves visual puzzles and passes visual IQ tests. It also understands natural language instructions. The new model is what’s known as multimodal AI, which means it uses different instruction sets, from text to audio and video. Mixing media is a key step in building artificial general intelligence (AGI) that can perform tasks in a manner approximating human performance. Examples from a Kosmos-1 research paper show it can effectively analyze images, answering questions about them. Continue reading Microsoft Unveils AI Model That Comprehends Image Content