Uh-oh, X’s Grok AI can now ‘understand’ images

New “Vision” Grok will be available to testers and select users.

X’s new Grok “Vision” can process complex visual information.
Credit: Jaap Arriens/NurPhoto via Getty Images

Elon Musk’s AI chatbot can now “understand” images, including information-riddled diagrams and charts. Sorry, doesn’t everyone use the platform once known as Twitter for multi-disciplinary research and optimizing their work flows??

Introduced as Grok-1.5V — Or Grok 1.5 “Vision,” the company’s “first-generation multimodal model” — the bot will be able to not only respond to your uploaded pictures and screenshots but also reason through complex documents, science diagrams, charts, screenshots, and photographs, the company says. Additionally, Grok-1.5V will gain “real-world spatial understanding” to better understand the physical world depicted in the images uploaded by its users.

“Advancing both our multimodal understanding and generation capabilities are important steps in building beneficial AGI that can understand the universe,” the company wrote in its’ announcement. “In the coming months, we anticipate to make significant improvements in both capabilities, across various modalities such as images, audio, and video.”

Example use cases include translating a diagram into Python code, turning a child’s drawing into a bedroom story, pinpointing the largest object among a group of many, and telling a driver if they have enough space to drive around an obstacle.

Grok-1.5V is released along with xAI’s RealWorldQA, an image and prompt dataset designed to test other GenAI models against Grok’s real world reasoning.

Tweet may have been deleted

Competition is the least of Grok’s worries, however. Despite xAI’s continued investment, Grok has yet to stick with early users and staff — a new report alleges its own developers struggle to use the slow xAI API. That same report, published by Fortune this week, highlighted X employee concerns about Musk suggesting Grok write paid user’s posts for them, despite warnings from developers and staff. Last week, Grok came under fire for generating fake news headlines from an alternate reality where Iran had assailed Tel Aviv with a military arsenal — not its first time.

While GenAI chatbots hallucinating realities and generating fake news is par for the course, Grok’s gaffe is indicative of yet another site wide issue. The bot, a par for the course response to ChatGPT from Musk, is integrating into a platform that has slowly whittled away at its defenses against AI gone bad. Combined with X’s all around poor reputation for moderation and the CEO’s own refusal to address misinformation in aid of the site’s “citizen journalists,” Grok occupies a precarious spot in the platform’s besieged information ecosystem.

Grok-1.5V will be available to early testers and select users soon.