FRIDA’s robot arm attempts to bring DALL-E-style AI art to real-world canvases

By Brian Heater – February 7, 2023

A version of this article was first published on TechCrunch

One could make a very reasonable argument that FRIDA (Framework and Robotics Initiative for Developing Arts) is as much a thought experiment as it is a research project. Certainly it butts up against similar questions around art and creativity as AI projects like DALL-E and ChatGPT — though the question is arguably even more in your face when it’s a robot arm doing the painting on a real-world canvas.

I recognize this is all extremely subjective, but at this point in the process, I’d go out on a limb and say the Carnegie Mellon University Robotics Institute’s project has some catching up to do with software-based AI systems. Even so, it’s fascinating to watch the project, which (obviously) gets its name from renowned Mexican portrait painter Frida Kahlo.

“FRIDA is a project exploring the intersection of human and robotic creativity,” says CMU professor Jim McCann. “FRIDA is using the kind of AI models that have been developed to do things like caption images and understand scene content and applying it to this artistic generative problem.”

The system currently requires some input, including text descriptions and existing images, though in a world so tied to literal interpretations, it can work a bit more abstractly. In one instance, the team played the robot the all-time banger that is ABBA’s “Dancing Queen.”

Though they also attempt to cut questions of job automations off at the pass. For one thing, humans are required for not just inputs, but actual paint mixing (though, to be clear, it’s realistic to assume that’s fairly automatable). The team suggests that it’s ideal as a collaborator.

Perhaps the most interesting piece of this is the system’s imprecision. The goal of such robotics is generally getting things as accurate as possible. Here, however, the system is allowed to make mistakes and adjusts the remainder of the painting accordingly, using an overhead camera to monitor its own progress. Speed isn’t a priority either. Each painting takes hours to complete.

Some of FRIDAs paintings #Robotics #creativeAI pic.twitter.com/5AX7lA8oxW

— FRIDA Robot Painter (@FridaRobot) December 5, 2022

FRIDA’s artwork can currently be seen on a CMU-run Twitter account.