Expanding artistic frontiers in artificial intelligence

Dr. Mohammed Elhoseiny, assistant professor of computer science at KAUST, has carved a career out of teaching machines the art of creating art. After finishing his doctoral degree at Rutgers University in 2016, Elhoseiny went on to work for Adobe Research, Baidu Research, Facebook and now KAUST.

His latest research paper, Creative Walk Adversarial Networks: Novel Art Generation with Probabilistic Random Walk Deviation from Style Norms, was accepted at the premiere conference on computational creative artificial intelligence (AI), the International Conference on Computational Creativity (ICCC) 2022.

The paper covers the work of Elhoseiny and his team VISION CAIR on the use of Creative Walk Adversarial Networks (CWAN) for novel, or original, art generation. CWAN learns about existing art styles in its training by being exposed to a large repository of paintings from various art movements and styles, from 5000 years ago to present times.

“CWAN tries to learn from those existing patterns, to come up with other new patterns that may be interesting and exciting, and generate something that, hopefully, people will like and find interesting,” Elhoseiny said. “At the same time, it deviates from what is out there by using ‘random walk deviation’ methods to find something new and bring originality to the creation.”

The team’s method produces art that is much more likable compared to other existing AI approaches—a conclusion supported by the experimental results, including ratings, of 341 unique human evaluators. The findings demonstrate that CWAN can generate novel art that is significantly preferable compared to other state-of-the-art methods, such as StyleGAN2 and StyleCAN2.

What does this mean for the art world? Elhoseiny said, “Many artists have adopted AI art technology already. In general, this shows AI art can be a useful tool in the artist’s toolbox, to help produce unique art that has more meaning to the viewer than current AI machine art.”

Elhoseiny does not think AI art will replace artists. “It’s similar to the invention of photography,” he commented. “People used to depict reality with photography, but artists began to create new art movements from these photographic techniques.”

Getting machines to talk ‘affectively’ about artworks

VISION CAIR has also been working on teaching machines to use more affective language when communicating about visual art, whether it is existing art or AI-generated art. An upcoming paper on the topic, It is Okay to not be Okay: Overcoming Emotional Bias in Effective Image Captioning by Contrasted Data Collection, will be presented at the Conference on Computer Vision and Pattern Recognition (CVPR) 2022, among the top annual computer vision events in the world.

“Affective communication of visual art is the next step for us,” Elhoseiny said. “What is at the heart of most of our efforts in this direction is to move the wheels towards human-compatible AI that takes emotion into consideration.”

Emotionally intelligent responses by machines is the aim, but he said the team has found some obstacles along the way.

“There is a human bias toward positive language at the moment, so we proposed a new way to balance and mitigate this bias. We call the resulting data set ArtEmis 2.0.”

Reflecting on the connection of ArtEmis 2.0 to CWAN, Elhoseiny added that an interesting future scenario is teaching CWAN to generate art grounded a full range of emotional responses, which he said is a significant yet challenging task to achieve.