Microsoft’s Bing AI Chatbot to soon let users post images, ask related questions

Users can search for an image, such as a crocheted teddy bear, for example, and ask Bing, “How do I make this?” The AI tool then suggests “how to crochet” websites and other kinds of craft projects to explore. 

The visual search feature, Mehdi says, will be available in the coming weeks to months. It uses OpenAI’s GPT-4 system, which enables so-called multimodal features that blend text and images. 

Other features that will soon be rolled out to Bing includes the ability to use third-party services like OpenTable and Wolfram Alpha through Bing chat and get answers that include videos and charts, Mehdi said. 

At a Manhattan demo, Bing representatives showed off some features in hands-on demonstrations, showing how a user can type in, say, “Find me a dinner reservation for two in New York City tonight,” and get a link to the reservation service OpenTable. 

“Chat really is the thing that is resonating with people and is in fact transforming search,” Mehdi said. He added that about 70% of people trying out the chat features are using them for search-related tasks, and Microsoft finds those customers perform a lot more searches. 

Last month, Microsoft CEO Satya Nadella said that Bing app installations have quadrupled since the launch of the AI-powered product, and that Bing has increased its market share in the US market in the quarter, without offering specifics. 

The executive said that it has made progress but is still working on issues like misinformation and incorrect or incomplete answers with Bing.  

“There’s a lot of work to review technology and plans against,” Mehdi said. “Is it harmful? Is it transparent? Is it inclusive? Does it not have bias? We’re going to do the same thing on the images, and adding multimodal adds another degree of complexity.”