Microsoft puts AI in the heart of Windows 11 with Windows Copilot

“There’s so much depth in this product from the ’90s on,” said Panay. “Our job is to responsibly, of course, push it forward to users that need both the past and need to get to the future […] We have so many kinds of users that use Windows in its simplest form, like browsing, Mail, and Office […] And then we’ve got the hardcore devs who get right down to the depth of the product. And I think both are so vital to the platform. But now I think every developer can move forward as an AI developer and every user can be a power user on Windows. I think it just makes Windows that much better.”

Windows Copilot is currently a text-only tool, but Panay envisions it evolving into something you can interact with on your own terms. It has the potential to be the powerful voice assistant that Cortana, the Siri competitor Microsoft pushed for years, never amounted to. (And to be fair, pretty much every virtual assistant ended up being a disappointment. Siri remains confounding and inaccurate, while Google’s Assistant is mainly useful for preemptively delivering information, rather than dealing with voice commands.)

The simple, text-based approach to AI also makes sense for Microsoft. For many users, Windows Copilot will be the first time they’ve interacted with any sort of generative AI product. Better to let general users warm up with a few text strings before overwhelming people with voice commands. And hopefully by the time Microsoft adds voice support, it’ll be closer to the Star Trek computer than the frustrating assistants of yore. (I’m dreaming of the day when I can shout at my computer to look up information as I’m writing feverishly on deadline, or ask it to transcribe and summarize a meeting.)

Windows Copilot, just like Bing Chat, will also support the same third-party plugins that OpenAI’s ChaptGPT uses. That means any developer will be able to easily connect their apps to AI, an essential move to make these products more useful. Every software platform needs a thriving third-party ecosystem to survive — just think of what iOS or Android would be like if you were stuck using only Apple or Google’s apps.

“AI is going to be the single largest driver of innovation for Windows in the years to come,” said Panay. “It’s going to change the way you work, change your interaction models to make it easier. It’s going to understand so much about what you need.”

Panay stressed that Microsoft is also focused on making your AI interactions secure and private. Windows Copilot could end up juggling some sensitive information if you start asking questions about specific health questions, for example. A potentially bigger concern is misinformation: Windows Copilot and Bing Chat can answer your questions confidently, but it may not always be accurate. And it’s a bigger problem than standard web searches since it’s unclear where AI chatbots are pulling information from.

“We have to approach AI with innovation but also optimism,” Panay said when I asked about how Microsoft is planning to deal with AI misinformation. “To your point, we have to prioritize both people’s safety and privacy. Also, at the end of the day, if you do it in a frame of humility, where we’re always learning, this is going to be vital to your point.”

It’s clear that Microsoft still has plenty to learn from Windows Copilot. The preview period, which kicks off next month, is a chance for the company to see how it’s used in the wild, and to make adjustments before it’s officially released. Panay says that the Bing team is already working to make sure its results come from grounded references, and that they’re also holding to Microsoft’s responsible AI standards.

Microsoft

According to Shilpa Ranganathan, a Microsoft Corporate Vice President leading the Windows team, the company has been talking a lot about how Copilot results are displayed, as well as communicating the uncertainty of some answers to customers. “We’re going to build it into the experience as a learning experience for us as well […] I don’t want to take a path that allows us to lose trust with customers,” she said. “We believe that’s most important. I’d rather say, hey, we’re not 100 percent sure. Help us make this better or if this didn’t help you, I would like feedback now so that we never show this to another customer again.”

Panay hopes to get to a point where “as a product maker, I have the confidence that what we’re handing to our customers is exactly what they need when we launch into the full availability [of Windows Copilot].” He added, “It’s about humility. If you give it time and the right innovation, then we can get to that point of creating the guardrails needed to keep it safe and remove the misinformation.”

Given the state of the web today, where less reputable websites employ a variety of SEO tricks to rank higher in search results, it’s unclear if we’ll ever be truly rid of AI misinformation. The next content war will be entirely focused on websites vying for placement within AI search results. The problem for users is that it’ll be harder to tell if bad information is coming from a seemingly reputable source, rather than an obviously scammy website filled with junk content.

Windows Copilot has the potential to fundamentally reshape the way we use Windows. But hopefully, it won’t erode user trust in the process.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission. All prices are correct at the time of publishing.

“There’s so much depth in this product from the ’90s on,” said Panay. “Our job is to responsibly, of course, push it forward to users that need both the past and need to get to the future […] We have so many kinds of users that use Windows in its simplest form, like browsing, Mail, and Office […] And then we’ve got the hardcore devs who get right down to the depth of the product. And I think both are so vital to the platform. But now I think every developer can move forward as an AI developer and every user can be a power user on Windows. I think it just makes Windows that much better.”

Windows Copilot is currently a text-only tool, but Panay envisions it evolving into something you can interact with on your own terms. It has the potential to be the powerful voice assistant that Cortana, the Siri competitor Microsoft pushed for years, never amounted to. (And to be fair, pretty much every virtual assistant ended up being a disappointment. Siri remains confounding and inaccurate, while Google’s Assistant is mainly useful for preemptively delivering information, rather than dealing with voice commands.)

The simple, text-based approach to AI also makes sense for Microsoft. For many users, Windows Copilot will be the first time they’ve interacted with any sort of generative AI product. Better to let general users warm up with a few text strings before overwhelming people with voice commands. And hopefully by the time Microsoft adds voice support, it’ll be closer to the Star Trek computer than the frustrating assistants of yore. (I’m dreaming of the day when I can shout at my computer to look up information as I’m writing feverishly on deadline, or ask it to transcribe and summarize a meeting.)

Windows Copilot, just like Bing Chat, will also support the same third-party plugins that OpenAI’s ChaptGPT uses. That means any developer will be able to easily connect their apps to AI, an essential move to make these products more useful. Every software platform needs a thriving third-party ecosystem to survive — just think of what iOS or Android would be like if you were stuck using only Apple or Google’s apps.

“AI is going to be the single largest driver of innovation for Windows in the years to come,” said Panay. “It’s going to change the way you work, change your interaction models to make it easier. It’s going to understand so much about what you need.”

Panay stressed that Microsoft is also focused on making your AI interactions secure and private. Windows Copilot could end up juggling some sensitive information if you start asking questions about specific health questions, for example. A potentially bigger concern is misinformation: Windows Copilot and Bing Chat can answer your questions confidently, but it may not always be accurate. And it’s a bigger problem than standard web searches since it’s unclear where AI chatbots are pulling information from.

“We have to approach AI with innovation but also optimism,” Panay said when I asked about how Microsoft is planning to deal with AI misinformation. “To your point, we have to prioritize both people’s safety and privacy. Also, at the end of the day, if you do it in a frame of humility, where we’re always learning, this is going to be vital to your point.”

It’s clear that Microsoft still has plenty to learn from Windows Copilot. The preview period, which kicks off next month, is a chance for the company to see how it’s used in the wild, and to make adjustments before it’s officially released. Panay says that the Bing team is already working to make sure its results come from grounded references, and that they’re also holding to Microsoft’s responsible AI standards.

Microsoft

According to Shilpa Ranganathan, a Microsoft Corporate Vice President leading the Windows team, the company has been talking a lot about how Copilot results are displayed, as well as communicating the uncertainty of some answers to customers. “We’re going to build it into the experience as a learning experience for us as well […] I don’t want to take a path that allows us to lose trust with customers,” she said. “We believe that’s most important. I’d rather say, hey, we’re not 100 percent sure. Help us make this better or if this didn’t help you, I would like feedback now so that we never show this to another customer again.”

Panay hopes to get to a point where “as a product maker, I have the confidence that what we’re handing to our customers is exactly what they need when we launch into the full availability [of Windows Copilot].” He added, “It’s about humility. If you give it time and the right innovation, then we can get to that point of creating the guardrails needed to keep it safe and remove the misinformation.”

Given the state of the web today, where less reputable websites employ a variety of SEO tricks to rank higher in search results, it’s unclear if we’ll ever be truly rid of AI misinformation. The next content war will be entirely focused on websites vying for placement within AI search results. The problem for users is that it’ll be harder to tell if bad information is coming from a seemingly reputable source, rather than an obviously scammy website filled with junk content.

Windows Copilot has the potential to fundamentally reshape the way we use Windows. But hopefully, it won’t erode user trust in the process.

All products recommended by Engadget are selected by our editorial team, independent of our parent company. Some of our stories include affiliate links. If you buy something through one of these links, we may earn an affiliate commission. All prices are correct at the time of publishing.