 |
Image Credit: Microsoft |
Microsoft is keen on pushing generative AI to the center stage of Windows and the computers that operate it. During the Build developer conference this week, Microsoft introduced a fresh range of Windows devices named Copilot+ PCs in two significant keynotes. Alongside this, they also revealed features powered by generative AI, such as Recall, which aids users in locating apps, files, and other previously accessed content. Copilot, the generative AI from Microsoft, is set to be more deeply embedded into the Windows 11 user experience. Additionally, new Microsoft Surface devices are in the pipeline.
Here's a summary of all the significant revelations from the first two days of the conference.
Windows Volumetric Apps for Meta Quest Headsets
 |
Image Credit: Microsoft |
Microsoft is introducing Windows Volumetric Apps, essentially spatially conscious, interactive VR applications, to Meta Quest headsets. Through a collaboration with Meta, Microsoft plans to provide Windows 365 and local PC connectivity to Quest headsets, allowing developers to expand their applications into the 3D realm.
In Tuesday's keynote, Microsoft demonstrated a digitally exploded 3D view of an Xbox controller as seen from a Meta Quest 3 headset — a digital entity that the user could interact with using their hands. "We are intensifying our collaboration with Meta to ensure Windows offers a premier experience on Quest devices," stated Pavan Davuluri, CVP of Windows and devices at Microsoft, during the demonstration.
Developers have the opportunity to sign up for a preview to gain access to Microsoft's new volumetric API.
Introducing Copilot+ PCs: Microsoft’s AI-Centric, Premium Windows Hardware
Copilot+ PCs represent Microsoft's vision of AI-centric, premium Windows hardware. All of them come equipped with dedicated chips known as NPUs to facilitate AI experiences like Recall. They are shipped with a minimum of 16GB RAM, coupled with SSD storage.
 |
Image Credit: Microsoft |
The inaugural Copilot+ PCs will be equipped with Qualcomm’s Snapdragon X Elite and Plus chips, which, according to Microsoft, provide up to 15 hours of web browsing and 20 hours of video playback battery life. Chip manufacturers Intel and AMD are also dedicated to developing processors for Copilot+ devices, in collaboration with a variety of manufacturers, including Acer, Asus, Dell, HP, Lenovo, and Samsung.
The starting price for Copilot+ PCs is $999, and some models are available for preorder today.
Microsoft’s New Surface Devices: Performance and Battery Life at the Forefront
 |
Image Credit: Microsoft |
Microsoft's recently revealed Surface devices, namely the Surface Laptop and Surface Pro, are centered around performance and battery life.
The newest Surface Laptop, available with either a 13.8- or 15-inch display, has been revamped with a "modern lines" design and slimmer screen bezels. It boasts a battery life of up to 22 hours and is touted to be up to 86% faster than its predecessor, the Surface Laptop 5. Additionally, it supports Wi-Fi 7 and features a touchpad with haptic feedback.
Regarding the new Surface Pro, Microsoft claims it to be up to 90% faster than the previous generation Surface Pro (the Surface Pro 9). It comes with a new OLED display with HDR, supports Wi-Fi 7 (with an optional 5G), and features an improved ultrawide front-facing camera. Its detachable keyboard, now reinforced with extra carbon fiber, also provides haptic feedback.
Windows 11’s Upcoming Recall Feature: A New Era of User-Friendly Search and Privacy
 |
Image Credit: Google |
The upcoming Recall feature in Windows 11 has the ability to “remember” applications and content that a user has accessed on their PC, even if it was weeks or months ago. For instance, it can assist users in locating a Discord chat where they were discussing potential clothing purchases. Users can utilize the timeline of Recall to “rewind” and see what they were working on in the recent past, and delve into files like PowerPoint presentations to uncover information that might be relevant to their searches.
Microsoft explains that Recall can establish connections between colors, images, and more, enabling users to search for virtually anything on their PCs using natural language, a feature not unlike the technology used by the startup Rewind. Developers will have the opportunity to enhance Recall by incorporating contextual information into their applications. Importantly, Microsoft assures that all user data linked with Recall is maintained privately and stored on-device — it is not used for training AI models.
Microsoft further elaborates: “The snapshots are your property; they remain locally on your PC. You have the option to delete individual snapshots, modify and erase time ranges in Settings, or pause at any moment directly from the icon in the System Tray on your Taskbar. You also have the ability to prevent certain apps and websites from ever being saved.”
Windows and Copilot+: Embracing Artificial Intelligence for Enhanced User Experience
Windows now incorporates more artificial intelligence than ever before, with some features being exclusive to the new Copilot+ PCs.
A novel feature, known as Super Resolution, has the ability to rejuvenate old photos by automatically enhancing their resolution. Additionally, Copilot now possesses the capability to analyze images, providing users with inspiration for creative compositions. With the introduction of a feature named Cocreator, users can not only generate images but also instruct the AI model to adapt to their drawings, enabling them to modify or restyle the image.
 |
Image Credit: Microsoft |
In other developments, Live Captions with live translations has the capability to convert any audio that is played on a PC — be it from YouTube or a local file — into a language selected by the user. Initially, Live translations will offer support for approximately 40 languages, including English, Spanish, Mandarin, and Russian.
A distinct yet related new feature in Microsoft Edge provides real-time video translation on various platforms such as LinkedIn, YouTube, Coursera, Reuters, CNBC, Bloomberg, and more. Slated for release in the near future, this feature — which facilitates the translation of Spanish into English and English into German, Hindi, Italian, Russian, and Spanish — converts spoken content into another language through both dubbing and live subtitles.
Team Copilot and extensions
Team Copilot represents the newest addition to Microsoft's expanding Copilot suite of generative AI technology. It seamlessly integrates with Teams, Microsoft's videoconferencing application, assisting in the management of meeting agendas and facilitating note-taking that can be co-authored by anyone in the meeting. Furthermore, it extends its functionality to Loop and Planner, Microsoft's platforms for collaboration and planning, enabling the creation and assignment of tasks, tracking of deadlines, and notification of team members when their contribution is required.
 |
Image Credit: Github/Microsoft |
In news related to Copilot, Microsoft has initiated a private preview of Copilot Extensions. This allows developers to enhance the capabilities of GitHub's code-generating tool, GitHub Copilot, by integrating it with third-party applications and skills. Launch partners for this initiative include DataStax, Docker, and LambdaTest. While these extensions will be available in the GitHub Marketplace, developers also have the option to create their own private extensions for seamless integration with their internal systems and APIs.
Windows Copilot Runtime
The capabilities such as Recall and Super Resolution are powered by the Windows Copilot Runtime, a compilation of approximately 40 generative AI models that constitute what Microsoft refers to as "a new layer" of Windows. Working in conjunction with the semantic index, a vector-based system specific to an individual Copilot+ PC, the Windows Copilot Runtime enables generative AI-powered applications — including those from third parties — to operate without necessarily requiring an internet connection.
 |
Image Credit: Microsoft |
Davuluri stated on Tuesday that "[The runtime] is comprised of ready-to-use AI APIs like Studio Effects, Live Captions translations, OCR, Recall with user activity and [more], which will be accessible to developers in June."
Microsoft has announced that CapCut, the widely-used video editor from TikTok's parent company ByteDance, will utilize the Windows Copilot Runtime and the accompanying new Windows Copilot Library, a collection of APIs and AI development tools, to accelerate its AI features. Furthermore, Meta will incorporate the aforementioned Studio Effects into WhatsApp to provide features such as background blur and eye contact during video calls.
Upgraded bot builders
Microsoft's Azure AI Studio, a component of the Azure OpenAI Service, offers a toolkit that enables users to integrate an AI model and construct an application that can process and interpret data. This service will soon extend its capabilities to allow developers to build applications using pay-as-you-go inference APIs. These APIs provide developers with the ability to access and refine generative AI models that are hosted on Azure's infrastructure. This service, referred to by Microsoft as "model-as-a-service," is set to launch with models from Nixtla and Core42.
In the related product suite, Copilot Studio, Microsoft is introducing Copilot agents. These are AI bots described by the company as being capable of autonomously managing tasks that are customized to specific roles and functions. Copilot Studio offers tools that link Copilot for Microsoft 365, the AI-assisted "copilot" in applications like Excel and Word, with third-party data. With the ability to recall and understand context, Copilot agents can navigate a variety of business workflows. They learn from user feedback and seek assistance when they encounter situations that are beyond their understanding.
Snapdragon Dev Kit
 |
Image Credit: Microsoft |
Qualcomm has introduced a new development kit targeted at developers who are creating applications for Copilot+ PCs that are equipped with Arm chips.
Priced at $899.99, the Snapdragon Dev Kit for Windows is comparable in width, height, and length to Apple's Mac Mini. It is equipped with Qualcomm's Snapdragon X Elite chip, complemented by 32GB of RAM and 512GB of storage, along with a generous amount of I/O. The Dev Kit is compatible with Wi-Fi 7 and Bluetooth 5.4. Furthermore, it can support up to three 4K monitors simultaneously through its various USB-C and HDMI ports.
Phi-3
Microsoft has unveiled a new addition to its Phi family of generative AI models, known as Phi-3-vision. This model is capable of performing general visual analysis and reasoning tasks, such as interpreting charts and images. It has the ability to process both text and images and is efficient enough to operate on a mobile device.
Phi-3-vision is currently available in a preview version, while its text-only counterparts — Phi-3-mini, Phi-3-small, and Phi-3-medium — have been officially released and are now generally accessible.
Partnership with Khan Academy
In collaboration news, Microsoft is partnering with Khan Academy. As part of this partnership, Microsoft will provide Khan Academy with access to its cloud computing infrastructure at no cost. This will enable educators in the U.S. to freely access Khan Academy’s AI-powered tools. Additionally, the two companies will work together to explore ways to enhance AI applications for math tutoring using generative AI, as announced by Microsoft on Tuesday.
FAQ
What are Windows Volumetric Apps for Meta Quest Headsets?
Windows Volumetric Apps are immersive applications designed for Meta Quest Headsets, allowing users to experience a more interactive and 3D environment. These apps leverage volumetric technology to create a sense of depth and presence, enhancing virtual reality experiences on Meta Quest devices.
What are Copilot+ PCs?
Copilot+ PCs are a new line of computers from Microsoft that come with advanced AI capabilities integrated directly into the Windows operating system. These PCs feature exclusive AI tools, such as Super Resolution for photo enhancement and enhanced image analysis capabilities.
What is the Recall feature in Windows 11?
The Recall feature in Windows 11 is designed to help users "remember" and access applications and content they’ve previously worked on, even after weeks or months. It allows users to rewind their activity timeline, search for files using natural language, and maintains all data privately on the device.
How does Copilot+ enhance the Windows experience?
Copilot+ enhances the Windows experience by integrating powerful AI features into the operating system. These include image enhancement, creative inspiration tools, and advanced translation services. Copilot+ PCs are specifically designed to utilize these AI capabilities to their fullest potential.
What is Team Copilot and how do extensions work?
Team Copilot is a collaborative AI feature that allows multiple users to work together using AI assistance. Extensions enable developers to add contextual information to enhance the functionality of Team Copilot, making it a versatile tool for team-based projects and workflows.
What are the upgraded bot builders in Windows?
The upgraded bot builders in Windows offer enhanced tools for creating and managing AI-driven bots. These improvements allow developers to build more sophisticated and responsive bots, facilitating better user interactions and more efficient automation processes.
