How Microsoft’s developers are utilizing AI

Microsoft envisions a scenario in which AI manages everything on your computer and performs tasks on your behalf in the background. However, to achieve this goal, the company must develop the necessary tools and persuade its own developers that AI can genuinely fulfill these ambitious promises.

Earlier this year, Microsoft CEO Satya Nadella disclosed that approximately 30 percent of the code for “some of our initiatives” is generated by AI, and I’ve been eager to find out exactly how Microsoft’s developers are implementing the technology since then. I’ve been consulting with sources and company executives to gain a deeper understanding of how AI is being utilized by Microsoft developers. Some employees have expressed skepticism regarding whether AI agents can entirely replace human functionality, which may leave developers to rectify the errors of these automated agents.

When I request further details from the company, however, Microsoft emphasizes its initial successes in employing AI internally.

“We aim to investigate areas of developer toil and identify inefficiencies,” states Amanda Silver, a CVP within Microsoft’s CoreAI division who leads products for the company’s Apps & Agents platform, in a conversation with Notepad. “Part of our focus is to determine both where and how we can implement [AI].”

Microsoft houses over 100,000 code repositories, ranging from new projects to legacy codebases that have been operational for over 20 years. “We encompass nearly every programming language, architecture, and lifecycle stage conceivable, which reflects a broad spectrum of our customers,” remarks Silver. That presents a substantial amount of code for AI to potentially engage with, especially as Microsoft advances beyond mere code completion towards increased automation through AI agents.

In May, Microsoft integrated a coding agent directly into GitHub Copilot, enabling developers to delegate tasks to it. The agent then establishes its own development environment, operates in the background, and creates draft pull requests. “Our data shows that developers save around 30 minutes on simple tasks, over half a day on medium tasks, and two weeks on intricate tasks,” explains Silver. Microsoft’s developers are employing it for tedious and time-consuming tasks such as debugging and enhancing documentation for applications and services.

Microsoft assesses developer hours saved, incidents reduced, or estimated hours saved for tasks to compile these statistics. “Additionally, we monitor the actions executed by agentic capabilities, including the number of pull requests it contributes,” states Silver.

Evaluating the effect of AI on developer productivity is an area Microsoft seems to be intensely focused on internally, even though some studies suggest AI may make seasoned developers slower. Some employees, who prefer to remain unnamed, believe that Microsoft leadership is not satisfied with the current frequency of AI usage by developers. There’s an internal push to encourage developers to prioritize AI for all tasks, although I hear that adoption is not always organic.

“It necessitates some intentional engagement for the mindset shift to take hold,” acknowledges Silver. While Microsoft’s developers might sidestep GitHub Copilot Chat because it operates in a separate window, the agentic mode and coding agent are integrated into the workflow of developers. “It encourages habitual use and modifies behavior,” notes Silver.

Microsoft asserts that 91 percent of its engineering teams utilize GitHub Copilot, but sources have indicated that, in certain areas of the company, overall AI tool adoption is significantly lower – approaching the 51 percent of developers who reported to Stack Overflow that they now use AI tools daily in a professional context.

Silver enumerates several teams that have accelerated their productivity with AI. The Xbox team recently employed Copilot’s app modernization agent to update their primary Xbox service from .NET 6 to .NET 8. “They experienced an 88 percent decrease in manual migration efforts,” she states, condensing “months of work down to mere days.” Microsoft’s discovery and quantum team also utilized the Copilot agent to transition a Java application to the latest version, achieving a similar “reduction in effort needed, thanks to the AI agent automatically identifying deprecated APIs, recommending fixes, and highlighting security vulnerabilities.” The company’s “ES Chat” agent, designed to answer inquiries regarding Microsoft’s engineering systems, has enabled engineers to save “46 minutes per task compared to using traditional search methods.” Additionally, Microsoft has employed AI agents to assist Site Reliability Engineers (SRE) in responding to system and application outages, having already saved over “10,000 operational hours.”

These time savings indicate that AI is increasingly responsible for producing Microsoft’s code as opposed to solely human input, but Silver refrains from quantifying the extent to which AI contributes to Microsoft’s codebase. She argues that tracking everything is challenging since AI is woven into code generation, review processes, test generations, and deployment workflows. “The agents essentially become an integral part of the engineering system itself,” states Silver. “This is one of the reasons it’s difficult to assign a precise figure to the lines of code generated through AI.” I also gather that publicizing a number that appears too high or too low would hinder Microsoft’s marketing strategies, both internally and externally.

Nevertheless, I don’t question the intricacy of the endeavor. A human engineer might submit code while Copilot is operational within their editor, or the engineer might copy and paste AI-generated code into their workspace. It’s reasonable to assert that AI is prevalent in portions of Microsoft’s developer output. One merely needs to examine the codebases of Aspire, Typescript Go, and Microsoft’s Agent Framework to observe that Copilot significantly contributes to all of them.

The AI systems are also not infallible. Silver notes that engineers scrutinize their contributions. A source within Microsoft relayed that certain tools do not perform as expected. “ES Chat saves me time by avoiding its use,” the individual joked.

Microsoft’s rapid move toward AI agents coding for developers has raised concerns among some staff about the future. I’ve talked to engineers in Microsoft’s CoreAI division worried about the increasing reliance on autonomous AI agents, particularly as they take on tasks typically assigned to junior developers. There’s considerable anxiety within the industry, and at Microsoft, that junior developer positions may be vanishing, leaving experienced developers to oversee the output generated by AI tools.

With Nadella’s vision of transforming Microsoft into a company centered on AI agents undertaking tasks, it seems probable that fewer humans will be engaged in coding down the line. Silver remains optimistic that AI will merely enable developers to delegate mundane tasks, allowing them to concentrate on creativity instead.

“No developer entered this field because they wanted to be assigned an extended coding maintenance migration project,” claims Silver. “They aspire to be at the forefront, to create, and to innovate. These are the kinds of responsibilities they want to delegate to AI to return to creative endeavors.”

I’m always eager to hear from readers, so please feel free to comment here, or you can contact me at [email protected] if you’d like to discuss any other topics. If you’ve been made aware of any of Microsoft’s confidential projects, you can reach out via email at [email protected] or communicate with me privately on the Signal messaging app, where I’m tomwarren.01. I’m also on Telegram as tomwarren, should that be your preferred method of communication.

Thank you for subscribing to Notepad.