Up to 1,500 beta testers may have had files deleted by a bug in Windows 10, all without triggering an examination by Microsoft, according to data from the company.
The bug, which erased user files in the Documents, Pictures, Music and Videos folders, forced Microsoft to take the unprecedented move of pulling the Windows 10 1809 feature upgrade from public distribution.
Four days after announcing 1809's release, Microsoft barred access to the upgrade via Windows Update, told those who had installed it to stay off their PCs and warned users who had downloaded but not installed it to trash the disk image.
Although Microsoft claims that 15 million users run the Insider builds - previews of progress handed to volunteers - and some reported the file deletion problem using Insider's Feedback Hub app, Microsoft's engineering teams either did not notice the reports or if they did, discounted them.
It's unclear how many Insiders reported data loss, whether general or specific to the Documents et al folders. Microsoft implicitly confirmed that reports had been received, but not noticed because they had not been sufficiently "upvoted" within the Feedback Hub app.
Much like a social media or discussion forum post can be seconded by others - sometimes by clicking a literal thumbs-up icon - to increase its visibility or signal agreement, Feedback Hub offers an "Upvote" button for that purpose. Microsoft uses the upvoting, along with other, secret elements to prioritise the feedback for its engineers to evaluate.
Changes to the Hub synced with the claims of the Insider failure. "To help us better detect issues like this ... we have added an ability for users to also provide an indication of impact and severity when filing User Initiated Feedback," John Cable, director of program management in the Windows servicing and delivery group, wrote in an 9 October post to a company blog.
"We expect this will allow us to better monitor the most impactful issues even when feedback volume is low."
The number of testers who encountered data loss - as opposed to the number who reported that to Microsoft - can be calculated using the company's own numbers. "While the reports of actual data loss are few (one one-hundredth of one percent of version 1809 installs), any data loss is serious," stated Cable in the same post.
One-hundredth of one percent - 0.0001 or 0.01 per cent - multiplied by 15 million (Microsoft's figure for the number of Insider participants worldwide) equals 1,500.
That so many experienced a problem without raising a red flag in Redmond bolstered criticism of Insider, and more generally, Microsoft's testing process and philosophy.
"This had to come up by the Insiders," said Chris Goettl, product manager with client security and management vendor Ivanti, in a recent interview. "How did (1809) not get held up, how did Microsoft not pause it?"
Microsoft has yet to restart distribution of 1809 to the general Windows 10 audience two weeks after the plug was pulled. However, it has seeded the Insider testers with a build that the company claims fixes the file deletion flaws.