Increase AI performance through machine learning and produce training data at scale
Deep Learning is opening a world of possibilities for enterprises across every industry. However, as companies look to implement their deep learning solutions, they are realizing that even more important than the quality of their model, is the quantity and quality of their data.
In 2018, Microsoft acquired Semantic Machines, intending to leverage the technology to power their next-gen conversational artificial-intelligence (AI). Targeting calendaring as the first application, Microsoft focused on training the AI to schedule users’ meetings, review daily agendas, and manage attendee responses and conflicts.
However, with any novel application, there was limited training data available for the AI and Microsoft quickly focused on data production as one of its most important initiatives. Looking to ramp up a global operation in a short time frame, the team zeroed in on several key priorities to ensure they could meet their data requirements as they scaled, including:
- Improving data production, quality, and relevancy
- Tailoring data production to mitigate identified gaps in their AI knowledge base
- Streamlining data production processes and standardizing best practices
- Developing production strategies to accelerate product and feature development
Microsoft engaged The Spur Group’s Data and AI team to scale their AI training capacity and deep dive into their data production processes and practices. With the goal of building a streamlined AI training operation, we set out to expand and scale the AI’s capabilities quickly and efficiently, to meet Microsoft’s rapidly growing data needs.
Partnering with Microsoft’s leadership team, we helped develop their vision for large-scale data production. After defining their ideal data production operation, we built a phased approach to effectively scale their data production team.
Establishing a team of experienced data scientists, we performed a deep dive into Microsoft’s data production operations. Prioritizing driving progress while we supported their transformation, we spent the initial phase producing data as well as identifying operational opportunities side-by-side with Microsoft’s leadership.
“The Spur Group’s partners showed an impressive level of dedication and flexibility from the start. We were tackling something that hadn’t been done before, and not only did they build a team unlike anything we’d created before, but more importantly, they acted as an invaluable thought partner for us as we developed our strategy and vision.”
Microsoft | Principal Program Manager
During the following months, we identified a set of recommendations to improve production efficiency. We accelerated the AI’s training utilizing machine teaching strategies and worked in lockstep with Microsoft’s top AI researchers to identify gaps in the AI’s knowledge base and response generation.
Operating across 100+ categories of product functionality, the team significantly fast-tracked the development of the AI engine. Performance improvement doubled in the team’s focus areas and we built a library of documentation and trainings, standardizing data production practices across a team of over 50 agents. In addition, the team implemented advanced machine learning strategies, such as Markov Chain algorithms, to automate data categorization and triage.
We drove significant improvements across the Microsoft team’s key priorities: Data Management, Process Optimization, and Training. The result was a rapidly evolving conversational AI engine, and scalable data production and AI training operations within 6 months.
- Oversaw data management across 100+ product functionality categories
- Improved functionality performance by an average of 14%
- Performed 36+ data migrations to improve model utilization and acceptance
- Built automated solutions, reducing manual process overhead by up to 96%
- Increased data collection relevancy, reducing downstream operational costs by 33%
Training and scale
- Created a documentation library consisting of dozens of trainings, process maps, best practices, and other reference materials
- Expanded production capacity by training 18 incremental specialists and 12 net-new annotators in data production best practices and processes
“We would not be in the position we are today without The Spur Group. They brought a level of expertise and professionalism that was above and beyond. Our partnership significantly accelerated the evolution of our AI engine and progress toward our product vision.”
Microsoft | Senior Manager, Learning and Development