Introduction

Updated: 14th January 2025 – The guidance has been updated by Microsoft and the Enhanced Search which uses Copilot Semantic Index does not require every user to have a Microsoft 365 Copilot license.

https://learn.microsoft.com/en-us/microsoft-copilot-studio/knowledge-copilot-studio

This article discusses some of the things to consider when deciding whether to use Azure AI Search as the data source for Copilot Agents.
The AI world is moving quickly, and Microsoft is also moving quickly with constant updates and changes to its propositions.

However, one of the issues that we have found at iThink 365 when building Knowledge Copilots for our customers is that the quality of the Copilot and the user experience are directly related to the quality of the search results underpinning it.

Generally, we have used Azure AI Search as our search engine which we have seen has provided dramatically better results for our customers. The reason is the type of search technology underpinning the Copilot. Azure AI Search is built for AI workloads and can be configured to use several different search technologies including Semantic, Keyword, Vector and a hybrid of these search technologies.

However, the use of Azure AI Search does increase the cost of running the Copilot because you are paying for Copilot Studio, Azure Open AI and Azure AI Search resources. Whilst some people would say that this is not worth it due to the increase in cost, it really is the only solution for getting the results that customers expect.

The reason for the Azure AI Search approach is that the base Copilot Studio uses the basic Microsoft 365 search engine when Copilot is accessing knowledge held in SharePoint. This experience uses Keyword based searches to give results and we have found that the experience for customers has not been as good.

However, it is important to keep up to date with Microsoft and their changes. Microsoft have been listening to the feedback from their customers, MVPs and partners and so at Ignite 2024 they announced the introduction of Copilots built on Copilot Studio to use the Microsoft 365 Copilot Semantic Index. This has the potential for really improving the experience for users and brings the experience more inline with what we have seen with Azure AI Search.

There is something to be aware of, to be able to take advantage of the Microsoft 365 Copilot Semantic Index your organisation and users need to be licensed for Microsoft 365 Copilot.
Without the licenses, your Knowledge Copilot will not be able to use the premium search capabilities and therefore Copilot Semantic Index.

 
Questions to help make your decision

So, our decision on whether to use Azure AI Search or not is based on the following questions:

• Does one user have a Microsoft 365 Copilot license?
• Are the documents being accessed with knowledge in them, either PDF, DOCX or PPTX and are they less than 512MB in size?
• Is all the information that needs to be accessed held in supported documents in Microsoft SharePoint?

If the answers to these questions are yes, then my suggestion is to do a proof of concept using Copilot Studio and the premium search experience and test to see if the user experience is good enough so you do not need to use Azure AI Search.

However, if the answer is no to either or following testing the user experience then I would recommend using Azure AI Search to give our customers the experience that they deserve.

Conclusion

I hope that this helps you understand when to use which approach with your Microsoft Copilot solutions, in particular those with Knowledge Copilot and Copilot Agents.