Tag / cloud

by Simon Doy

Teams AI Library Blog Series: Episode #2 – Teams AI Library App using Azure AI Search – Ready your Stack!


Introduction

This blog post is part of a series of posts describing the setup of a Microsoft Teams application that uses Team Toolkit, and Microsoft Teams AI library and extends it to allow you to deliver an application that can reason over SharePoint content using Azure Open AI Services.

If you have come to this and have not read the introduction post, then please starter there. It will give you more context on what we are trying to achieve.

As mentioned previously the solution will require the following components to be set up:

  • Azure Open AI Service
  • Azure AI Search
  • Setup of an Index for SharePoint content.

There is an assumption being made that you already have your SharePoint content setup that you wish to be reasoned over with your GPT model hosted in Azure Open AI.

Steps to delivery

The steps required to deliver the solution are as follows:

  • Setup of Azure Open AI Services.
  • Setup of Azure AI Search.
  • Setup of SharePoint Index.
  • Or optionally Setup of Azure Blob Storage Index

Setup of Azure AI Search

The Azure AI Search needs to be set first up, in this article we will configure it through the Microsoft Azure Portal.

Please check out the Create a search service in the portal – Azure AI Search | Microsoft Learn article for instructions on setting up the service.

Please note that depending on the region that we create the Azure AI Search within, then different regions have different functionality, you can use the Azure Products by Region | Microsoft Azure page to find out whether Azure AI Search has the correct features. I have had good success with France Central.

We want our Azure AI Search to be able to support the Semantic Ranking feature, since I set up the Azure AI Search service back in December, the rollout has continued and delivered this feature into most of the major Azure Data Centres now.

Setup Azure Open AI Service

The setup of the Azure Open AI Service requires that your subscription is enabled for Azure Open AI Services. This is to help manage the demand for AI services in the various Azure Data Centre. To request access you will need to fill out the Azure Open AI Service request form to get access.

Once you have filled out the form, it may take a few hours or so to receive a reply that your form has been processed.

Follow the instructions, How-to: Create and deploy an Azure OpenAI Service resource – Azure OpenAI | Microsoft Learn, to set up the Azure Open AI Services.

Setup of the Azure AI Search Index

The following guides will describe the process of setting up the search index for either SharePoint or Azure Blob Storage.

SharePoint Index Configuration

The SharePoint Index configuration uses the following process, SharePoint indexer (preview) – Azure AI Search | Microsoft Learn.

I have created a GitHub Repository, azure-ai-resources, https://github.com/SimonDoy/azure-ai-resources which has a Postman collection which will make it easier for you to set up your SharePoint indexer.

Please follow the instructions in the README, https://github.com/SimonDoy/azure-ai-resources/blob/main/azure-ai-search-postman/README.md

I want to highlight two things to make sure you have success when you set up the indexer.

  • The instructions discuss the permissions that your Microsoft Entra ID Application needs to have. Make sure you use Microsoft Graph and not SharePoint permissions.
  • Understand the limitations of SharePoint Indexer which is still in preview. It does not work well with changes to documents and document library names.
    • I prefer the Azure Blob Storage Indexer for this reason.

Once you have the search index setup, you can check its progress by browsing your Azure AI Search and checking the status of the index.

To do this:

  • Browse https://portal.azure.com
  • Find your Azure AI Search instance
    • I like to pin the Azure AI service to the left navigation using the All services menu item, search for Azure AI and click on the star to favourite.
  • Click on Indexers
  • Check status of your index using the name that you provided when setting up the index.

Next Steps

This article explained how to set up the Azure AI infrastructure for this solution. You have now got the base of your solution with an Azure Open AI Service, Azure AI Search Service, and set up the process for indexing the content in SharePoint.

The next steps are to start configuring the Teams AI Library application and bring in our Teams AI extension which will allow us to access the Azure AI Search Service and use that information to reason over the data in Azure AI Search.

by Simon Doy

Teams AI Library Blog Series: Episode #1 – Setup A Teams AI Library-Based Application to Reason Over SharePoint Content with Azure AI Search


Introduction

This article is the product of discussions that occurred at a recent Microsoft 365 PnP Dev community call.

On the call. several discussions were asking to have content around setting up a Teams app. The Teams App should be able to use Generative AI (Ideally with the Teams AI library) that can reason over content in a SharePoint Document library.

Back in December, I built a little app from one of the Teams AI library samples to do just that. I had been thinking about authoring this article for a couple of months but thought that someone else would do it.

Anyway, I have not really found anyone who has put something out there quite like this. Yes, there are lots of articles about Teams AI and Copilot Studio but not quite like this one, so here it is.

The Teams AI library is a fascinating toolset and enables developers with relative ease to be able to create Teams-based Chatbots which integrate with Open AI or Azure Open AI Services to deliver solutions using Generative AI.

The nice thing about the Teams AI library is the way that you can build the prompts to deliver clever applications. Do not believe me? Check out the Twenty Questions sample.

Last year, one of the projects that I wanted to build, was to create a bot that can consume SharePoint content via Azure Open AI Services through Azure AI search. The problem was that when I came to look at this, no abstract or example referenced allowed you to use Azure AI Search with Teams AI library.

This article is part of a series of blog series which will describe how to get this all set up and show you how an extension to the library to be able to use Azure AI Search. As I started authoring this article, I realised that it would be quite overwhelming as one article, so I have split it up into a blog series.

To help consume this content there will be the following articles:

Let’s get into it.

Solution overview

The beautiful thing about the Teams AI Library is that it handles a lot of the complexities of chatbots for you.

For example, AI-powered chatbots work better if they have the history of the conversation included as part of the prompt. This way they are more natural to use, less annoying and behave in the way that you would expect.

Of course, the Teams AI Library has a mechanism for reasoning over content. It uses the Retrieval Augmentation Generation (RAG) pattern to take a user’s natural language prompt, extract the intent and pass that to a data source for extract of results and bringing those results back to the LLM for a natural language response to the user.

In Teams AI Library these are called data sources and there is an interface provided that you can implement.

However, the samples that I have seen implemented are for a local file system of files. Now, I want to be able to use the data source from an Azure AI Search for these queries. By doing this we can access content hosted in SharePoint, Azure Blob Storage and a whole lot more using third-party connectors.

So, I built this abstraction, the AzureAISearchDataSource which implements the DataSource interface and allows you to use Azure AI Search as the data source.

What is more, I have shared this library with you via my GitHub repository. This has the current implementation and will be updated as required.

In the rest of this post, I will explain the steps that are needed to build and deliver this solution. The solution includes the setup of:

  • SharePoint Document Library.
  • Azure AI Search
  • Azure AI Search Indexing
  • Building the Microsoft Teams application.

I will say that the landscape is moving quickly so where possible I have referred to Microsoft articles to help describe the steps. Hopefully, that way if something changes, you’ll still be able to continue setting up the solution.

However, if it is not clear please reach out to me.

I am assuming that you already have your SharePoint content setup that you wish to reason over, with your GPT model.

Next Steps

In our next post, we will start you on your journey by setting up the Azure AI infrastructure for your solution.