Tag / microsoft

AI generated with the prompt please create an image which has the title "How to solve: Copilot Studio Agents, Managed Solutions, and Knowledge Sources that you cannot change" The image is for a blog post and is related to development, Copilot Studio, configuration that cannot be changed, frustration and then enlightenment for the developer. If you could feature me as a manga character that would be great. here is my photo.
by Simon Doy

How to solve: Copilot Studio Agents, Managed Solutions, and Knowledge Sources that cannot be changed


Introduction

When you build and deploy a Copilot Studio Agent with Knowledge as a managed solution, we have a problem.

The problem is that the Knowledge Source Url cannot be changed!

You see, when you configure a Knowledge Source to use information, for example, from SharePoint, you provide a URL for where that knowledge exists by pointing to a SharePoint site or document library.

Ideally, you are using different sources in Development, Testing and Production. When you deploy your Agent in a Managed Solution, the knowledge source is managed, and the URL cannot be changed.

Hopefully, the Copilot Studio product team will allow us to use Environment Variables to configure these URLs in the future, but until that happens, what options do we have?

We could use the same URL in all environments, but it’s not great if we want to do proper testing.

Also, if our development and test environments are built in separate tenants, then we have a problem. This is often the issue when building solutions for customers, where we deploy into our customer tenants.

A Solution

We can utilise DevOps and Build and Release Pipelines to export a solution, unpack it, modify the URL that the Knowledge Source uses, and then repackage the solution for deployment into the target environment.

To get started, read the article on the Power Platform Build and Deployment Pipelines that I built a few years ago. These have been extended to enable support for configuring Copilot Studio Knowledge Support Urls.

I have updated the Github Repo with the latest versions of the pipelines, so you can get access to all the code from there. If you use the instructions in my article above then will be able to get everything set up. However, there are some new parameters that I need to make you aware of.

New Tooling Overview

The pipelines are held in the dev-ops folder, and there is a new folder called tools, which has a PowerShell script called Update-PowerPlatformSolutionKnowledgeSite.ps1

This PowerShell script takes three parameters, and you will need your managed solution as a zip file that has been extracted into a subfolder to test.

The parameters are the following

  • DirectoryPath – this is the path to the extract solution zip.
  • KnowledgeSourceName – this is the name of the KnowledgeSource within the zip file.
  • SiteUrl – this is the URL that we should use for the new KnowledgeSource.

The script will look for an appropriately named folder which has the following naming convention

search value *[AgentName].knowledge.[KnowledgeSourceName]_*

e.g. cr633_copilotForHr.knowledge.HumanResources_XHInmH4G9R5qccpXQxHCs

Once it has found that folder, it will look inside for a data file.

The data file has the following structure, including a site: parameter. The script simply modifies that entry with the provided URL and saves the file.

Now, we simply pack the solution file back and deploy it.

Fortunately, for you, all the pipelines have been updated to do all this great work. You just need to configure them!

Configuring the Pipelines

I am going to presume that you have got the pipelines up and running using the other series of blog posts.

So, to get Pipelines configuring your Knowledge Sources, do the following:

  • open /dev-ops/i365-powerplatform-solution-release-pipeline.yml
  • Find the environment stages in the pipeline (see screenshot above). There are two. Change the parameters to the following
    • KnowledgeSourceUpdate: true
    • KnowledgeSourceName: [the name of your knowledge source, see above about the script to find out how to get that name]
    • KnowledgeSourceSiteUrl: [the URL to your SharePoint site that you wish to change it to for that environment]
  • Save your pipeline and commit changes to git.
  • Run the build pipeline to get all the tools into the assets for your release.
  • Run the release pipeline (the one we have just changed).

Now, when your Agent is deployed, your knowledge URL will be changed.

Here is an agent in Development.

Here is the agent being deployed by the pipeline.

Here is the agent deployed into Production

Conclusion

In this article, we discussed the challenge faced in configuring the knowledge sources when deploying Copilot Agents built on Copilot Studio with managed solutions.

I highlighted an approach for making changes to the URL of these knowledge sources.

I hope that it helps you and let me know how you get on!

Gemini generated image that shows the title of the blog post Tackling Content Filtered Errors in Copilot Agents. It should a robot picking up an agent as thought to rearchitect the Copilot Agent.
by Simon Doy

Tackling ContentFiltered Errors in Copilot Agents – Rethinking Copilot Agent Architecture


Introduction

So I’ve been using Copilot Agents more and more everyday, whether that is personal or in my worklife to help with my personal workflow.

I have been spending time looking at how I can embed AI into my daily routines.

In particular, I have been looking at how I can use agents to make me more productive and efficient. One area that I spend a lot of time on is keeping up to date with what’s going on in the world. This has been something that I’ve been using agents to do, horizon scanning!

So, horizon scanning is the process of looking into trends. What’s going on with the latest news for me that is the latest general news, business news, tech news, but also using it as well to help me keep up to date with AI and technology. Of course, I am constantly trying to keep up to date with Microsoft 365.  Also, I want to keep an eye on white papers and research from various outlets such as Google, Microsoft, Open AI, Gartner, Mckinsey, etc.

Since GPT-5 has been launched and has more capabilities in terms of research and reasoning, I’ve really been spending more time trying to use these models with these agents as I get better results.

Now, one of the challenges recently has been that when I build these agents using Copilot Studio, I am looking to get content sent to me in the morning. Copilot Studio has triggers which can be executed for all sorts of reasons, and I have been using the daily scheduling trigger, which fires off every morning. This workflow calls a Copilot Agent and gets a result. Unfortunately, I have been getting errors when those agents run. These errors are Content Filtered errors or exceptions, and they come about when Microsoft’s Responsibility AI detects an issue and kicks in because it thinks there is an attack occurring against the AI.

Here is an overview of the Agent Flow

Being an MVP, I am fortunate to be able to get access to the Copilot Studio Product Team. So I reached out and explained the issue I was seeing. They reviewed one of my agents, and they said that’s an issue in the way that I’m asking the Copilot Agent to execute the agent. From the Responsibility AI perspective, it looks like an attack on the system. The reason is that the prompt being run is trying to manipulate the output, and so it looks like I am trying to manipulate the AI to do something it wasn’t instructed to do. Therefore, it’s being picked up as an attack, and so, you know, I need to not do that.

So, this got me thinking. I need to rethink how I architect these agents. Copilot Studio, as you are probably aware, have the concept of topics. Topics allow you to have an agent which can support multiple capabilities within one agent. For each topic, you configure the topic by describing how the topic should be detected and used. This is used by Copilot Studio’s orchestration engine to understand which topic to trigger.

This allows the building of an agent that supports multiple capabilities, each with their individual workflows or sub-processes.

In my example, I had an agent that had two topics. One topic for getting the latest news, and another topic for researching the latest research and white papers. These topics were being executed by an internal trigger which executes an Agent Flow. The Agent Flow calls the Copilot Agent with a prompt that states whether it’s the latest news or the latest research that I want. It was this that was triggering the ContentFiltered Error and meant that I was not getting any information back.

So this has got me re-thinking my approach and now I have changed the Agent so that it is now two Agents, one for getting the latest news and the other Agent gets the latest research.

All the details of what the agent should do are in the Agent instructions, and I simply call the Copilot Agent with the prompt, “Please execute your instructions”, and away it goes.

Now, since these changes have been made, the Agents have been working reliably for the past few days.

Conclusion

So, when you are thinking about the architecture of your agents, think about how they are going to be executed. Look at having multiple agents rather than using topics, when you are having external systems or processes calling an agent from outside rather than directly from the Copilot Studio agent.

So rather than having one agent with say 5 topics, you would have 5 agents, one for each topic. If you wanted to be able to access the agent from one place, then you could look at building a main agent that about the other five agents and each of those agents would represent a topic.

This is where my thinking is going these days when architecting these solutions. There are certain challenges and considerations to think about when building architectures with child agents, so it might be that they are not needed, but it depends on how the users need to interact with your “main” agent..