Connecting SharePoint Online to On-Premise Databases with SharePoint Framework (SPFx)


Introduction

Recently we seem to be getting involved in projects after they have gone awry. One of our partners reached out asking for help with an issue they had whilst migrating a SharePoint 2010 environment to SharePoint Online.

The problem was that the customer made extensive use of Dataview Webparts with their SharePoint 2010 environment. These web parts displayed important information used by the business. The Dataview Webparts connected directly to a SQL server hosted within the customer database.

Of course when the web parts were migrated into the cloud, they did not work as they were unable to connect to the SQL Server. This ended up stopping the migration until a solution was found.

The customer did not want to move the data into the cloud due to several reasons but the most important one being all of the other systems dependent on it.

Several options were discussed which will be discussed in the next session.

The approach

So several options were looked at, and as moving the database was quickly ruled out, we came up with these two:

  • Build a solution with PowerApps and use the On-Premise data gateway.
  • Build a solution with SharePoint Framework using a REST API use Azure Hybrid Connections

Whilst the PowerApps solution would take less time, the licensing cost of the PowerApps solution ended up ruling it out due to its total cost of ownership (TCO).

So, the SPFX solution was chosen. The architecture was to use SPFX webparts which connected to a REST API hosted in Azure App Services. The clever part was using Azure App Service Hybrid Connections which allowed us to connect from Azure back into the customer network without the need to reconfigure complex firewalls.

To help visualise the solution, let’s take a look at the architecture.

We ended up having two Azure Hybrid Connection Services running. One for the active live environment and another for the disaster recovery environment.

The data being accessed was sensitive so the REST Api had to be secure. It was configured so that it was backed by Azure AD using an Azure AD Application, implemented with OAuth authentication using JWT Bearer tokens. The SPFX connected to the REST API and authenticates using Open Id Connect to ensure that only authenticated and authorised users can access the API. Further protection was provided by setting the API behind Azure API Management.

Authentication

For the SharePoint Framework web parts to be able to authenticate with the REST API there are a couple of steps that need to be performed:

  • Configure the SharePoint Framework solution to request permission to access the REST API
  • Authorise the request made by SharePoint Framework to access the REST API.

To configure the SharePoint Framework solution take a look at this Microsoft post which provides a good guide (see section Configure the API Permission Requests).

The second part is performed by going into the SharePoint Admin centre and approving the request. Now the point to make is that the user accepting the request needs to be a SharePoint Administrator and also grant admin consent to Azure AD Applications. Basically a Global Admin has this role, so we worked with the IT team to ensure a privileged user did the authorisation. Be mindful of this when deploying to the customer as that will take some discussion and time to organise!

The screen to authorise SharePoint Framework solutions.

Another point to make here is that the name of the Azure AD Application configured in the SharePoint Framework needs to be map to the name of the Azure AD Application configured in Azure AD. This is configured as mentioned above in the SharePoint Framework solution. When I first looked at this I set the resource to be the resourceid for the Azure AD Application rather than the name of the application.

Hopefully mentioning this will mean you do not waste your time getting this right.

"webApiPermissionRequests": [
      {
        "resource": "HROnline Api",
        "scope": "user_impersonation"
      }
]

Azure Hybrid Connections

The Azure Hybrid Connection is setup in two places.

  • Azure App Service hosted in the cloud
  • Hybrid Connection service – running as a Windows server within the network.

The hybrid connection service establishes a connection to the Azure App Service through Azure Relay which is built on top of Service Bus.

Diagram of Hybrid Connection high-level flow

To setup the Hybrid Connection in Azure App Service you must be running at least the Basic Tier or above.

There are some limitations to the types of connection that the technology supports. The transport mechanism needs to be TCP based and does not support UCP. For this solution a .NET SQL Client was used which is supported and works really well.

For information on setting up the Azure Hybrid Connection see the following Microsoft article.


Performance

One of the areas that we wanted to ensure was the performance of the applicatoin. So we put together a POC was put together to prove the approach and also check performance. The performance has been very good and provided that the REST ApI is developed with some thought, it performed better than expected.

There was plenty of thought that went into the API. A few of the optimisations we made were

  • making sure that we had support for paging and limiting the number of records retrieved at one time.
  • Using Dapper and performing filtering at the SQL layer rather than pulling the data down and filtering in the API

Conclusion

This solution enables SharePoint Online solutions to access data hosted On-Premises and it does work really well. To be honest we were surprised how well the solution performed.

Most importantly, he partner and customer were really happy with the end result too.

I hope that people find this post useful, if there is an aspect that you would like more information on then leave a comment and I’ll see what I can do.

SharePoint Online Search Results and PnP Search Web Parts suddenly stops working


Introduction

We have been working at iThink 365 with one of the NHS Trust in the UK to move their SharePoint 2010 environment to SharePoint Online.

One of the tools that we have made a lot of use of is the excellent PnP Search Webparts. If you have not checked these out then you must do so, more information here (https://microsoft-search.github.io/pnp-modern-search/).

Here is an example of something that was built with them.

The strange thing we were seeing is that every once in a while the search results would stop working. No one seem to do anything or at least change anything but they would stop working.

The search results used custom search result sources (site settings -> manage result sources) and Microsoft SharePoint search rather than Microsoft Graph.

Resolution

After a lot of trying out different queries in the search result sources page the issue was down to one main issue.

Here is the original result source query which looked like this:

The fixed version ended up being this, do you see the subtle difference?

Note: the double quotes around the path.

On a couple of occasions the ” were stripped out of the query when a user went into the search results and we think that an administrator was saving the result query rather than cancelling and this seemed to break the search web part.

Anyway hope that helps someone as it took us a few hours to work out what was going on!