Connecting SharePoint Online to On-Premise Databases with SharePoint Framework (SPFx)


Introduction

Recently we seem to be getting involved in projects after they have gone awry. One of our partners reached out asking for help with an issue they had whilst migrating a SharePoint 2010 environment to SharePoint Online.

The problem was that the customer made extensive use of Dataview Webparts with their SharePoint 2010 environment. These web parts displayed important information used by the business. The Dataview Webparts connected directly to a SQL server hosted within the customer database.

Of course when the web parts were migrated into the cloud, they did not work as they were unable to connect to the SQL Server. This ended up stopping the migration until a solution was found.

The customer did not want to move the data into the cloud due to several reasons but the most important one being all of the other systems dependent on it.

Several options were discussed which will be discussed in the next session.

The approach

So several options were looked at, and as moving the database was quickly ruled out, we came up with these two:

  • Build a solution with PowerApps and use the On-Premise data gateway.
  • Build a solution with SharePoint Framework using a REST API use Azure Hybrid Connections

Whilst the PowerApps solution would take less time, the licensing cost of the PowerApps solution ended up ruling it out due to its total cost of ownership (TCO).

So, the SPFX solution was chosen. The architecture was to use SPFX webparts which connected to a REST API hosted in Azure App Services. The clever part was using Azure App Service Hybrid Connections which allowed us to connect from Azure back into the customer network without the need to reconfigure complex firewalls.

To help visualise the solution, let’s take a look at the architecture.

We ended up having two Azure Hybrid Connection Services running. One for the active live environment and another for the disaster recovery environment.

The data being accessed was sensitive so the REST Api had to be secure. It was configured so that it was backed by Azure AD using an Azure AD Application, implemented with OAuth authentication using JWT Bearer tokens. The SPFX connected to the REST API and authenticates using Open Id Connect to ensure that only authenticated and authorised users can access the API. Further protection was provided by setting the API behind Azure API Management.

Authentication

For the SharePoint Framework web parts to be able to authenticate with the REST API there are a couple of steps that need to be performed:

  • Configure the SharePoint Framework solution to request permission to access the REST API
  • Authorise the request made by SharePoint Framework to access the REST API.

To configure the SharePoint Framework solution take a look at this Microsoft post which provides a good guide (see section Configure the API Permission Requests).

The second part is performed by going into the SharePoint Admin centre and approving the request. Now the point to make is that the user accepting the request needs to be a SharePoint Administrator and also grant admin consent to Azure AD Applications. Basically a Global Admin has this role, so we worked with the IT team to ensure a privileged user did the authorisation. Be mindful of this when deploying to the customer as that will take some discussion and time to organise!

The screen to authorise SharePoint Framework solutions.

Another point to make here is that the name of the Azure AD Application configured in the SharePoint Framework needs to be map to the name of the Azure AD Application configured in Azure AD. This is configured as mentioned above in the SharePoint Framework solution. When I first looked at this I set the resource to be the resourceid for the Azure AD Application rather than the name of the application.

Hopefully mentioning this will mean you do not waste your time getting this right.

"webApiPermissionRequests": [
      {
        "resource": "HROnline Api",
        "scope": "user_impersonation"
      }
]

Azure Hybrid Connections

The Azure Hybrid Connection is setup in two places.

  • Azure App Service hosted in the cloud
  • Hybrid Connection service – running as a Windows server within the network.

The hybrid connection service establishes a connection to the Azure App Service through Azure Relay which is built on top of Service Bus.

Diagram of Hybrid Connection high-level flow

To setup the Hybrid Connection in Azure App Service you must be running at least the Basic Tier or above.

There are some limitations to the types of connection that the technology supports. The transport mechanism needs to be TCP based and does not support UCP. For this solution a .NET SQL Client was used which is supported and works really well.

For information on setting up the Azure Hybrid Connection see the following Microsoft article.


Performance

One of the areas that we wanted to ensure was the performance of the applicatoin. So we put together a POC was put together to prove the approach and also check performance. The performance has been very good and provided that the REST ApI is developed with some thought, it performed better than expected.

There was plenty of thought that went into the API. A few of the optimisations we made were

  • making sure that we had support for paging and limiting the number of records retrieved at one time.
  • Using Dapper and performing filtering at the SQL layer rather than pulling the data down and filtering in the API

Conclusion

This solution enables SharePoint Online solutions to access data hosted On-Premises and it does work really well. To be honest we were surprised how well the solution performed.

Most importantly, he partner and customer were really happy with the end result too.

I hope that people find this post useful, if there is an aspect that you would like more information on then leave a comment and I’ll see what I can do.

Issue with Microsoft Forms and Uploading File Questions and deleted folders


Introduction

We have been working on a solution with a local university and part of the solution makes use of Microsoft Forms and uses the file attachment feature.

By the way, this is only available to forms which are authenticated and is not available for anonymous forms.

Anyway the forms are associated with a Microsoft Teams group and one of the team did some tidying up of the default Shared Documents/Documents library in the Team site.

A few minutes later all the Microsoft Forms were unable to upload files….

So it turns out that Microsoft Forms uses the default Document Library and creates a Apps/Microsoft Forms/[Form Name]/[Question Name]/ folder which it uses to upload the files to.

Investigation

I’ll be honest that I had not really thought about the fact that the document library was being tidied up and the impact that might have had. The folders in question were moved to another Team’s channel so I thought, “well I will move it back”.

Unfortunately, that did not fix the issue and when the Microsoft Form loaded up it told me that it could not “create an upload file folder in OneDrive for Business”.

I had another idea, I will restore the folders from recycle bin. I had to be honest I thought this would work.. alas it did not.

So it seems that Forms includes a reference to the Id for the folder and I did not have a way of changing that.

So the solution….

Solution

So the solution was to recreate the file upload questions and then update the Power Automate flows consuming them.

Not really a great solution to be honest, but if anyone has worked out a proper solution, I would love to know. I will have to do some more digging when I get time… not sure when that will be πŸ™‚