SharePoint 2016/2013/Online: How to Optimize SharePoint Custom Pages Using HTML5 IndexedDB API

In this article we will discuss another obvious performance issues with SharePoint Solutions involving large volume of data transactions surfacing SharePoint Custom Pages.

This could become more prominent if we have strict governance in place and we are not allowed to make use of advanced server side options (Custom Web Service End Point, MTA Enabled Modules etc.).

In one of the recent assignment I came across a similar scenario where I need to crawl data from an external Web Service end Point and surface data on SharePoint Pages. Since the anticipated data volume was huge and traditional caching approaches like Cookies wont’ work due to size limitations.

In pursuit of the solution I have gone through the “HTML5 Web Storage APIs” that allows you to setup an In-Browser Transactional Database System called “IndexedDB”.

Here is a quick introduction of IndexedDB for details I must recommend you to visit IndexedDB

“IndexedDB is a transactional database system, like an SQL-based RDBMS. However, unlike SQL-based RDBMSes, which use fixed-column tables, IndexedDB is a JavaScript-based object-oriented database. IndexedDB lets you store and retrieve objects that are indexed with a key; any objects supported by the structured clone algorithm can be stored. Operations performed using IndexedDB are done asynchronously, so as not to block applications.”

I also want to thanks to “Raymond Camden” for his detailed research on Storage Limits for IndexedDB and believe you must refer this link to understand the limits carefully before getting into concrete implementations.

Now let’s try to understand the implementation details by using following diagram:

Solution Architecture Diagram & Explanation

1

In this solution the SharePoint Page will try to look for the required data in Local Indexed DB created to support this page. If data is not found in local database, page will issue the request for data from SharePoint List.

Since we are dealing with “100,000” Items present in SharePoint List, I made use of “REST API + OData Continuation” data access strategy to overcome SharePoint List Threshold Limits. This mechanism will access only 100 List Items at a time and it is safe to extend this limit up to 2000 items per fetch.

Each fetch will a JSON Object that will be persisted into Indexed DB as individual record. I opt this strategy to reduce the page load time. If the number of items are not much you can add each item as separate record.

Every subsequent data call will be automatically diverted to the local database as primary source.

Additionally we can add “Auto Refresh Modules” to keep the local database fresh with SharePoint List Changes and sync the changes with Indexed DB “Asynchronously”.

Ideally speaking for a complete solution “Auto Refresh Modules” are must to have.

So this all about execution summary for this solution.

Now let’s have look at implementation details as follows-

I have created a SharePoint List with two columns and “100,000” Items added to it as shown below.

Demo

This list will be acting as data source for the page. In actual scenarios this source could be a Web Service End Point which can provide voluminous data on demand.

2

3

Before getting into code let’s see how this Page will behave on execution. Demonstrating the page in action will be helpful later when we get a deep dive in code.

If we run the page we will see this page took about “3 minutes” to get execution completed.

The first execution cycle will include the following actions:

  1. Initialize IndexedDB Database
  2. Query SharePoint List
  3. Add REST API Response to IndexedDB
  4. Load page with data from IndexedDB

Since we are adding data to the store asynchronously, overall application will remain functional even it is taking 3 minutes to complete.

4

Following screen shot showing data adding to IndexedDB asynchronously

5

We can also review the Indexed DB initialized as the part of this request using “Developer Tools or F12 Key” with in the browser as shown below-

6

We can explore each item in the each of the JSON Object as shown below-

7

Now refresh the page to see the execution again and we can see roughly “1 second” to complete the page request.

The subsequent execution cycle will include the following actions:

  1. Query IndexedDB for data
  2. Load page with data from IndexedDB

So we can see how we can trim the execution path by using a well-defined strategy.

8

Code Analysis

Let’s do the code analysis to understand the concrete implementation.

In Step 1 we are enclosing some of the literals as variables and will refer theses variables later in the code

9

In Step 2 we are checking if respective Indexed Database is initialized already or not and if not Initialize the Database. In this demos let’s call this database as “Products”

10

In Step 3 “onsuccess” event handler will get executed and database object will get stored in a global variable “SharePointOptimization.sharePointStore”. This variable will be acting as start point for all the operations on the database in future.

In Step 4 default error handling module is assigned as callback function to “onerror”, “onblocked”, “onabort” event handler

11

In Step 5 we are querying SharePoint List using REST API

12

In Step 6 we are making use of OData Continuation Techniques to overcome SharePoint List Threshold restrictions.

In this step we also call “AddDataToStore” function that will add SharePoint List Items coming as JSON Object to the Local Indexed Database.  It is important to recall that in this demo I am storing 1 JSON Object as 1 record in database and each object contains information for 100 List Items.

13

In Step 7 we are adding JSON Objects to IndexedDB. In order to do that we need to perform following operations-

  • Initialize Transaction with Read Write Operation Permissions
  • Get Handle on “Products” Database inside IndexedDB Data Stores
  • Call asynchronous “add” method to add JSON Object to “Products” Store

In Step 8 we are calling “QuerySharePoint” function to query data from SharePoint List in case data is not available in Local Database.

14

Steps 9, 10, 11 explains about “ReadSPStore” function where we will read the data from Local Data Store (IndexedDB)

In Step 9 following operations are performed-

  • Initialize Transaction with Read Operation Permissions
  • Get Handle on “Products” Database inside IndexedDB Data Stores
  • Call asynchronous “count” method to get total number of JSON Object available in “Products” Store

In Step 10 following operations are performed-

  • Check for get count request status
  • If success Initialize Indexed DB Cursor by calling asynchronous “openCursor” function

In Step 11 following operations are performed-

  • Check for get cursor request status
  • If success read the record from IndexedDB and add to the local array variable
  • Call “continue” function as long as there are items left in local store
  • Once all data is read and save to the local array pass this array to “RenderUI” function to render this data on the interface as required

15

In Step 12 we can plug any UI engine to produce more intuitive UI as applicable, for the sake of this demo I am writing out the Count of Store records * 100 (since each record contains 100 Items) to show the total number of items stored in the local store.

16

Steps 13, 14, 15 show you a helper function to check if local store contains required data or not. It helps to decide if we need to read data from Local Store or SharePoint List

“GetProductCount” function is quite similar to the “ReadSPStore” function except it perform a lesser number of operations

17

In Step 16 we will initialize Local SharePoint Store by calling “InitializeSharePointStore” function

18

In Step 17 we can see some of the UI elements to build a basic UI for this demo

19

Point of caution

Before implementing this mechanism make sure you have identified all the compatibility issues around this corner.

I would recommend you to refer the following site every now and then to make sure you are using features supported by the targeted browsers.

http://caniuse.com/#search=IndexedDb

20

Since I have made use of artifacts which are compatible with SharePoint Online Development Guidelines so we can use this approach with pages hosted in SharePoint Online as well.

That is all for this demo.

Hope you find it helpful.

 

SharePoint 2013 : Batch Exception Handling

Batch Exception Handling in JSOM is a new concept in SharePoint 2013 that enables us to execute Try Catch and Finally code blocks in the same way as we do in Server Side Code i.e. executing with in the same Server Call.

For instance, while working with Server Side code if we need to create a new list then we need to first check its existence and if the list does not exists only then code will create it.

Point to notice here is this whole logic will gets executed in the same Server Call.

On the other hands if we try to perform the same action while working with JavaScript Client Object Model (JSOM) we need to issue two separate calls to the server, one is to check for the existence and other is to load or create the list.

Using new Batch Exception Handling constructs “startScope, startTry, startCatch, and startFinally we can achieve this in single JSOM Call to the Server.

In this article we will see the implementation details for Batch Exception Handling using SharePoint Hosted App.

  • Create a new project using “App for SharePoint 2013” Project Template

1

  • Specify the Host Web URL
  • Specify “SharePoint Hosted” as Hosting Model

2

  • Wait while Visual Studio configure Project for you

3

  • Prepare UI using HTML Elements decorated the desired CSS using Default.aspx (App Start Page)

Step 1: Specify the Page Title

Step 2: Add HTML to present the UI Elements for “Result Panel” and decorate the element with the required CSS of your choice

Step 3: Adding container to hold the button to execute Batch Exception Handling Process

Step 4: Add a HTML Button, specify its Label and decorate it with the required CSS

4

  • Write code to implement the logic for Batch Exception Handling by modifying App.js JavaScript File

Step 5: Bind the event handler with the Button using dynamic binding in “document. Ready” function

5

Inside the Event Handler function “batchExceptionHandling”

Step 6: Instantiate SharePoint Context and get the instance of the current web (all old school thing)

Step 7: Instantiate the Exception Handling Scope

Step 8: Start a Scope block by calling startScope() method

Step 9: Start a Try block by calling startTry() method

Step 10: Inside the Try Block, try to load the instance of the list in our case list is “My-Custom-List”

Step 11: Dispose the Try Block by calling dispose() method

Step 12: Start the Catch Block by calling startCatch() method

Note: This catch block will be executed only when action in try block gets failed, like in this case if we try to load “My-Custom-List” and it is not available in the SharePoint Site the operation will get failed and catch block gets executed.

Step 13: Since we get an exception while loading the list that means list is not available, so adding the new list with the same name

Step 14: Dispose the Catch Block by calling dispose() method

Step 15: Start the Finally Block by calling startFinally() method

Step 16: Load the list now, since by this time we will be having list available anyways (Created if not exists or loaded if exists)

Step 17: Dispose the Finally Block by calling dispose() method

Step 18: Dispose Scope Block by calling dispose() method

6

Step 19: In the success callback function check for the exception occurred during the function call by calling “get_hasException()” method, if the method returns True that means we had created the list else we already had a list that we have loaded.

7

With this step we are all done with the coding part and now the next step is to launch the app and see some action.

  • Build and deploy the solution

8

  • Enter the Credentials when asked for Authentication

9

After successfully authenticating the User, we can see the Start Page for the App

10

Open the SharePoint Manager and browse the App Web -> Lists and see if we got the list “My-Custom-List” created or not, in our case we do not have any such list created

11

Now click the Button “Batch Exception Handling” and see the Result Panel

First time when we click the button, JSOM code executes within the Exception Handling Scope, and see if the list is already available in the Site or not, if not it will be added by the code present in catch block as shown below

12

We can confirm the same in SharePoint Manager as well-

13

When we click the button afterwards the code in Try Block gets executed and successfully loads the list due to which in the Success Callback function “get_hasException()” function returns False and we get an alternative message in the Result Block as shown below –

14

I found this technique really effective when we have a scenario for implementing Fall Back mechanisms without involving multiple Server Calls for the same request.

Hope this simple implementation could help you to understand the concept of Batch Exception Handling.