The blog of Wictor Wilén

SharePoint 2013: Enabling PDF Previews in Document Libraries with Office Web Apps 2013

Mon, 29 Apr 2013 00:42:00 PDT

Introduction

A couple of weeks back I blogged about the March Update for Office Web Apps 2013 and also how you could use that update to show PDF previews in a SharePoint 2013 Search Center. Since then I’ve received a lot of requests on how to enable PDF Previews in a Document Library, which isn’t there by default. Of course it is not a WAC thing, it’s a SharePoint 2013 thing – but the SharePoint 2013 updates (up until now at least) does not provide this capability either.

In this post I will show you that it can be done. It’s a JavaScript thing and can be done using a Content Editor Web Part added on all pages where you want the PDF previews or as Farm solution which uses a delegate control and a custom JavaScript file.

Build the PDF Preview solution

I assume that you are familiar with SharePoint 2013 development and knows what a delegate control is. What you need to do is create a new empty Farm solution project. In this project we’ll create a new web control that will add a JavaScript (which we will create in just a minute) to the page. The implementation should look like below:

[MdsCompliant(true)]
public class PdfPreviewControl: WebControl
{
    protected override void OnPreRender(EventArgs e)
    {
        ScriptLink.RegisterScriptAfterUI(this.Page, "wictor.pdfpreviews/previews.js", false);
    }
}

The JavaScript file, added as a Layouts file, is what makes the magic happen. We’re using the Script-On-Demand features in this script to make sure that the scripts aren’t executed before the SharePoint filepreview.js file is loaded. Once that file is loaded two new JavaScript objects are created; the filePreviewManager and the embeddedWACPreview. To enable the PDF previews we only need to add the PDF extension to these objects and specify the previewer objects and the dimensions. In this case I use the same settings as the other Word previewers.

function WictorsPdfPreviews() {
    SP.SOD.executeOrDelayUntilScriptLoaded(function () {
        filePreviewManager.previewers.extensionToPreviewerMap.pdf = 
            [embeddedWACPreview, WACImagePreview];
        embeddedWACPreview.dimensions.pdf= { width: 379, height: 252}
    }, "filepreview.js");
    notifyScriptsLoadedAndExecuteWaitingJobs("wictor.pdfpreviews/previews.js");
}
WictorsPdfPreviews();

Now we need to make sure that this control loads the JavaScript on all pages. This is done by adding a new Empty Element SPI and creating a Control element pointing to the web control, like this:

<Control 
  ControlAssembly="$SharePoint.Project.AssemblyFullName$" 
  ControlClass="Wictor.PdfPreviews.PdfPreviewControl" 
  Id="AdditionalPageHead" 
  Sequence="100"/>

As you can see we’re adding this control to the AdditionalPageHead, which means that we will have it on every page. Do not forget to add the web control as a Safe Control in the project!

The final thing we need to do is to modify the Feature that was automatically created when the Empty Element SPI was added to the project. You can scope it to whatever you like, but I want it for all Document Libraries in my farm so I set the scope to Farm. The image below shows all the files in the project.

Deploy and Test

Now all we have to do is to deploy the solution. Once the solution is deployed and the Farm feature activated we can navigate to any document library and upload a PDF file. Note that you have to be on at least the March 2013 update of Office Web Apps Server and that you have enabled the WordPDF application (see previous blog post). Once you have the PDF file in the library you can click on the ellipsis button and see the PDF Preview:

Disclaimer

As always when it comes to stuff that is not documented. I do not give any guarantee that this will work on your machine(s) or after any upcoming SharePoint patches.

Summary

Enabling PDF Previews are not (yet) a default feature of SharePoint 2013 but can easily be added to your SharePoint farm – if you’re allowed to use Full Trust solutions. If you don’t feel like you want to do some hacking yourself you can download the WSP here and deploy it yourself to try it out.

Recertified as Microsoft Certified Solutions Master (MCSM) for SharePoint

Wed, 17 Apr 2013 02:28:00 PDT

Yesterday I got the really cool news that I completed all recertification requirements for the Microsoft Certified Solutions Master: SharePoint certification. Couldn’t be a happier SharePoint professional right now!

What is the MCSM and what about MCM?

The Microsoft Certified Master (MCM) program has during the latest year transitioned into the Microsoft Certified Solutions Master (MCSM) program. It is not only a change in name but also a change made to adapt to the new world order. The program is not longer focusing on one specific version of the product but instead focus on what’s in the market at the current moment and specifically it covers both on-premise and cloud solutions. This is good in many senses – this allows the program to always be current, always use the latest techniques and technologies etc. The MCM was a certification without expiration date (well eventually the product cease to exist, but you still have the cert) whereas the MCSM has a three year life span and you must recertify to stay on top.

The first MCSM : SharePoint rotation!

I was fortunate to be able to participate in the rotation called U3, or Upgrade 3, which was the first MCSM rotation for SharePoint. It was two weeks on site, in the always sunny and warm Seattle, in January. I had the opportunity to spend these two weeks in a class room with the finest SharePoint professionals there is. We had great instructors, awesome labs, and fantastic discussions over the two weeks duration. It all led up to one written exam, called the Knowledge Exam, and one hands on lab, called the Qualification Lab. As always the QL was basically doing a couple of weeks worth of deep dive SharePoint work in about 8 hours time. Taken into account that this was a beta it was just about the hardest work I’ve ever done – but it was pure fun. And I made it! Phew…

Congrats to all my other friends that made it through this rotation and best of luck to the ones who will have the joy of doing the exams one more time! You can do it!

Introducing Open WOPI - an open WOPI Client for SharePoint, Exchange and Lync

Mon, 15 Apr 2013 10:50:14 PDT

Today at the SharePoint Evolutions 2013 Conference I announced my latest pet project called Open WOPI. Open WOPI is an open WOPI client that allows you to extend SharePoint 2013, Exchange 2013 and Lync 2013 with file previews and editors for any type of file formats.

The project is now (at least very, very soon) available to download from openwopi.codeplex.com and is published under the Ms-PL license. This is currently an early beta (or what you would like to call it) but will be improved over time.

Standards

Open WOPI is based on the [MS-WOP] protocol, published by Microsoft, and used by Office Web Apps Server 2013, SharePoint 2013, Exchange 2013 and Lync 2013.

File format support

Currently Open WOPI has support for the following formats:

GPX - Uses Bing maps
TXT - Viewing and editing
VSDX - Thumbnail viewing

More formats are in the works…

Documentation

Not much yet, but I’ll try to add that over the next few weeks.

More information

For more information, feedback and ideas about the project please refer to the Codeplex site: openwopi.codeplex.com. I’d like to hear what file formats you would like Open WOPI to support.

The slides from the presentation at the SharePoint Evolutions conference and links can be found here.

Contributors

Thanks to Sam Dolan, aka Pinkpetrol, for the really cool logo for Open WOPI.

SharePoint 2013: Enabling cross domain profile pictures

Tue, 09 Apr 2013 02:07:00 PDT

Just discovered a really interesting and just awesome nugget in SharePoint 2013 that solves a problem that have been annoying me for a long time. The problem manifests itself when you’re having multiple URL’s for your SharePoint farm or when using SAML or Forms based login (like in Office 365 and SharePoint Online) and you’re using the profile pictures on sites not residing on the My Site Host Web Application (or host named site collection). Then the user profile picture is not shown, you get the default image not found image or you’re prompted to authenticate with the My Site Host.

Let’s take an example. Assume I have one site at intranet.contoso.com and the My Site host exists on mysite.contoso.com. I have not configured any Internet Explorer zones or anything and I’m promted to log in at each location. This is how the Newsfeed Web Part will look like on intranet.contoso.com, if I cancel out the authentication prompt or if I’m using some forms based login:

You see, no fancy picture of Mr administrator! There’s a couple of ways to solve this using IE Zones, anonymous access etc, but none are perfect and comes with consequences.

So how can I get the picture to be shown without messing with security, cross domain issues etc. Fortunately I guess I was not the only one that was annoyed by this (most likely everyone using Office 365 as well) so the SharePoint team has added a new feature to SharePoint that allows us to show profile pictures cross-domain.

It’s a very simple operation and just requires some basic PowerShell skills. Basically all you need to do is to set the CrossDomainPhotosEnabled property on the SPWebApplication object to true, like this:

asnp Microsoft.SharePoint.PowerShell
$wa = Get-SPWebApplication http://intranet.contoso.com
$wa.CrossDomainPhotosEnabled = $true
$wa.Update()

Now the Newsfeed, in the sample above, will look like below. And I was not prompted for any authentication or anything! Isn’t that sweet! And it works very well on Host Named Site Collections as well.

Basically what happens behind the scenes is that the request for the user picture is sent via a “proxy” .aspx page called userphoto.aspx which takes a couple of parameters; URL of the picture, the account name (or e-mail) as well as the picture size (S, M or L). This page will return a JPEG stream of the user profile picture without crossing any domains on the client side.

I hope this little nugget will save you and your customers a lot of time and annoyance..

SharePoint 2010 Web Parts in Action as the Manning Deal of the Day

Thu, 04 Apr 2013 22:58:00 PDT

If you still haven’t picked up on my book about SharePoint Web Parts – SharePoint 2010 Web Parts in Action, then this is the chance for you. Today (5th of April) the book is featured as the Manning Deal of the Day. All you have to do is browse to http://www.manning.com/wilen/ and then use the dotd0405au promotion code. This will give you 50% percent discount, for you non-math-geniuses that’s half off the full price! And since you now saved a couple of bucks, there’s another Manning book with the same deal and promotion code today and that’s PowerShell in Action 2nd edition.

PS: Even though the name of the book implies SharePoint 2010, it’s fully applicable to SharePoint 2013 – very little has changed in the Web Parts space.

Well, what are you waiting for, go get it…

Speaking at SharePoint Evolutions Conference 2013

Thu, 04 Apr 2013 00:04:00 PDT

In less than two weeks I’m speaking at the SharePoint Evolutions Conference 2013 in London. It is as exciting for me as it is for all attendees. The conferences held by Combined Knowledge has proven over the last years to be the highlight of SharePoint conferences around the world – everything from the venue, to the people, to the sessions and to the parties are surgically planned and executed – no one leaves without a smile on their face and pumped with knowledge!

As a speaker being selected for this conference is the by far the proudest moment of any SharePoint professional and I’m so glad that I’m back for another year, hanging around with the best of the best, and not only speaker wise – the speakers at the Evolutions conference are the people that knows the product inside out and work with the product in real projects.

This year I’m having one session called Extending SharePoint, Exchange and Lync 2013 with a custom WOPI Client. I will walk you through the WOPI protocol, which is what Office Web Apps Server 2013 (WAC) is built upon, and show you how to make your own WOPI client for custom file formats, for viewing, editing, previews etc. This will be an interesting session and show you a really interesting “partner opportunity”. I’ve prepared a good set of demos, so be there!

Looking forward to seeing old and new faces at this conference. If you haven’t registered yet – there’s still a chance for you to get your very own ticket.

Renewed as SharePoint Most Valuable Professional (MVP) for 2013

Mon, 01 Apr 2013 07:56:00 PDT

I just received the confirmation that I am renewed as SharePoint MVP (Microsoft Most Valuable Professional) for my fourth consecutive year. It’s an honor being chosen among all the professionals around the world, especially now when SharePoint is getting more and more widespread and is being adopted by more and more companies worldwide.

I’d like to take the opportunity to say thanks all my colleagues at Connecta, that put up with me, and all my friends around the world that I’ve learnt to know throughout these years. I’ll continue to write obscure blog posts and show up at conferences, and I will continue to organize the Swedish SharePoint User Group meetings.

Thank you all for the support!

SharePoint 2013 Managed Metadata field and CSOM issues in 2010-mode sites

Wed, 27 Mar 2013 17:01:00 PDT

Introduction

SharePoint 2013 introduces a new model on how Site Collections are upgraded using the new Deferred Site Collection Upgrade. To keep it simple it means that SharePoint 2013 can run a Site Collection in either 2010 mode or 2013 mode, and SharePoint 2013 contains a lot of the SharePoint 2010 artifacts (JS files, Features, Site Definitions) to handle this. When you’re doing a content database attach to upgrade from 2010 to 2013, only the database schema is upgraded and not the actual sites (by default). The actual Site Collection upgrade is done by the Site Collection administrator when they feel that they are ready to do that and have verified the functionality of SharePoint 2013 (or you force them to upgrade anyways). But, the Site Collection admin might have to upgrade sooner than expected for some sites.

Upgrade troubles

This post is all about what has happened to us when our Office 365 SharePoint Online (SPO) tenant had been upgraded (and right now the Site Collection upgrade is disabled for us so we can’t do anything about it). The limitations of customizations in the 2010 version of SPO was very limited (to be kind) and we embraced JavaScript (which people have despised for a decade and now suddenly think is manna from heaven). We also leveraged Managed Metadata in a lot of lists, libraries and sites. We’ve built Web Parts using JavaScript CSOM to render information and also .NET CSOM stuff running in Windows Azure doing background work. Once our tenant was upgraded to 2013 all of the customizations using Managed Metadata stopped to work…

JSOM sample with Managed Metadata

I will show you one example of what works in SharePoint 2010 and what does not work in SharePoint 2013 when you’re site is running in 2010 compatibility mode.

Assume we have a simple list with two columns; Title and Color, where Color is a Managed Metadata Field. To render this list using JSOM we could use code like this in a Web Part or Content Editor or whatever.

var products;
function loadProducts() {
	document.getElementById('area').innerHTML = 'Loading...';
	var context = new SP.ClientContext.get_current();
	var web = context.get_web();
	var list = web.get_lists().getByTitle('TestList');
	products = list.getItems('');
	context.load(products, 'Include (Title, Color)');
	
	context.executeQueryAsync(function() {
		var collection = products.getEnumerator();
		var html = '<table>'
		while(collection.moveNext()) {
			var product = collection.get_current();
			html +='<tr><td>'
			html += product.get_item('Title');
			html += '</td><td>'
			html += product.get_item('Color').split('|')[0];
			html += '</td></tr>'			
		}
		html += '</table>'
		document.getElementById('area').innerHTML = html;
	}, function() {
		document.getElementById('area').innerHTML = 'An error occurred';
	});
}
ExecuteOrDelayUntilScriptLoaded(loadProducts, 'sp.js');

As you can see a very simple approach. We’re reading all the items from the list and rendering them as a table, and this html table is finally inserted into a div (with id = area in the example above). This should look something like this when rendered:

The key here is that Managed Metadata in 2010 JSOM is returned as a string object (2010 .NET CSOM does that as well). This string object is a concatenation of the Term, the pipe character (|) and the term id. So in the code sample above I just split on the pipe character and take the first part. There was no other decent way to do this in SharePoint 2010 and I’ve seen a lot of similar approaches.

Same code running in SharePoint 2013 on a SharePoint 2010 mode site

If we now take this site collection and move it to SharePoint 2013 or recreate the solution on a 2010 mode site in SharePoint 2013. Then we run the same script, then this is what we’ll see, something goes wrong…

You might also see a JavaScript error, depending on your web browser configuration. Of course proper error handling could show something even more meaningful!

Something is not working here anymore!

What really happens is that the CSOM client service do not return a string object for Managed Metadata but instead a properly typed TaxonomyFieldValue. But that type (SP.Taxonomy.TaxonomyFieldValue) does not exist in the 2010 JSOM. Remember I said that SharePoint 2013 uses the old 2010 JavaScripts when running in 2010 compatibility mode. Unfortunately there is no workaround, unless we roll our own SP.Taxonomy.TaxonomyFieldValue class (but that’s for another JS whizkid to fix, just a quick tip to save you the trouble – you cannot just add the 2013 SP.Taxonomy,js to your solution).

So why is this so then?

If we take a closer look at what is transferred over the wire we can see that when running on SharePoint 2010 the managed metadata is transferred as strings:

But on SharePoint 2013 it is typed as a TaxonomyFieldValue object:

It’s a bit of a shame, since the server is actually aware of that we’re running the 2010 (14) mode client components! (SchemaVersion is what we sent from the CSOM and LibraryVersion is the library used on the server side)

I do really hope that the SharePoint team think about this for future releases – respect the actual schema used/sent by the Client Object Model!

Of course, this JavaScript solution will not work as-is when upgrading the Site Collection to 2013 mode. That is expected and that’s what the Evaluation sites are for.

What about .NET CSOM?

We have a similar issue in .NET CSOM, even though we don’t get a SharePoint CSOM runtime exception. Instead of returning a string object you will get back a Dictionary with Objects as values – but if you’re code is expecting a string you still get the exception. So in 99% of the cases it will fail here as well.

Summary

Deferred Site Collection update might be a good idea and you might think that your customizations will work pretty well even after an upgrade to SharePoint 2013, just as long as you don’t update your actual Site Collections to 2013. But you’ve just seen that this is not the case.

Happy easter!

SharePoint 2013: Enabling PDF Previews with Office Web Apps 2013 March 2013 update

Wed, 13 Mar 2013 08:06:00 PDT

In my last post (still smoking fresh) I showed you how to update your Office Web Apps 2013 farm to the March 2013 update, connect it to SharePoint 2013 and being able to view PDF documents in the browser. What I didn’t explain or show in that post was how to enable the PDF Previews in Search – but I’ll do it now.

Pre-requisites

Before you start fiddling with this, you need to make sure that you have the March 2013 update of Office Web Apps Server 2013 (WAC) installed and connected to your farm – if you don’t know for sure, ask your admins – sometimes they know…if they don’t give them the link to my previous blog post.

Note: You don’t have to have a patched SharePoint 2013, this will work on the RTM bits.

Default PDF Search Experience

SharePoint 2013 natively supports crawling PDF documents, through the new document parsers. That is you don’t have to fiddle with any custom PDF IFilters etc. The native PDF document parser is a good enough solution, but have some room for improvements.

The Search Experience and display in SharePoint 2013 is based upon Display Templates. Display Templates decide how the result should be shown and how the fly-out of the result should look like. For Office documents, when SharePoint 2013 is connected to a WAC farm, SharePoint displays inline previews which you can use to skim through the results really quick. For PDF this is not the case – not even if you use a WAC farm with the March 2013 update (even though the WOPI binding supports the interactivepreview action). This is a sample on how a PDF document could look like in SharePoint 2013 Search:

Enabling PDF Previews in the Search Result

Since we have the opportunity to modify the Display Templates and create our own Search Experience we can very easy modify the fly-out/hover panel of the PDF results to show the interactive preview. We can do this in two different ways…

Create a new Result Type

The easiest and fastest approach to enable our previews for PDF documents is by creating a custom Result Type. This is done by going to Site Settings > (Search) Result Types and then finding the PDF Result Type. Choose to Copy the Word Result Type. This will create a new Result Type.

Give the new Result Type an appropriate name, “PDF with Preview” for instance. Then scroll down to Actions and in the “What should these results look like?” drop-down, choose to use the Word Display Template.

Then just click Save, go back to your Search Center and search for a PDF document and voilá – we have PDF Previews.

Modifying the PDF Display Templates

The second approach, which is a bit more advanced is to actually modify the PDF Flyout Design Template. You do this by going to Site Settings > Design Manager then choose 5. Edit Display Templates. Locate the PDF Flyout item by filtering on the Target Control Type column and use SearchHoverPanel, then scroll down to PDF Hover Panel. To modify the PDF Hover Panel, use the Word Hover Panel as a template. I’m not going in to all the details on how to modify the actual HTML file (see my similar post on how to achieve this here). But once you’ve modified it, make sure to save it to the gallery so the JS file is generated, and publish it. Now (if you made the correct updates) you will have Custom Search Previews for PDF documents.

Summary

You’ve now seen how you quick and easy can enable PDF Search Previews in SharePoint 2013 using the March 2013 update for Office Web Apps 2013. All you need to do is either create a custom Result Type and use the Word Display Templates or modify the default PDF Display Template. I really hope that Microsoft makes this a standard feature for upcoming SharePoint 2013 releases.

Office Web Apps 2013: Patching your WAC farm with no downtime

Wed, 13 Mar 2013 03:03:00 PDT

I’m really glad to see some patches being rolled out for Office 2013, SharePoint 2013 and Office Web Apps 2013. There’s some really important fixes and some very interesting fixes that I’ve been waiting for. In this post we’ll take a look at the first Office Web Apps 2013 (WAC) update – specifically we’re looking at how to patch your WAC farm to minimize the downtime. If you follow my instructions you will have zero downtime (except for a brief moment where Excel stuff will not be accessible).

Background and preparations

For this sample I will have a SharePoint 2013 (RTM) farm connected to a WAC 2013 (RTM) farm. The WAC farm is load balanced using NLB, like illustrated on the right.

On order to update our WAC farm we need to download the March 2013 patch for Office Web Apps 2013. You can find it at Microsoft Download center and it is called “Update for Microsoft Office Web Apps Server 2013 (KB2760445)”. The KB article does not reveal much of what has changed, but if you take a look at an earlier patch KB2760486 (March 5th) you’ll notice some really cool things. If you don’t have time to read it – I’ll show you some of them at the end of this post. Ok, you do not have to install KB2760486, you only need the KB2760445 one.

Patching the first WAC Server

After downloading the patch and copying it to our WAC machines we can start the patching process. We would like to do this without the users even noticing it. First of all we need to take one of the servers out of rotation in the WAC farm. You do this using the load balancer. In my case I’m using Microsoft NLB and using PowerShell this is an easy task. Next thing to do is just remove the WAC server from the WAC farm – that is also a one-line PowerShell command. WAC Servers should only be patched with the binaries installed, not when they are participating in a WAC farm.

These two lines does the trick:

Get-NlbClusterNode -NodeName $env:COMPUTERNAME | Stop-NlbClusterNode -Drain
Remove-OfficeWebAppsMachine

NOTE: You MUST remove the machines from the old farm before patching and then later create a new WAC farm (follow the instructions thoroughly). You cannot patch a running WAC Farm!

Once this is done the server is no longer receiving requests and it’s no longer a part of the WAC farm and we can start patching this machine. All our end-user requests are now going to the remaining server(s) in the WAC farm.

To patch the machine start the downloaded .exe file (wacserver2013-kb2760445-fullfile-x64-glb.exe) and just follow the instructions; accept license agreement etc. You will be asked to close any PowerShell (IDE) windows that has the Office Web Apps module loaded and you might get asked to reboot once the update is finished (there’s of course no harm in doing it anyways).

When the patching is done (and you rebooted) we need to create a new WAC farm, on the patched server. You do this using the same way as you did for the RTM version. Use the same server name as your old farm, same settings and the same certificate.

Patching the rest…

Once the new farm is created you have a few options depending on how many servers you have in your WAC farm. For instance if you have more than two, then you can take another one out of rotation, patch it and join it to the new and patched farm and then you flip the load balancer so it points to the new WAC farm. Depending on your load balancer you might have a brief moment where end-user request might be served by both versions. A bit funky situation but it works, as the illustration below shows:

In my case I have two servers, so once I created the new farm I just put the patched server into the NLB rotation and then immediately take the RTM server out of rotation. Once the RTM WAC Server is taken out of rotation I remove WAC from that machine and start the patching process. All requests to the WAC farm are now on our patched server(s).

As you can see, simple PowerShell operations this time as well, and this can (and should) be automated.

# On the new patched WAC Server
Get-NlbClusterNode -NodeName $env:COMPUTERNAME | Start-NlbClusterNode

# On the RTM WAC Server
Get-NlbClusterNode -NodeName $env:COMPUTERNAME | Stop-NlbClusterNode -Drain
Remove-OfficeWebAppsMachine

Once the other server(s) are patched, you need to join it to the newly created (and patched) WAC farm – just as you normally join a server to a WAC farm. And then you’re done. You’re farm is now patched and ready! But wait, there’s some stuff you have to do on the SharePoint side to enjoy all the new cool features.

Configuring new WOPI bindings

One of the most interesting new features in the March 2013 update of Office Web Apps is the ability to view PDF documents! Yup, you heard me right! The March 2013 update contains a couple of new WOPI bindings that enable viewing and previewing (even mobile views) for PDF documents. These new WOPI binding has an application name called WordPdf. If you try to use the New-SPWOPIBinding cmdlet and just specify the WAC server name, you will get an error – since you already have those WOPI bindings. What you need to do is to specify the name of the application as an extra parameter to the command, like this:

New-SPWOPIBinding - ServerName wac.corp.local -Application WordPdf

Another approach could be to remove all the bindings and then add them all back again – of course this would cause a few seconds of downtime, and that’s not how I roll…

Once you’ve run that command (and waited a couple of seconds) you can start previewing PDF documents in your Document Libraries. And now we got yet another good reason not to install the crapware called Adobe Reader!

Now, your end users can start taking advantage of all the new features in the March 2013 update.

New Excel WOPI Bindings!

If you’re examining the WOPI bindings provided by the March 2013 update really carefully, you’ll notice that the Excel application has six new bindings, three each for the two new actions syndicate and legacywebservice. We cannot use these two bindings in SharePoint 2013 RTM – the New-SPWOPIBinding cmdlet does not accept those actions. They are not a part of the well-known WOPI action values (as specified in the WOPI specification available when this post is written) – and I have at the moment no clue of what they are doing…stay tuned…

Summary

You’ve just seen how simple patching an Office Web Apps 2013 farm is. Compare that to patching SharePoint! By using the load balancer effectively you can make sure that the end-users can continue to use the Office Web Apps features while you’re doing the patching. Enjoy!

SharePoint 2013: Intelligent Promoted Results and Best Bets

Sun, 10 Mar 2013 09:50:00 PDT

Introduction

The search engine and search experience in SharePoint 2013 has been totally re-written, since its predecessors. In SharePoint 2010 we had something called Best Bets or Visual Best Bets if you worked with FAST for SharePoint. A best bet was a promoted result that was triggered by one or more keywords, used by the search admins to promote certain links or banners for specific search queries. In SharePoint 2013 this is now called a Promoted Results and the procedure of creating them is different and so much better – there’s more ways to trigger on, more ways to render the results etc, but the actual shown result isn’t that smart, until now…

In this post I will show you how to create an even smarter and more intelligent Promoted Result – a best bet that actually uses the search query to do something interesting with. In this sample I will let users enter simple math questions and then we let the promoted result calculate the math question for you (just as the big search engines on the interwebz does).

Creating Promoted Results aka Best Bets

A Promoted Result is created either on Search Service Application, Site Collection or Site level using the Search Query Rules option. To create a new Site Collection scoped Promoted Result you navigate to Site Settings > Search Query Rules. Next thing to do is to choose which scope to create this Query Rule for, choose Local SharePoint Results in this case. You have the option to further narrow down the Query Rule using segments and topic categories, but that’s not relevant for this exercise. Next thing to do is to click on New Query Rule. The Add Query Rule page contains quite a few options to make sweet things happening. First we have to give it a name and then we have to specify the query conditions. Query Conditions is what is used to trigger this promoted result. In order to trigger on our “mathematical” question we need to use a regular expression. To use a regular expression as the Query Condition you have to select Advanced Query Text Match and then write a regular expression.

I’m no regular expression savant, but this regular expression works for this blog post. Anyone with a cooler or smarter or more complex regex – feel free to post it in the comments.

\d+\s*[/\+\-\/*]\s*\d+

Using a custom page as the Promoted Results

Next up is to add the Promoted Results Action to this Query Rule. This is done by clicking on the Add Promoted Result further down on the Add Query Rule page. When adding a promoted result you will be asked for a Title, a URL and a description. In our case the URL is of interest – we want to render a page containing our logic to calculate the math query.

I let the URL point to a page (an .aspx page we’ll create in a few seconds) and then I pass two query string parameters to it; IsDlg=1 to get rid of the chrome in the page and query={SearchTerms} to pass the actual query to the page. I also check the check box Render the URL as a banner instead of as a hyperlink so that the promoted result will be rendered as an iframe instead of a link.

Seem simple, doesn’t it, but wait – this will not work until you followed all the steps in this post. There is no thing out of the box that allows you to pass the search query to the Visual Best Bet.

Now click Save on the Add Promoted Result page and then Save on the Add Query Rule page.

Creating the custom promoted results page

Now, before we test this (and it also takes a couple of seconds before this promoted result can be used – don’t ask me why!) let’s create our page that should calculate the result from the query. For this demo we create a fairly simple .aspx page with a JavaScript section and a small HTML snippet, as shown in the snippet below:

<%@ Page Inherits="Microsoft.SharePoint.WebPartPages.WebPartPage, Microsoft.SharePoint, 
    Version=15.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c" 
    MasterPageFile="~masterurl/default.master" Language="C#" %>
<asp:Content ContentPlaceHolderID="PlaceHolderAdditionalPageHead" runat="server">
    <script type="text/javascript">
        function getQueryStringValue(name) {
            var qs = window.location.search.substring(1);
            var pairs = qs.split("&");
            for (var i = 0; i < pairs.length; i++) {
                var pair = pairs[i].split("=");
                if (pair[0] == name) {
                    return pair[1];
                }
            }
            return '';
        }
        _spBodyOnLoadFunctions.push(function () {
            var query = getQueryStringValue("query")
            try {
                document.getElementById("result").innerHTML = eval(query)
            } catch (e) {
                document.getElementById("result").innerHTML = "error"
            }
        });
    </script>
    <style>
        #s4-ribbonrow {
            display:none;
        }
    </style>
</asp:Content>

<asp:Content ContentPlaceHolderID="PlaceHolderPageTitleInTitleArea" runat="server">
    Intelligent Best Bet
</asp:Content>

<asp:Content ContentPlaceHolderID="PlaceHolderMain" runat="server">
    <h1>SharePoint Search Calculator</h1>
    <h2>The result is: <span id="result"></span></h2>
</asp:Content>

The first section contains a simple JavaScript that extracts the Search Query from the query string and then uses the JavaScript eval() method on it to “calculate” the value. In the PlaceHolderMain I added a span element which is where the calculator writes the result.

So let’s upload this page to the Documents library in the Search Center – which was the location I specified when creating the Promoted Results. If we now search for something that should be triggered by the Query Rule you will see the page rendered as a banner, but it will not calculate the query as we expected.

Passing the search query to the custom promoted results page

So, why aren’t our result rendering properly? It all comes down to that the default Display Template for Best Bets/Promoted Results does not pass the search query to the iframe. In order to fix this we need to modify this Display Template. The Display Template that we need to edit is the Item_BestBet.html file.

First we add two lines, directly following the var isVBB = … line, like this

var isVBB = ctx.CurrentItem.IsVisualBestBet;

var url = ctx.CurrentItem.Url
url = url.replace('{SearchTerms}',ctx.CurrentItem.ParentTableReference.Properties.QueryModification)

The code is crafting the URL to use in the iframe by retrieving the URL from the CurrentItem and then replacing the text {SearchTerms} with the actual search terms used.

Next we need to modify the line where the iframe is rendered.

<iframe id="_#= $htmlEncode(id + Srch.U.Ids.visualBestBet) =#_" 
    class="ms-srch-item-visualBestBet" title="_#= $htmlEncode(ctx.CurrentItem.Title) =#_" 
    scrolling="no" frameborder="0px" 
    src="_#= $urlHtmlEncode(url) =#_"></iframe>

Modify the src attribute of the iframe element so that it uses our new url property instead of the default URL value from the current context. Save the file and go back to the Search Center…

See it in action

Now we can take it for a test drive. Search for simple mathematical questions and see the results being rendered as a promoted result.

Summary

I’ve just shown you how to make the Promoted Results aka Visual Best Bets more intelligent by creating a Query Rule that promotes a web page that does operations on the search query. To get this all to work the secret ingredient was to modify the Display Template for promoted results to allow it to pass the search query to the promoted results page.

I can really see this being extended to provide the users with even more interesting “intelligent” promoted results.

SharePoint 2013 Central Administration Productivity Tip

Wed, 06 Mar 2013 14:24:00 PST

Here’s a short post with a tip that I find very useful.

In many scenarios you have several SharePoint 2013 installations to handle – it might be different farms, production environments, testing, staging, development etc. Do you know which Central Administration you’re working in at the moment? They all look the same, SharePoint Blue, the regular Status Bar warning that you’re running out of disk space etc. Unless countermeasures are taken you don’t know what environment you’re in unless you take a look at the URL – which in many cases is just another server name and port. It’s very easy to make a mistake and make a change in the production environment instead of in the test or dev environments.

So, how do we keep track of what Central Administration site we’re actually working on at the moment? One way could be to change the theme of the Central Admin site. But, was the production CA red or was it the one with the dog in the background? I’ve got a better tip for you!

Here’s how I’ve done to keep track of the Central Administration sites. I take advantage of the Suite Link Bar in the upper left corner. By default it says just “SharePoint” – yes we know it’s SharePoint.

By modifying the Web Application property that controls this text we can easily change it to something more friendly and appropriate for the specific farm, like below.

It’s a very simple PowerShell operation to accomplish this. You just retrieve the Central Administration Web Application object, then update the SuiteBarBrandingElementHtml property and set it’s value to something that tells you which Central Administration site this is:

asnp microsoft.sharepoint.powershell
$ca = Get-SPWebApplication -IncludeCentralAdministration | `
    ?{$_.IsAdministrationWebApplication -eq $true}
$ca.SuiteBarBrandingElementHtml = "<div class='ms-core-brandingText'>Central Admin: FarmA Production</div>"
$ca.Update()

You should leave the div element with the ms-core-brandingText class, to get decent formatting of the text, but inside it you can add whatever HTML you like (I’m thinking the marquee or blink tag…)..

That’s it. I hope I saved a few kittens from being slaughtered…

SharePoint 2013: Personal Site instantiation queues and bad throughput

Tue, 05 Mar 2013 00:18:00 PST

In SharePoint 2013 the way Personal Sites (aka My Sites) are created have been totally rebuilt to support the new way of utilizing the Personal Sites. In this post I will go through how Personal Sites are provisioned, asynchronously, and bust a couple of myths about how interactive Personal Site instantiations should be “prioritized” and increase throughput.

Background

Personal Sites or My Sites were previously created “on-demand”. When a user went to his or hers non-existing My Site the provisioning started while the user waited for the site to be created, painfully watching the spinning animated gif. This worked fine in SharePoint 2010, and earlier, but now with SharePoint 2013 so much more are depending on the user having a Personal Site – everything from the social stuff in SharePoint 2013 (yes, not all SharePoint customers have yet wandered down the Yammer road) to the really great Office 2013 interaction (SkyDrive whatever). Requiring users to “manually” creating their Personal Sites is no longer a good option…

Note: this post is valid for SharePoint 2013 RTM – and hopefully some of the myths/bugs/features/stuff that is explained in this post will be fixed in upcoming releases…

Asynchronous Personal Site instantiation

In SharePoint 2013 the Personal Site is no longer created synchronously while the users wait for the site to be created. Instead all Personal Site provisioning are handled asynchronously by the Timer Job service (OWSTIMER.exe). So how does all this work? Whenever a user visits the My Site Host for the first time they are forced to create a Personal Site (depending on the My Site creation settings in the User Profile Service Application). The Personal Site instantiation are then added to one of possibly three queues (more about these in a bit). A set of timer jobs picks up the instantiation requests from these queues, creates the Personal Site and finally sends an e-mail to the user. The benefits of this new approach are many; the user can continue to do other stuff in SharePoint (real productive work instead of facetweeting for instance), the provisioning are not hogging the w3wp.exe process, we avoid performance bottlenecks, we should(!) see improved Personal Site creation throughput etc. There are of course drawbacks of this approach, such as it could potentially take some time before the personal site is created, since we have to wait for the timer job etc. But all in all this is something that is necessary (especially for the cloud) and something is very good.

The Instantiation queues

I mentioned that there are three queues for the Personal Site instantiation.

The Interactive Request Queue
The Second Interactive Request Queue
The Non-interactive Request Queue

First of all – the difference between the Interactive and Non-interactive requests are basically that all “requests” to create a Personal Site through the web user interface are called interactive, while those requests coming from the Office client are non-interactive. The reason of having a second interactive request queue is that requests coming from the user interface – users actually wanting to facetweet – should have a higher priority and a higher personal site provisioning throughput.

Each queue are (by default) polled every minute, by three timer jobs – one per queue, Each Timer Job will retrieve five (5) items from the queue and process them one at a time. According to some (ahem, SPC12) presentations this timer job are executed on every WFE, but that’s not what happens in real life. The Timer Jobs are based on the SPWorkItemJobDefinition Job Definition Type. This is a really nice timer job implementation that has a queue per content database. In this case the queue exists in the Content Database where the My Site Host is residing. This also means since the lock type of the job is Content Database per Web Application – only one server will run this timer job. So, to sum it up – for each queue and every minute a total of 5 Personal Sites can be created, which means about 300 Personal Sites per hour per queue – if you have the hardware that can handle that! One might think now that since there are two interactive queues, we can get a throughput of 600 Personal Sites an hour, well…no…

The second (and useless) interactive queue

The idea with the second interactive queue is that interactive personal site requests should have a higher throughput, and that is a good idea. But unfortunately something went wrong when this was implemented. Stop reading now if you don’t like digging deep into SharePoint internals and just accept the fact that this implementation is flawed…

The three queues, or rather the three Timer Jobs are created per Web Application through a hidden feature called MySitesInstantiationQueues. This feature creates the timer jobs and also configures an object in the config database (SPPersistedObject) which contains a value of how many queues that should be used. The ~~funny~~ thing is that this is an internal definition (the SPPersistedObject) and it sets the number of instantiation queues to 1, not 2. This means that we’re hardcoded to only have one instantiation queue, and the second interactive timer job just don’t do anything since it’s queue is always empty – even though we could use reflection to modify it, but then we’re in unsupported territory. So there goes the idea of bumping up the number of Personal Sites created asynchronously.

You can very easily see this in the ULS logs:

Also, if we take a look directly at the configuration object we’ll see that it is configured only for one queue:

So the idea of having servers provisioning up to 600 Personal Sites an hour we instead have 300 Personal Sites per farm per hour. Imagine a new corporate portal launch spiced with social features for 20.000 users and about 5% of the users visiting the site at launch. This will give us 1.000 users in the queue and a potential queue time of 3:20 hours, at least – not that great in my opinion. Just imagine if you did some marketing of your new launch and only a couple of more users tried to use the social features…

Well, the good thing is that we don’t have the web servers hogged with creating thousands of Personal Sites, prohibiting real work.

Increasing the throughput anyways…

This section should really be classified under the unsupported things you shouldn’t do, ever, unless you need to… But for a large Enterprise Social deployment using SharePoint 2013 it might be a necessity, to not piss the end users off by having them waiting four hours to get started using SharePoint 2013.

If you don’t want to provision all the Personal Sites in advance, which in most cases are a dumb idea, you could instead of just relying on the queues to provision the Personal Sites you could read the queue yourself and create the personal sites “manually”. You need to find the database where the My Site is hosted and then query the ScheduledWorkItems table for items. Query for where the Type is E94A6CAA-B0F5-4897-B489-585CA50C7803 (which is the id of the first interactive instantiation queue, the second queue has it’s own Type Id – but you will never see that in here :-)). Find those with the InternalState equal to 8 – those are the ones waiting to be processed. Using this information you can use PowerShell or similar to spread out the load on other servers to create the personal sites.

To create the actual personal sites you could use the CreatePersonalSite() method of the UserProfile object, this method will bypass the queue and immediately create the site.

Summary

By now you should be familiar with the process of how SharePoint provisions Personal Sites in SharePoint 2013. It’s a better approach than in previous versions. The idea of having multiple queues and allowing several servers to help out with provisioning the personal sites is good, but currently flawed. Hopefully this will get fixed in upcoming cumulative updates. As it is now it’s not working for a large enterprise rolling out SharePoint Enterprise Social, so you have to be very careful when planning a roll out of Enterprise Social in SharePoint 2013…or use Yammer…

SharePoint 2013 and Unified Access Gateway (UAG) 2010 Service Pack 3

Tue, 26 Feb 2013 15:19:00 PST

Last week the Forefront team finally released Service Pack 3 for Forefront Unified Access Gateway (UAG) 2010. This is a long awaited release for us working with SharePoint 2013 and for those using non-legacy operating systems and browsers. In this post I will show you how to publish SharePoint 2013 Host Named Site Collections through UAG 2010 SP3 and consume the published site using Internet Explorer 10 on Windows 8.

What’s new in UAG 2010 Service Pack 3

Before diving into the interesting pieces let’s take a look at some of the new features of Service Pack 3. First of all, to be able to install SP3 you need to have your UAG at SP2 level – SP3 is NOT cumulative.

The most interesting features in SP3 are support for Windows 8, Windows Phone 8, Windows 8 RT and Internet Explorer 10 (classic and Metro) as client, support for the Office 2013 client and support for SharePoint 2013 as the published application. For SharePoint 2013 the URL Rule Sets has been modified to handle the new types of URL’s and request patterns used in SharePoint 2013, and new endpoint policies are added for SharePoint 2013. If you’re doing an 2010-2013 upgrade, you need to look over any custom URL Rules and policies. There’s also a number of improvements with regards to ADFS publishing.

Get all the nitty gritty details from the KB article: KB2744025.

Infrastructure

Ok, now you know what’s new! Let’s start publish our SharePoint 2013 sites through UAG. First of all before even installing UAG you need to think about your infrastructure, network, DNS and certificates (there it is again – certificates!). The picture below shows my demo environment. As you can see I have a UAG 2010 SP3 server with dual NIC’s (actually it have three – but we can safely ignore that). One NIC is connected to the Corporate Network and the other one to the External Network.

DNS

Before starting our publishing and SharePoint 2013 configuration we need to plan our DNS namespace. We need at least one publically available (that is a real) domain name. For this demo I will use three:

trunk.corp.local – this will be used for my UAG Trunk Portal
sp.corp.local – this will be a SharePoint 2013 site
teams.sp.corp.local – this will be another SharePoint 2013 site

In order to make it efficient for the end-users I do prefer to have alignment of URL’s internally and externally. In this case it means that I have the second two domains above registered in the internal DNS as well. There is no need to register the Portal/Trunk domain internally.

Certificates

When publishing sensitive stuff such as SharePoint content you most likely would like to use HTTP over SSL, HTTPS, to encrypt the data flying throughout the internet. This is not only a good option to do externally, you should do it internally as well – even more now with SharePoint 2013 and the authorization flow over HTTP. Utilizing HTTPS internally also gives you the benefit of URL alignment for internal and external users and you don’t have to worry about sending an unreachable link. And we don’t have to do SSL termination.

What you essentially need is one or two certificates – one from a trusted root authority to use on the UAG server and one for SharePoint, which can be a local CA issued certificate. You could use the same though. You should use wildcard certs or SAN certificates here – so in the end it comes down to manageability and costs of certificates.

In this sample I use one certificate, issued by my local CA. So any external clients, not joined to the domain will get a certification error (unless they have the CA root cert as a trusted root authority – which my external demo machine have). Specifically the Office clients have issues with not trusted certs.

The certificate is a SAN certificate covering sp.corp.local, trunk.corp.local and *.sp.corp.local. The certificate is installed on the SharePoint 2013 machine and on the UAG 2010 machine.

SharePoint with HTTPS

Before starting the UAG configuration let’s take a quick look at how the SharePoint 2013 farm is configured. In the SharePoint farm there is one Web Application using SSL on port 443. This Web Application is used to host Host Named Site Collections. This installation has two; sp.corp.local and teams.corp.local.

Install and update UAG 2010 to SP3

Installing Unified Access Gateway 2010 and patch it to Service Pack 3 is quite easy. But first of all – UAG 2010 SP3 does not support Windows Server 2012, so you have to dust off your old Windows Server 2008 R2 DVD’s and get used to that annoying Start button in the lower left corner.

Before inserting the DVD though, make sure that the OS is patched – yea, this machine should be a secure box and should have all the latest security hotfixes etc. Secondly make sure that you have the two required NIC’s wired in. Third – configure the NIC’s. If you’re uncertain on how to configure the NIC’s there is a good TechNet Wiki article called “Recommended Network Adapter Configuration for Forefront UAG” that is highly recommended as a starting point.

Once all that is set, insert the DVD and follow the wizard to install UAG. If your DVD is a UAG SP1 you need to apply the following patches, in this order, to get up on Service Pack 3.

Once that is done a reboot is not out of order. Especially if you fiddled with the NIC’s and haven’t rebooted since.

Initial UAG configuration

Once you’re up on UAG Service Pack 3 it’s time to do the first time configuration of UAG. It’s a three step wizard that automatically starts when you fire up the UAG Management console the first time – and you can’t bypass it.

The first step is to connect UAG to your different NIC’s. As I said you need two of them – one for the internal network and one for the external network. In the Network Configuration Wizard (Step 1 in the UAG Getting Started Wizard) you have to choose which adapter to connect to the UAG Internal and External zone respectively. A good practice is to name your network adapters (in Windows) and give them describing names so they are not called just “Local Area Connection 1” and “Local Area Connection 2” – very easy to make stupid mistakes then…

The image below shows how I connected my “Corporate Network” NIC to the Internal UAG zone and the “External Network” to the External UAG zone. The machine have a third NIC – which I leave unassigned in this case.

Click Next when you’re done and finish the wizard. Step 2 in the Getting Started Wizard let’s you specify the UAG topology – leave it as as a single server and finish the wizard. Step 3 is the obligatory – do you want Windows Update and do you want to send usage data to Microsoft – I’ll leave it up to you to figure out the settings here.

Note: you can always get back to the Getting Started Wizard by choosing Admin > Getting Started in the UAG Management app.

Now all is set to start publishing some applications.

Creating a UAG Trunk

All applications in UAG are published in Trunks. A trunk consists of a portal and one or more applications. Each trunk have a set of shared settings between the different published applications – such as a public host name, certificate, URL Rules, endpoint policies, authentication and authorization settings etc.

Since we would like to publish an HTTPS SharePoint 2013 application we start with creating a new Trunk under the “HTTPS Connections” node in the UAG Management App.

This will start yet another wizard (YAW). In Step 1 choose to create a Portal trunk. In Step 2 we need to specify the name, public host name and IP of the trunk.

Step 3 is used to configure how the end-user will authenticate. Click Add to select (at least) one Authentication Server. If you don’t have any servers in the server list you need to add one. You have a lot of options here – but in most cases you will add an Active Directory server and connect it to your forest. You also need a service account for this – that needs permissions to retrieve user information and change password.

In Step 4 it’s time to select the external certificate. Hopefully you followed the instructions and have already registered the certificate on the UAG box. Choose the appropriate certificate and click next. If you’re like me and using a dummy domain (corp.local) you will get a warning here – it’s safe to ignore, we’re only writing a blog post.

The fifth step allows you to select what kind of endpoint protection you would like to use. Choose between UAG access policies or NAP. Choose UAG and continue to step 6, where you choose the endpoint policies – keep the default ones and continue. Then review your settings and click finish.

That’s it, you now have a trunk. All you now need to do is to save and then activate your new configuration. This is done through the cog wheel button in the management console.

You should now be able to test if you can access the trunk from a remote machine. The first time you access the trunk URL you will be asked to download and install the Forefront UAG client components. Depending on your configuration you will have limited access to your applications unless you install it. These components does a number of client security validations and they are also responsible for cleaning up your machine when you log out. Once logged in you should see an “empty” portal saying “No applications defined”.

If you don’t see anything like this or get errors here – it’s most likely related to certificates. Make sure that you have a correctly issued certificate registered with the Trunk. Also check the certificate chain and make sure that all the certs in the chain are trusted.

Add an application to the trunk

Now the time has come to publish our SharePoint 2013 Site Collection. In the Trunk you created under Applications click the Add button. YAW! In Step 1 you specify what kind of application you want to publish. You will find SharePoint 2013 under Web – it’s called “Microsoft Office(!) SharePoint Server 2013”.

Click next, give the application a name. In Step 3 leave all the endpoint policies as-is – you could easily spend a day fiddling with these if you have the time or need. In the fourth step choose to publish an Application Server. You should choose this if you have one server or a SharePoint farm behind some kind of load balancer. Step 5 is where you configure the actual SharePoint 2013 application. Add the name of your internal SharePoint 2013 site and specify the public host name. In my case my SharePoint 2013 site is published using HTTPS internally so I need to specify HTTPS/443 here – no SSL Termination.

Step 6 – Authentication. Choose an authentication server (use the same as you created and used for the Trunk). Here you also specify how rich clients authenticate with your application – in my case I choose to use both Rich Client integration and a Office Forms Based AuthN.

In the seventh step you configure how the application appears in the portal and in the eight and last step you can configure which users actually have access to your application, but just leave the default settings for now – that is everyone who can log in can also access the application (note, that this doesn’t mean they bypass any security in SharePoint).

Once you’re done with the wizard you should have two applications listed in your trunk – the portal and your newly added SharePoint site:

Now save and activate your new configuration.

Take it for a test-drive…

Close down your browsers, if you did test to log in to the Trunk earlier, and browse to the public portal URL once again. If you don’t see your application – log out of the portal and log in again.

You should see something like this:

Now, click on the application link and see how beautifully your SharePoint 2013 site is published through UAG. Test how you can drag and drop documents, open them in Office clients, edit lists in Quick Edit mode and work just perfectly!

Publishing the second SharePoint 2013 Site Collection is exactly the same – you add a new Application to the trunk and follow the wizard.

Summary

In this post I showed you how to install, patch and configure Forefront Unified Access Gateway (UAG) 2010 Service Pack 3 to be able to publish SharePoint 2013 Web Applications and Site Collections. It’s a pretty straightforward approach but involves some crucial steps. If you previously published SharePoint 2010 applications using UAG you noticed that there is no difference with regards to setting it up. The difference are in the details, specifically in the URL rules applied and a new set of policies.

SharePoint 2013: SharePoint Health Score and Throttling deep dive

Mon, 11 Feb 2013 14:33:00 PST

The SharePoint Health Score was introduced in SharePoint 2010 and plays an even more important role in SharePoint 2013. The Health Score determines the health of a SharePoint server/web application on a scale from 0 to 10, where 0 is the most healthy state. SharePoint automatically starts throttling request once the Health Score is to high. The Health Score is can be calculated using many parameters, such as memory usage, concurrent requests etc. In this post I will give you some details on how the Health Score works, how you can troubleshoot it and how you can use it and how you can configure it.

Note: this article contains some registry changes that never ever should be done on a production server, unless told by Microsoft support...or whatever...

What is the Health Score?

Let’s start with some background facts. Stop whining, I know plenty of peeps have already blogged about it but I think it is good to have all the facts in this same blog post.

The Health Score is as an integer between 0 and 10, where 0 represents a really good health of the server, or rather Web Application, and 10 is where you do not want to be. SharePoint continuously calculates the Health Score and for each and every request the Health Score is sent as an HTTP Response Header, called X-SharePointHealthScore.

This Health Score is used by (or at least should be used by) client applications to determine the health of the web application and then use it to make a decision on whether it should stop hammering the web application or not. If the Health Score reaches up to 10, SharePoint itself will start to throttle requests (which we’ll talk about in just a bit).

Worth noticing here is even though this article is written for SharePoint 2013, most of it is valid on SharePoint 2010 as well.

How the SharePoint Health Score is calculated

So, how does SharePoint calculate this Health Score? When a SharePoint Web Application spins up, internally a new thread is created (you can see it in debuggers with the name SPPerformanceInspector). This thread regularly reads performance counters and calculates the Health Score. Each time it calculates the Health Score it will log this to the Trace Logs (Category=Http Throttling, Level=Verbose):

“The current health score for web application is X”

Performance Counters

The Health Score is calculated from a set of Performance Counters. By default SharePoint 2013 (and SharePoint 2010) uses two performance counters for this:

Memory/Available MBytes
ASP.NET/Requests Current

You can easily retrieve these performance counters by using the following cmdlet:

Get-SPWebApplicationHttpThrottlingMonitor http://app.contoso.com

You can also use the Set-SPWebApplicationHttpThrottlingMonitor cmdlet to modify the current monitors.

Buckets, what is that!?

When running the cmdlet above, to retrieve the performance counters used to calculate the Health Score, you should have noticed that each associated performance counter has a property called AssociatedHealthScoreCalculator.

This is an array of values, or a set of Buckets, that SharePoint uses to calculate the Health Score. There are 10 buckets in each of those (you can have fewer). Each bucket represents a Health Score value and corresponds to a lower limit of the performance counter. For instance if you have 24 current ASP.NET requests – you will have a Health Score of 4 and if you have less than 200 MB of RAM available you will have a Health Score of 6. The Health Score of the Web Application is equal to the largest Health Score of all measured performance counters.

Adding your own ,monitor

To add your own performance counters to monitor you need to use the API. Here’s a sample on how to add the CPU usage as a monitored object. You should not add this one into any production environment, but it can be used in testing scenarios when you would like to see what happens when the Health Score is high.

$throttle  = (Get-SPWebApplication http://app.contoso.com).HttpThrottleSettings
$throttle.AddPerformanceMonitor("Processor", "% Processor Time", "_Total", 10, $true)
$throttle.Update()

This will create a new monitor for the Web Application that checks the processor time on the machine. It will only have one bucket. So if the processor time is larger than 10 it will yield a Health Score of 10.

Refresh Interval

The Health Scores are by default updated every 5 seconds. You can get (or set) this setting by looking at the HttpThrottlingSettings of a Web Application like this:

$throttle = (Get-SPWebApplication http://app.contoso.com).HttpThrottleSettings
$throttle.RefreshInterval

To avoid having spikes in performance counters creating high Health Scores, the actual value is calculated from an average of a number of samples, by default 12 samples. This can be configured using the NumberOfSamples property on the HttpThrottleSettings.

How SharePoint uses the Health Score

Now you know a little bit more on how the SharePoint Health Score is calculated. So what is it used for then? SharePoint internally uses it for mainly two reasons; HTTP throttling and for the Request Management Service.

HTTP Throttling

Whenever the SharePoint Health Score reaches the value of 10, SharePoint 2013 starts its throttling mechanism to protect the servers. As soon as the Health Score is below 10, the servers stops the throttling. The actual throttling is divided into two steps – the first and second stage. After receiving a Health Score of 10 and SharePoint enters throttling it will start with the first stage and requests will start to be throttled. After one minute of throttling the throttling enters the second stage and starts throttling even more. What’s being throttled in the different stages depends on a set of throttling classifiers. Out of the box (in SharePoint 2013) there is only one classifier used – which checks if the current request is a search crawler. This classifier starts throttling request directly in the first stage – all other requests are handled as usual. When the throttling enter the second stage all requests will be throttled (default, if you have any custom classifiers some requests might be allowed).

Throttled requests are met with the Error message above: “The Server is busy now. Try again later”, and the HTTP Response: “503 Service Unavailable”.

When entering the second stage of throttling, you will see the following lines in the ULS logs. The logs also includes the “Excessive” counters.

To the Server is Busy response SharePoint also adds two Response Headers; one called “Retry-After” which contains a numeric value which should be taken as an indication (by clients) when a retry operation should be done (it’s always has the same value as the Health Score refresh interval) , and a second one “SharePointError” which always has the value of “2”.

By the way, if you notice the nice little error icon in the Busy error. That image is not throttled, why? Well, I can tell you…there’s actually an exclusion for that specific image with regards to resource throttling. Requests to that image will never be throttled.

HTTP Throttling of POST Requests?

There’s this myth out there that POST and PUT requests are never throttled. Well, that is to good to be true! My tests and reverse engineering skills shows that all requests are throttled in SharePoint 2013 (I would be glad to be proven wrong :). Fortunately Office 2010 and later has the “Upload Center” which will retry any save operations and eventually succeed once throttling has stopped.

But, using one of the built-in classifiers we can actually make sure that our precious documents are saved regardless of HTTP Throttling, and this is how this is achieved.

$classifier = New-Object Microsoft.SharePoint.Utilities.SPHttpUserAgentAndMethodClassifier 
    -Args "","POST","HttpMethodMatch", "Never"
$throttle = (Get-SPWebApplication http://app.contoso.com).HttpThrottleSettings
$throttle.AddThrottleClassifier($classifier)
$throttle.Update()

First of all we create a new classifier of the type SPHttpUserAgentAndMethodClassifier, when creating that object we pass in “POST” as HTTP method, make sure it’s using the HttpMethodMatch method to match the method :), and finally we tell it to never throttle these requests. We add this classifier object to the throttle settings of the web application and badabing, POST operations are not longer throttled.

Turning off HTTP Throttling

Ok, so you’re fed up with this throttling and want to turn it off. It’s an easy operation and you can do it from Central Administration or from PowerShell. In PowerShell just update the PerformThrottle property of the HttpThrottleSettings of the web application. And in Central Administration you select the Web Application for which you want to turn off throttling, then select General Settings > Resource Throttling, then at the bottom of the dialog you will find HTTP Request Monitoring and Throttling.

HTTP Throttling and automagic garbage collection

The HTTP Throttling is actually a quite smart implementation, and internally tries to fix your machine if throttling occurs. One thing that is important here is if you start fiddling with the different performance counters used for calculating the Health Score, make sure that you do not remove the monitor that monitors memory usage. That plays a special role in the throttling scenario. If that specific monitor is considered an “Excessive” monitor, which means that it has an Health Score of 10, it will actually force a Garbage Collection.

Request Management

SharePoint 2013 introduced the new Request Management Service, which is used to “redirect” requests to specific servers based on rules and/or health. When using health based rules, it is the SharePoint Health Score that is used to determine the health of a server. Whenever a machine receives a request it will update the Health Score in the Request Management service, which makes the Health Based Request Management a very efficient way to “load balance” the requests for optimal performance.

For more information about Request Management read Spencer Harbars series about it.

How to use the SharePoint Health Score

Whenever you’re building a client that interacts with SharePoint 2013, for instance a SharePoint App, a Windows 8 app or a Windows Phone app, then you should strongly consider always take the Health Score in consideration. You will receive the X-SharePointHealthScore response header for each and every CSOM, REST or web service call to SharePoint. If the value is 10 (and SharePoint starts throttling) then you will most likely not be able to get data. If your application is constantly polling SharePoint for information, perhaps you should increase the delays between your calls when the Health Score raises above a predetermined number.

Andrew Connell has a small code sample on how to leverage this in a Silverlight app.

The SPPerformanceInspector

You could also use the Health Score in traditional SharePoint Farm solutions (timer jobs, or other long running or resource consuming operations) using the SPPerformanceInspector object. Here’s a quick sample how you can show the data in a Web Part:

SPPerformanceInspector pi = 
    SPPerformanceInspector.GetInspector(SPContext.Current.Site.WebApplication);

this.Controls.Add(new LiteralControl(
    String.Format("Current Health Score: {0}<br/>", pi.HealthScore)));
this.Controls.Add(new LiteralControl( 
    String.Format("Is throttling: {0}", pi.IsInThrottling())));

This Web Part retrieves the performance inspector object and then prints out the current Health Score for the current Web Application on the current machine. Also this Web Part will print out if throttling is currently going on (which you never should see anyways, since you’re throttled!).

How to debug the SharePoint Health Score

To wrap this little post up I will give you two great debugging tips – they both involve editing the registry (you’re warned!). First of all if you’re building a remote application that communicates with SharePoint and you want to test how it behaves at different Health Scores. Then there’s a neat trick to give a server a constant Health Score value. You do this by adding a new DWORD key called ServerHealthScore to the HKLM\SOFTWARE\Microsoft\Shared Tools\Web Server Extensions\15.0\WSS hive and then do an IISRESET. You can give this key any value from 0 to 10 and the Response Header will always show that value. It will not invoke throttling though.

It will of course log a warning in the ULS telling you how bad this really is:

The second tip is a cool debugging feature, and its also a small regkey hack. In the same hive as above create a new DWORD key called DebugThrottle and give it the value of 1.

Once again you need to do an IISRESET. After that you can browse to any page and then append debugthrottle to the address, for instance: http://site.contoso.com/debugthrottle. This will give you a sweet looking dashboard with all the details you need about the health of the server/web application. You can see the monitors and their Health Score history, the throttling stage, your classifiers etc.

Summary

There you have it! All you need to know about the SharePoint Health Score and HTTP Throttling. As you can see it’s a really nice feature if used correctly – by you, by third party vendors (I’m pretty sure 99% of the vendors do not care about this though) and by SharePoint itself. Just one final word of warning – if you’re fiddling with monitors and classifiers make sure that you thoroughly test them, it’s really easy to get things funked up here…

Office Web Apps 2013: Securing your WAC farm

Sun, 10 Feb 2013 09:16:00 PST

With this new wave of SharePoint, the Office Web Apps Server (WAC – I don’t like the OWA acronym, that’s something else in my opinion) is its own server product, implementing the WOPI client protocol, which allows a client to retrieve documents from SharePoint on the behalf of the user. Documents will flow from the WOPI servers (SharePoint, Lync, Exchange etc.) to the Office Web Apps Server – this means that potentially confidential information will be transferred from the SharePoint environment and stored/cached on another server. This could result in unnecessary information leakage and compromise the enterprise security.

In this post I will walk through a number of steps that you can do to properly secure your Office Web Apps 2013 farms. And you should seriously consider and implement most of these methods.

Note: this post focuses on the Office Web Apps Server and not a WOPI client in general (but if you’re building your own you should consider security as well!).

The WOPI protocol specification and security

Note: I will not cover how WOPI clients and servers implements the server to server authentication and authorization.

WAC runs as Local System

To start with it is very important to know that the Office Web Apps Server 2013 runs as the Local System and Network Service on the machine it is installed on. There is no service account or anything! This means that you cannot protect your systems using dedicated accounts etc., like you do with SharePoint, SQL and other applications.

The images below shows the Office Web Apps Windows Service, which runs as LocalSystem.

And this image shows some of the applications pools in the IIS on an Office Web Apps machine.

The advantage of using these local accounts is that it makes installation and configuration easier. But it is very important that you are aware of this configuration.

SSL is a requirement!

Exposing the Office Web Apps Server over HTTPS should be a requirement in my opinion. There is no reason not to. Having it on HTTP will only cause trouble for you; for instance if your SharePoint uses https you will not be able to render the iFrame containing the document (aka WOPI Frame) since you’re not allowed to show http content in an https environment. But first and foremost you’re sending data in clear text.

So what about SharePoint on HTTP then? Well, if you’re using SharePoint 2013 you should seriously consider running that over HTTPS as well – that IS a best practice. SharePoint 2013 leverages several technologies that sends tokens and credentials over the wire, OAuth for instance, so in order to have a secure environment make sure you use HTTPS for SharePoint as well. If you are running SharePoint on HTTP you must fiddle with the security settings in SharePoint to allow OAuth over HTTP – and this is not a good thing.

Certificates are king!

Any WAC farm running on SSL must have a certificate for the HTTPS endpoint. You can use self-signed, issue certificates using a Domain CA or buy a certificate. When you’re creating the WAC farm, using New-OfficeWebAppsFarm, you can/should specify the certificate.

For any SharePoint, WAC and even SQL installations nowadays certificates are more and more important. If you’re on the verge of deploying these in your organization you should consider deploying a Domain CA – which is not a lightweight task.

Securing the communication using IPSec

If you for some reason do not run HTTPS on SharePoint and/or WAC you could consider implementing IPSec. Unfortunately there is no button in the Control Panel that says “Use IPSec”. This is something that requires careful planning and testing. So going SSL might be an easier way. But consider the scenario where you have an internet facing web site which leverages WAC and using the HTTP protocol – then you should consider using IPSec for the communication between SharePoint and Office Web Apps Server.

Firewall considerations and requirements

When setting up your Office Web Apps Farm you should also configure the firewall for the WAC machines. Office Web Apps uses four different ports. It uses 80 and 443 for HTTP and HTTPS, that’s used by the end-users and the WOPI Server/Client communication. Internally Office Web Apps uses port 809 (HTTP) and 810(HTTPS) for communication between the WAC machines. I’ve only seen 809 in use, which is the default. There is no way (I’ve found at least, but internally WAC has a switch to use port 810) to configure WAC to use port 810 and if you do find a way, it’s likely unsupported. The things sent over the wire using the admin channel (809) is mainly health and configuration information for the WAC farm, but it would be nice to be able to secure this channel as well (IPSec!).

When installing WAC the Windows firewall is configured to allow incoming TCP connections on port 80, 443 and 809.

As always it is a good practice to evaluate these default rules and if you’re not using port 80, disable that port. For port 809 it might also be a good practice to make sure that it only allows incoming connections if they are secure (i.e. implement IPsec).

Preventing rogue machines

So far we’ve been talking about how to secure information being transmitted from and to the Office Web Apps farm. Let’s take a look at Office Web Apps farm security from another angle. Joining a new WAC machine to an Office Web Apps Farm can be quite easy. The only thing that you need is local administrator access on the WAC machine that is the master (the Get-OfficeWebAppsMachine gives you the master machine). Depending on how you’re having your (virtual) metal hosted this might be a problem, too many sysadmins have to much permissions out there. If you have this access then you can easily join a rogue machine to the WAC farm and take control over it, without the users/client knowing anything about it.

There are a couple of methods you can and should use to protect the WAC farm. And the error messages below can also be a good troubleshooting reference…

Master Machine Local Administrator

If the account trying to create the new WAC machine does not have local admin access on the machine specified when joining the WAC farm you will simply get an “Access is denied”.

As a side note; if you’re not running the cmdlet using elevated privileges you will get an “You must be authenticated as a machine administrator in order to manage Office Web Apps Server”.

Using the Firewall

I already mentioned the firewall. If the machine joining the WAC farm cannot access the HTTP 809 channel the New-OfficeWebAppsMachine will throw a “The destination is unreachable error”.

This is a fairly easy way to protect the farm, but if the user has local admin access on the master machine it can easily be circumvented.

Certificate permissions

If you’re using a domain CA, make sure that you protect the private key using ACL’s, or if you’re buying a certificate make sure to store the certificate private key in a secure location. If you’ve specified a certificate when the Office Web Apps farm was created, which you should have, then the user cannot join the new machine – regardless of local machine admin, since the user cannot install the certificate locally. The error message shown is “Office Web Apps was unable to find the specified certificate”.

Using an Organizational Unit in Active Directory

The way that Microsoft recommends to secure your WAC farm is to have a dedicated OU in Active Directory where the computer accounts for the WAC farm is located. When joining a new machine to the farm the cmdlet verifies that the account is in the OU specified by the WAC configuration. If it’s not then you’ll see the “The current machine is not a member of the FarmOU”.

The Farm OU is specified when creating a new WAC farm or using the Set-OfficeWebAppsFarm/ cmdlet. The only caveat with this OU is that it has to be a top level OU in Active Directory. Creating that OU in your or your customers AD might cause some headache, but if you want to use the FarmOU as protection method for your farm it has to be this way. That’s the way it is designed!

Also having all the WAC servers in a OU gives you other benefits; such as using Group Policies to control the WAC servers.

Limit the WOPI Server and host access

Now we’ve seen how we protect the farm from rogue machines and data tampering. Another issue with the WAC farm in it’s default configuration is that any WOPI Server can use it. Might not be a big problem for most of the internal installations, but what if you’ve designed a WAC farm and someone with a huge SharePoint collaboration implementation connects to your WAC farm. It can sure bring it down. Or if you’re exposing your Office Web Apps farm on the internet anyone on the internet can potentially use it.

For this purpose there’s a cmdlet called New-OfficeWebAppsHost. This cmdlet allows you to specify host names that will be accepted by the WAC farm. The cmdlet interprets any domain with a wildcard. For instance the following cmdlet will allow all WOPI Servers on contoso.com (www.contoso.com, extranet.contoso.com etc.) to contact the WAC farm:

Set-OfficeWebAppsHost -Domain "contoso.com"

Do not forget to do this!!

Summary

You’ve seen quite a few ways how to protect your WAC farm from information leakage, rogue machines, undesired excessive usage etc. Using HTTPS and certificates together with a dedicated OU in Active Directory will give you the most secured WAC Farm. Hopefully you also understand a bit more on how Office Web Apps Server works internally. It’s a magnificent and simple server product, but it should be handled with care.

Sharing a Workflow Manager 1.0 farm between multiple SharePoint 2013 farms

Thu, 10 Jan 2013 14:47:00 PST

SharePoint 2013 introduces a whole set of new and improved features. One of the things that is both new and vastly improved is the Workflow platform. Workflow is no longer a part of the SharePoint core infrastructure, but instead a separate server product. Even though ye olde Workflow platform, 2010 style, is still in the product for backwards compatibility. SharePoint 2013 leverages the Azure service called Workflow Manager 1.0. (Not the cloud version but a local on-premises installation).

The Workflow Manager 1.0 server application should be installed on a separate set of servers – not on the SharePoint servers (remember; just because you can doesn’t mean you should). This introduces for some organizations more investments in physical and virtual hardware. So, what happens if you have several farms? Do you need several Workflow farms? That can be expensive! Well, as always in the SharePoint world - It depends! If you come to the conclusion that you could use the same Workflow Farm and hardware for managing all your SharePoint Farms workflows then you can actually use the same Workflow Manager 1.0 farm. Usual disclaimers apply here – it’s all up to you and your organization to decide upon designing your workflow infrastructure for this scenario, remember this can make backup/restore and DR a bit problematic since now your farms are sharing the same workflow databases etc.

Let me show you a multi-tenant Workflow farm, how it works and how you do to set it up properly…

Multi-tenancy in Workflow Manager 1.0

Azure Workflow Manager 1.0 is multi-tenant aware through Scopes. “A scope is a named and securable container for Activities, Workflows, Instances, configuration and child Scopes.” – that is directly stolen from the Workflow Manager documentation. Everything that runs inside Workflow manager belongs to a Scope. By default a Root Scope is created when you set up the Workflow Manager farm. You can see details about the Root Scope by navigating to the HTTP/HTTPS port of your farm. The URL should look something like this: http://wffarm.corp.local:12291 for HTTP and https://wffarm.corp.local:12290 for HTTPS.

Connecting SharePoint 2013 to the Workflow Manager 1.0 farm

I’m not going to dive in to how to configure the Workflow Manager farm; it’s a wizard or PowerShell thingie and it’s very well documented – you can’t fail! Once you have the Workflow farm up and running you MUST install the Workflow Client 1.0 components on all the Web Server in your SharePoint 2013 farm (you can get the bits using the Web Platform Installer).

All you have to do then is to register the Workflow Manager 1.0 farm with the SharePoint 2013 farm using the Register-SPWorkflowService cmdlet. This will register the SharePoint 2013 farm with the Workflow Manager and vice versa and it will also create the Workflow Service Application Proxy in SharePoint.

You execute the cmdlet like below and pass in a URL to a site in your SharePoint 2013 farm (any site will do) and the URL to your Workflow Manager farm (if you’re using HTTP you also need to add the AllowOAuthHttp parameter).

Register-SPWorkflowService 
    -SPSite https://teams.corp.local 
    -WorkflowHostUri https://wffarm.corp.local:12290

Not only will this cmdlet do the things mentioned above, it will also create a new Scope in the Workflow Manager farm (under the Root Scope) called SharePoint. You can verify this by navigating to Workflow Manager URL and append /SharePoint, like this: https://wffarm.corp.local:12290/SharePoint. In the XML returned you can see the URL to the SharePoint 2013 metadata endpoint (https://teams.corp.local/_layouts/15/metadata/json/1) under the SecurityConfigurations element. Now, that was one SharePoint 2013 farm connected to the Workflow Manager farm.

So, let’s jump on over to our second SharePoint 2013 farm, that we now decided should use the same Workflow farm! If we run the same cmdlet in that farm (using a different SPSite parameter URL of course) we get a nice error that says “An existing scope named “SharePoint” already exists in the workflow server”.

You see, this scope is already taken by some other farm (the one we previously connected)! The clever one might think that oh, I’ll just use the Force parameter of that cmdlet and override this. Well, if you’re that smart then you will hijack that Scope from the other farm. The Scope will be recreated and the other farm will have a non-functional Workflow Service (the security configuration will now point to the SharePoint farm that hijacked the Scope).

Note: the Force flag might be useful if you really want to re-create the Workflow connection using the same Scope name.

Instead you should use the ScopeName parameter of the Register-SPWorkflowService cmdlet. That parameter will create a new Scope in the Workflow farm, and with that create an isolated container for this new SharePoint farm. So on our second farm we run this PowerShell cmdlet:

Register-SPWorkflowService 
    -SPSite https://farmb.corp.local 
    -WorkflowHostUri https://wffarm.corp.local:12290
    -ScopeName FarmB

Once this command is issued your Workflow Manager farm will have two child Scopes under the Root Scope; SharePoint (the first farm we connected) and FarmB (the second one with the ScopeName parameter). You can verify the second scope by browsing the the Workflow farm and append the Scope name to the URL, like this: https://wffarm.corp.local:12290/FarmB.

Now you have two SharePoint 2013 farms using the same Workflow Manger 1.0 farm, and each SharePoint farms workflows are isolated using the Scopes in Workflow Manager. Cool!

Summary

I’ve just shown you how easy it actually is to share a Workflow Manager 1.0 farm between several instances of SharePoint 2013 farms. Now it’s up to you to decide on weather this is an appropriate way for your organization to build your infrastructure or not. Remember to plan your resources and your DR strategy!

Summing up the year of 2012 and embracing 2013

Sun, 30 Dec 2012 03:24:00 PST

Only one day and a few hours left of the year of 2012 and the time has come for me to make my (now traditional) summary post. I’ve done it now for six years (2011, 2010, 2009, 2008, 2007 and 2006) and it’s always fun to look back at what has happened during the last 12 months.

This year has been an interesting year to me and contained some really interesting milestones, happenings and events.

Writing

This year I’ve written basically the same amount of posts as in 2011, even though a new SharePoint version has been released. Basically it comes down to that I’ve been working a lot and haven’t had time to finish up some of the posts I’ve started and I’ve been busy with other commitments in the community. Anyways, I hope you enjoyed what I’ve written – I try to keep the quality up rather than the quantity.

What’s really cool is that I now on average have more that 1.300 daily subscribers to my blog, a 30% increase since last year. Thank you!

The most popular posts this year has been:

How Claims encoding works in SharePoint 2010 – a post you should read IMO and still valid for SharePoint 2013
Visual Guide to Azure Access Control Services authentication with SharePoint 2010 – part 1 – The first part of six posts in the series.
SharePoint 2013 – Introduction to the Minimal Download Strategy – MDS – An MDS overview.
SharePoint 2013 – Claims is the new black – the post title says it all
SharePoint 2013 – A look at the hardware and software and other requirements – my pov on the hw and sw requirements

The most popular post (from all years) is still the “Fix the SharePoint DCOM 10016 error on Windows Server 2008 R2” post.

No books this year – done that once…

MVP again..

I was re-awarded the Microsoft Most Valuable Professional, MVP, award for SharePoint for the third consecutive year. I’m honored by still being a part of this group of community contributors.

Microsoft Certified Architect – MCA

What I’m most proud of this year is that I passed the Microsoft Certified Architect for SharePoint, MCA, certification. This is a certification, currently only held by five individuals worldwide, which proves that you can “architect” a business and technical solution based on SharePoint and related technologies. What I do like about this certification is that it is not just a test or a lab (you get those tests while doing the MCM/MCSM, which is a pre-requisite for the MCA) but instead it is a mix of interview, business case, portfolio, presentations and Q&A’s, where you really must show that you understand business requirements, budgets, time plans etc. You can read more about my take on the program here – What is a Microsoft Certified Architect?

Conferences and travels

I got my fair share of travelling this year as well, a bit less than last year – which my family appreciated though. I had the opportunity to speak at a couple of conferences – where the International SharePoint Conference 2012 was the highlight. This conference was something extraordinary for both the attendees and us speakers. I really enjoyed working together with the team on the dev-track building out our solution and sessions and it was so fun having the whole team on the front row supporting (and a little bit of heckling) each other during the three days. I will be back next year in London!

I also had a blast at the first SharePoint conference in Croatia, and I hope I’m invited again and as usual our local SharePoint and Exchange Forum – which just keeps growing!

Microsoft arranged the SharePoint Conference 2012 in Las Vegas to unveil the spanking brand new SharePoint version…well it turned out that the product was released a month ahead, so the conference didn’t have much new information. In my opinion this was a pretty bad conference – the depths of the sessions was to low, to many of the speakers should not have been allowed on stage, and what a total disaster when all Microsoft talked about was how good the cloud is and they had no Internet connectivity. I really hope that the conference team get their stuff together and rethink a lot of things for the next conference!

The Cloud

I can’t really write a summary post of 2012 without talking about the Cloud. Microsoft (and other vendors) really put all their money on the cloud this year – nothing new with this, it’s been going on for years, but this year it’s more clear than ever. Well, the cloud is nothing new – it’s just a new name for the Internet, Application Service Providers, etc etc, it’s more of a marketing term.

Since the release of Office 365 the transition of SharePoint to the cloud has started (BPOS was first but SharePoint wasn’t cloud ready at all at that time). With this new wave of SharePoint and 365 the SharePoint cloud offering is even more evident – and we don’t know now where it will end, I’m not sure even Microsoft knows that yet.
We’ve been running 365 for a year and a half now and we still suffer a lot from different strange issues (that could be a post on it’s own!). Hopefully once upgraded the service will be more stable and useful.

I might sound a bit doubtful about the cloud – and I am. For the vast majority the cloud (read 365) is a great option instead of hosting and managing their own instances. But for large enterprises (the clients I’m normally working with) 365 is not even an option…

Only time will tell…

Predictions

This is the part I really enjoy writing and thinking about at this time of the year – looking back at my last years predictions and looking in the crystal ball for the upcoming year.

Last years predictions wasn’t that good – Silverlight is till not dead (someone hooked it up to a CPR machine) and all the browser vendors seems to be on some kind of honeymoon. My wish that 2012 was a bit less cloudy failed miserably! The only thing I think I got right was that Windows Phone would have momentum and by looking at some stats there’s some truth in that.

So what do I think about next year?

Cloud, cloud and cloud! The cloud marketing will continue, it will be shoved down our throats, we’ll be so sick of it by the end of 2013. But this will be the way forward. It will be really interesting to see what will happen to traditional “on-premises” products – will they vanish? Just take a look at how Microsoft has discontinued a lot of products this year!
Identities, certificates, federation! I think one of the most important things for next year and the years following are having a good identity infrastructure and identity management in place. This should be a priority for all companies and architects working in our business. It’s also the basic requirement to get any cloud services to work properly.
Apps, devices and integration! For the cloud, hosted services etc to work the integration story must be better. We’ve seen a good start with Windows 8 (RT), Windows Phone 8, Office 2013 etc – but it’s not fully there yet. I think that this is what will be improved over the next 12 months and that might also be the key differentiator between the three large “ecosystems” – Windows, Linux/Android and the fruit camp.
SharePoint community! The SharePoint community has grown really large and it has reached some kind of “critical mass”. I’m not saying it’s to big or anything but I see the community being more divided than ever splitting up in different directions with different specializations, aspects, mind-sets etc. The number of conferences, user group is ever increasing. This year the community will change (due/thanks to the cloud) – we will see a big change in focus, we will see SharePoint professionals being Azure, Windows 8/Phone, JavaScript professionals.

What do you think?

Thank you and a happy new year!

With that I would like to wish you all a Happy New Year and say thank you for 2012. I‘ve been having my doubts about 2013, but now I ‘m really looking forward to it. I have some cool events and travels planned, I know that my work at Connecta will contain some interesting opportunities and I know that the demand for SharePoint is ever increasing…

See you on the other side!

SharePoint 2013: Unable to install the pre-requisites on a disconnected machine

Mon, 03 Dec 2012 14:44:00 PST

You all know that you need to install the SharePoint 2013 pre-requisites before installing SharePoint 2013 – this is done either online or offline using the pre-req installer. All the requirements are listed in the Hardware and Software requirements for SharePoint 2013 Technet article.

Once in a while you need to do the installation on a disconnected machine, that is a machine that is not connected to the interwebz. Then you typically Bing for a fancy script that downloads all the pre-reqs for you and you run the pre-reqs installer in unattended mode. That is where you pass in the local path to all the downloads either through a file or through the command line (I’m not going to cover that – it’s all over the web and even in the TechNet article mentioned before).

Trouble in paradise?

One thing that you might stumble upon when doing a disconnected install of the SharePoint 2013 pre-reqs is that your scripts or the pre-req installer just keeps asking you to run the installer again and it just says that you need to restart your server and run the installer again. After doing that for a couple of times you most likely start inspecting the log files and you will see stuff like this:

2012-12-03 21:38:40 - Installing Microsoft WCF Data Services 5.0
2012-12-03 21:38:40 - "\\server\SP2013\PrerequisiteInstallerFiles\WcfDataServices.exe" /quiet
2012-12-03 21:42:50 - Install process returned (-2146762486)
2012-12-03 21:42:50 - [In HRESULT format] (-2146762486)
2012-12-03 21:42:50 - Last return code (-2146762486)
2012-12-03 21:42:50 - Install needs restart

We’ll it looks like it needs a restart – but what about that HRESULT code? Let’s turn it into a hexadecimal representation – 0x800B010A. Ahaaa! Ya’ll know your HRESULTs, right. It starts with 0x800B01?? so it must be some certificate issue, and it sure is.

As you can see it fails when installing the WCF Data Services 5.0. Let’s run that installer by itself. When we do that we’ll see an error that clearly says that it is a certificate error – A certificate chain could not be built to a trusted root authority.

The installer log file for WCF Data Services also tells it right in our face:

[0B48:0C78][2012-12-03T21:52:06]: Error 0x800b010a: Failed to verify certificate chain policy status.
[0B48:0C78][2012-12-03T21:52:06]: Error 0x800b010a: Failed to get certificate chain for authenticode certificate.
[0B48:0C78][2012-12-03T21:52:06]: Error 0x800b010a: Failed to verify signature of payload: wcf_dataservices

So there’s something wrong with our certificate validation chain here…

These darn certificates!

Having disconnected machines means one or two things with regards to patching – either they are not patched at all or the admins control the patching themselves and don’t let the right ones through. The problem here is that the machine experiencing this problem does not have its root certificates updated. It’s an easy fix – make sure you let those updates from Windows Update through or update the root certificates manually.

A manual update, which should only be your last resort, can be done by downloading the Update for Root Certificates for Windows XP [April 2013] (KB931125). Yup, they are for Windows XP but works perfectly on a Windows Server 2008 R2 (that’s where I tested it). Download the rootsupd.exe file and run it on your server before the running the pre-req installer and all should be running fine from now on.

Summary

A short explanation and solution for something that hopefully isn’t that common – since all of you have patched servers right?

SharePoint 2013: Building your own WOPI Client, part 4, now Search enabled

Wed, 03 Oct 2012 04:06:00 PDT

Well, I thought I should write another episode of my Building your own WOPI Client series, here’s the links to the previous episodes part 1, part 2 and part 3. This time around we’ll dig into another of the different actions that a WOPI Client can surface – the interactivepreview mode.

Background

As you’ve seen in the previous posts we can build a viewer and an editor for C# files, to be used in document libraries for instance. What if we would like to lift up and enhance our custom file formats in search, just like Office Web Apps Server does with the Office files. We’ll you can do that very easy, and you should! In this post I’ll show you how to surface the preview mode in a Search flyout. This also means that we’re going to take a look at the new SharePoint 2013 Search Engine, the Design Manager and some funky HTML syntax.

Configure indexing for C# files

The first thing we need to do is to actually make SharePoint indexing our C# (.cs files). This is done in two steps. First we need to tell SharePoint 2013 Search to include the .cs file extension and then we need to tell the search engine what filter to use. To include .cs files in the index there is two ways to do it, using PowerShell or using the Central Administration. In Central Administration you go to the Search Service Application and choose File Types, then click on New File Type and type in cs. That’s it. Using PowerShell the following snippet works (as long as you only have one SSA):

$ssa = Get-SPServiceApplication | ?{$_.TypeName -eq "Search Service Application"}
New-SPEnterpriseSearchCrawlExtension -name cs -SearchApplication $ssa

Once that is done we need to fiddle a bit with the registry (usual regedit disclaimers apply here). Start regedit.exe and navigate to HKLM\SOFTWARE\Microsoft\Office Server\15.0\Search\Setup\ContentIndexCommon\Filters\Extensions and choose to add a new Key. Give the key the name “.cs” (without quotations) and then set the default value to the following (fancy crafted) Guid: {0FF1CE15-002C-0000-0000-000000000000}. All you now have to do is to restart the Search Engine like using the PowerShell below and then start a full crawl:

Restart-Service OSearch15

Now when you search for any content in your .cs files it should look something like this:

It’s something, but not what we want…

Building the WOPI Preview

Before we start customizing the UI let’s build the interactiepreview mode of the WOPI client. Just as we did in the previous post, when we added the edit mode, we need to add a new action to the discovery file and then remove and add the bindings again (for that PowerShell, see previous post). This is the action I’ve added to my WOPI Client:

<action name="interactivepreview"
        ext="cs"
        default="false"
        urlsrc="http://wacky.corp.local/csharp/preview.aspx?&lt;ui=UI_LLCC&amp;&gt;&lt;rs=DC_LLCC&amp;&gt;" />

As you can see that the urlsrc points to a preview.aspx web form. That web form is essentially a copy of the viewer.aspx, with the exception of the menu which I have removed. In a real world scenario you might consider having very different viewers for the normal view mode and the interactivepreview mode. You want this process to be fast and you might just generate a thumbnail or something.

When all is compiled and deployed, and you have removed and re-added the WOPI bindings, it’s time to hook this up to search.

Creating the Display Templates

Let’s build the search interface. The SharePoint 2013 Search Interface does not any longer require extensive XSLT skills but instead plain ol’ HTML hacking. Every Result Type in SharePoint 2013 Search can have it’s own Display Templates, and there’s two of them; the normal item view and the flyout/hover view. The easiest way to create new Display Templates is to copy an existing one. For this sample let’s use the Word Display Templates.

You will find the Display Templates in the new Design Manager (which exists in a Search Center or a site with the Publishing features enabled). You access the Design Manager through the Site Settings menu. Once in the Design Manager you should choose Display Templates in the menu on the left hand side. Then I usually filter on Target Control Type, to narrow down on the Display Templates used for Search; choose SearchResults and SearchHoverPanel. Now scroll down to the Word Item and Word Hover Panel. You’ll see two of each but we’re only interested in the HTML files for now. Download those two files to your local disc and name them Item_Cs.html and Item_Cs_HoverPanel.html respectively.

Now open these two files in your favorite HTML tool – for me that is Visual Studio 2012. There’s a lot of weird stuff in here as you will notice, lots of comments and JavaScripts. I’m not digging into all the details here, that’s for Randy Drisgill or someone else to do.

Let’s do some simple string replacement. In both files replace “Word” with “Cs” as the first thing. Next update the mso:MasterPageDescription to something more appropriate and the title tag, perhaps replace “Microsoft Word” with “C#”. In the Item_Cs.html file locate the second div having an id equal to "_#= $htmlEncode(itemId) =#_" and remove everything inside it and replace with this instead:

<div class="ms-srch-item-title">
  <h3 class="ms-srch-ellipsis">
    <a class="ms-srch-item-link" href="_#= ctx.CurrentItem.Path =#_">_#= ctx.CurrentItem.Title =#_</a>
  </h3>
</div>
<div class="ms-srch-item-body">
   _#= Srch.U.processHHXML(ctx.CurrentItem.HitHighlightedSummary) =#_
</div>
<div id="_#= $htmlEncode(hoverId) =#_" class="ms-srch-hover-outerContainer"></div>

What this does is that we’re first rendering the title of the document with the value ctx.CurrentItem.Title. All these JavaScript variables are enclosed in __#= … =#_. Then we output the hit highlighted summary using a built-in method (Srch.U.processHHXML). Finally there’s an empty div which will contain the flyout (The flyout is defined in the Item_Cs_HoverPanel.html file). There is one final thing that we need to do and that is to update the hoverUrl JavaScript variable in this Item_Cs.html file. It should point to the Item_Cs_HoverPanel.js file like this:

var hoverUrl = "~sitecollection/_catalogs/masterpage/Item_Cs_HoverPanel.js";

Notice that it points to a JS file, not the HTML file. And also that it’s located directly in the masterpage gallery and not in a subfolder (compared to the original Word html file).

The Item_Cs_HoverPanel needs no more changes than replacing Word with Cs and updating the title and description.

To add these Display Templates to the Design Manager, just drag and drop them into your browser where you downloaded the original Display Templates. The files will end up as Draft version so go ahead and publish both of the HTML files.

Once the files are approved, the Design Manager will create two JS files from these HTML files and that’s what’s used behind the scenes when rendering the search results.

Create a Result Type

The final thing we have to do is to tell the Search Center to actually use our newly crafted Display Templates when finding C# files. For that we need to create a new Result Type. Go to Site Settings and choose Search Result Types under Site Collection Administration. Click on New Result Type to start creating it. Give the Result Type an appropriate name, for instance “C# Files”. Under Conditions choose Show more conditions and select the FileExtension property and set it so that it “Equals any of…” and enter “cs” in the text box. Under Actions choose the C# item in the drop down and then simply click Save. You should now have a Result Type looking like this:

Test the stuff…

All that is left is now to test if all this works. Go to the search page of you Search Center and search for something you know exists in your C# files, that by now should have been indexed. You should then see something like this:

You see the items (using the Item_Cs Display Template) and if you hover over one of the search results then you see the preview from the WOPI Client being generated (using the Item_Cs_HoverPanel). Isn’t that nice, huh?

Summary

In this post you’ve seen yet another way to leverage a WOPI Client, this time in search, using a preview mode of the files. You have also seen how “easy” it is to generate a custom search result design using the Design Manager, Display Templates and Result Types. I’m really looking forward to seeing WOPI Clients for all kinds of different formats out there…