Will Code for Nuts

Forcing the SharePoint object model to reload the 12 hive

2009-10-13T11:34:00.007+02:00

Or: A bit of insight into the inner workings of the SharePoint object model

This isn't necessarily something many have found themselves battling, but I most certainly did yesterday. As I was trying to deploy a feature containing a site definition, and subsequently create a site using the site definition, using my new SPDeploy application, it all abruptly came to a nasty halt as I added the site (using SPSiteCollection.Add()):

File or arguments not valid for site template 'SomeTemplate#0'

Having triple-checked the template, feature, solution and just about everything else (yes, SPDeploy also does the equivalent of stsadm -o execadmsvcjobs), it eventually dawned on me how SharePoint goes about things when you create a new site.

Just about any call in SharePoint will go through whatever object model class you're dealing with, such as SPSiteCollection, then to a SPRequestManager class, on to an SPRequest, SPRequestInternalClass and eventually into an arcane COM class (which appears to be part of the old SharePoint 1.0 structure, when COM, ISAPI and such was the shizznit). This COM class exposes hundreds of functions, ranging from list administration to tepmlate lookup.

The issue in my case was two-fold. First of all, any thread in an application which uses the SharePoint object model, will access a set of SPRequest objects (based on whether you need an authenticated or non-authenticated instance) from the SPRequestManager class. These are per-thread request objects, which execute on a first-come, first-served basis. In figuring this out, I began experimenting with releasing the SPRequests for the main thread in the SPDeploy tool - which would cause them to be reinitialized in a subsequent call. I verified that this would not hurt other stateful objects, such as already lookup'd SPWebApplication instances; as all functions in these classes will acquire their SPRequest instance from SPRequestManager. And as I said: the SPRequestManager would create a new instance if an old one wasn't found.

After typing up the reflection madness to clear the current thread's SPRequest objects, I gave the application another go. But still got the error - doh.

This confirmed my worst fears: the SPRequestInternalClass COM object, which is obviously native code, did its own caching. Pulling out Sysinternals' ProcMon and watching loads from the filesystem beginning with "web server extensions\12" backed this up further: web templates, and the rest of the hive, is only loaded once - on demand - then cached.

In realizing this, the solution became pretty clear, and given the nature of SPRequestManager; not too shabby at all. For tools which require adding stuff to the 12 hive (such as deploying features with web templates), and then *use* those templates without re-executing the tool: you have to recycle the SPRequests in SPRequestManager, and recycle the OWSSVR.dll COM library.

This *also* applies for PowerShell scenarios, where you during one session call into one of the SharePoint objects, such as e.g.:

([type]"Microsoft.SharePoint.Administration.SPWebApplication")::Lookup($uri);

Doing so will load the OWSSVR.dll into your current session, and if you happen to e.g. trigger a function from the object model which scans the 12 hive for tepmlates; any subsequent script based addition to this folder, and then attempted usage of the additions, will fail with the error noted above.

The workaround I apply in SPDeploy, which may look intrusive, but is completely harmless in the single-tool scenario, is:


[DllImport("kernel32.dll")]
private static extern IntPtr FreeLibrary(IntPtr library);

[DllImport("kernel32.dll")]
private static extern IntPtr GetModuleHandle(string lpModuleName);

[DllImport("kernel32.dll")]
private static extern IntPtr LoadLibrary(string lpFileName);

public static void RecycleOwssvr()
{
    PropertyInfo threadCtx = typeof (SPFarm).GetProperty("ThreadContext", BindingFlags.Static | BindingFlags.NonPublic);
    var tbl = (Hashtable) threadCtx.GetValue(null, null);
    PropertyInfo requestProp = typeof (SPFarm).GetProperty("RequestNoAuth", BindingFlags.Static | BindingFlags.Public | BindingFlags.NonPublic);
    requestProp.GetValue(SPFarm.Local, null);
    tbl.Remove(typeof (SPFarm).Assembly.GetType("Microsoft.SharePoint.SPRequestManager"));

    IntPtr p = GetModuleHandle("OWSSVR.DLL");
    FreeLibrary(p);
    string stsadmPath = SPUtility.GetGenericSetupPath("ISAPI");
    p = LoadLibrary(stsadmPath + @"\OWSSVR.DLL");
}

And that will work beautifully.

One note, though; the timer service has to be running for this recycling to work, otherwise some other internal component will freak out come second-go.

SPDeploy update

2009-10-11T23:01:00.003+02:00

CodePlex link: http://spdeploy.codeplex.com

I just uploaded a new binary and source package to the CodePlex site, with some structural, config and visual improvements.

First of all, web application extensions are now supported, meaning that web applications you either lookup or create as part of the configuration can be extended to intranet / extranet / custom zones. A custom rolemanager, membershipprovider and ssl usage can be specified through the attributes. See the xsd schema in the Schemas folder for more info on which attribues are available for the WebApplicationExtension node.

Further, the logging now looks a lot better, thanks to clearer language and some more elaborate code behind the output formatting. I've also refurbished a lot of the general processing code, to make it easier to extend for possible project helpers. I've got more changes coming later, but with a pretty limited amount of time on my hands, I can't be specific as for when.

There are still some hidden general behavior settings being tossed about in the background, which aren't obvious unless you turn to the code. I'll document and clear those up in a future (probably the next) release.

Introducing SPDeploy - a quick way to (re)deploy dev and test structure to SharePoint

2009-10-09T14:37:00.003+02:00

CodePlex link: http://spdeploy.codeplex.com

About

SPDeploy will, based on schema validated xml configuration, lookup or create content in a local SharePoint farm, such as WebApplications (with content database and iis site / application pool), Sites, Webs. It will also handle wsp solution installations to the farm; further deployments of installed wsps into web applications; activation of features on all feature-accepting levels of the structure.

Folder, list item and document support is planned, but not as a replacement for WSPs, features or plain common sense.

With SPDeploy you can quickly, and painlessly, reset and redeploy your test or development environment, as well as quickly add development / third party features to the mix - without incorporating them into your production ready wsps, or extending large powershell / batch scripts. The point is to encourage proper integration testing in your development environment, and bringing structure to your favorite SharePoint build server.

Important! Deployment or upgrades into a production or even UAT environments should be made with as strict a setup as possible, with (if possible) all the structure definition, creation and library deployment packaged into definitions, features and solutions. I do not suggest using SPDeploy for this. Test with SPDeploy - ship with wsps and upgradable features!

Developer / extension notes

All data classes in the project are generated from XSD, and generic configration runners make it easy to expand with custom sub nodes. Add an accepted sub node to the XSD, rebuild the project to generate new data classes, and create a Processor derived class to deal with the new sub node type. A new sub node processor will automatically get the result from the lookup or creation of the node above (like a SPWeb, if you're adding processors under the XML's Web node).

Sample xml configuration data

If you reference the schema provided in the source package, Visual Studio will even give you autocompletion and validation of the configuration as you go.


<?xml version="1.0" encoding="utf-8" ?>
<Deployment xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="Schemas\SPDeploy.xsd">
  <Solutions>
    <Solution FileName="C:\path\to\a\Solution.wsp"/>
  </Solutions>
  <WebApplications>
    <WebApplication Action="Lookup" Name="SharePoint - 80">
      <SolutionDeployments>
        <SolutionDeployment Name="Solution.wsp"/>
      </SolutionDeployments>
      <Sites>
        <Site Action="Create" Url="/" Title="RootWeb" LCID="1044" Template="STS#1" OwnerLogin="somedomain\user">
          <RootWeb>
            <Features>
              <FeatureActivation Guid="fba9e8df-b758-4197-8768-169ef727d5bb" />
            </Features>
            <Webs>
              <Web Action="Create" Title="RecursionWeb" LCID="1044" Template="STS#1" Url="RecursionIsFun">
                <Webs>
                  <Web Action="Create" Title="RecursionWeb" LCID="1044" Template="STS#1" Url="RecursionIsFun">
                    <Webs>
                      <Web Action="Create" Title="RecursionWeb" LCID="1044" Template="STS#1" Url="RecursionIsFun">
                        <Webs>
                          <Web Action="Create" Title="RecursionWeb" LCID="1044" Template="STS#1" Url="RecursionIsFun">
                          </Web>
                        </Webs>
                      </Web>
                    </Webs>
                  </Web>
                </Webs>
              </Web>
            </Webs>
          </RootWeb>
        </Site>
      </Sites>
    </WebApplication>
    <WebApplication Action="Create" Name="TestApp" Header="vm-dev" Port="8009" Database="TestContent" PoolName="TestPooly" CreatePool="true">
      <SolutionDeployments>
        <SolutionDeployment Name="Solution.wsp"/>
      </SolutionDeployments>
      <Sites>
        <Site Action="Create" Url="/" Template="STS#1" LCID="1044" OwnerLogin="somedomain\user" Title="TestWeb">
          <RootWeb>
            <Webs>
              <Web Action="Create" Title="SubWeb" LCID="1044" Template="STS#1" Url="SubWebUrl">
                <Features>
                  <FeatureActivation Guid="fba9e8df-b758-4197-8768-169ef727d5bb" />
                </Features>
              </Web>
            </Webs>
          </RootWeb>
        </Site>
      </Sites>
    </WebApplication>
  </WebApplications>
</Deployment>

Assigning dynamic ids to HTML and Script elements - Client Side!

2009-10-05T23:35:00.012+02:00

This question popped up on Twitter yesterday:

How do you usually handle duplicate IDs for html elements and scripts in SharePoint Web Parts, scripts and controls?

Which is a natural concern, with all the different web parts and scripts that's being injected SharePoint portals these days. To end Monday off (in my neck of the woods anyway), I thought I'd make a quick post on what I do in my projects.

Script naming convention

For scripts, I have a few guidelines. First of all, my scripts follow a naming convention based on my organization handle, library concern, project name and so forth. Looking back to my previous couple of posts, e.g. the Script Loader (http://www.codefornuts.com/2009/10/pieces-of-my-core-javascript-library.html), this name would be grep.scriptloader - as that's a core piece of script for me. For a script specific to a web part, I'd name it similar to the webpart, e.g. grep.webparts.navigationcontrol.

As for the actual script definitions, two matters apply: is it a class, or a single instance?

For single instances, such as the Script Loader above, I declare it like this:


if (!window.grep) window.grep = {};
if (!grep.someSingleInstanceClass)
{
    grep.someSingleInstanceClass=
    {
        instanceVar: "baz",
        foo: function() {},
        bar: function() {}
    }
}

So basically I check if it's already been assigned, and if it hasn't, I assign it to a global variable. The above code would then be accessed such as


grep.someSingleInstanceClass.foo();

For multi instance classes, I follow regular javascript oo syntax, with the same has-it-been-defined-clause, such as:


if (!window.grep) window.grep = {};
if (!grep.someClass)
{
    grep.someClass = function(ctorVariable)
    {
        this.someVar = ctorVariable;
        this.DoWhatever();
    }

    grep.someClass.prototype.DoWhatever = function()
    {
       // ...
    }

    grep.someClass.prototype.AnotherFunction  = function(input)
    {
       // ...
    }
}

Then access that such as


var foo = new grep.someClass("some value for the ctor");
foo.AnotherFunction(42);

HTML element lookups, and dynamic ids

For elements, I use another trick. Since giving the elements a specific id can easily crash with other web parts of the same kind, or even other web parts / controls, I tend to avoid using ids as much as possible, and rather rely on class lookups (where the class has been named similar to the web part). For such lookups to be possible, however, they must be based somewhere - you can't just search for all instances of 'css class foo' in all of the body tag.

Imagine the following scenario:


<div>This element will be added several times, by other instances of the same web part</div class="webPartInstanceFrame">

<script type="text/javascript">
    new grep.webparts.someFrameManager(id of the previous div);
</script>

The script convention noted further up would make sure that the class' code will only be added once, and we feed that the id of the div, so that the class can make css class based jQuery (or similar) lookups to find the elements it needs within the frame. But what's the id of the div? Assigning a static id to it would crash with other instances of the same web part, and using a class would be just as hopeless.

That's where the trick applies, and the following is a fully functional example:


<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
<script type="text/javascript">
    function renamePrevious(tagType, id)
    {
        $(document.body.lastChild)
            .prev(tagType + ":first")
            .attr("id", id);
        return id;
    }
</script>

<div>I want a dynamic ID!</div>

<script type="text/javascript">
    var dynamicId = renamePrevious("div", "SomePrefix" + new Date().valueOf());
    /* renamePrevious should be passed as an argument in dynamicId's place, to avoid all variable assignments! */
    document.getElementById(dynamicId).innerHTML += " - OK!";
</script>

The trick works by looking up the script block's element in the html dom. This element is the last added element, as the script block executes, even if it's not at the bottom of the page. Based on this script element, it searches its previous siblings for the first match of a given tag type, then assigns a new id.

You could do a lot better with the actual dynamic id than the millisecond postfix shown here, such as an incrementing (or even random) number which is added to a string, then checking if that already exists (in which case we increment / re-random and try again). I leave that to your imagination, though.

As for the example further up, the frame could be identified and put into action such as:


<script type="text/javascript">
    new grep.webparts.someFrameManager(
        renamePrevious("div", "E" + new Date().valueOf()));
</script>

And that would enable us to avoid any and all name crashes for our component.

Connecting SharePoint UI Elements: SPDrag and the DropBox

2009-10-02T16:42:00.006+02:00

So the last few posts have focused on some of my core javascript libraries. Nothing of the fancy stuff, just basic paper and glue type things. To make it all come together, I'll bring back up something I mentioned on Twitter about a month ago, but haven't spoken much of since: Two libraries of mine called SPDrag and DropBox.

SPDrag

The first of the two essentially makes standard SharePoint ui elements, such as files, list items and so forth, draggable. With a library building on jQuery, jQuery UI and my own Lambda framwork, it allows a developer to connect various of the SharePoint elements to his own widgets. An example of such a widget would be the Hierarchical Navigation replacement I've mentioned time and again (jQuery ajax powered, lazy-loaded SharePoint quick launch replacement). Another example is the DropBox.

The Lambda Framework, described here is a key player in making this work.

DropBox

One of the basic features missing in SharePoint 2007 has for me been the possibility of picking things up, taking them to another location, and dropping them into place. Be that files, folders, tasks or whatnot - I've wanted to pull them about. It's no joke that file copy / move in default SharePoint is as fun as watching grass grow. Even with webdav, it's nowhere near as pleasant as it should have been.

Enter the DropBox. Pick up a file. Pick up an item. Pick whatever you want, and put it in the DropBox. Then poke around the portal at your own whimsical liking; pull the item from the DropBox and place it in a compatible container. File copy made smooth, task assignment made friendly.

So am I going to release this now? No, I'm afraid not. I still have some work to do on both libraries, and I've yet to decide in which shape and form the release will take on. I'm obviously thinking OSS, but that's futile unless someone actually wants to contribute ;-)

Anyhoo, here's the screencast of several of the said libraries acting together. Please watch, and supply me with ideas if the concept appeals to you!

Pieces of my core javascript library: The Lambda Framework

2009-10-02T16:11:00.005+02:00

Following up on my previous post, about the Script Loader piece of my core library, here's a couple more central pieces for the stuff I'm about to tell about in posts to follow.

On the spot in this post is the lambda library. Simply put, a lambda here means an expression with placeholders for parameters to come. As such, it's not unlike the closures we already have in javascript, but this does up the stakes a little bit.

Closures allow us to define inline functions, and bind expressions to said function's parameters as well as those of the outside scope, ex:


function doSomething()
{
    var someGreeting = "Hi";
    var closure = function(name) { alert(someGreeting + " " + name); }
 closure("yourself"); // Should alert "Hi yourself"
}

... which is all old news.

With the following lambda library, I could construct the following fulent expression tree:


 var verification = _.first().is(10);
 alert(verification(10)); // Should alert 'true'
 
 // The same as a closure:
 // var verificationClosure = function(x) { return x == 10; }
 
 // A more advanced verification (although hardly the best we can do):
 var advancedVerification = _.and(_.first().is(10), _.second().isnt(_.first()));
 alert(advancedVerification(10, 10)); // Should alert 'false'
 alert(advancedVerification(10, 20)); // Should alert 'true'

This may seem like utter gibberish, but in certain cases it can be useful in order to define a relatively clear set of rules for an operation to take place. Specifically I'm using this in a library which establishes relationships between abstracted gui elmements. What I'm able to do is say, with an expression tree, that:
Given squares A and B; B can be dragged into A if B isn't already a child of A, and it has a certain inherent property.

This is actual code from my DropBox SharePoint plugin (I'll showcase that later), which uses another library of mine, SPDrag (that essentially makes standard SharePoint UI elements drag'n'droppable).


    grep.spdrag.connect(
        /* target container */ $("table.ms-listviewtable"),
        /* conditions */ [
            _.and(
                _.item().is_dropbox_item(),
                _.target().is_document_library(),
                _.not(_.item().is_child_of(_.target()))
            )
        ],
        /* action on drop */ function(element)
        {
            self._deployFileToDocumentLibrary(element);
        });

Connect, in other words, is a function which accepts three parameters: a target container (in this case the a SharePoint list view), a condition for it to allow an element to be dragged onto it, and what's supposed to happen if they are dropped there.

The second parameter, the conditions, is a further abstracted lambda expression tree, which introduces elements specific to the SPDrag library. item, the dragged element; target, the target container; and a HTML DOM type function called is_child_of which is here used to disallow dropping elements from the target container back onto the target container.

Arguably, the syntax can get complex, especially with the logical operators preceding the operands rather than being between them, but it has also the ability to be very clear and precise on the domain operation at hand, avoiding as much of the function / closure / parameter clutter as possible.

Without further ado, here's the lambda core, which can be (and are) abstracted into domain specific lambda expressions. In the case of the above item element, that's actually a specialization of first in the base lambda library, but that's for the library implementing the domain specific lambda to know only. As the end user, you'd only have to consider item representing the item.


if(!window.grep) window.grep = {};
if(!window.grep.lambda)
{
    window.grep.lambda = window._ = 
    {
        nth:function(x){return window.grep.tools.extend(function(){return arguments[x-1];},this);},
        first:function(){return this.nth(1);},
        second:function(){return this.nth(2);},
        is:function(x){var c=this;return function(){return (typeof(x)=="function"?x.apply(this,arguments):x)==c.apply(this,arguments);}},
        isnt:function(x){var c=this;return function(){return (typeof(x)=="function"?x.apply(this,arguments):x)!=c.apply(this,arguments);}},
        gt:function(x){var c=this;return function(){return (typeof(x)=="function"?x.apply(this,arguments):x)<c.apply(this,arguments);}},
        lt:function(x){var c=this;return function(){return (typeof(x)=="function"?x.apply(this,arguments):x)>c.apply(this,arguments);}},
        and:function(){var a=arguments;return function(){for(var x=0;x<a.length;++x)if(!a[x].apply(this,arguments))return false;return true;};},
        or:function(){var a=arguments;return function(){for(var x=0;x<a.length;++x){if(a[x].apply(this,arguments))return true;}return false;};},
        not:function(x){return function(){return !x.apply(this,arguments);};}
    };
    
    window.grep.lambdapredicate = function(handler)
    {
        return function()
        {
            var outerArgs = arguments;
            var ctx = this; 
            return function() 
            { 
                var args = [ ctx.apply(this, arguments) ];
                for(var i = 0; i < outerArgs.length; ++i)
                {
                    args.push(typeof(outerArgs[i]) == "function" ? outerArgs[i].apply(this, arguments) : outerArgs[i]);
                }
                return handler.apply(this, args);
            }
        }
    }
}

This is an excerpt from the tool library, which contains the extend function used in the above lambda lib.


if (!window.grep) window.grep = {};
if (!window.grep.tools)
{
    window.grep.tools = {
        extend: function()
        {
            var r = arguments[0];
            for(var i = 1; i < arguments.length; ++i)
            {
                for(var x in arguments[i])
                {
                    r[x] = arguments[i][x];
                }
            }
            return r;
        }
    }
}

Pieces of my core javascript library: The Script Loader

2009-10-02T12:38:00.010+02:00

I'm about to release a few more open source SharePoint solutions, including a core javascript library feature. The CoreJS is a required feature for the hierarchical navigation component, which will also be properly released as OSS on CodePlex before long.

A central piece of the CoreJS library and activation feature, is the Script Loader. This is in fact the only piece of the library which is automatically injected (as a delegate control) into any page in the site collection it's activated for.

The Script Loader's purpose is pretty much exactly what it sounds like: it's a general purpose way of loading scripts from javascript code. A sample scenario would e.g. be if you need a few external utility library, but you don't know if it has been loaded yet. Using the script loader, this would amount to:


function loadComplete()
{
  alert("The scripts are loaded, and we're good to go!");
}
  
grep.scriptloader.loadScripts(
  "http://www.json.org/json2.js",
  "myJsonAjaxLibrary.js",
  loadComplete
  );

The Script Loader would ensure that all the scripts are loaded in order, so in this above scenario, where the fictional myJsonAjaxLibrary relies on json2.js to do its thing, myJsonAjaxLibrary would not be loaded at all until json2 is present and ready. Additionally, we can supply a function as a parameter to loadScripts, which is called once all previous tasks have completed.

Making more calls to loadScripts would queue up even more scripts to be loaded. Imagine the following turn of events:


// On top of your script
grep.scriptloader.loadScripts("http://www.json.org/json2.js");
  
// ... Meanwhile (actually later) in some distant part of the same script
grep.scriptloader.loadScripts(
  "myLibrary.js",
  function() { alert("myLibrary loaded"); },
  "http://www.json.org/json2.js",
  function() { alert("json2 loaded"); });

It's pretty common for different parts of your script setup to require different libraries, and yet other times they even require the same scripts (whew). The script loader library does two things here: It will load all scripts in order, even between calls to loadScripts. In the above case, myLibrary would *not* be loaded until json2.js has been loaded from json.org; since that was specified in the first call to loadScripts. Second, it it will check each source url, to make sure that it hasn't already been loaded. If it's already loaded; it will be skipped.

In the above case, the actual sequence of events would be:

load json2
load myLibrary
alert(myLibrary loaded)
alert(json2 loaded)

Here's the full script, ready for inclusion. If you spot anything off, you'll have plenty of time to suggest changes once I've moved this and other core javascript libraries, plus the SharePoint activation feature, to a project on CodePlex!


if (!window.grep) window.grep = {};
if (!window.grep.scriptloader)
{
  window.grep.scriptloader =
  {
      nocache: false,

      loadScripts: function()
      {
          for (var i = 0; i < arguments.length; ++i)
          {
              var arg = arguments[i];
              switch (typeof (arg))
              {
                  case "function":
                  case "string": window.grep.scriptloader._queue.push(arg); break;
                  case "object": if (arg.length > 0) for (var x = 0; x < arg.length; ++x) window.grep.scriptloader._queue.push(arg[x]); break;
              }
          }
          if (!window.grep.scriptloader._active)
          {
              window.grep.scriptloader._active = true;
              window.grep.scriptloader._processNextQueueItem();
          }
      },
  
      /* Private interface */
  
      _active: false,
      _queue: [],
      _loaded: {},

      _processNextQueueItem: function()
      {
          if (window.grep.scriptloader._queue.length > 0)
          {
              var item = window.grep.scriptloader._queue.shift();
              if (typeof (item) == "function")
              {
                  item();
                  window.grep.scriptloader._processNextQueueItem();
              }
              else if (typeof (item) == "string")
              {
                  if (!window.grep.scriptloader._loaded[item])
                  {
                      window.grep.scriptloader._loaded[item] = true;
                      window.grep.scriptloader._loadScript(item, window.grep.scriptloader._processNextQueueItem);
                  }
                  else
                  {
                      window.grep.scriptloader._processNextQueueItem();
                  }
              }
          }
          else
          {
              window.grep.scriptloader._active = false;
          }
      },
  
      _loadScript: function(src, onload)
      {
          var js = document.createElement('script');
          if (window.grep.scriptloader.nocache)
          {
              js.src = src + "?" + Math.random();
          }
          else
          {
              js.src = src;
          }
          js.type = 'text/javascript';
  
          if (js.readyState)
          {
              js.onerror = function() { alert("Error loading script: " + this.src); };
              js.onreadystatechange = function()
              {
                  if (js.readyState == "loaded" || js.readyState == "complete")
                  {
                      js.onreadystatechange = null;
                      if (onload)
                      {
                          onload(src);
                      }
                  }
              };
          }
          else
          {
              js.onload = function()
              {
                  if (onload)
                  {
                      onload(src);
                  }
              };
          }
  
          var head = document.getElementsByTagName('head')[0];
          if (head)
          {
              head.appendChild(js);
          }
          else
          {
              document.body.appendChild(js);
          }
      }
  }
}

Dear SharePoint Team: ItemDeleted? Seriously?

2009-09-30T11:10:00.007+02:00

ItemDeleted - a would-be handy event receiver, but in reality no more useful than a box full of used matches.

An analogy, if I may, to explain the state of affairs

While away on business, you're phoned up by your home town police, and given the brief message:
- Someone very dear to you just passed away.
Caught slightly off guard, but immediately inquisitive, you might ask:
- Who?!
Which is rewarded with a mere:
- I don't know.
- What do you mean "I don't know"? For God's sake man, someone just died, and you can't tell me who?
- No.
- Well do you know anyone who could know, then?
- No.
- Great .. Thanks a heap.

What I'm saying is ...

ItemDeleted knows *nothing* about the item that was deleted, apart from its now invalid index into the list. While the event receiver properties does contain the fields BeforeProperties and AfterProperties, the former of which one would expect to contain some information about the state of affairs *prior* to the delete taking place; both are null.

Next up, there's ItemDeleting. After all, that signals the same possible state change, and unlike it's dimwitted brother 'Deleted, it *does* have the BeforeProperties initialized. Upon discovering this, you might decide to use this receiver to process item deletions, such as in my current project - to signal item changes across farm boundaries. What you're not considering, then, is that ItemDeleting is just an indication of what might happen. Any number of receivers following yours may set SPItemEventProperties.Cancel to abort the delete, thus possibly invalidating the state of your application (parts of it thinking an item is deleted, other parts knowing it's not).

So what can we do, then?

Parse the RecycleBin?

Possible, but even if we did iterate all items in there (possibly *a lot* of overhead), we don't have any idea at all what the items title, name or guid might have been.

Store event handler instance / static data in ItemDeleting, then check in ItemDeleted?

Unlike the RecycleBin, this would work, and it would give a reliable indication of what item was deleted, but it'd cost an arm and a leg efficiency-wise. Event handler instance data is a no-go, I should add, since event handler classes are initialized once per call to a receiver function (ugh). You could use an interlocked static field, but 1. it would be very inefficient in any environment with more than zero users (locking and unlocking the field for each call), 2. I would grant you *no* guarantees in a multi-frontend farm setup, or even a multi-worker process environment.

Store state info in the SPItemEventProperties.ReceiverData from ItemDeleting, then process it in ItemDeleted?

Once again: would require inefficient thread interlocking, possibly not visible across frontends / processes, and certainly the most ugly solution you'd ever put into production code. You should be ashamed.

Store data in [web properties / custom list / user profile / own database / an usb drive] from ItemDeleting, then process it in ItemDeleted?

Sure, that would work, and all apart from the usb drive it would probably be accessible from all processes / frontends. Your major pain in the neck here, though, is the lingering fact that ItemDeleted *may not be executed*. Imagine installing some third party WillProtectYourDataAndCrap wsp solution, which essentially will cancel all ItemDeleting events. Then imagine your custom table / properties / whatever, suddenly containing 50.000 could-once-have-been deleted entries, which were never processed or removed since ItemDeleted never executed. At that point: welcome a custom SPJobDefinition to clean up every x minutes. Also give yourself complementary slap on the back, since you've just implemented the worlds most over-engineered and likely-to-break-down-horribly "What was just deleted?"-mechanism.

Man, wouldn't it have been great if ItemDeleted could just tell you what was just deleted?

Filling a SharePoint web service gap: Fetching current username, using client-side only jQuery Ajax

2009-09-18T09:36:00.007+02:00

I'm stumped as for why no such service was included in the first place, seeing as the code is a one-liner, and I imagine it quite useful to most service clients.

Having first settled on a custom service (code here) to deliver the user name, deploying that isn't always an option. If you've only got client-side access to the portal, you *need* to do it with a script..

So I came up with this solution, which should work in most cases:


window.userInfo = 
{
    /* Public interface */
    getCurrentUsername: function(callback)
    {
        $.get(
            "/_layouts/userdisp.aspx?Force=True", 
            null, 
            function(data)
            {
                var username = window.userInfo._findUsernameFieldText(data);
                callback(username);
            }, "html");
    },
    
    /* Private interface */
    _findUsernameFieldText: function(pageData)
    {
        var fields = $(pageData).find("table.ms-formtable td#SPFieldText");
        for(var i = 0; i < fields.length; ++i)
        {
            field = fields.get(i);
            for(node = field.firstChild; 
                node.nextSibling != null; 
                node = node.nextSibling)
            {
                if(node.nodeType == 8 && /FieldInternalName=\"UserName\"/.test(node.nodeValue))
                    return $(field).text().replace(/(^[\s\xA0]+|[\s\xA0]+$)/g, '');
            }
        }
        return null;
    }
};

What I realized, as I pondered this yesterday, is that there are obviously a lot of pages in the portal which lists the current user's info - including the user profile page. Extracting the username should be no harder than asynchronously loading the page (using jQuery) and parsing the resulting form looking for a field commented with 'FieldInternalName="UserName"'.

Should the userdisp page be violently customized, this would be less likely to work, but being a _layouts page; such a customization is unlikely.

Granted the above code, you can test it by pasting that (in a script tag) plus the following into the html source of a content editor web part:


<script src="http://www.google.com/jsapi"></script>
<script type="text/javascript">
    google.load("jquery", "1");
    google.setOnLoadCallback(function() 
    {
        userInfo.getCurrentUsername(function(usr) { alert("[" + usr + "]"); });
    });
</script>

Full source code download here.

Forcing SharePoint into asynchronous AJAX-like submission, using jQuery

2009-09-17T22:32:00.006+02:00

A while ago I posted a screencast on Twitter, of me navigating around a WSS 3.0 document library, without any synchronous reloads (full page refreshes): changing sort settings, creating folders and deleting files - all updated in place. If you didn't catch that, here's the screencast:

The demo looks pretty cool (he said confidently), and I feel that these are features SharePoint is lacking. With SharePoint 2010, this will all have been improved majorly, so it's admittedly not the most useful feature to be adding now - a mere few months before the release of '10. For the sake of (mis)using JavaScript and jQuery, though - let's stride on!

The code behind the demo is pretty intrusive - I'll be the first to admit that - and it's in no way complete. What it does, is override the usual SharePoint document library submit behavior, with a jQuery asynchronous load. The loaded data, which would be the page returned from the server, is chopped up and updated at the relevant spots on the existing page. If the user opens a folder in a document library, the asynch call loads the page with the view of the folder, then strips all the stuff which isn't the view of the folder, and replaces the current folder view: straight forward DOM stuff.

For this hack to (kinda) work, I also override some standard SharePoint APIs which returns state information (e.g. which document library folder we're currently viewing). I spent a minimal amount of time completing this, so I have no doubt that some state info will be missing or wrong in the code below, but it does seem to work fairly well for folder navigation, creation and item deletion. Sorting, however, is another matter entirely.

So all in all, there are quite a few kinks. The create folder popup is included mostly for show, and looks horrible. I didn't adapt this to work with the upload form, although that improvement would be pretty simple to complete. One of the more prominent issues, however, is the scripts which are returned along with the asynchronous loads. IE8 freaks out somewhat by these, and throws an error at you when you e.g. delete an item in a library. The fix is to ignore scripts in the parts of the page we keep (the view of the folder, etc.), but that's another thing I didn't look into.

All in all, I consider this a curiosity, rather than something of actual value. I'm positive someone with more time on their hands than me could improve it a ton, but then again - why bother with SharePoint 2010 coming up :-)

Excuses aside, here's the important part of the script:


var wssHook =
{
    /* Public interface */
    
    setupWSSHooks: function()
    {
        this.hookWSSApi("FilterFieldV3");
        this.hookWSSApi("GetSource");
        this.hookWSSApi("GetUrlKeyValue");
        SubmitFormPost = this.hookedSubmitFormPost;
        STSNavigate = this.hookedSTSNavigate;
    },
    
    /* Private interface */
    
    hookWSSApi: function(api)
    {
        this[api] = window[api];
        var body = window[api].toString();
        eval("window[api]=function" + body.substring(body.indexOf("(")).replace("window.location.href", "window.realurl?window.realurl:window.location.href"));    
    },
    
    hookedSubmitFormPost: function(url, forcesubmit, getonly)
    {
        window.realurl = url;
        var postvars = {};
        if(!getonly)
        {
            $(document.forms[MSOWebPartPageFormName]).find("input").each(function(id, element) { postvars[element.name] = element.value; });
        }
        else
        {
            postvars = "";
        }
        $("table#MSO_ContentTable>tbody").load(url + " table#MSO_ContentTable>tbody", postvars, 
            function(resp, status, req) 
            { 
                var location = req.getResponseHeader("Location");
                if(location)
                { window.realurl = location; }
            });
    },
    
    hookedSTSNavigate: function(url)
    {
        var shown = false;
        var dialog = $("<div></div>");
        dialog
            .dialog({
                    width: "80%", 
                    height: "300", 
                    modal: true,
                    close: function(event, ui) { dialog.remove(); SubmitFormPost(window.location.href, false, true); }
                })
            .append(
                    $("<iframe frameborder='0' width='100%' height='100%' src='" + url + "' />")
                        .load(function(){ 
                                if(shown) { dialog.dialog('close'); return; }
                                shown = true;
                                var c = $(this).contents();
                                var ctrls = c.find("td.ms-bodyareaframe>table").clone();
                                ctrls.find("script").remove();
                                c.find("table#.ms-main:first").remove();
                                c.find("form#aspnetForm").append(ctrls);
                            })
                );
    }
}

You can download the entire thing, including jQuery / jQuery-UI loads from the Google CDN, here.

If you want to give it a test run in your own portal; head to a document library, add a content editor web part and add the code downloaded from the link above as the html source.

Dealing with abducted SharePoint features

2009-09-17T13:19:00.006+02:00

If you've ever gotten an error message when deploying a feature with stsadm or WSPBuilder, resembling the following:

The feature '...' uses the directory "NewFeatureName" in the solution. However, it is currently installed in the farm to the directory "OldFeatureName". Uninstall the existing feature before you install a new version of the solution.

... or you've loaded up SharePoint Manager 2007, checked the feature definitions, only to find one or more features without the green "ALL IS OK" icon, an error reading "Object reference not set to an instance of an object" when you try to open them, and no option to delete.

... or you've otherwise been slapped a message indicating that you've once had a feature installed somewhere, but although its files are now gone, SharePoint won't allow you to deploy a new one with the same guid.

The solution is relatively simple, and can be automated with a lookup of the local farm's feature definitions, deleting those who throw an "not found" error when you attempt to access its properties. Kind as I am, I've already thrown this together for you, and made it available as a download here (Grep.SharePoint.Tools.CleanFeatures). The source code follows below:


namespace Grep.SharePoint.Tools.CleanFeatures
{
    using System;
    using System.Linq;
    using Microsoft.SharePoint;
    using Microsoft.SharePoint.Administration;

    internal class Program
    {
        private static void Main(string[] args)
        {
            if (!args.Contains("/D"))
            {
                Console.Out.WriteLine("Listing missing features. Invoke with /D to delete as they are found.");
            }
            else
            {
                Console.Out.WriteLine("Deleting missing features.");
            }

            SPFarm farm = SPFarm.Local;
            foreach (SPFeatureDefinition feature in farm.FeatureDefinitions)
            {
                try
                {
                    SPFeaturePropertyCollection p = feature.Properties;
                }
                catch (SPException e)
                {
                    if (e.ErrorCode == -2146232832)
                    {
                        Console.Out.WriteLine("Not found: {0} ({1})",
                                              feature.Id,
                                              feature.RootDirectory.Substring(feature.RootDirectory.LastIndexOf(@"\") + 1));
                        if (args.Contains("/D"))
                        {
                            feature.Delete();
                        }
                    }
                }
            }
            Console.Out.WriteLine("Done");
        }
    }
}

Find who's not mutually following you on Twitter, using jQuery Ajax / JSON

2009-09-17T10:23:00.017+02:00

Admittedly not the most useful use of the Twitter API, nor of jQuery, but nevertheless here it is.

Two different APIs are used; one to check the statuses of all friends of user X, which supplies us with 1. friends, 2. screen names and 3. user ids. The second API retrieves all user ids of the user X's followers. The two resulting lists of these APIs are diffed against each other, and voila - you've got the illoyal ones.

No username or password required to do this (unlike some other services I've seen), as long as your profile isn't protected (in which case you can easily modify it to supply a user / pass). It's completely harmless to run, as long as you don't hammer the Twitter server. The code will spend one API call (out of your total 150 per hour) per 100 friends of the user you target, and another single call to fetch all the followers. If you have more than 10k friends: don't run this.

Here's a fully functional demo. Feel free to try it on your own screen name. Again, no password or cross site scripting takes part here, so it's completely harmless. Input a screen name in the input box, and click the "check" button.

If you want to toy around with the code, it can be found here.

SharePoint logger bug workaround

2009-09-10T12:50:00.009+02:00

So yesterday I blogged about a bug in the SharePoint tracelog code, which would result in a ridiculous and unnecessary exception, should you happen to create a site collection, with feature file deployment, from a WCF service. If you haven't read that - feel free to do that now: 'Tis a SharePoint bug, it is!

Anyhoo, after giving up on having it fixed by the powers that be anytime soon, and not wanting to be too intrusive (ie. patching standard code, or injecting a hotfix at runtime), I went with a much more ... logical ... solution.

Allow me to demonstrate.

While this approach from yesterday didn't work:

Click it to view larger version.

This most certainly does:

In layman's terms: In the WCF service I now - rather than create the site coll directly - 1. Create a new thread, which creates the site coll, 2. Wait for the thread to end, and 3. Return. Makes perfect sense (actually it does). The stack rewind done in the trace api will stop at the entry into the worker thread, and thus never get to the DynamicMethod in WCF, which breaks it all apart.

Time for coffee.

SharePoint Conference 2009, I'm coming for you!

2009-09-10T09:42:00.005+02:00

Last year it was hosted in Seattle. I had a great time going there, and did catch some interesting presentations, but can't say that the technical content brought me to bits in awe. This year, with the community and contributors having matured for even longer, I'm thinking it'll be a blast. With the upcoming release of SP2010, and the posibility of presentations on that as well, I really don't see how it can fail. If worst comes to worst, there's always poker and booze, right? Or juice and pancakes; whatever suits your fancy.

Should *you* find your way to Vegas at the same time (18th of October =>), feel free to slap me a tweet/mail, so we can formally exchange our hellos!

'Tis a SharePoint bug, it is!

2009-09-09T14:32:00.019+02:00

I ran into this a few weeks ago, but had to delay dealing with it, for the sake of [somegoodreason]. Now I'm back on track, and I'm obviously going to whine about it to you.

The background for this problem is simple:

I create a site collection from a site definition, using code. This sitedef contains a couple of aspx files, which are put on the base of the new site's root web. To put the files there, I have Module tags in a feature's elements xml. The module tag contains a Name attribute, which is required, and used by SharePoint to look up the library it is to put the files in.

The thing is, though - ghostable files on the root of a web don't go in a library. So as SharePoint prepares the site, it fails to locate a non-existing library (which it wouldn't use anyway), and takes note of the error, but then carries on, as the library *really* wasn't required (told you).

Here's me explaining that by use of poor penmanship and malplaced creativity:

Click it to view larger version.

So while that scenario works out ok, what I'm actually doing, is calling the create-sitecollection code via a WCF service. And that plays out as follows (gore included):

Click it to view larger version.

The problem is rooted in SharePoint's logging code, which includes a routine to step back through the call stack, taking note of the where the failed call originated from. In this case, that would pass through the calls made by SharePoint to look up the library, deploy the file, open the elements, [...] and eventually into the service - which is internally in WCF an invoked DynamicMethod.

SharePoint's logger essentially does this:

StackFrame frame = trace.GetFrame(i);
builder.Append(frame.GetMethod().DeclaringType.ToString());

And that doesn't sit well with the DynamicMethod, which isn't really your run of the mill method (it's a dynamically compiled at runtime), and sure as batman doesn't have a DeclaringType. Calling the ToString() on a null reference ... not good at all.

Thus, SharePoint logs itself to bits and pieces, and I'm left none the wiser at the other end of the line.

So the moral of the story is:

Null references are bad
Assuming that all methods are declared somewhere, in a language which supports emitting and compiling code at runtime - also bad
Logging events that aren't actually errors, and which no part of the application cares about either way, even when logging is turned off - well you'll figure that one out yourself.

Update: I made a new post (and obviously an illustration) on a workaround: here

SharePoint hierarchical Quick Launch replacement using jQuery ajax

2009-08-10T02:25:00.016+02:00

(See here for a screencast of the in-development version)
(Solution (wsp) download available here)

The navigation control features (among other things):

Collapsible tree-view of the entire site collection
Lazy-loaded (jQuery ajax + json) sub-site structure
Node title editing, hiding and reordering (also ajax)
Customizable UI

Unlike many other ajax based controls, this doesn't post-back to the page itself, but rather to a standalone web service which is deployed with the feature. Consequently, this control will *not* cause a heap of server-side overhead (instantiation of many unnecessary controls, and so forth) caused by the slight horror that is e.g. plain ASP.NET Ajax.

The resulting Grep.SharePoint.Navigation control solution can be downloaded here, and taken for a test drive. At some point, I'll make a CodePlex site for it.

The setup is pretty easy. You need to activate two features, and the control will automatically replace the default quick launch navigation.

Once you've installed the solution, and deployed it to a site, head over to the web application features:

Make sure you select the right web application (use the drop-down in the top-right corner of the feature page to change this), and not e.g. your central admin web app. Nothing will be harmed if you happen to activate it for the wrong application - the navigation control simply won't work.

Once you've selected the correct web app, enable the extension feature:

Next, head over to the site you want to test it on, navigate to the site features (not site collection features), and enable the Navigation injection feature:

You can undo the navigation replacement at any time, by deactivating this very feature.

To edit the navigation, you can - as a site collection administrator - press the link above the navigation control, seen here:

When node editing is activated, unchecking the check box will hide the node from display, changing the name will rename the node (but not actually rename the underlying web / list). To restore default settings for the node, press the arrow button next to it.

Also, a style sheet is shipped along with the control, in your 12\TEMPLATE\LAYOUTS\Grep.SharePoint.Navigation folder - feel free to play around with that.

Don't take it from me .. So here's a book

2009-07-31T15:08:00.004+02:00

I've read a few SharePoint books, and I've poked through even more. Most of them, it should be no surprise, aren't really of much worth to the most dedicated of us. All too many books - and I find this especially so for SharePoint - seem to be mere paraphrases of what's already available on MSDN; hardly worth paying hard earned pigeons for.

A couple of months ago, after a pretty hefty pause from all technical offline references, I spent some time reading Building the SharePoint User Experience. Originally thinking this was going to be another "here's a list - it can hold stuff, but don't ask me how" kind of book, I was sort of a skeptic. What I found, however, was that it actually went pretty deep in subjects I had been pondering myself, not to mention all the stuff I had no idea about.

The best part, though, is the fact that it dares point out all the horrid, horrid stuff you're bound to encounter with SharePoint. The stuff that many others will happily ignore, so as not to displease anyone. Originally being a C++ dev, I'm used to stuff being awkward, ugly and downright painful - so I want to be told, rather than stumbling ass backwards into every trap there is. In short, this book saved my behind a few times in a library I was deploying at the time, so I feel I'm doing you a favor by giving you a heads up here.

The author (Bjørn Furuknap) also maintains a very helpful blog, at furuknap.blogspot.com, so you should at least give that a go.

Hovering Bing Search using jQuery

2009-06-11T22:48:00.017+02:00

Taking the Bing API for another test drive, I wrote up a jQuery / JSON Ajax driven website tool as well. The following little thing will show a semi-transparent hovering box as you select text on a web site. If you click the box, a Bing search will be made, and the top 10 results will be shown in the popup.

Feel free to give it a go at the example page. The initial JavaScript is shown in full shortly, but for updated source code and examples, I suggest you head over to the HoverBing CodePlex project I just created.


(function()
{
    getSelectedText = function()
    {
        if(window.getSelection){
            return window.getSelection().toString();
        }
        else if(document.getSelection){
            return document.getSelection();
        }
        else if(document.selection){
            return document.selection.createRange().text;
        }
    }   

    alternate = function(counter, norm, alt)
    {
        return counter % 2 == 0 ? alt : norm;
    }

    createHoverBingObject = function(settings) 
    {
        HoverBingObject = 
        {
            // Settings
            appId: settings.appId,
            numResults: settings.numResults,
            sources: settings.sources,
            title: settings.title,
            
            activateSearchPopup: function(x, y)
            {
                var popup = $("#SearchPopup");
                if(x && y)
                {
                    popup
                        .css({
                            cursor: "pointer",
                            left: x + 5,
                            top: y + 5
                        })
                        .show();
                    HoverBingObject.repositionToFitScreen();
                    HoverBingObject.fadePopupByDistance(x, y);
                }
                popup.data("SearchPopup_State", "waiting");
                $().bind('mousemove', HoverBingObject.mouseMoveSearchPopup);
            },

            create: function() 
            {
                HoverBingObject.initSearchPopup();
                $().mouseup(function(e) {
                        var selText = getSelectedText();
                        var popup = $("#SearchPopup");
                        var state = popup.data("SearchPopup_State");
                        if((state == undefined || state == "hidden") && 
                           selText.length > 0)
                        {
                            popup.data("SearchPopup_Query", HoverBingObject.washQueryText(selText));
                            HoverBingObject.activateSearchPopup(e.pageX, e.pageY);
                        }
                        else if(state != "stapled")
                        {
                            HoverBingObject.deactivateSearchPopup();
                        }
                    });
                $().mousedown(function(e) {
                        var popup = $("#SearchPopup");
                        var state = popup.data("SearchPopup_State");
                        if(state != undefined && state == "waiting")
                        {
                            HoverBingObject.deactivateSearchPopup();
                        }
                    });
            },

            deactivateSearchPopup: function()
            {
                var popup = $("#SearchPopup");
                popup.find("span").empty();
                popup
                    .fadeTo(0, 0)
                    .hide()
                    .data("SearchPopup_State", "hidden");
                $().unbind('mousemove', HoverBingObject.mouseMoveSearchPopup);
            },

            fadePopupByDistance: function(mouseX, mouseY)
            {
                var popup = $("#SearchPopup");
                var pos = popup.position();
                pos.left += popup.width() * 0.5;
                pos.top += popup.height() * 0.5;
                var dist = Math.round(Math.sqrt(Math.pow(mouseX - pos.left, 2) + Math.pow(mouseY - pos.top, 2)));
                popup.fadeTo(0, Math.max(1 - dist / 500, 0));
            },
            
            initSearchPopup: function()
            {
                // Create box
                var box = document.createElement("div");
                $(box)
                    .attr("id", "SearchPopup")
                    .attr("class", "HoverBing")
                    .css({
                            position: "absolute",
                            opacity: 0
                        })
                    .mouseenter(function() {
                            var state = $("#SearchPopup").data("SearchPopup_State");
                            if(state == "waiting") 
                            {
                                HoverBingObject.stapleSearchPopup();
                            }
                        })
                    .mouseleave(function() {
                            var popup = $("#SearchPopup");
                            var state = popup.data("SearchPopup_State");
                            if(state == "stapled") 
                            {
                                HoverBingObject.activateSearchPopup();
                            }
                        })
                    .data("SearchPopup_State", "hidden");
                
                // Create header control
                var header = document.createElement("div");
                $(header)
                    .attr("class", "Header")
                    .html(HoverBingObject.title);

                // Create content control
                var content = document.createElement("span");
                $(content)
                    .attr("class", "Content")
                
                // Append header and content to outer box
                $(box)
                    .append(header)
                    .append(content);
                    
                // Add outer box to body
                $(document.body).append(box);
            },

            mouseMoveSearchPopup: function(e)
            {
                HoverBingObject.fadePopupByDistance(e.pageX, e.pageY);
            },

            onClickSearchPopup: function()
            {
                var popup = $("#SearchPopup");
                var query = popup.data("SearchPopup_Query");
                var contents = popup.find("span").empty();
                $.getJSON("http://api.search.live.net/json.aspx" + 
                            "?AppId=" + HoverBingObject.appId + 
                            "&Market=en-US&Query=" + query + 
                            "&Sources=" + HoverBingObject.sources +
                            "&Web.Count=" + HoverBingObject.numResults + 
                            "&JsonType=callback&JsonCallback=?", 
                            HoverBingObject.onSearchResultsReceived);
            },

            onSearchResultsReceived: function(data)
            {
                var popup = $("#SearchPopup");
                var contents = popup.find("span").empty();

                if(data.SearchResponse == null) 
                {
                    contents.html("No search results returned");
                    return;
                }
                
                $.each(data.SearchResponse.Web.Results, function(i ,item) {
                        contents.append(
                                $(document.createElement("a"))
                                    .attr("class", alternate(i, "ResultLine", "ResultLineAlt"))
                                    .css({
                                            display: 'block'
                                        })
                                    .click(function() {
                                        HoverBingObject.deactivateSearchPopup();
                                    })
                                    .mouseenter(function() {
                                            $(this).addClass("ResultLineHover");
                                        })
                                    .mouseleave(function() {
                                            $(this).removeClass("ResultLineHover");
                                        })
                                    .text(item.Title)
                                    .attr("href", item.Url)
                                    .attr("target", "_blank")
                            );
                    });
                    
                HoverBingObject.repositionToFitScreen();
            },
            
            repositionToFitScreen: function()
            {
                var popup = $("#SearchPopup");
                var pos = popup.position();
                if(pos.left + popup.width() > $(window).width())
                {
                    popup.css({left: $(window).width() - popup.width()});
                }
                if(pos.top + popup.height() > $(window).height())
                {
                    popup.css({top: $(window).height() - popup.height()});
                }
            },

            stapleSearchPopup: function()
            {
                $("#SearchPopup")
                    .data("SearchPopup_State", "stapled")
                    .fadeTo(0, 1)
                    .one('click', HoverBingObject.onClickSearchPopup);
                $().unbind('mousemove', HoverBingObject.mouseMoveSearchPopup);
            },
            
            washQueryText: function(text)
            {
                return escape(text.replace(/[^'"A-z0-9]/g, "+")).substr(0, 100);
            }
        }
    }
    
    window.HoverBing = function(options)
    {
        settings = jQuery.extend({
                title: "Click to Bing Search",
                appId: "96AE4D816B34AA03F44EEBC53F4C23F9A146C011",
                numResults: 10,
                sources: "web"
            }, options);

        if(typeof(HoverBingObject) == 'undefined')
        {
            createHoverBingObject(settings);
            HoverBingObject.create();
        }
    }
})();

And there you have it. To use it, link the jQuery-1.3.2 javascript, include the above JavaScript, and the CSS styles found on the CodePlex site linked from the top, then activate it by using a jQuery ready handler such as:

$(function() { HoverBing(); });

Writing a Silverlight powered Bing news web part for SharePoint

2009-06-11T15:41:00.011+02:00

Microsoft recently unveiled their Bing search engine, and swiftly followed up by exposing an API free of request count limits. Grabbing that opportunity to write a really simple news fetcher web part for SharePoint, I set out to attach a Silverlight client to the Bing API, using Windows Communication Foundation (WCF).

Getting started with Bing

Essentially all you need to use Bing in your .NET apps, is an API id, which is available from the Bing developer site. Specifically, you can create an app id here.

The developer site also hosts a bunch of other resources, such as how to get started with other clients (using JSON or XML and so forth), so if you're interested - check that out.

Setting up your client

I chose to write a client in Silverlight, for a bunch of reasons, really. First of all I like how easy it is to wrap Silverlight apps together in Visual Studio, and second, it's really simple to make them shine using Blend. So while I could use jQuery, or just about any other technology, Silverlight pulled me in solely on how easy it is to get something working.

If you wish to test the Bing api from a console app, after creating your app id, you can do so by:

Create a new console project in Visual Studio.
Add a service reference to the Bing web services (http://api.search.live.net/search.wsdl?AppID=YourAppIdHere).
Name the service namespace BingService, for simplicity.
Slap the following code in there (be sure to replace the app id there as well):


namespace TestApp
{
    using System;
    using BingService;

    internal class Program
    {
        private static void Main(string[] args)
        {
            var bingClient = new LiveSearchPortTypeClient();
            SearchResponse response = bingClient.Search(new SearchRequest
                                                            {
                                                                AppId = "YOUR APP ID HERE",
                                                                Query = "einaros",
                                                                Sources = new[] {SourceType.Web}
                                                            });
            foreach (WebResult result in response.Web.Results)
            {
                Console.Out.WriteLine(result.Title + ": " + result.Url);
            }
            Console.WriteLine("Done");
            Console.ReadKey();
        }
    }
}

Running that should give you a bunch of url's you'd never wish to browse to.

Going all Silverlighty

Now that you've got the basics going, it's time to pull out your Silverlight. If you haven't installed the developer sdk / runtime and such, now would be a great time to do so. Head over to the Silverlight developer site for instructions on how.

With the environment setup, create a new Silverlight application in Visual Studio. Be sure to answer "yes" when you're asked whether to create a new test project as well. You'll need this to access the web services.

If you named your project anything like BingSilverNews, you should now see two projects; BingSilverNews and BingSilverNews.Web. The latter you won't have to touch at all, so feel free to collapse that.

The main project, however, should get the same service reference as previously added.

With the service reference added, head to your Page.xaml markup, and add a simple templated listbox as follows:


<UserControl x:Class="BingSilverNews.Page"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" 
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
    <Grid x:Name="LayoutRoot" Background="White">
        <ListBox x:Name="NewsResults">
            <ListBox.ItemTemplate>
                <DataTemplate>
                    <StackPanel Orientation="Vertical">
                        <TextBlock Text="{Binding Path=Title}" FontWeight="Bold"/>
                        <TextBlock Text="{Binding Path=Snippet}" TextWrapping="Wrap" Width="300"/>
                        <HyperlinkButton NavigateUri="{Binding Path=Url}"><TextBlock Text="Read More"/></HyperlinkButton>
                    </StackPanel>
                </DataTemplate>
            </ListBox.ItemTemplate>
        </ListBox>
    </Grid>
</UserControl>

If you're entirely unfamiliar with Silverlight, you may want to give Scott Guthrie's tutorial a go. There's really not much complexity to the above construct though.

What we're doing is adding a ListBox, and calling that "NewsResults", to ensure that we've got somewhere to put our results.

The Silverlight listbox however knows precious little about Bing, news results or anything remotely like that. It knows how to stack items in a neat little list, and that's it. To simplify handling of results from Bing, we're going to instruct it on how to handle each line of the results, using an item template.

Notice the DataTemplate above. The contents of that block will be repeated for each item in the Bing search result array. It's got a stack panel, which basically puts UI elements on top of, or alongside, each other, and two text blocks and a hyperlink. The key part of making the template work, is obviously tying the output to something from the active Bing search result line. That's where the Bindings come in. If you look to the title textblock, the "{Binding Path=Title}" simply makes the template fetch the value for the textblock's text attribute from the Title field of whatever object is being processed. Repeat that explanation for the rest of the template's controls, and we can move on. I'll soon get back to where the listbox (and the templates) get the data from.

Now turning to the codebehind, we're going to create a search service instance once more, and do a timed search every 30 seconds or so.


namespace BingSilverNews
{
    using System;
    using System.Threading;
    using System.Windows.Controls;
    using BingService;

    public partial class Page : UserControl
    {
        private readonly LiveSearchPortTypeClient _bingClient;
        private readonly Timer _updateTimer;

        public Page()
        {
            InitializeComponent();

            _bingClient = new LiveSearchPortTypeClient();
            _bingClient.SearchCompleted += OnSearchCompleted;
            _updateTimer = new Timer((state) => OnTimer());
            _updateTimer.Change(new TimeSpan(0), new TimeSpan(0, 0, 0, 30));
        }

        private void OnSearchCompleted(object sender, SearchCompletedEventArgs e)
        {
            if ((e.Result != null) &&
                (e.Result.News != null) &&
                (e.Result.News.Results != null))
            {
                // We've got news!
                NewsResults.ItemsSource = e.Result.News.Results;
            }
        }

        private void OnTimer()
        {
            if (CheckAccess())
            {
                _bingClient.SearchAsync(new SearchRequest
                                            {
                                                Sources = new[] {SourceType.News},
                                                Query = "Microsoft",
                                                AppId = "YOUR APP ID"
                                            });
            }
            else
            {
                Dispatcher.BeginInvoke(OnTimer);
            }
        }
    }
}

The source should resemble that of the first search client we made way up. The only difference here, is that we do it on a timer, and assign the result to a Silverlight ListBox, rather than just writing each result to console.

Worth noting is the fact that Silverlight requires an asynchronous service call model. That means that each call you make to a service, will return immediately. Only upon call to a predefined callback will the remote call actually have happened (and any return value be brought back). In this case, that's happens through the _bingClient.SearchCompleted property, which is told to execute OnSearchCompleted every time a search completes. The actual search call is now called _bingClient.SearchAsync(), rather than Search. Other than that, the search client behaves pretty much the same as in the previous app.

The timer does have some voodoo to it. Since it's happening on a separate thread from the UI, and we're updating UI controls as a result of the service call, we have to jump about a bit. This resembles the InvokeRequired semantics from WinForms: Only the UI thread can update it's controls - anything else will result in an exception, to avoid concurrent (and crashing) updates to the same controls. Because of this, we have to check for access to the UI on each OnTimer call, and if not granted, have the UI dispatcher schedule to call OnTimer once more in its own context. In other words, the UI and timer will run in different threads. Once the timer is signaled on it's thread, it will tell the UI to execute the _bingClient.SearchAsync call in its own thread.

Upon completion of each search, the search result array (noted by e.Result.News.Results), which has items of the NewsResult class, each holding Title, Snippet and Url fields, is assigned to the listbox. What will happen is that the listbox processes the array, one item at a time, and instantiates the contents of the xaml's DataTemplate for each one. Remembering that the template adressed the Title, Snippet and Url fields in its bindings, it's now obvious where these come from.

Compiling and running the app should result in something like the following(only my results are localized for Norwegian Bing):

Using this in SharePoint

Getting Silverlight going in SharePoint is a slightly discomforting process. That's why someone took the time to write the Silverlight in SharePoint Blueprints. While all the steps there aren't strictly required, it does get the job done.

Once you've made the necessary changes to your portal, I also suggest you adobt the brilliant WSPBuilder to speed up the process of writing (and deploying) webparts. There's a walkthrough of the WSPBuilder Visual Studio add-in here, which should get you going even quicker.

Once you've got this all setup, wrapping up the Silverlight web part is quite simple. Create a new WSPBuilder project, and add a new WSPBuilder Web Part with Feature item to that project. Name the project (and webpart) something meaningful, for your own good.

In my example setup, I've expanded the structure of the WSPBuilder project as follows:

Note the LAYOUTS and BingNews folders added under the TEMPLATE folder. Putting the .xap-file from your Silverlight project here (or better yet, adding a pre-build step to copy it to that project folder, from your Silverlight project's output folder), will make the WSPBuilder solution deploy it to the relative layouts folder of portal you deploy to. The .xap *is* the Silverlight, so this step is required for anything to work properly beyond here.

You'll also want to change the .NET runtime version for the project to 3.5, and reference the System.Web.Silverlight assembly, as well as System.Web.Extensions.

Heading back to the codebehind of your new webpart project, it needs a few (but not many) changes:


namespace Grep.SharePoint.BingNews
{
    using System;
    using System.Collections.Generic;
    using System.ComponentModel;
    using System.Runtime.InteropServices;
    using System.Web.UI;
    using System.Web.UI.SilverlightControls;
    using System.Web.UI.WebControls;
    using System.Web.UI.WebControls.WebParts;

    [Guid("your guid")]
    public class BingSearch : WebPart
    {
        private bool _error;
        private ScriptManager _scriptManager;

        public BingSearch()
        {
            ExportMode = WebPartExportMode.All;
        }

        /// <summary>
        /// Create all your controls here for rendering.
        /// Try to avoid using the RenderWebPart() method.
        /// </summary>
        protected override void CreateChildControls()
        {
            if (!_error)
            {
                try
                {
                    base.CreateChildControls();
                    var sl = new Silverlight();
                    sl.ID = ID + "_SL";
                    sl.Width = Unit.Percentage(100);
                    sl.Height = Height;
                    sl.Source = "~/_layouts/BingNews/Grep.Silverlight.BingNews.xap";
                    Controls.Add(sl);
                }
                catch (Exception ex)
                {
                    HandleException(ex);
                }
            }
        }

        /// <summary>
        /// Clear all child controls and add an error message for display.
        /// </summary>
        /// <param name="ex"></param>
        private void HandleException(Exception ex)
        {
            _error = true;
            Controls.Clear();
            Controls.Add(new LiteralControl(ex.Message));
        }

        /// <summary>
        /// Ensures that a script manager exists on the page.
        /// </summary>
        /// <param name="e"></param>
        protected override void OnInit(EventArgs e)
        {
            if (!_error)
            {
                try
                {
                    base.OnInit(e);
                    _scriptManager = ScriptManager.GetCurrent(Page);
                    if (_scriptManager == null)
                    {
                        _scriptManager = new ScriptManager();
                        _scriptManager.ID = "ScriptManager1";
                        Controls.AddAt(0, _scriptManager);
                        if (Page.Form != null)
                        {
                            Page.Form.Controls.AddAt(0, _scriptManager);
                        }
                        else
                        {
                            throw new Exception("No form tag found on current page. ScriptManager cannot be added.");
                        }
                    }
                }
                catch (Exception ex)
                {
                    HandleException(ex);
                }
            }
        }

        /// <summary>
        /// Ensures that the CreateChildControls() is called before events.
        /// Use CreateChildControls() to create your controls.
        /// </summary>
        /// <param name="e"></param>
        protected override void OnLoad(EventArgs e)
        {
            if (!_error)
            {
                try
                {
                    base.OnLoad(e);
                    EnsureChildControls();
                }
                catch (Exception ex)
                {
                    HandleException(ex);
                }
            }
        }
    }
}

This really looks a lot like the default WSPBuilder template, with a couple of modifications. I've removed the web part properties (you could keep them, though, and use those to pass keywords to your Silverlight Bing client, using the Silverlight control's InitParams), and I've added an OnInit override which adds a ScriptManager to the page - if that's not there already. The Silverlight control requires a ScriptManager to spawn Silverlight instances on the fly, so this is also a quite required step.

I've also changed the webpart's base class to System.Web.UI.WebControls.WebParts.WebPart (which is recommended by Microsoft, for all new web parts).

Other than that, all it does is create a Silverlight control instance, setup the source, and initialize the Silverlight's widt / height from the webpart's own width / height, and add it to the control stack.

Building the source, wrapping a WSP (using the WSPBuilder project context menu), and deploying it to your SharePoint portal. Then activating the feature, and adding the webpart, should add a simple little Bing News box to your portal. Expanding it further, I leave up to you. Being Silverlight, you can add all kinds of fancy effects to the UI, if you please, and even test it from an outside-of-SharePoint web project.

New and Improved Template Library Connector

2009-06-01T23:59:00.009+02:00

If you're new to the Template Library Connector, here's the nutshell version:

Grep's Template Library Connector adds the option to use a SharePoint document library as a template library for other document libraries. The templates are hierarchically shown in the "New" menu submenus of the document libraries, and instantiates exactly like common document templates. Changes to the template library are instantly available through the "New" menu of *all* linked document libraries.

There's an older, a bit more technical post available here, with install instructions.

Downloads (source and binary) is available at the CodePlex project site: Grep's Template Library Connector.

News for v1.5 include:

Full WSS support
No more webpart to connect document libraries: all config done from the list's settings page.
Will work correctly with webpart pages hosting multiple document library views (ListViewWebParts)

From v1.0:

Connect any document library to another document library, with one serving template documents to the other.
Not require any administration of new templates, other than uploading them to the template library - Instantly making them available to all document libraries linked with the template library.
Allow document libraries and template libraries to reside in different sites.
Supports Content Types in the template and document libraries, with workflow connections, columns, data information panels and the other goodies that provides.
Make the templates instantly, and hierarchically, available from the "New" menu in the document library.

Demonstration Video

Using Document Libraries as Template Libraries

2009-05-30T14:04:00.039+02:00

Download the source / wsp at CodePlex

Update June 1st: Version 1.5 released. The new version no longer requires a webpart, nor a lot of configuration, so I've updated the outlines seen in this post. See this new post for more info on the update.

Background

SharePoint document libraries are capable of handling many day-to-day tasks, right out of the box; there's no question about that. When you approach the need or wish for simple to maintain template libraries, that's an entirely different story.

A usual approach is to configure content types, and a set of hard-to-maintain URL links between these and actual documents in other locations. Administrative tasks, such as adding and removing templates, quickly becomes next to impossible, as you'll have to create new content types, and add /remove these manually for each document library you want the templates available in.

Another issue is the fact that the "New" menu for document libraries will show all content types at the root level. You can't categorize them in submenus, so large amounts of templates will be everything but a smooth ride for the end user.

Fixing things with a simple feature

Realizing that the aforementioned widespread pain is completely unnecessary, I set out yesterday to write a feature which can:

Connect any document library to another document library, with one serving template documents to the other.
Not require any administration of new templates, other than uploading them to the template library - Instantly making them available to all document libraries linked with the template library.
Allow document libraries and template libraries to reside in different sites.
Make the templates instantly, and hierarchically, available from the "New" menu in the document library.

.. And make it all available to all of you, complete with source code, pre-built solution/feature, instructions and documentation.

Installation and Usage

Download the latest source and install packages here.

Unpack and run the install.bat file from the root folder. This will install the solution, and deploy it to all site collections. If you want more fine-grained control, feel free to install and deploy the .wsp-file on your own.
Open a SharePoint portal with document libraries you want to template enable.
Open the Site Settings, and navigate to the Site Feature page.
Enable "Grep's SmartDocument Template Library Connector"
Open a document library you want to expose templates in, and open it's settings.
Navigate to the "Template Library Connector settings" entry, under "General Settings".
Make sure you select "Yes" to enable template library connections, pick a document library to act as a template library, and save.
Navigate back to the document library, and verify that the "New" menu is wired up properly.
Adding new documents to the selected template library will instantly make new templates available from the "New" menu.
You can optionally add a column called "Description" to the template document library, for additional text under each template, in the "New" menu view. If you wish to call this something other than "Description", the column name can be changed through the webpart settings shown above.

And that should be it. If you've got any questions or problems, feel free to drop me a note through the comments here.

Getting Technical

Writing a solution like this takes a few steps, and a bit of tinkering with undocumented SharePoint libraries. Here's a rough cut of what's covered in the source:

Utilizing delegate controls to install handlers for all opened pages
Find all ListViewWebParts on the webpart page, and retrieve entries from its toolbar.
Adding items to ListViewWebPart's menus
Adding submenus to the "New" menu
Adding submenus to submenus
Utilizing the default SharePoint document template JavaScripts to create new documents
Retrieving hierarchies from SharePoint document libraries
Writing and exposing list settings pages

A few of these steps, such as adding menus and submenus, are powered by extension libraries I've included in the source code. A simple thing like adding a submenu to a submenu isn't easily and readily available in the SharePoint object model, but with the extension libraries in the source project, it's ridiculously simple!

Code Pieces

I won't be covering everything here, but some key parts of the functionality, which I haven't seen elsewhere on the web, will be brought forward.

Extending the LiewViewWebPart's "New" menu

The ListViewWebPart has a toolbar, but this isn't immediately accessible from its object model. Why this is so is oblivious to me, as I see many possible benefits of extending the default views.

Ignoring the ListViewWebPart's attempt at hiding it, and the fact that even toolbar access functions are internal, I extended the System.Web.UI.Control class to give me the following edge:

  1: public static T FindChildByType<T>(this Control self)
  2:     where T : class
  3: {
  4:     var stack = new Stack(self.Controls);
  5:     while (stack.Count > 0)
  6:     {
  7:         var control = stack.Pop() as Control;
  8:         if (control is T)
  9:         {
 10:             return control as T;
 11:         }
 12:         foreach (Control child in control.Controls)
 13:         {
 14:             stack.Push(child);
 15:         }
 16:     }
 17:     return null;
 18: }

The approach is pretty straight forward: it will do a stack-based iteration of the control class, until a control of the target type is found.

Including this allows you to call a function such as FindChildByType() on any class deriving from System.Web.UI.Control. Doing just that on the ListViewWebPart will, with no further hassle, let you access its inner NewMenu.

Once you get an instance of the NewMenu class (which derives from the superbly named ToolBarMenuButton class), you can call functions such as AddMenuItem and AddMenuItemSeparator (see MSDN). There's however no function to add submenus (or submenus to submenus, for that matter), so that will require another extension.

  1: public static SubMenuTemplate AddSubMenuAt(this ToolBarMenuButton self, int index, string id, string displayName,
  2:                                            string imageUrl, string description)
  3: {
  4:     SubMenuTemplate child = CreateSubMenu(id, displayName, imageUrl, description);
  5:     self.MenuTemplateControl.Controls.AddAt(index, child);
  6:     return child;
  7: }

The key part in the above code is the call to CreateSubMenu, which returns a SubMenuTemplate. The latter is a contruct from the Microsoft.SharePoint.WebControls namespace, which is used for several menus around the SharePoint UI, such as the Site Settings menu you get in MOSS when the "Office SharePoint Server Publishing" feature is enabled.

  1: private static SubMenuTemplate CreateSubMenu(string id, string displayName,
  2:                                              string imageUrl, string description)
  3: {
  4:     var child = new SubMenuTemplate();
  5:     child.ID = id;
  6:     child.Text = displayName;
  7:     child.Description = description;
  8:     if (string.IsNullOrEmpty(imageUrl))
  9:     {
 10:         imageUrl = "/_layouts/images/MenuNewItem.gif";
 11:         child.ImageUrl = imageUrl;
 12:     }
 13:     else
 14:     {
 15:         if (imageUrl.IndexOf("/") == -1)
 16:         {
 17:             imageUrl = "/_layouts/images/" + imageUrl;
 18:         }
 19:         child.ImageUrl = imageUrl;
 20:     }
 21:     return child;
 22: }

With these functions, along with a few more included in the source project, such as extending SubMenuTemplate to allow addition of nested submenus; adding menus and submenus is easy as pie.

Adding meaningful menu items

Getting to add menu items is one thing, but actually adding something meaningful is what obviously makes it useful. In this webpart, the goal was to make it possible to instantiate document (templates) from other document libraries, and that turns out to be surprisingly simple if you study how Microsoft does regular template instantiation. The javascript function called "createNewDocumentWithProgID" is the key here, and it accepts such parameters as source file url and target folder url. If the menu items are passed calls to this javascript in its click handler, with the correct urls, document instantiation is in the bag as well.

In Closing

So that's about it. Feel free to check the code, use the webpart, develop it further and share alike. For any questions or comments, use the comment feature on this blog.

Effective SharePoint UI prototyping using jQuery, FireBug and Greasemonkey

2009-05-18T14:11:00.020+02:00

Demo script download available at the end of the post.
Updated May 19th, with clearer code examples.

Background

Customizing SharePoint's user interface, or even updating your own interfaces, can be a tiresome thing to do. Usually you'll have to deal with multiple master pages, many master page overrides, multiple css files and a wide array of dynamic code from XSLTs or web parts. All this makes it quite difficult to maintain a clear understanding and overview of the UIs you're customizing, or what the impact of your changes will be.

To add to the already high stress levels, many changes - especially those where you're deploying changes to your own web parts - will require recompiles, iis resets, feature reactivation or similar time consuming steps.

In this article I want to propose an alternative approach to UI changes, using jQuery prototyping. While jQuery isn't necessarily something you'd want to use to do all SharePoint UI restructuring, it does come in handy if you want to make simple runtime enhancements to the current UI, or even want to extend your custom parts with simple asynchronous behavior, or neat visual effects.

To accomplish this, in a way which will require no recompilation, redelpoyment or other nasty things, and overall keep the time between change and prototype visualization as low as possible; we'll be using the FireFox addons FireBug and Greasemonkey. The former is a brilliant DOM / CSS / Html / Network explorer, built into FireFox' interface, with lightning quick structure inspection and navigation. Greasemonkey, on the other hand, can inject custom javascript into any loaded page, without making any serverside changes.

The combination of these tools will allow us to write advanced javascripts in our favorite javascript editor, and see the results immediately, without having to upload any files to the server. A simple browser refresh will suffice.

Requirements

FireFox (3.0.10 is the latest at the time of writing)
FireBug
Greasemonkey
Knowledge of JavaScript and HTML / DOM.

Installing

Download and install FireFox, if you haven't done this already.
Start FireFox, and open the Addons config screen, available from the Tool menu.
Activate "Get Addons" function on the top bar, then search for and install FireBug and Greasemonkey.
Restart FireFox.
Navigate to a SharePoint portal you wish to prototype, and confirm that you've got a set of FireBug / Greasemonkey icons in your browser status bar, such as:
Right click the Greasemonkey icon, make sure it's set to enabled, and click New User script:
Fill in values for the namespace and script title, such as:

The "Includes" box should be the url to the page you wish to prototype. This can be changed later, so stick with the default value for now. In other words, don't copy this from my example above.
Next you'll be asked to select which editor to use for script editing. This can be any text editor of your choice, such as notepad, or in my case, Visual Studio.
Once the editor selection is completed, it will be opened, you'll see an empty script. To test that you've got it all setup correctly, you can try adding an alert statement, and reload the page.
Next click the FireBug icon, and activate the HTML function:
If you're unfamiliar with FireBug, play around with the Inspect function, to see how various UI elements correspond to the HTML source code. Selecting an element also enables you to immediately see the active CSS styles for said element.

Prototyping something useful

Assignment: Making the Quick Launch on the left collapsible.

"Useful" is relative, I agree, but this is at least example use of how to hotplug jQuery into the mix, and how you can use that to experiment with UI changes you'd later wish to deploy to a master page, or portal page.

From here on we'll concentrate mainly on the script opened in your chosen editor. As of now, all the content should be a commented "UserScript" section - which is good. If you've entered anything else, feel free to remove that now.

So, without further stalling, lets code.

Loading scripts (such as jQuery) at runtime

Greasemonkey lets us inject custom scripts, but rather than messing any more around with it's configuration, we'll do the rest of the script loading with code. Since this is a prototype, and we'll be doing all of the prototyping in FireFox; loading scripts is simple.

The only complicating step, is the fact that Greasemonkey will execute Greasemonkey scripts in a sandbox, isolating certain functionality from the scripts already loaded in the browser. The reasoning here is that Greasemonkey, and its scripts, will execute with greater privileges than those loaded by the browser. To prevent browser-loaded scripts from getting unwanted access from your custom Greasemonkey scripts, and possible do naughty stuff on your system, Greasemonkey forces us to take certain actions to access the stuff outside the sandbox. If you wish to read more about that, search for "unsafeWindow" in your favorite search engine.

I've put together a set of functions, which will load custom scripts in order, and map the objects from these into the namespace available from the Greasemonkey sandbox. The functions looks as follows, and you can go ahead and input these in your prototype script. Reading and understanding them are optional steps -- we'll soon be getting to the interesting stuff.

  1: function mapUnsafeObjects(objectArray) {
  2:     if (objectArray == null) {
  3:         return;
  4:     }
  5: 
  6:     for (var i in objectArray) {
  7:         var objectName = objectArray[i];
  8:         window[objectName] = unsafeWindow[objectName];
  9:     }
 10: }
 11: 
 12: function loadScript(url, objectArray, loadCallback) {
 13:     var js = document.createElement('script');
 14:     js.src = url;
 15:     js.type = 'text/javascript';
 16:     js.wrappedJSObject.onload = function() {
 17:         mapUnsafeObjects(objectArray);
 18:         loadCallback();
 19:     };
 20:     document.getElementsByTagName('head')[0].appendChild(js);
 21: }
 22: 
 23: function loadScripts(scriptArray, completeCallback) {
 24:     if (scriptArray.length == 0) {
 25:         completeCallback();
 26:     }
 27:     var scriptEntry = scriptArray.shift();
 28:     loadScript(scriptEntry.url, scriptEntry.objects, function() { loadScripts(scriptArray, completeCallback); });
 29: }

So these are the only utility functions you need to load external scripts. You can reuse them in any prototyping scripts you feed into Greasemonkey, to load whatever external content you need.

Here's an example on how to load the jQuery, jQuery UI and JSON2. We won't be needing more than the first in the following prototype, but if you wish to play around with jQuery UI (which has some cool drag and drop features, among other things), or the ajax + JSON functionality in jQuery, the others will be handy.

  1: loadScripts([
  2:              {url: "http://jquery.com/src/jquery-latest.js", objects: ["jQuery", "$"]},
  3:              {url: "http://jquery-ui.googlecode.com/svn/tags/latest/ui/minified/jquery-ui.min.js", objects: []},
  4:              {url: "http://www.json.org/json2.js", objects: ["JSON"]}
  5:             ], onLoadComplete);

What this will do, is load one script at a time, making sure it's completely loaded before skipping to the next in line. For each script loaded, an array of objects is mapped into the sandbox namespace, for easy access in the prototype script. Once all scripts are loaded, the callback function named at the end will be called.

First jQuery action

Thus, the onLoadComplete function will be where our action will take place. Once this is called, jQuery will be fully loaded, and available for use. To demonstrate this, create a callback such as follows:

  1: function onLoadComplete() 
  2: {
  3:     $("a").css({ fontSize: '18pt' });
  4: }

Once you save this, along with the previous code snippets, in the Greasemonkey script opened in your chosen browser, and reload the portal, all html anchor elements will get a ridiculous font size. Not too useful, but it at least confirms everything is working properly. If the jQuery syntax is unfamiliar to you, now would be a good time to read up on some basic examples at jQuery.com.

Collapsible Quick Launch

This is when FireBug will come in handy. Writing the code is simple, and Greasemonkey is great, but without a proper tool to navigate the html, we'd be banging heads into walls before long.

With the FireBug window opened, and pointed to "HTML" mode, hit Inspect and select the section just below a main quick launch link, as shown in this screen shot:

Doing this tells us that the quick launch main menu items have IDs such as "zz2_QuickLaunchMenunN", where N is the menu item index. Tapping into this with jQuery is pretty simple, with a selector such as $("tr[id^=zz2_QuickLaunchMenun]"). What we'd like to do with this, is to attach a click handler to collapse / expand the table row below it, if the following table row isn't another main menu item.

Adding such a handler, in a pretty straight-forward manner, would look like:

  1: function onLoadComplete() {
  2:     $(function() {
  3:         $("tr[id^=zz2_QuickLaunchMenun]")
  4:             .next("[id=]").toggle()
  5:             .prev().click(function(evt) {
  6:                 $(this).next().toggle();
  7:                 evt.preventDefault();
  8:             });
  9:     });
 10: }

This function will install a handler to be run once the DOM tree is loaded and ready for manipulation: the "$(function(){})" syntax. This handler will, noted by line number:

3: Select all tr nodes with an id beginning with "zz2_QuickLaunchMenun". Hence forth called "main items".
4: For each of these main items, select the next sibling, if it has an empty id attribute. The empty id is an important point, as we don't want one main item to collapse the following main item, in the case of there being no child menu between the two. If there's no child menu, lines 5-8 will do nothing more for this main item. In this case, the selector from line 3 will bring us to the next main item instead.
4: Toggle (collapse) the child menu.
5: Select the sibling's previous item, which brings the selection state back to the main item
5: Install a click handler to the main item.
6: Use the toggle() function from jQuery to collapse or expand the sibling (child menu).
7: Simply prevents the click handler from bubbling further on to other nodes down the tree.

If jQuery is still new to you, you may just have noticed that jQuery selectors are cascading. Selecting one node, then doing another selector expression on that, will make a relative selection. That's how we, in the previous example, can move from the parent to the child, then back to the parent.

Saving and reloading the page, should confirm that the above code is working. When you click the background of the main quick launch items, the section will collapse or expand.

To wrap things up, we're going to add another piece of UI code, to animate the font size of the quick launch menu items, as they are mouse hovered. Expand / replace the function to resemble the following:

  1: function onLoadComplete() {
  2:     $(function() {
  3:         $("tr[id^=zz2_QuickLaunchMenun]")
  4:             .mouseenter(function(evt) {
  5:                 $(this).find("a").animate({ fontSize: '14px' });
  6:             })
  7:             .mouseleave(function(evt) {
  8:                 $(this).find("a").animate({ fontSize: '11px' });
  9:             })
 10:             .next("[id=]").toggle()
 11:             .prev().click(function(evt) {
 12:                 $(this).next().toggle();
 13:                 evt.preventDefault();
 14:             });
 15:     });
 16: }

Saving the script and reloading the page will actually uncover a bug in our previous selector, which wrongfully assumed that only the main items of the quick launch menu have the previously mentioned IDs. As it happens, sub items also employ this ID. While the click handler won't be affected (due to the sibling check), the mouseenter and mouseleave hover effect will be. Hovering the menu will now animate everything, while we want only the main items to be affected.

Pulling FireBug back up real quick will show that the items we actually want to attach to, have a parent table with an ID of "zz2_QuickLaunchMenu". To fix the issue we can thus take advantage of a more specific jQuery selector syntax: $("#zz2_QuickLaunchMenu > tbody > tr[id^=zz2_QuickLaunchMenun]").

This tells jQuery to look for a tr node with an id which starts with "zz2_QuickLaunchMenun", that has a direct parent element of type tbody, which in turn has a direct parent element with id "zz2_QuickLaunchMenu". Saving and reloading the page will confirm that the hover effect now only affects the main menu items.

Where to go from here

What I've demonstrated is a way to install and setup a hotpluggable SharePoint UI development environment. jQuery, and various other scripts, have vast possibilities for expanding SharePoints UI, and essential to exploring these, is a way to doing so quickly, without redeploying a bunch of stuff to the development environment (possibly polluting that, with nonsensical experiments).

Once you're done with the prototyping, you would obviously have to deploy jQuery, or whatever scripts you use, to the portal. Either in shape of a master page change, a new aspx page, web parts page template or similar. How you should deploy it depends largely on what you're installing, and what you want it to override.

There's no doubt in my mind that SharePoint development can benefit from such hotplug prototyping, however. Both the previously blogged about CodeConsole web part, and this simple prototyping environment, can - if used with care - simplify your life as a SharePoint developer or designer. Any opinions on how it could be done differently, are much obliged of course!

Downloads

MyProtoType.js

The final example code, with a decent looking expander icon next to the collapsible rows.

CodeConsole Minor Update

2009-05-18T13:51:00.004+02:00

The CodeConsole Web Part just got a minor update, to allow simple inclusion of other assemblies and namespaces. This came in handy when I was debugging some SPContext-dependent functionality in a few libraries I deployed to a portal's bin dir.

In the new version, the source / wsp of which is still available here, the syntax is updated to support '@' prefixing for namespaces, and '#' for assemblies. Such as:

@MyLibrary
#Some.NameSpace.From.MyLibrary
MyClass foo = new MyClass();
output.Write(foo.SomeFunction());

Where the first MyLibrary reference would be an assembly in e.g. the bin folder of the current web application, or the GAC. No .dll suffix required.

CodeConsole Web Part for SharePoint

2009-05-13T15:52:00.019+02:00

About the CodeConsole Web Part
Download Source Code + WSP

This is a web part I've found myself wanting several times, when I've been either too lazy to compile and deploy code, or haven't had the development tools available on the target SharePoint box.

In short, the CodeConsole Web Part ..

Executes C# code, entered into a text area, on the server. The code can access the SharePoint object model, and other .NET / ASP.NET objects.
Displays output from the execution in an output text area.
Uses jQuery to make asynchronous requests to a custom .asmx service on the server.
Uses JSON to serialize / deserialize data, should more complex structures be required by someone who decides to use it.

A Screen Shot of the Web Part in action:

Click the image for a larger version
Update: Uploaded a demonstration screen capture video, in case a screen shot isn't enough. Check it out.

In this shot, the input frame is white, and the output frame is black. The code passed through the input box will have the following using directives available to it by default:

using System;
using System.IO;
using System.Text;
using Microsoft.SharePoint;

This can easily be extended in the attached code, otherwise you'll have to use fully qualified names in the code.

For output redirection, the code entered will have an object called 'output' available. This is a System.IO.TextWriter, meaning it will have such functions as Write and WriteLine. The above example screen shot uses this output object, so turn to that if you're confused.

Installing the web part is pretty straight forward, but will (due to the jQuery / JSON use) involve a few manual config steps. See my previous blog post for more information on that. Within the source code package, there's a folder called "wsp", within which you'll find a precompiled solution file, which can be deployed to a working SharePoint 3.0 portal.

About the implementation

The code isn't complex at all, and I'm sure many would point out that I could have done things a little bit differently here and there (such as rewriting how the web part is displayed, and taking optional configuration input to decide whether or not to link the json / jquery javascripts - which may not be necessary, if they've already been imported by another webpart, or in the master page).

Essential files all reside in the TEMPLATE\FEATURES\CodeConsoleWP folder, where:

CodeConsole.cs is the web part's source file. This emits all ui and javascript, including references to jQuery and JSON. Most essential of which is the following javascript. Yes, it's simple. Feel free to extend it. I'm sure intellisense would be a nice addition ;)


$(function(e){
    var input = $("#CodeConsole_input");
    var output = $("#CodeConsole_output");
    $("#CodeConsole_execute").click(function(be) {
        output.text("Compiling, please wait.");
        var code = {'code' : input.text()};
        var jsonStr = JSON.stringify(code);
        $.ajax({
            type: 'POST',
            url: '/_vti_bin/CodeConsoleSVC.asmx/ExecuteCode',
            data: jsonStr,
            contentType: 'application/json; charset=utf-8',
            dataType: 'json',
            success: function(msg) { output.text(msg.d); },
            error: function(xhr, msg) { output.text("Ajax Error:\n" + msg + "\n" + xhr.responseText); }
        });
    });
});

CodeConsoleSVC.cs is the web service source file, which basically looks like:


namespace CodeConsoleWP
{
    using System;
    using System.IO;
    using System.Web.Script.Services;
    using System.Web.Services;

    [WebService(Namespace = "http://tempuri.org/")]
    [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
    [ScriptService]
    public class CodeConsoleSVC : WebService
    {
        [WebMethod]
        public string ExecuteCode(string code)
        {
            using (TextWriter stringWriter = new StringWriter())
            {
                try
                {
                    Dynamic.ExecuteCode(code, stringWriter,
                                        new[] {"System", "System.Text", "System.IO", "Microsoft.SharePoint"});
                    return stringWriter.ToString();
                }
                catch (Exception ex)
                {
                    return "Error:\n" + ex.Message;
                }
            }
        }
    }
}

Most interesting in the above web service, which will be deployed to the /_vti_bin/, along with other SharePoint services, is the call to Dynamic.ExecuteCode. This is a very simple class I typed up, which generates an assembly in-memory, referencing all assemblies of the hosting assembly (the web part, meaning SharePoint assemblies and such will be referenced), wrapping the code, executes it and returns an optional return value.

Dynamic.cs


internal static class Dynamic
{
    public static object ExecuteCode(string code, TextWriter output, string[] namespaces)
    {
        using (CodeDomProvider provider = CodeDomProvider.CreateProvider("C#"))
        {
            var parameters = new CompilerParameters
                                 {
                                     GenerateExecutable = false,
                                     IncludeDebugInformation = false,
                                     MainClass = "temp",
                                     GenerateInMemory = true
                                 };

            foreach (AssemblyName refasm in Assembly.GetExecutingAssembly().GetReferencedAssemblies())
            {
                parameters.ReferencedAssemblies.Add(Assembly.Load(refasm).Location);
            }
            string usingDirectives = "";
            foreach (string ns in namespaces)
            {
                usingDirectives += "using " + ns + ";";
            }
            CompilerResults results = provider.CompileAssemblyFromSource(
                parameters, 
                @"namespace Dynamic {" + 
                usingDirectives + 
                @"public class Code { public object Run(System.IO.TextWriter output) { " + 
                code + 
                "; return null; } } }");
                
            if (results.Errors.Count > 0)
            {
                var sb = new StringBuilder();
                foreach (CompilerError error in results.Errors)
                {
                    sb.AppendLine(String.Format("{0},{1}: {2}", error.Line, error.Column, error.ErrorText));
                }
                throw new CompileException(sb.ToString());
            }
            else
            {
                object obj = results.CompiledAssembly.CreateInstance("Dynamic.Code");
                object ret = obj.GetType().InvokeMember("Run", BindingFlags.InvokeMethod, null, obj,
                                                        new object[] {output});
                return ret;
            }
        }
    }

    public class CompileException : Exception
    {
        public CompileException(string s)
            : base(s)
        {
        }
    }
}

Compiling and installing

Upon build, the source will be compiled, and put along with the service endpoint in a SharePoint solution file (wsp). This is taken care of by the build scripts in the DeploymentFiles folder. This file will end up in the CodeConsoleWP\wsp\ folder. To install the web part, just deploy this solution to the SharePoint server. Be sure to follow the instructions in my previous post, to add json/ajax support to SharePoint, though - otherwise the service call will fail miserably.

If you've got any questions, drop me a note in the comments here, on twitter, or by email.

Download Source Code + WSP

Enabling JSON serialization and jQuery posts for SharePoint web services

2009-05-13T11:38:00.005+02:00

While making the finishing touches to a new web part I'm publishing here shortly, I found myself wanting to make JSON enabled jQuery AJAX requests to custom .asmx services I'd deploy to the _vti_bin service root of my SharePoint portal. The incentive is simple; jQuery is quite nice for quickly typing up dynamic UIs, and by connecting that to a service backend, I can do even more fun stuff (which will become clear in my next post). An alternative to jQuery would be to stick with ASP.NET Ajax, but to be honest, I find that a lot less comfortable, when my code is mostly client side (javascript), and I'm deploying it all to SharePoint.

The overhead of using ASP.NET ajax for simple stuff like this is another story altogether, and while I don't have any specific numbers, I'm sure you'd see quite a bit more data on the wire (large scripts, xml encapsulation, etc.), compared to simple jQuery ajax post requests with JSON for data encapsulation. If you're interested in that part, throw in a few google searches - I'm sure it's been thoroughly covered elsewhere.

Anyhoo, the point here is jQuery/JSON and whatnot, so if that's what you're after; keep reading.

What I soon discovered after attempting to make simple $.ajax calls from jQuery, was that HttpPost isn't enabled for the _vti_bin folder. Trying to call /_vti_bin/MyService.asmx/SomeMethod will result in some less-than-intuitive error page (especially when seen through the deserialized error callback of jQuery's .ajax()). This is essentially caused by the default _vti_bin config not allowing HttpPost service calls. Your options are either to change the configuration, which I'll soon outline, or make SOAP envelope calls. I find the latter fairly tedious, having to construct fairly large XML structures in strings, rather than just calling an url like that of SomeMethod above.

Fixing the config, to allow simple service call urls, is pretty simple. Open the web.config from your "Web Server Extensions\12\ISAPI" folder, and flick the "remove" keyword of HttpPost to "add". The system.web configuration should look like:

<system.web>
    <webServices>
        <protocols>
            <remove name="HttpGet" />
            <add name="HttpPost" />
            <remove name="HttpPostLocalhost" />
            <add name="Documentation" />
        </protocols>
    </webServices>
    <customErrors mode="On"/>
</system.web>

At this point, you'll be able to call the web services through HTTP Post requests, even from jQuery, such as:

$.ajax({
    type: 'POST',
    url: '/_vti_bin/MyService.asmx/SomeMethod',
    data: '{}',
    contentType: 'application/json; charset=utf-8',
    dataType: 'json',
    success: function(msg) { alert(msg.d); },
    error: function(xhr, msg){ alert(msg + '\n' + xhr.responseText); }
});

Still there's the case of JSON, however, so the JavaScript above would still trigger the error callback, with your web service's return value encapsulated in XML. Should you wish to use XML for data transport, you'd be all set now.

To enable JSON, you'll first of all need the .NET 3.5 framework installed on the server. If you haven't already done this (2008 should have it pre-installed), go do so now. Once that's in place, there's a few quick changes which need to be made to your web application's web.config - not dissimilar to the general steps to make ASP.NET Ajax working.

Within <system.web><httphandlers> tag, add the following line, right after the initial <remove>:

<add verb="POST" path="*.asmx" validate="false" type="System.Web.Script.Services.ScriptHandlerFactory, System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"/>

Within the <safecontrols> tag, add the following at the bottom:

<SafeControl Assembly="System.Web.Extensions, Version=3.5.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" Namespace="System.Web.UI" TypeName="*" Safe="True" />

This should pretty much be all you need, and you should be all set for using jQuery and JSON to query your own custom _vti_bin deplyed services, from web parts, pages and so forth.

My next post will provide an example of this combination, calling a custom .asmx service from jQuery, with JSON serialization for parameters / return value, neatly wrapped up in a WSP.