Fail Forward Fast

Bumper Cars – Bumper Workflows – run on a thread currently executing workflow

2012-05-01T08:42:03-07:00

Workflow ID=e23514ca-ac35-4aa2-8f3b-33a8e2fbe966 attempting to run on a thread currently executing workflow ID=29797226-38d1-4f25-ba10-da9fe458af21. This workflow will be run at a later time.

If you've ever gotten this error before in the ULS, you'll probably be scratching your head. Basically, your workflow has collided with another running workflow instance on the same process and it got kicked over to be ran via timer job. If you're impatient like me, this is annoying, since the workflow timer job runs only every five minutes. To resume your workflow, simply find the timer job and click Run Now.

How do you avoid the issue all together? For starters, stop processing your workflows on your WFEs, as can be seen by reading this post. Then, start your Workflow Timer Service on several of your application servers (image below) and change them to process workflows every 1 minute (image above) to make them process near continuously.

Not only will you not run into collisions, but your web front ends (aka, end user experience) won't be bogged down in workflow processing.

Phil

Why the cloud fuss? Because little fishes can now have the same tools as big fish

2012-04-04T18:40:54-07:00

As one of the largest cloud providers in the world, Microsoft, isn't stupid when they say "we're all in" when it comes to cloud computing. If you read my previous five posts in this series, you may think that they're "all in" simply because the writing is on the wall – the cloud is the future of computing.

However, they're all in because there's also a lot of money to be made.

Take a look at Microsoft's 2011 annual report. In it they discuss how 80% of small businesses using Office 365 (Email/Exchange, SharePoint, Lync, and Office) had never purchased one of those products previously. Microsoft is tapping a market they were never able to penetrate before. There's a pretty simple reason when you think about it.

First – if you were a small business, would you rather spend $100,000 on hardware, software, and professional services to configure a SharePoint deployment, when you could get the same thing for $6 / user / month? It's virtually the same product, whether it's on-premise or in the cloud, and yet there exists a financial chasm between the two.

Also, why would Microsoft target small businesses? Is there money to be made in selling to small businesses at $6 a user? You bet. In fact, since 80% of all fortune 500 companies already own SharePoint, for example, there aren't many other markets besides small business to even target in the first place. The enterprise beast was slain by Microsoft decades ago. Small businesses are all that's left to conquer.

Plus the small business market is bigger (and worth more to Microsoft) than you'd think. Consider these interesting statistics from Uncle Sam.

How important are small businesses to the U.S. economy?

Small firms:

Represent 99.7 percent of all employer firms.
Employ half of all private sector employees.
Pay 44 percent of total U.S. private payroll.
Generated 65 percent of net new jobs over the past 17 years.
Create more than half of the nonfarm private GDP.
Hire 43 percent of high tech workers (scientists, engineers, computer programmers, and others).
Are 52 percent home-based and 2 percent franchises.
Made up 97.5 percent of all identified exporters and produced 31 percent of export value in FY 2008.
Produce 13 times more patents per employee than large patenting firms.

Check out that 2^nd bullet – small businesses represent HALF of all employees in the U.S. economy.

HALF.

Half of the jobs in America is a pretty big market. Half of the economy represents quite the opportunity for any cloud wannabes. Consider again that 80% of Office 365 businesses hadn't ever purchased email, collaboration, or connectivity products from Microsoft previously. It's pretty clear that Microsoft was letting opportunities "die on the vine", as they say.

Not anymore.

In fact, Microsoft's cloud offerings are taking off like a you wouldn't believe. They reported that in the first 30 days of Office 365 being released, over 50,000 businesses signed up.

50,000!

Fifty thousand small businesses all of a sudden had a cost effective way to purchase enterprise products. Previously only the largest companies in the world had that opportunity. Now, with the cloud, a ma and pa shop with five employees can have email, IM, and document sharing access for only $50 / month. Take that times fifty thousand and you got millions of reason to get excited about the cloud.

Well – this concludes my "Why the fuss" series. I'm sure I could keep going. Perhaps I will again, but these represent overwhelming reason in my opinion to conclude that the cloud is real, and represents a fundament shift in Information Technology in the coming decade. The IT world will look quite different in ten years. You best be ready.

"Why the fuss?" Blog series:

Why the Cloud Fuss? Because the PC’s foot is already in the grave

2012-03-29T07:00:00-07:00

This blog series is all about why Cloud computing is such a big deal. Yesterday I discussed how on-premise cloud computing is already facing exodus, and today I'm extending that discussion to the PC.

Yep – the PC is dying… in fact the PC will be dead by 2014, Gartner says:

By 2014, "the personal cloud will replace the personal computer at the center of users' digital lives".

Two years isn't very far away.

Megatrends in mobility

One of my favorite megatrends listed in the Gartner study was #5, aka the trend of mobility and the need for a "wherever and whenever you want" end user experience:

Megatrend No. 5: The Mobility Shift — Wherever and Whenever You Want
Today, mobile devices combined with the cloud can fulfill most computing tasks, and any tradeoffs are outweighed in the minds of the user by the convenience and flexibility provided by the mobile devices.

How true. The trouble is this shift in technology is moving way faster than CIO's and software companies can keep up with. Younger generations especially are demanding a mobile and social "cloud-based" connectivity. For a CIO, there are things you can start doing today to get ready for this paradigm shift that's taking place, such as:

Start planning for products such as Unified Access Gateway and Direct Access to enable anywhere secured access from mobile devices to your internal applications without a VPN connection.
GET OFF INTERNET EXPLORER 6 and WINDOWS XP! J - ok, this one should be obvious, but you'd be surprised how many companies are still on this archaic technology… there's probably no hope for them if they're already so far behind.
Check your WAN connectivity – as users' are more and more cloud-based, you're going to need more and more bandwidth
Start your "Mobile & Cloud Readiness" portfolio planning – TODAY
Go back to the Enterprise Architecture drawing board, and redraw it to map better to changing end-user habbits
Start playing "what-if" scenarios with your business architecture (users and what they're trying to do), your information architecture (your information and where it is), and see how those scenarios impact to your technical architecture (aka, the cloud).

Your employees have already started shifting to the cloud – whether you like it or not.

Embrace and extend beats passivity in this situation.

"Why the fuss" Blog series:

Why the cloud fuss? Because the on-premise exodus has already began

2012-03-28T07:00:00-07:00

In yesterday's post I discussed how there's a chasm between on-premise IT infrastr
ucture and the cloud. It's a financial chasm. The fact is, why would you spend $100k on hardware and another $100k on software when you can get the same service in the cloud for $5 / user / month?

Yes, there's a chasm, and the exodus to the cloud has already started.

A recent Gartner study shows that at the onset of 2012, 20% of all businesses have ZERO IT assets.

ZERO.

Zero is not a big number. How do these companies survive without a significant investment in IT assets? They survive because they can get the same thing at a fraction of the cost in the cloud.

And it's not even just a financial issue. Not only does it come at a fraction of the cost – the cloud comes with a fraction of the risk. The cloud promises a peace of mind that CIO's are eating up like candy.

Downtime? Not my problem.

Backups? Not my problem.

Disaster Recovery? Not my problem.

High availability? Not my problem.

Compliance and security? Not my problem.

My job as CIO is to be a business enabler, with minimal investment and risk. The cloud is how I do this…

"Why the fuss" Blog series:

Reference:

Gartner, (2010). Gartner Highlights Key Predictions for IT Organizations and Users in 2010 and Beyond. Referenced from http://www.gartner.com/it/page.jsp?id=1278413

Why the cloud fuss? Because your CIO is spending a fortune

2012-03-27T07:00:00-07:00

Think back with me to the early 1900's.

Back in those good ole' days, manufacturers didn't have electrical companies to supply them with the so often taken for granted utility, electricity. Since no one was around to sell them electricity, they needed to make their own.

Ever wonder why so many old plants were built by rivers? To harness the power of the river.

That's all changed now. In fact, nobody wants to be near a river today because of cost of land (or cost of environmental impact studies). The point is… nobody cares because today we have this neat thing called the power grid.

The Impact of a Shared Utility

If you want to spin up a new manufacturing plant that makes bicycles, there's no need to waist millions of dollars building a nuclear reactor. Simply pay your utility bill on time and you're good to go.

This is what's happening today to the information technology world. Cloud computing is to IT today, what electricity was to manufacturers in the early 20^th century.

Your CIO hates recreating the wheel

The promise of shared infrastructure is why Cloud Computing was ranked as your CIO's #1 and #2 priorities. Why?

Because 70% of your CIO's budget is spent maintaining infrastructure

Yep – it's true. Your CIO's primary expenditure is to recreate the wheel, year after year. Consider this chart of IT expenditures:

What % of this budget do you think is impacted by the essential out-sourcing of infrastructure spending? It's pretty easy to see the 70% is that pie.

Cloud computing is as impactful to IT as the electrical grid was to manufacturers

Do you doubt the impact? Consider that the majority of IT jobs are going to radically change over the next 10 years as companies continue to adopt cloud computing, buying software versus building it, and continued outsourcing. Why would you play $1,000,000 in software and hardware, when you can use the cloud for $5 / user / month?

There's a chasm between on-premise and the cloud. In 10-20 years, on-premise as we know it today, will be long gone. It will be replaced by cloud computing. IT Infrastructure is becoming a commodity, controlled by the largest "utility" providers in the world… such as Microsoft, Google, Amazon, and Apple. I'd make it a goal to be on their payroll if you want to keep your infrastructure job.

What's the silver lining?

The benefit to this nightmare for some is that IT is no longer going to be a cost center for businesses. Rather, it will become a business enabler.

Business enabler is a better descriptor. Technology is fast becoming the primary medium for innovation, especially when you consider the saturation of social media with younger generations.

Cloud computing, and specifically the applications built in the cloud, will be the primary foundation of innovation for generations to come.

Just look at the cloud-based services such as Facebook, Office 365, Windows Azure, iCloud, Amazon Web Services, etc. No point in fighting it. It's here.

Embrace it.

Seek out jobs where you can assist with the transition. It's coming – and companies are looking for cloud leadership.

"Why the fuss" Blog series:

Why the cloud fuss? Because Cloud Computing has Priority

2012-03-26T07:00:00-07:00

What's with all the fuss lately about "the cloud?" We've heard about the cloud before. Is it real this time?

As some of you know, last fall I finished my book that featured "the cloud." It was about how to deploy SharePoint to the cloud, but the cloud is much bigger than just SharePoint.

THE CLOUD IS A BIG DEAL

All this week I'm going to do a series on why the cloud is a big deal. The cloud truly is a paradigm shift in the Information Technology world.

Is the fuss being taken seriously?

Yes. In fact, your boss cares a ton about the cloud:

A Gartner study in 2010 shows that across the 1,600 CIOs polled, "cloud computing" ranked as their #2 priority for the year.

Consider this with me… the National Institutes of Standards and Technology define the cloud has having the following five characteristics:

On-demand self-service, AKA users of a cloud should be able, to at their Lieser, consume cloud resources and have those resources available to them within moments.
Broad network access, AKA "the cloud" should be widely available, across the Wide Area Network as well as across many platforms and devises. The cloud has no favorites.
Resource pooling, AKA the cloud is virtualized, AKA foundational resources are abstracted away from those consuming them
Rapid elasticity, AKA the cloud can grow and shrink capacity to meet demand – automatically, as well as self heal, etc.
Measured service, AKA the consumer of the cloud gets a service level agreement (SLA) and they get their money back when that SLA isn't met.

Now – an easier way to say all that is to simply look at the following diagram:

That's essentially what those five points are getting at. With the cloud being the #2 priority of CIOs, you really get the feeling that the cloud is here to stay this time. What's a CIO's #1 priority?

Virtualization

You heard it correctly, virtualization. The cloud REALLY takes the top two spots on the CIO's list, since the cloud – for most businesses – is essentially virtualization + automation. Yes, characteristics 1, 2, and 5 are cloud too, but 3 and 4 are the coolest features of the cloud.

With "all the fuss" about the cloud, and since your boss cares a lot about it, you ought to care a lot about it to. But first, **WHY** does your boss care so much about the cloud? More on that tomorrow!

"Why the fuss" Blog series:

Reference:

Gartner, (2010). Survey of nearly 1,600 CIOs shows budgets to be at 2005 levels in 2010. Referenced from http://www.gartner.com/it/page.jsp?id=1283413

PowerShell Script to Delete Closed SharePoint Web Parts

2012-03-23T07:00:00-07:00

I am working at a client today who had literally thousands of closed web parts in their SharePoint sites. If you don't know already, it's always better to delete a web part rather than close it. Closing a web part doesn't delete it (albeit, it gets removed from the page). The closed web part remains in the database and it possess usability and performance concerns if it gets out of hand. In fact with SharePoint 2007, closed web parts take the same amount of resources as open web parts, so if you have 50 closed web parts on the page you can see a pretty big performance issue. Fortunitly this is no longer the case with SharePoint 2010.

Anyways, I saw this nice script from Brian Jackett that lists closed web parts. I extended it to also delete those web parts as well, since after all that's the point. Of course there's risk of data loss here, but when you have 3,000+ closed web parts accross your farm and users are complaining that pages are slow... a strong hand is needed. Enjoy!

Add-PSSnapin Microsoft.SharePoint.Powershell

$site = new-Object Microsoft.SharePoint.SPSite("[YOUR URL]")

$site.AllWebs | ForEach-Object {

$currentWeb = $_

$pages = $currentWeb.Files | Where-Object {$_.Name -match ".aspx"}

$title = ""

$pages | ForEach-Object {

$currentPage = $_

$webPartManager = $currentWeb.GetLimitedWebPartManager(

$currentPage.ServerRelativeUrl,

[System.Web.UI.WebControls.WebParts.PersonalizationScope]::Shared)

$array = New-Object System.Collections.ArrayList

$webPartManager.WebParts | ForEach-Object {

if($_.IsClosed)

{

$array.Add($_.Title)

Write-Host "'$($_.Title)' on $($currentPage.ServerRelativeUrl) is closed"

}

$array | ForEach-Object {

For($i=0;$i -lt $webpartmanager.WebParts.Count;$i++) {

if($webpartmanager.WebParts[$i].title -eq $_)

{

$wp = $webpartmanager.WebParts[$i]

if($wp.IsClosed)

{

Write-Host "Deleting $($wp.Title)..."

$webpartmanager.DeleteWebPart($wp)

break;

}

$currentWeb.Dispose()

}

$site.Dispose()

Create a hustle culture to grow your business and retain your employees

2012-03-22T07:00:00-07:00

Hustle: to play a game or sport in an alert aggressive manner.

One of the things I value most for people that work on a project with me is a sense of urgency. I just LOVE a good hustle. It reminds me of football in high school – the coach demanded that the kids hustle. Some did. Some didn't. Some were lazy.

A team that has a sense of urgency is a team that gets things done.

They're fast.

They're efficient.

They're focused.

They're appreciated.

THEY WIN.

It's the kind of team I want to be a part of. It's the kind of team I strive to create.

Sense of urgency doesn't mean PANIC

Does that mean I want to create a panic stricken culture? BY NO MEANS! Panic occurs in tandem with fear. I don't like fear. Fear is a poor motivator – long term. You can use fear to get things done, sure. But then your employees quit because you're a jerk.

A healthy sense of urgency is more like a good hustle. It's what separates you from your competition. A good hustle is why you won the business. It's why your customers keep calling you back for more. And, I argue it's why your employees are happy.

I believe a foundational tenant of a growing business and happy employees is to create a hustle culture.

A hustle culture is good for business

The thing about a good hustle is that everybody benefits from it when everybody is doing it. The football team that hustles wins the game. Customers get a better product at a lower price. Revenues are higher because customers are promoters. Teammates feel supported by their teammates. Hustle creates comradery through shared accomplishment and contribution. Comradery benefits employee retention. New employees sense the hustle culture and quickly understand they need to hustle too. The cycle continues.

Hustle is the engine of growth

Average doesn't grow a business or make a happy team. Average means average customer satisfaction. Average means average retention. Average means average culture. Average is status quo.

Lazy destroys – relentlessly. Root out the lazy. Encourage the average. Celebrate those who hustle.

A Hustle Culture is key to Happy Employees

Here's the deal… I'd rather have an employee hustle for 40 hours than an average employee work 50 hours. In America there's this wacked out notion that everybody needs to work 50 hours a week to be successful.

I don't buy it. People choose to work 50 hours per week most often because they choose not to hustle. The rare few hustle for 50+ hours per week, but I wouldn't call that a good thing. That just leads to a burned out employee, not a happy employee.

If you hustle for 40 hours and spend the other 10 hours with your family or at recreation – you'll be a happier person. Happy people tend to be content. Content people tend not to quit. Happy and content people tend to hustle. It's cyclical. I could elaborate – but it's really that simple. 10 hours per week for a life time has a pretty big effect on the individual and the culture.

Another key point is that hustling for 40 hours is definitely less stressful than working 50 hours per week over the long term. Working hard is better than working long – because how we use our time is where we derive so much of our happiness. Let 40 be enough, but give the 40 you have all you got (and encourage your teammates to do likewise).

How did that football coach get them kids to hustle?

Ever wonder how coach got all those kids to hustle? Do you think it's because he yelled at them? Nope. He painted a vision of winning, and sold those kids on that vision. He taught them that winning is more than just beating the competition. Winning is best when it's shared. Winning is about the feeling you get when you and your 50 buddies all did something extraordinary. Winning is more about peer celebration, and it's through this sense of shared accomplishment that you create a hustle culture in your team and within your company.

The winning metaphor is sometimes a bit hard to translate into a business context. For example, a project doesn't have an opponent. There is no winner. However, the sense of shared accomplishment is present, and little things that encourage it is how you get average people to hustle.

The following are some hustle tips:

Give a battle cry often
Create short term and long term TEAM goals
Have a vision statement for your team
Talk about those goals on a daily basis
Have frequent team celebrations
Give your team a focus… one thing that is the foundation for how you measure their success
make their focus about THEM, not the customer, projects, etc.
Lead by example – demonstrate what a good hustle looks like

The thing is I could probably come up with a few dozen more tips. I think leading by example, and doing 20 small things that create shared accomplishment are really the foundations of building a hustle culture. As a manager, the key to your success is to find a way to facilitate shared accomplishment on a daily basis. As you do this, your team will begin to instinctively hustle more and more.

How do you encourage your team to hustle?

Getting workflows to process on an app server, where they belong

2012-03-21T07:00:00-07:00

By default SharePoint workflow instances aren't processed on an application server in the SharePoint farm. Instead, they're processed on the web front end that initiated the workflow. This is a bummer for two reasons:

This isn't WEB traffic… it's application processing.
You typically only want WEB traffic on your **WEB** front ends

Web requests can be negatively impacted when junk is running on your web front ends, besides web requests of course. Translation: Workflow processing can slow page response times, which makes for a bad end user experience.

When I build farms I try to isolate web traffic on my web front ends, and therefore move all services onto my application servers. But how do you get workflows to process on an application server, when by default they always kick off from within the w3wp process from whence they came?

Microsoft SharePoint Foundation Workflow Timer Service

The funny thing is I actually lied. There's a timer service that processes workflow jobs. The thing about it is the timer job only runs when the threshold of queued jobs (across the farm) is greater than 15, the default. So, job number 16 gets queued and sent to the workflow timer job for processing, which runs every 5 minutes by default.

The trick, then, is to set the threshold from 15 down to 0. This will force all workflow processing to be handled via the timer job. Then, simply make sure that the timer service is only started on the application servers. Start the service on all the application servers to setup a round robin effect.

Lastly, run the following PowerShell cmdlet to set the threshold:

Set-SPFarmConfig –WorkflowPostponeThreshold 0

Pretty slick huh?

Bottom line – if you have a workflow heavy farm, change the threshold to 0 to get that blasted CPU pressure off your web front ends.

The Chasm between OOB, SharePoint Designer and Visual Studio Workflows

2012-03-20T07:00:00-07:00

Wouldn't it be great if there ever were a one-size fits all tool for building custom workflows for SharePoint? Alas! There isn't L. Here's why:

The OOB workflows are useless
SharePoint Designer is a toy
Visual Studio is just too danged hard

The out-of the box workflows are useless

Ok – useless was a strong word. If you're seeking basic approval via task and/or email, you'll find value out of a few of the out-of-the-box workflows within SharePoint. You may wonder… has anyone complained to Microsoft for the apparent lack of useful out-of-the-box workflows? The basic answer is yes, people have complained, but it makes a lot of sense when you think about why there's only a handful of OOB workflows. I mean really, try to think up as many one-size fits all business processes as you can. Needless to say, your list will be short – which is why there aren't a lot of "good" out-of-the-box workflows.

SharePoint Designer is a glorified toy

I'm sure this will turn some heads… but it's true. SharePoint Designer workflows are only good for one-off basic workflows. Enterprise BPM? Not a chance. Here's why SharePoint Designer workflows lack luster:

Buster #1) They're sequential… meaning they can only progress forward. Think of a business process within your company. Chances are that business process needs to go "back in time" at some point. A SharePoint Designer workflow can't do this. It can only move forward (aka, A > B > C > D, where A is the start and D is the end). Any serious business process shouldn't be attempted in SharePoint Designer. The only way to "fake it" is to start a new workflow instance before the first instances completes. The better approach is to build a state machine in Visual Studio. The image below shows the two different types and how they would look on a diagram:

Buster #2) You can't easily integrate with other lines of business apps and/or data. Here's a platform buster: Another line of business application needs to send a message to an inflight workflow, and thereafter the SharePoint workflow needs to respond to the calling application. Designer workflows can't really integrate well (out-of-the-box) with other systems external to SharePoint. Now, you can use BCS with Workflows but really all you can do is CRUD (create, review, update, and delete) data, YOU CAN'T RESPOND TO EXTERNAL EVENTS! This is a significant limitation to SharePoint Designer workflows. For instance, look at the figure below. This figure shows several workflow instances in SharePoint that interact with some Line of Business application. They call into the system, and then sit and wait for a response from that system. The red arrows reflect the response, where the line of business app wants to notify the waiting instance that it has either finished processing or that a human has performed some action. You can't cross this boundary in SharePoint Designer workflows.

Buster #3) Parent Child workflows: In SharePoint Designer there is no way (out-of-the-box) to have a parent workflow spawn child workflows and wait for those workflows to finish executing (exception – you can spawn the OOB workflows). I see so many companies build a huge workflow that tries to do everything and fails. Plus, you then get no reusability in your workflows. Rather, I suggest building smaller, reusable workflows and having parent workflows invoke those child workflows, as seen in Figure 4:

Visual Studio is just do danged hard for most people

Ok – now I've let you think that Visual Studio is your best bet for creating custom workflows for SharePoint. Well, the truth is, is you're a .NET developer and you have a limited budget, you're right – it is your best option. But the reality is most IT-Professionals aren't developers. So how you do cross the chasm when OOB, Designer, nor Visual Studio fit the bill? This is where 3^rd parties come in.

My take on Workflow ISVs

The goal when looking at 3^rd party workflow tools is to consider what your business driver is:

Enterprise business process management
Simply filling the void between SharePoint Designer and Visual Studio

Knowing which group you fall into makes a huge difference. First, if you're looking for enterprise BPM (LOB integration, etc), you'll be spending a lot more money than if you simply wish SharePoint Designer was more intuitive and had more bells and whistles. BPM tools that come to my mind are K2 and Global 360, and I believe you can spend a 100k without trying too hard. These are big bucks tools for big BPM problems.

On the other hand you have Nintex workflows for the common folk to use, and yet it provides a ton more features and capabilities than SharePoint Designer. What's more is Nintex is much more reasonably priced. Agile Point is another promising provider in this space, albeit I have yet to research their differentiator.

In the end you have balance maintainability and flexibility, as the following image shows. This is even true for these 3^rd parties I mentioned. For example, Nintex is easy to use, and much less expensive, than say K2, but it isn't as robust.

These are the guidelines that I've used to advise my clients on 3^rd party workflow tools. Do you know of any compelling differentiators that are worth mentioning?