There is a dire need for better technical managers in the world.

As a basketball coach I would look at a player as a freshman, evaluate their skills, and then spend the next two years working with them on a specific set of drills so that they would fulfill a role for my team in the future. For example, if I knew I needed them to get rebounds we would focus on those drills primarily. By the time they became seniors they would fill a need for the team to be successful.

How many managers these days have such foresight that they can say “Look, PowerPivot is coming, and some people keep asking about cubes, maybe I should send some people for some BI training”. Very few, because most managers are not in their role to have foresight or to innovate, they are there because they can execute orders given to them by others.

And therein lies the bulk of the problem, poor managers lead to poor quality employees which lead to everyone shaking their head and saying “how do you claim to have five years’ experience as a DBA and not know what a heap is?”

I was very fortunate that when I was taking my first baby steps as a DBA. I had a wonderful manager to watch over me. Let’s call him Vinny, because that’s his name anyway. Vinny was more than just my manager, he was my coach. He encouraged me to be a sponge. He helped me get sent off to conferences like PASS (and even got me to Barcelona once, I have no idea how that happened). And he started to work with me on the necessary soft skills I would need as my career progressed.

I said I was fortunate to have Vinny, but it’s more like I hit the Manager Lotto with him. Unfortunately, not everyone is as lucky. Including me, because when Vinny retired a few years back none of my subsequent direct managers had the same coaching ability that Vinny did. As I was shuffled from one manager to another over time I found that Vinny was truly a treasure. Without him I doubt I would have progressed as far and as fast as I did.

So, when I see questions about trying to find SQL Server experts I sit right up and yell at my iMac screen and say “WE’RE RIGHT HERE!” But I am also the same guy who likes to yell “I’M AS MAD AS HELL AND I’M NOT GONNA TAKE THIS ANYMORE!” every time I hear the neighbor’s dog barking at night.

I have little doubt that many managers are looking at their current staff and thinking “boy, if I only had a guy/gal that could do everything I need, then we’d be making progress. Why can’t I find that expert?” And if I was within reach I would slap them in the back of the head and tell them to wake up.

It’s almost comical that people are sitting around debating the idea about where the SQL Server experts have gone and yet very few people are trying to do something about it. Did they think we all went on strike, emulating a story line from Atlas Shrugged? If you were alone in a forest and couldn’t find food, would you turn to your friend and say “man, why isn’t there a KFC nearby?” No! You roll up your sleeves and take matters into your own hands! You would go find the food you need. So why not go and find the experts?

You want to know where the SQL Server experts have gone? I’ll tell you where they have gone:

They Are Sitting In Front Of You

Unless you don’t have a DBA in your shop then chances are you have an expert right there in front of your face every day. Have you ever tried listening to your DBA when they tell you that the unique constraint that the developer implemented doesn’t scale beyond 100 rows? Or did you simply look their way and say “just make it work and don’t touch the code”.

Vinny would listen to the DBA. So should you.

Chances are you have a DBA that is already a generalist in many areas and a specialist in one or two. Try finding out what areas are their specialty and see if you can complement them with others. Yeah, that’s right, others. Which reminds me…

There Is A Crapload Of Stuff To Know

Check out this list: SQL Server, SQL Server Management Studio, SQL Server Analysis Services, SQL Server Integration Services, SQL Server Reporting Services, SQL Server Configuration Manager, SQL Server Profiler, SQL Server Notification Services, T-SQL, and SQL Azure. Noticing anything in common? Yeah, they all have the letters SQL in them. Which means, by default, that your DBA knows all of them inside and out, backwards and forwards, right?

Wrong.

Just because it says SQL doesn’t mean your DBA will know everything well enough to have an immediate answer for whatever problem you need fixed immediately. That doesn’t mean they are not an expert. In fact, the best Senior DBAs are the ones that can open the manual and understand something well enough in a reasonable amount of time that they have a chance at helping you with your problem. So, if they take a few hours to figure out something in SSIS that doesn’t mean you start questioning their skills as if you are thinking about taking away the car keys from grandpa.

Also, when you go looking for help, try to refrain from listing everything under the Sun in your job description. I bet a LOT of people frustrated about the perceived lack of SQL Server experts are the same people that want someone to step in as their DBA/Exchange admin/Sharepoint admin/IIS guru/AD expert. I laugh out loud when I get emails from recruiters telling me that they have my dream job, doing those five things and having over 700 databases under my control. I like to tell them that they are the ones dreaming. Not only don’t I want that job, but neither does anyone else, unless you plan on paying for five salaries as well.

Vinny knew this, and that’s why we set up our team in a manner to have complementary skills, and we would reassess those skills periodically to make certain we had enough bases covered.

So don’t be surprised that the only people showing up to the interview are the ones that aren’t qualified. See, the ones that are qualified are too smart to show up for a nightmare job description. And, quite frankly, if you have to resort to recruiters to find yourself a good DBA then you might as well be ordering brides from Russia. If you want to find where the real SQL Server experts are these days, everyone knows that…

We Are Hanging Out On Twitter

Look, if you want to find the best DBAs, doesn’t it make sense to go to where they are, as opposed to waiting for them to come find you and your craptastic job description? If you are actively interviewing strangers for an open position with your company then you are not doing anything different if you were to place an ad for a date on Craigslist. You have about the same chance at happiness.

If you truly want to find the experts then go to where the experts are. Attend a local user group meeting. Go to a SQL Saturday. Join PASS and attend the PASS Summit. Get hooked on Twitter and use the #sqlhelp hashtag. We are not hiding from anyone. We’re right here, we love to help each other, and we love to talk about new opportunities all the time. It helps us to understand more about where we are, where we are going, and the paths that are available.

Vinny knew this as well. He understood that networking was a key part in my success, mostly because I was working alone in my office and I would need some type of virtual water cooler.

If you want to find the SQL Server experts stop expecting us to show up out of nowhere. We aren’t going to show up to interview for some awful job description. Your best bet to land an expert DBA? Buy us dinner and get to know us a bit before you ask us to go steady. I know you’ve got the money, because you haven’t spent anything on your…

Training Budgets

So, you are standing there, lamenting about how you can’t find any SQL Server expert yet you have people on staff right now, perhaps even a DBA. Or maybe you have someone who is interested in becoming a DBA. Go and send them to a training class! Look ahead to the future, plan what you will need to get done, and send one of your employees to get the necessary training. You can send them to the PASS Summit where they will not only learn technical skills but they will increase their network as well.

Vinny knew this. He not only sent me to the PASS Summit, he had me take a pre-con seminar. He knew that I would learn a lot from the seminar and make a friend or two, and that those friends would serve as a valuable resource when I needed help. Vinny knew all of this, and so should you.

Over the years my network has expanded greatly. And as a SQL MVP I get access to the greatest help support imaginable. Back when I was still a production DBA, if I needed help I could get it in a very short amount of time. It made me very efficient at my job. I didn’t need to spend hours trying sixteen different solutions before deciding on a course of action. I could ask a question, get sixteen answers, and then decide where to focus my efforts.

And all because I had a manager that wasn’t afraid to send me to whatever training I needed. If you are asking about where all the SQL Server experts are, ask yourself if you have been doing all you can to train the people you already have. Chances are you haven’t because you’re no Vinny. That’s OK, few people are. But if you are a manager of a DBA then you should really be able to…

Know The Difference Between Good and Bad

If you have no idea what a DBA does and you find yourself imitating Bob from Office Space by asking them “What, exactly, would you say it is that you do here”, then you shouldn’t be managing the DBA. It’s that simple. If you have no idea if the person reporting to you is doing a good job or not, then they shouldn’t be reporting to you. And you shouldn’t be scratching your head wondering why you can’t find an expert, either, because you have no frakkin idea if they are good or bad.

Vinny knew the difference (and yet he still kept me around at the start). That’s why he could give me the necessary course corrections from time to time. Every so often we would talk about some of the latest trends, where I wanted to take a deep dive next, or something that was related to my job. He would match my interests with the needs of the company and keep me on track for growth. And as time went on I would find myself willing to crash through thicker and thicker walls for him.

He never looked at my desk and said “why can’t I find an expert?” He knew he had one in the making, and he was going to coach me all the way there. And he did.

And that’s how I got here. Because of Vinny.

Apparently there are too few Vinny’s in the world, and that’s why there is a perception that there aren’t enough SQL Server experts.

Don’t Kid Yourself

I saw some people mention the possibility that there are simply more open positions available than people to fill them. Don’t kid yourself, the US of A is at about 10% unemployment right now. There is not a glut of open jobs.

What we have is a dearth of people like Vinny that can take someone like me and make them into an expert.

You want more experts? Hire better managers. Hire people that can inspire others. Promote people that can lead and innovate as opposed to just executing orders. You’ll be surprised at the results.

Tomorrow at noon (ET) will be the second meeting for the Hartford SQL Server User Group (HSSUG). That’s right, we meet during the day because we prefer to be home at night instead of, well, at a user group meeting. Weird, I know. And I bet some people would rather we met at night because they can’t get the time off from work but we know we can’t please everyone. So we decided to please ourselves first and worry about others later.

Anyway, the pizza starts flowing at noon and the first speaker takes the stage at about 1PM. And who do we have lined up for that slot?

Installing SQL Server 2008 on a Windows Server 2008 R2 Failover Cluster

Abstract: As DBAs we get to work on a lot of the features of the SQL Server product, but it is not uncommon for us to spend our entire careers without having a chance to work closely with clustering.  There are a lot of DBAs who are otherwise perfectly qualified in all respects for attractive job openings that absolutely require clustering experience. Chuck Boyce will walk through the basics of installing SQL Server 2008 on a Windows 2008 R2 failover cluster and show you some ways that you can obtain this skill on your own using a tool called VMLite.

Bio: Chuck Boyce is a Database Administrator who has worked for several multi billion dollar companies on the US East Coast. He began working with database technologies in 1998, working with Sybase on Unix platforms. Chuck began working with SQL Server in 2000.   Chuck has been awarded the Microsoft MVP award three times – twice for SQL Server. He is the former Executive Chair of the Professional Association for SQL Server (PASS) Chapter Program. He founded the Philadelphia PASS chapter and has served as both Track Lead and Co-Chair of the Philly Code Camp. He participates regularly in the SQL Server community on twitter and can be found there most days at http://twitter.com/chuckboycejr

After Chuck finishes up we will have a brief intermission to let people stretch their legs and their minds. We will get started up again after we all sit down and listen to our sponsor talk for about ten minutes. Well, everyone has to sit down except for me, because I’ll be the one speaking. I’ll also be handing out extra tickets for people that (1) attend my talk and (2) are active participants. What are the tickets for? Great question.

The door prize for tomorrow is my one remaining MSDN Ultimate subscription. It is a one year subscription that has a potential value of just under $12k. That’s right, one lucky attendee tomorrow is going to win an MSDN subscription.

After my talk is done we will have the second session of the day:

PowerPivot Demonstration and Discussion

Abstract: PowerPivot is a data analysis add-in for Microsoft Excel.  In this session we will discuss and demo the new SQL Server 2008 R2 PowerPivot feature.

Bio: Jon Gore is a BI Technical specialist from Microsoft Corporation.

After Jon is done we will hold the drawing for the prize and yes, you will need to be there to win.

Interested in attending? Go to the website to register otherwise bring your own lunch because we won’t be ordering for you unless we know you are coming. And by “register” I mean “click to join HSSUG”, so we can send you an invite to the meeting. We aren’t fancy technical here folks, just two guys that like to meet twice a year in a secluded spot for a few hours at a time.

And I really couldn’t argue with that. See, I am one of those people as well. In fact, had I not met Pat Wright (blog | twitter) and Allen Kinsel (blog | twitter) on my very first day at my very first PASS Summit, I may have never met anyone. [Believe it or not, I am a shy person by nature. Stop laughing.]

So…what can we do to change things?

Well, here is an idea I had, and it goes back to my college days. When I showed up for my freshman year I was greeted by members of the Orientation Committee. These were dedicated student volunteers who helped me move into my room, show me around campus, and introduce me to a handful of other freshman. I was assigned a Big Sister, an upperclassman who was willing to answer some basic questions even before I arrived (I seem to recall having a phone conversation with her a few weeks before we even met).

So, why not do the same thing for new people that are attending the PASS Summit? Have a PASS Orientation Committee (OK, we need a better name, I know).

From what I can tell we need the following things:

• Names of people that are attending for the first time (we have this data)
• Names of people willing to help facilitate a group discussion (we put out a call for help)
• A venue for everyone to meet (Welcome Reception seems like a good time to meet, if we can find a quiet spot)

What more do we need than that? The group leader would spend time talking with everyone, get to know a little bit about why each person has chosen to attend, can offer to help them connect to someone else in the Community (like send everyone with replication problems to talk to Kendal Van Dyke), and most importantly we can get these people to build a connection with someone else which is going to result in a more positive Summit experience and make it more likely to have those persons return next year.

Let me know if you think this idea has merit or not. If you think it has legs, then I’ll work to get it in place before November.

This morning I was reading up on row versioning (what? like you weren’t doing the same?) over at http://msdn.microsoft.com/en-us/library/ms345124.aspx and I came across this statement:

“The database administrator for a system is responsible for evaluating the impact on the system and applications of either of the row versioning–based transaction isolation schemes. The developer is responsible for understanding how to exploit the new isolation-level behaviors to build better applications.”

Wait a minute there…the developer is responsible for something? Do they know this? Has anybody told them yet?

So, let’s back up for a second here. Go and read a recent article by Brian Moran titled “Is There a Shortage of SQL Server Experts”. The title suggests one thing, “experts”, but then shifts to talk about a shortage of DBAs, and I’m not sure he meant to use that label but maybe he did.

Well, what about the developers? Shouldn’t they be experts as well?

My experience has been that the DBA is expected to be the expert in the shop, and that they need to play the Superhero role and come in at the 11th hour to explain everything in depth in terms that everyone (even managers) can understand. That ain’t easy. Worse still is when a developer spends eight months building something that doesn’t scale beyond the 100 rows they “tested” against and when you tell them they need to rethink their design you are met with something like “stop blaming my code, it isn’t the code!” (dude, yes it is, now sit down and shut up, I’m the expert here.)

So, who does what in your shop? Do your developers know that they should have an understanding of what row versioning is, and how it can be applied to build better applications? Or are you expected to know such concepts and to teach it to developers as well?

There are always a lot of new things to learn with SQL Server, but who in your shop is taking the time to stay on top of everything?

In short, the end result is that I had about 28 or so people with all their stuff ready to go. I had to choose four of those to be alternates. That decision was mine and mine alone (more on that later).

About two days after I had made the final schedule I was notified that I neglected to include a subset of Summit speakers. See, the Microsoft speakers are chosen after the Community speakers. That means I never invited them to take part in the upcoming 24 Hours of PASS event. And since they are as much a part of the Summit as anyone else, it seemed fair to include them. But the schedule was already set, so how could I get them involved? It didn’t seem fair to push four Community speakers aside in order to make up for my oversight. Two wrongs don’t make a right, right?

As luck would have it, we had already decided to switch up the schedule for the event. Instead of 24 straight hours we split the sessions into two days of 12 hours each which meant we had room to insert extra sessions. We decided to insert sessions at the end of each day so as to not disrupt the current schedule in any way. This way we can give a true preview of the Summit, we just needed 28 hours to get it all done!

We will get the registration open soon enough and will notify everyone when you can sign up for these extra sessions.

Now…about that session selection process…I want to change things entirely for next time. I am kicking around some ideas, but whatever we do will involve the Community in some way. Right now I am thinking we should have the Community pick the sessions and speakers they most want to hear. Not sure how to make it all happen, but the idea of a “24 Hours of PASS: Community Choice” sounds real good. Maybe we could do it American Idol style and have rounds of voting until we get down to 24 finalists…

Lastly, this is my 500th blog post. That’s a whole lot of writing without really saying anything. Thanks for reading and thanks to everyone that ever felt motivated to leave a comment.

I thought that was the craziest thing I had heard and I walked away from that dealer thinking he had gone mad. After all, who would buy something with such restrictions? Why would anyone make such a purchase? Who did this guy think he was, the cable company or some other type of utility that gets to set rates without any warning from one bill to the next?

I didn’t buy the car and came home to do some reading up on Oracle, since my job requires me to be able to support multiple database platforms now. I decided to dive into the Oracle Enterprise Manager (OEM), the Tuning Pack, and something called the Diagnostics Pack. I was in the middle of reviewing the PDFs when something caught my eye:

Licensing
These features may be accessible through Oracle Enterprise Manager , Oracle SQL Developer and APIs provided with Oracle Database software. The use of these and other features described in the product licensing documentation requires licensing of the Oracle Diagnostic Pack regardless of the access mechanism.

OK, so the diagnostic pack requires me to pay more money on top of the Oracle licensing I have already paid? Larry Ellison didn’t get enough dollars the first time around, so now he gets to charge a few dollars more if I want to know what is going on under the hood?

Digging around to find some additional licensing information and I found this nugget:

Note:
The Tuning Pack for Oracle Database requires a Diagnostics Pack for Oracle Database license. Therefore, customers interested in using any of the Tuning Pack functionality must license both of these packs.
Any and all methods of accessing pack functionality — whether through the Enterprise Manager Console, Desktop Widgets, command-line APIs, or direct access to the underlying data — require the Pack license.

Wow. So if I bought a 3rd party tool that monitors my Oracle instances using a command-line API, I will get charged an extra licensing fee by Oracle, not to mention the money I spent on the monitoring tool itself? Unbelievable.

Can you imagine paying extra for the privilege to using SQL Trace? Neither can I. How about if you run a  query against a DMV? How much extra would you pay then?

I have no idea at what point Oracle became as big and as strong as a utility company, but it is clearly the case. And some may argue that the tiered pricing is perfect for customers that install Oracle only to run some application and they rely on the application vendor for support anyway (if ever), but I’m not listening to that argument. I think that I am most concerned at this point for the fact that Microsoft is looking to get into the utility game by offering us the Azure platform. They already have the complicated billing process nailed down perfectly.

It just makes no sense. If I buy the car, I should be able to pop the hood from time to time and not be forced to pay extra.

Unless, of course, you have something to hide.

While I was in Nashville for two events, but this post is a recap for SQL Saturday so I am going to keep my focus on that event for now.

Diane debutante backseat of Jacky’s car…

I have been employed by Confio since June 7th, but in many ways this was my first real event as their Padawan. I spent a good chunk of time putting together my talk (“What Are You Waiting For?”) and even had the chance to rehearse it for two local user groups prior to presenting it in Nashville. In other words, I did everything possible to hit a home run for this talk. I treated this like my job, and my reputation, were on the line. I revamped how I built my slide decks, tossing out ideas I had used in the past. I changed my speaking style to be more interactive than previously (mostly because I knew I needed to keep people awake at 3:30PM on a Saturday).

The result say I had some success, I think. I had about 30 people attend, and 21 of them took the time to fill out a short sessions evaluation. The scale was from 1 (very poor) to 5 (great) in terms of quality and there was also a section for expectations (did not meet | met | exceeded). The people were also asked to leave their name and additional comments. Out of 21 tickets I received two ‘met’ and five ‘exceeded’ for expectations (including one person who wrote exceeded ++). For quality I received two 4′s and 19 5′s, giving me an average of 4.9 for the talk. I also had nine people take the time to write something to me personally, here are some of the quotes:

Good, practical, immediately usable solution to query tuning.

Very informative and entertaining, thanks!

Not a DBA, but thought this was helpful to create ‘structure’. Will share with my DBA!

Tom is a solid presenter.

You kept me awake!

Not sure, but since the person I kept awake gave me a 5 I am guessing that is a good thing.

And, if I go by the evaluations, I would like to think I had a good first showing as a representative of Confio Software. I’d also like to think I helped people become a little better at being a database professional.

Ain’t no rest for the wicked…

I was involved in other parts of SQL Saturday as well. I was asked to do an introductory DBA session at 8:30AM in the cafeteria. About 25 people showed up to listen to me talk about being a DBA Survivor. There was not a lot of code or examples in my talk, it is more of career advice. I often tell people how the book was written with the idea that I wish someone else would have written the book and handed it to me on the day I became a DBA. One person left the talk a bit disappointed because they were looking for more examples of (or about) code, but still took the time to tell me that I was a great presenter. So, mixed emotions there, I guess.

I was also invited to participate in the keynote along with Kevin Kline. This was my first keynote ever and probably the best one since I got to insult Kevin’s mom not once, but twice. (Hey, don’t look at me that way, she’s been asking for it!) I never got much feedback on the keynote but a few people told me it went well enough. I didn’t have much time to prepare for that one so a lot of my comments were really off-the-cuff. I did my best to keep my parts quick and concise, not sure if I succeeded in getting my points across in a short amount of time or not.

[The last time I had to speak to a group of adults in a cafeteria I was telling the parents of my basketball players that they were not allowed to ever question their son's playing time until they had watched a full week of practice. That crowd reaction was a bit, well, different than this one.]

My last event for the day was sitting as part of the “Stump the Experts” panel. I think most people were stumped by my being included in the panel, but I was the only person present that knew you could create 999 non-clustered indexes on a table. No, you don’t want to know how I know that trivial bit of SQL knowledge.

Anything you can do, I can do better…

This is my third SQL Saturday event, and my second as a speaker. I’d like to take a moment to post some suggestions for people looking to organize future SQL Saturday events.

First, figure out a way to welcome people to your city and event that is unique. Meeting in a restaurant is fairly ordinary. For Nashville we met at Kevin Kline’s house and had a cookout. It was amazing for Kevin to invite us into his home and watch him eat barefoot on his living room floor (nothing says “upscale hillbilly” quite like that image, right?). I was accosted by Tamara Clark who ran up and said “Hi, I’m Tamera, as in camera.” It was like some odd family reunion as everyone gathered around inside of someone’s house to talk food and SQL.

And it was awesome. Every other SQL Saturday organizer should take note. I’m looking at you, Kansas City, and I am hoping you can put together a “rib crawl” on Friday for lunch/dinner. I’ll fly in early on Thursday to eat my weight in meat-like products and mingle with other SQL geeks. Other cities should find something else that is fairly unique about them and look to incorporate that into the event. Raleigh could do a pig roast or show us how to chew tobacco. Orlando could show us why it’s called the Magic Kingdom. Lots of things make your city or region unique, find a way to showcase whatever that may be and help create a shared experience for everyone.

Next, when a person get’s his/her packet, they should be given instructions on what the packet contains. Walk me through everything I need to know, or point to a piece of paper that has all the important information. Point out where the speakers room is, and where the restrooms are. Tell people where the food is located, and what time the first event starts. Assume that the person in front of you knows nothing and is lost. Don’t assume that we are smart enough to figure it all out on our own. We aren’t.

Lastly, if you are doing a raffle at the end, you should make it clear at the very start if a person is allowed to win multiple prizes. I would rather spread the wealth as much as possible. If someone wins an MSDN subscription I don’t think they should also win addition software. But that’s just me. If you want to allow someone to walk away with more than one prize, that’s fine, just set your raffle up in a way that goes along with what your rules are.

But the funny part about all of that is when they do get the truth, they don’t always believe it. Even when Jessup admits to ordering the Code Red everyone in the courtroom is stunned. And when people don’t like the truth they start to dig deeper (or go off the deep end) and suspect that there is another reason to explain everything. Conspiracy theories abound on lots of topics:

We never landed on the moon.

Must have been a second shooter, no way one guy could pull that off alone.

A missile hit the Pentagon, not a plane.

The PASS Board meets at Area 51 and doesn’t like people who are critical of them.

I’ve been evaluating my peers in one form or another since I was 18 years old. It is never an easy thing. When someone gets turned down for anything they will want answers. They will want the truth.

Details for the PASS Election process can be found at http://elections.sqlpass.org/, and about the only thing you won’t find there are the transcripts/recordings of the phone interviews. When I first ran for the board in 2007 no such portal existed. Much of what you see there is the result of a lot of hard work that has been done very recently in an effort to be as transparent as possible about the process. Funny thing is the more information we are able to provide, the more information people want.

But the trouble we have here isn’t with the details included at that portal. The trouble here is that a line was drawn. And people want to know why the line was drawn. They want answers. They want the truth.

I have not had an opportunity to speak with the two candidates that were not included on the slate this year. The announcements came out yesterday and myself and the rest of the board were in transit to Nashville for a scheduled meeting. When we arrived in Nashville we didn’t turn on our laptops and start blogging. Instead we sat down and started having a very healthy discussion on “just what the hell happened”. Our conclusion? Our process isn’t perfect, but it also isn’t all that bad. We all agree that we can make improvements, but we haven’t agreed on what steps to take yet.

When you serve on the Nomination Committee, if you put forward a person’s name as a candidate then you do so with the assumption that they will be elected. So in order to get your name on the slate you must convince a majority of the committee members that you are able to effectively serve as a Director. It’s that simple, really. There is no mandate that a line must be drawn, it is just a matter of reviewing the candidates and answering this question: “Is this the right person at this time?” The scores help us see where the candidates rank against each other and help spawn conversations about the candidates, but in the end the answer to that question has to be “yes” by a majority.

Fail to convince a majority, and you fail to have your name on the slate. But that’s not the answer people want to hear right now. People want to know about how one particular candidate did not advance. And they are asking us to provide proof or evidence for why that candidate was not advanced. And we have to some extent. Have our efforts so far been perfect? Apparently not. The original emails to those two candidates were not well received, for example. But those emails also show that we have tried. Personally I believe that we shouldn’t send an email, we should place a phone call to the person and have a conversation.

And that’s where we are right now. I feel it would be *very* unprofessional for me to air any dirty laundry without talking to the candidates personally. So far one of the candidates has contacted me and asked for some feedback. I hope to speak with him next week. I don’t know if the other one will contact me, but I suspect we will talk at some point in the future. After those conversations happen, and if the people are willing to have all the sordid details posted on a blog for everyone to see, then I’ll do exactly that.

But what I will not do is simply post something as a knee-jerk reaction to events or be dragged out into an argument on the internet regarding what I believe to be a conversation that is best held in private first. I’m sorry if that doesn’t please all of you who want answers right now.

But after serving on the Board for the past two years I’d like to think that you want me on this wall.

We are human, and like all humans we are far from perfect. We are happy to learn from our actions and improve. And we are more than happy to listen to all recommendations for improvement.

As I progressed through my DBA career (and continue to progress, I might add) I came to realize that a LOT of issues are caused by problems that I usually equate with stubbing your big toe on something. In other words, a lot of issues can be cleared up with a few simple remedies. One of those remedies happens to be duplicate indexes. See, over time a database will get used and abused by a lot of people. Some of those people have an idea of what they are doing and some will not. And at some point everyone will think “hey, if I add some indexes then this thing will run faster.” So they go off and create some indexes, their particular query or statement runs faster for the time being, and they are happy.

Unfortunately they never see the likely side effects of the additional indexes. Possible side effects include, but are not limited to, the following:

• nausea
• loss of vision
• slower DUI (Delete, Update, Inserts) statements
• slower batch loads
• renal failure (from drinking heavily)
• nightly phone calls (problems sleeping)
• longer maintenance windows (increased appetite)
• larger databases and backups (weight gain)
• dizziness
• increased transaction log size (constipation)
• hiccups

OK, so maybe I exaggerated a bit there, I doubt you’ll get hiccups from adding more indexes. The main point here is that blindly adding indexes to your database is not always the right thing to do. And that is especially true if indexes already exist for you to use (or reuse). Years ago I had under my care a 300GB database that had roughly 45GB of space wasted by duplicate indexes alone. That’s a lot of additional overhead and cost for little to no benefit. And that’s when I came across the script I am sharing with you here. It helped us to identify duplicate indexes (BTW, wouldn’t it be nice if SQL Server didn’t allow for duplicate indexes to even be created?)

I normally don’t share scripts on this blog but I may start posting more of them in the future. For now I’ll share the standard disclaimer that Buck Woody (blog | twitter) uses:

Script Disclaimer, for people who need to be told this sort of thing:

Never trust any script, including those that you find here, until you understand exactly what it does and how it will act on your systems. Always check the script on a test system or Virtual Machine, not a production system. Yes, there are always multiple ways to do things, and this script may not work in every situation, for everything. It’s just a script, people. All scripts on this site are performed by a professional stunt driver on a closed course. Your mileage may vary. Void where prohibited. Offer good for a limited time only. Keep out of reach of small children. Do not operate heavy machinery while using this script. If you experience blurry vision, indigestion or diarrhea during the operation of this script, see a physician immediately.

/* This script will generate 3 reports that give an overall or high level
view of the indexes in a particular database. The sections are as follows:
1.  Lists ALL indexes and constraints along with the key details of each
2.  Lists any tables with potential Redundant indexes
3.  Lists any tables with potential Reverse indexes
*/
--  Create a table variable to hold the core index info
DECLARE @AllIndexes TABLE (
 [Table ID] [int] NOT NULL,
 [Schema] [sysname] NOT NULL,
 [Table Name] [sysname] NOT NULL,
 [Index ID] [int] NULL,
 [Index Name] [nvarchar](128) NULL,
 [Index Type] [varchar](12) NOT NULL,
 [Constraint Type] [varchar](11) NOT NULL,
 [Object Type] [varchar](10) NOT NULL,
 [AllColName] [nvarchar](2078) NULL,
 [ColName1] [nvarchar](128) NULL,
 [ColName2] [nvarchar](128) NULL,
 [ColName3] [nvarchar](128) NULL,
 [ColName4] [nvarchar](128) NULL,
 [ColName5] [nvarchar](128) NULL,
 [ColName6] [nvarchar](128) NULL,
 [ColName7] [nvarchar](128) NULL,
 [ColName8] [nvarchar](128) NULL,
 [ColName9] [nvarchar](128) NULL,
 [ColName10] [nvarchar](128) NULL
)

--  Load up the table variable with the index information to be used in follow on queries
INSERT INTO @AllIndexes
 ([Table ID],[Schema],[Table Name],[Index ID],[Index Name],[Index Type],[Constraint Type],[Object Type]
 ,[AllColName],[ColName1],[ColName2],[ColName3],[ColName4],[ColName5],[ColName6],[ColName7],[ColName8],
 [ColName9],[ColName10])
SELECT o.[object_id] AS [Table ID] ,u.[name] AS [Schema],o.[name] AS [Table Name],
 i.[index_id] AS [Index ID]
 , CASE i.[name]
 WHEN o.[name] THEN '** Same as Table Name **'
 ELSE i.[name] END AS [Index Name],
 CASE i.[type]
 WHEN 1 THEN 'CLUSTERED'
 WHEN 0 THEN 'HEAP'
 WHEN 2 THEN 'NONCLUSTERED'
 WHEN 3 THEN 'XML'
 ELSE 'UNKNOWN' END AS [Index Type],
 CASE
 WHEN (i.[is_primary_key]) = 1 THEN 'PRIMARY KEY'
 WHEN (i.[is_unique]) = 1 THEN 'UNIQUE'
 ELSE '' END AS [Constraint Type],
 CASE
 WHEN (i.[is_unique_constraint]) = 1
 OR (i.[is_primary_key]) = 1
 THEN 'CONSTRAINT'
 WHEN i.[type] = 0 THEN 'HEAP'
 WHEN i.[type] = 3 THEN 'XML INDEX'
 ELSE 'INDEX' END AS [Object Type],
 (SELECT COALESCE(c1.[name],'') FROM [sys].[columns] AS c1 INNER JOIN [sys].[index_columns] AS ic1
 ON c1.[object_id] = ic1.[object_id] AND c1.[column_id] = ic1.[column_id] AND ic1.[key_ordinal] = 1
 WHERE ic1.[object_id] = i.[object_id] AND ic1.[index_id] = i.[index_id]) +
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id], 2) IS NULL THEN ''
 ELSE ', '+INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id],2) END +
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id], 3) IS NULL THEN ''
 ELSE ', '+INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],3) END +
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id], 4) IS NULL THEN ''
 ELSE ', '+INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],4) END +
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id], 5) IS NULL THEN ''
 ELSE ', '+INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],5) END  +
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id], 6) IS NULL THEN ''
 ELSE ', '+INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],6) END +
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id], 7) IS NULL THEN ''
 ELSE ', '+INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id], 7) END +
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],8) IS NULL THEN ''
 ELSE ', '+INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],8) END +
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id], 9) IS NULL THEN ''
 ELSE ', '+INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],9) END +
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id], 10) IS NULL THEN ''
 ELSE ', '+INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],10) END  AS [AllColName],
 (SELECT COALESCE(c1.[name],'') FROM [sys].[columns] AS c1 INNER JOIN [sys].[index_columns] AS ic1
 ON c1.[object_id] = ic1.[object_id] AND c1.[column_id] = ic1.[column_id] AND ic1.[key_ordinal] = 1
 WHERE ic1.[object_id] = i.[object_id] AND ic1.[index_id] = i.[index_id])   AS [ColName1],
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id], 2) IS NULL THEN ''
 ELSE INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],2) END AS [ColName2],
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id], 3) IS NULL THEN ''
 ELSE INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],3) END AS [ColName3],
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id], 4) IS NULL THEN ''
 ELSE INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],4) END AS [ColName4],
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id], 5) IS NULL THEN ''
 ELSE INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],5) END AS [ColName5],
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id], 6) IS NULL THEN ''
 ELSE INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],6) END AS [ColName6],
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id], 7) IS NULL THEN ''
 ELSE INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],7) END AS [ColName7],
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id],8) IS NULL THEN ''
 ELSE INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],8) END AS [ColName8],
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id], 9) IS NULL THEN ''
 ELSE INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],9) END AS [ColName9],
 CASE
 WHEN INDEX_COL('[' + u.[name] + '].['+ o.[name] + ']', i.[index_id], 10) IS NULL THEN ''
 ELSE INDEX_COL('[' + u.[name] + '].[' + o.[name] + ']', i.[index_id],10) END AS [ColName10]
FROM [sys].[objects] AS o WITH (NOLOCK)
 LEFT OUTER JOIN [sys].[indexes] AS i WITH (NOLOCK)
 ON o.[object_id] = i.[object_id]
 JOIN [sys].[schemas] AS u WITH (NOLOCK)
 ON o.[schema_id] = u.[schema_id]
WHERE o.[type] = 'U' --AND i.[index_id] < 255
 AND o.[name] NOT IN ('dtproperties')
 AND i.[name] NOT LIKE '_WA_Sys_%'

-----------
SELECT 'Listing All Indexes' AS [Comments]

SELECT I.*
 FROM @AllIndexes AS I
 ORDER BY [Table Name]

-----------
SELECT 'Listing Possible Redundant Index keys' AS [Comments]

SELECT DISTINCT I.[Table Name], I.[Index Name] ,I.[Index Type],  I.[Constraint Type], I.[AllColName]
 FROM @AllIndexes AS I
 JOIN @AllIndexes AS I2
 ON I.[Table ID] = I2.[Table ID]
 AND I.[ColName1] = I2.[ColName1]
 AND I.[Index Name] <> I2.[Index Name]
 AND I.[Index Type] <> 'XML'
 ORDER BY I.[Table Name], I.[AllColName]

----------
SELECT 'Listing Possible Reverse Index keys' AS [Comments]

SELECT DISTINCT I.[Table Name], I.[Index Name], I.[Index Type],  I.[Constraint Type], I.[AllColName]
 FROM @AllIndexes AS I
 JOIN @AllIndexes AS I2
 ON I.[Table ID] = I2.[Table ID]
 AND I.[ColName1] = I2.[ColName2]
 AND I.[ColName2] = I2.[ColName1]
 AND I.[Index Name] <> I2.[Index Name]
 AND I.[Index Type] <> 'XML'

And there you go. When you come across a system that is having performance problems, use this script to help you locate possible duplicate indexes and see if removing them helps you to improve performance. Of course you need to perform some analysis on the indexes, their usage, the system, etc., before you just removing indexes without notice, but at the very least you are doing your part to help someone who just stubbed their big toe.

[Sorry that you had to find out this way Seacoast and SNESSUG. It's not easy finding out you were being used, but you were. And thanks for the bacon pizza, I can let myself out. Can we stay friends?]

Anyway, part of my talk has to do with how you define if something is a problem. Let’s take a quick example from my travels last week: You have a query that joins eleven tables and views. The code for this join is about as easy to read as anything written by Ayn Rand. The query takes 15 seconds to complete and while it is running it consumes memory and CPU resources. The question becomes: is this a problem?

Is it? Assuming that Paul Randal (blog | twitter) has convinced so many of you to answer everything with “it depends” allow me to interject right here and now and say “fine, what does it depend on?” And yes, there is only one right answer here. And rather than make you wait until Saturday to find out, I’ll go ahead and give you a sample of the knowledge I intend to share with you during my talk.

It depends on your definition of a problem.

Sit down and stop yelling at me. I don’t care about any technical details about what is going on under the hood. I don’t care about how eleven table joins are probably a good indication that the database needs some design work. I don’t care about writing T-SQL in a way that Dr. Seuss himself would be proud. None of that matters. Why? Because nobody knows if any of that is a problem unless it has been defined to be a problem. It’s that simple, people. Really.

An eleven table join is only a problem if you have it written down somewhere that eleven table joins are a problem. A query that takes 15 seconds to execute is only a problem if the end user doesn’t want to wait 15 seconds for the results. An unreadable T-SQL statement is only a problem if someone comes along and documents how to properly write and format T-SQL statements.

Without definitions for such things then all you have is Chaos. And that means Don Adams is your DBA. Nobody wants that.

Imagine if everyone just sat around and complained about things simply because they felt that it was a problem. But they have no evidence to back up their feelings, so they just complain about time, resources, unreadable code, etc. Wait, did I just describe every code review you have ever taken part in? Are you having flashbacks? Next time be prepared for your meetings. Find out if the customer thinks 15 seconds is too long, for example. Start with the user experience because that is always going to play a big part in how problems are defined. In fact, in my talk I list three common sources for definitions:

Performance benchmarks – Quest released a great poster to help you understand many of these benchmarks and you can get it at http://www.quest.com/backstage/images/promotions/SQLServer-Perfmonance-Poster.pdf. You’re welcome.

User experience – This is when the user doesn’t want to wait 15 seconds for the results and they tell you as much. Write down what they tell you is acceptable and not, and use that as how to define if something is a problem or not.

Business requirements – For when those rare moments when business requirements change, you have the opportunity to document them and refer to them as how to define if something is a problem or not.

This means that you already have three sources for how problems are defined in your shop. And if no one is complaining about that query taking 15 seconds then you may want to just close your eyes and believe that a problem doesn’t exist. After all, how long will it take you to redesign the database in order to write a more efficient join? And will that time (and expense) be worth the (at most) 15 second gain in run time?

Well…it depends…right?