Adobe recently announced general availability of a new product, Adobe AudienceResearch. This announcement builds upon Adobe’s Audience Certification Program to provide content publishers with an independent certification of key SiteCatalyst metrics. This enables publishers to accurately represent their audiences in the ad selling process for ads whether those ads appear on sites, in digital magazine editions, or in mobile applications.

Before today, audience measurement services relied almost exclusively on panel-based measurement systems that use the behavior of a few people to estimate the behavior of the entire population. The result is that trying to guess how big your audience really is often feels like placing a bet at the horse track.

AudienceResearch was built to address these concerns and provides the following key benefits to Adobe Certified Publishers.

Accurate

The Sample-method: The estimates of audience size are frequently much smaller than the actual size you see when you measure the whole population. Samples can be useful when all you need to do is compare the relative size of one site to another, but are not as useful when you need to know the actual size to forecast inventory and sell ads.

The Adobe Way: The better way is to simply measure each and every interaction from every individual on the site. Then you filter out the traffic you know is invalid based on known rules, and what’s left is your real, addressable audience.

Comprehensive

The Sample-method: Accuracy questions aside, if your audience is below a certain size threshold, no estimate is available at all because the panel members simply don’t generate statistically significant levels of activity. In addition, sample-based methods are typically skewed towards US residents with less representation from other geographic areas. Lastly, behavior from, sample-based methods rely on installed software and struggle to pick up at work activity where the collection software is not installed.

The Adobe Way: With tried and tested solutions for measuring interactions on all kinds of sites and on every major platform, Adobe can collect census-based information no matter where your audience chooses to consume your content. And when you collect each and every interaction, there is no minimum threshold for audience size.

Adobe can tell you how many people are in your audience, no matter how many there are, no matter where they live, or no matter where they are.

Consistent

The Sample-method: The actions of the few are used to predict the behavior of the many, and the actions of the few are quite varied. This means that an estimate of audience size has a confidence level that varies from property to property.

The Adobe Way: By having each Adobe Certified Publisher comply with some basic implementation standards, every interaction from every audience member is collected, processed, stored, and reported the same way. Every time.

Transparent

The Sample-method: As we’ve talked to publishers, we frequently hear concerns about the “black-box” nature of how panel data is turned into audience estimates. It’s hard to have confidence in a number without an understanding of how that number was derived.

The Adobe Way: Earlier this year, Adobe achieved accreditation with the Media Rating Council (MRC). This means that every year an independent auditing firm comes in to examine everything involved in the collection, processing, storage, and reporting of Certified Audience Data to ensure validity, reliability, and effectiveness. Audits are conducted in accordance with standards from the MRC and the Interactive Advertising Bureau (IAB).

Adobe’s verified compliance with industry standards means that everyone can get comfortable with the processes and procedures that Adobe uses to produce the Certified Metrics. There is no guesswork about where the numbers came from, because a new report in SiteCatalyst will account for every audience interaction page by page and tell you which ones were valid and which ones were not. If they are not valid, we’ll tell you why.

At Adobe, we also understand that your data belongs to you, and for that reason, use of AudienceResearch is entirely opt-in for customers.

Real-time

The Sample-method: In order to reach statistical significance, panel-based methods combine data for an entire month before releasing their data. This results in audience data not being available until half-way into the next month.

The Adobe Way: Adobe has always operated in real-time. Cleansing and validation of Adobe Certified Audiences is done in real time and is available within SiteCatalyst in moments.Metrics in AudienceResearch are updated once daily when the day’s processing is complete, so that you can get the data you need when you need it.

In the analytics realm you need to have a plan.  Whether kicking off an implementation, validating your web data against internal systems, or embarking on an analysis of some sort, you better know what you’re doing.

That being said, all things in business and in life should come in moderation, and web analytics is no different.

Design Stasis in (in)Action

Let me give you a real life example of design stasis.  Last year, I started working with a new client that was embarking on an effort to validate their web data against some internal systems to determine what degree of confidence could be instilled in their data, and improve it where they could (something I wholeheartedly agree with).

While we were popping the hood to tune the engine, they wanted to redo their page names in order to provide themselves (and their end users) with more relevant and contextual names for the content on their site.

The site was fairly shallow and consisted of less than 100 pages.  Those pages were spread across 2-3 subdomains that were considered different sites.  Each site had a few sections for trying, buying, learning, etc.  Finally, they had localized versions of the site in 7-8 different languages, and wanted the language reflected in the page names as well.  I took all this and laid out for them a fairly simple structure of country:site:section:page that was extensible through all the different countries and sites.  

Up to this point, I had spent just a few hours gathering requirements and validating the strategy against different scenarios they could come up with to see how well it handled exceptions.  It held up and I explained as long as they followed this structure they would be just fine.

Then it all went sideways.  For the next 2 months I fielded questions like: what should exact name for page Such-And-Such be?  Should we use 2 or three letter country codes?  Should we capitalize the Country Codes?  Should we use upper-case? Lower-case? Camel-case?  Our sites have short names, should we use the short one or the long one?  Should we use spaces or hyphens between words in the Page Name?

Diagnosis

Design Stasis is not an execution problem.  It’s indecisiveness - fear of making a decision because it might be wrong.

Design Stasis is the older sibling of Analysis Paralysis (it comes first after all) and this client had a classic case.  They got hung up tiny details that were tangential to the purpose for the project in the first place.  And they didn’t get hung up while doing the work, they got hung up while thinking about the work.  Design Stasis claims another victim.

In my experience, there are two major components to the problem, and one or both may apply.  Understanding why we’re being indecisive is the first step to overcoming however, so for your consideration I offer my diagnosis of these potential causes.

We want to create the perfect solution.  We want to impress our team, our boss, and our HiPPOs, and if we create the perfect solution to the problem we can achieve this goal.  All ambiguity will be removed and everyone will be happy.

We need consensus.  This is related to the above, but often times, there are a lot of cooks in the web analytics kitchen.  Lots of people have ideas about how things should work and you’ve got to sort everything out.

We are just plain scared.  Problems are not always clearly defined and it can be hard to feel comfortable about a solution.  In addition, I’ve seen a lot of the people out there doing online analytics have an inner feeling of being in over their heads, of not quite “getting it”.  They may (or may not) have an understanding of the big picture and the overall goal, but when it comes to executing on a plan, they just plain don’t know what to do.

Treatment

To be clear, I’m not advocating that you shouldn’t do your due diligence or that you should leap before you look.  I’m merely telling you that if you’re waiting for a perfect solution before you do anything, then you’re never going to get anything done.  Do your due diligence, explore the alternatives available within the limitations you face, choose the best alternative and move on.  Action beats inaction every time.

If you’ve fallen victim to design stasis, then I offer this.  The first thing you have to do is admit you have a problem.

Come to grips with the fact that there is no perfect solution.  There is no magic bullet that will solve all problems and involve no trade-offs.  You just have to do the best you can and get moving.  In these cases, Perfect is an unreachable ideal, you have to figure out what is Good Enough and run with it. Seth Godin just wrote about a very similar topic and concluded “…no matter what, don’t do nothing.”

You must have a master chef.  If you want to be successful, there needs to be someone who can make the final decision.  Read Brent Dykes post about executive sponsors or search the Omniture Blogs for a whole bunch of information on this topic.

Fear is a little harder because fear is a psychological thing.  Everybody has different reactions to it and different ways of dealing with it, but deal with it you must.  If inexperience or lack of expertise is causing you some discomfort, then there are dozens of resources to help you get what you need.

Free Resources

1) Connect with the web analytics community online by using the #omniture and #measure hashtags.  Be a lurker or join Twitter (if you’re not there already) and be active participant.

2) Follow @omniturecare on Twitter.  He’s one smartest Omniture guys around.  If you pay attention to who he talks to a lot, you’ll find a whole bunch of smart Omniture customers from around the world.

3) I’m not even going to try to make a list of really smart people blogging about web analytics, but there are dozens of them.  Use Google and Twitter and go find them.

4) Subscribe to the Beyond Web Analytics podcast (<- iTunes link) and visit the site while you’re at it.

5) Join the Yahoo Web Analytics Group: A group of web analytics professionals.  You can search the archives or ask your own questions.

6) Attend your nearest Web Analytics Wednesday.  Get to know people and ask for help.

7) Sign up for the Analytics Exchange.  You can join as a student to get a real problem that a real business is having and have a mentor help you through the analysis.  Learn all you can in the process, this is a fantastic resource.

Paid Resources

1) Omniture University: Official classes offered by Omniture to turn you into a ninja.  Classes are usually product-centric and will teach you the ins and outs of Omniture tools .

2) Sign up for the Web Analytics Association’s online courses offered through the University of British Columbia.  These classes cover web analytics topics in general and are not Omniture specific.

However, before we dive into the measurement, you’ll need a basic (and I do mean basic) understanding of how Test&Target works.  If you’re already a Test&Target customer or you’re familiar with what it does and how it works, skip down to the What’s Important section.

Test&Target

For the official version of what Test&Target actually is/does, check out the Omniture site, but here’s my over-simplistic version.

Test&Target is an A/B testing, multivariate testing, and content targeting tool that allows you to try out different versions of content/creative/offers on your site.  You can also see which segments of your audience respond to the different versions, then target the top performing versions to the segments that responded.  You can manage each step along the way, or put in your different versions of content and put it on auto-pilot.

Each campaign that you run can have one or more experiences you want your audience to see.  These experiences can be on one page or span multiple pages on your site.  They can be targeted to a fixed percentage of your audience, a specific segment of your audience, the entire audience, or some combination of the above.

When your pages load, a request is made to Omniture.  Omniture determines which version of content (if any) to show to that visitor based on the way your campaign is set up.  Then the content (or offer in Test&Target lingo) is returned to the page.

What’s Important

Omniture is not a retail shop (shocking I know), our revenue is driven by subscription contracts.  For Test&Target, those contracts are for Mbox Requests.  Mboxes (marketing boxes) are the areas of the site where Test&Target will place the alternate content.  Obviously we want to measure the number of Mbox Requests, but those are transactional and while the data itself is interesting, there’s nothing inherently fascinating about the measurement method.

Active Campaigns, however, is state-based and much more interesting in terms of how we measure it.  An Active Campaign is a campaign that is live and running right now on a Customer’s website. 

More Active Campaigns means tests are running, offers are being served, visitors are getting relevant content, and customers are seeing value from the product (all good things).  Fewer Active Campaigns means the opposite.  If a customer is not running many campaigns, that indicates they’re not using the product.  If they’re not using the product, it’s a safe bet that they’re not getting the promised value, which is something we like to avoid.

If a customer was running 22 campaigns yesterday and launched 2 more this morning, then our measurement will reflect the additional 2 campaigns and a growth rate of 9.1%.  Similarly, if 3 campaigns were turned off or completed, we would see the attrition rate (or negative growth rate if you prefer) of 13.6%.

Measurement Methodology

State-based metrics are inherently different than transactional metrics because we can’t just add them up to look at them.  We’re looking at these as of a point in time, and then trending them over time to see growth and attrition.  So we take a measurement every day and record it.  We probably won’t look at it every day, but daily granularity lets us aggregate to weeks, months, quarters, and years as needed.  If we only had weekly data we couldn’t aggregate properly.  Daily granularity also preserves our ability to drill into the data when we see abnormalities at the weekly or monthly level.

So where do we actually measure this?  For this kind of thing, we rely on plain old databases and timed processes (cron jobs for the technical folks in the audience).  At the end of every day, when all the day’s data has rolled in, there is a computer process to run a few queries on the back-end databases to collect the data we want.  There are a number of data points we collect and one of them is Active Campaigns per Customer.

Once the data is in our hands, there are a few options at our disposal get the data into SiteCatalyst.  You can use the standard XML-based Data Insertion API, the new PHP measurement class, or a Data Source of some kind.  Implementation will obviously vary depending on which route you go.  I went with the PHP measurement class in a timestamped report suite, picked a conversion variable (eVar) to hold Customer Name, and picked a conversion event to hold Active Campaigns.  We must also send in a Day Counter event (1 event per day) and I’ll get to that in more detail in a moment.

We then turned the Active Campaign event into a Numeric event instead of a Counter event (through the Admin Console).  Numeric events can count up by arbitrary numbers that you pass in, rather than just counting by one.  That means I spend only one server call per customer even if they’re running 25 campaigns.  These processes are all run automatically in the wee hours of the morning and all the data is processed and ready to be looked at by the time I roll out of bed.  

Reporting

There are a few things to keep in mind when you report on this kind of data, since it does differ from standard transactional data.  First and foremost, you must always keep granularity in mind.

For an example, above, is the number of Active Campaigns that one of the Omniture marketing teams is running for the month of October (to see the results of some Omniture campaigns, you can play this game).  Keep in mind the state-nature of the report, and you can see that they’ve got around 100 campaigns running each day with some growth in the latter end of the month.  If I look at this on a weekly basis then it would tell me that there are around 700, which is obviously not correct (in addition to the misleading partial week).

Monthly suffers the same problem, but to the tune of 30k Active Campaigns.  In fact, any time period greater than a day dramatically inflates the numbers.  If this is all we had, our analysis potential would be rather limited, but not to worry.  To the rescue comes the Day Counter event that I mentioned a moment ago.  By creating a calculated metric that divides Active Campaigns by Total Day Counters (700 / 7 or 31,000 / 31), we get an Avg Active Campaigns metric that can be freely applied to any time frame.

Looking at the same report with my Avg metric yields the following:

Much better.  Much easier to see the overall trend here with the growth displayed nicely (and accurately).  The number of Active Campaigns this team is running is increasing over time.  The fact that they are running more and more campaign indicates to me that they are realizing the value of the product and are trying different tests to increase their ROI.  My hope is that as they complete one test they get some interesting results which drives one or more additional tests to increase learnings about their audience or to target particular content to a segment of their audience.

If you have any questions about how state-based measurement could apply to your business or need some help interpreting the results, feel free to email: ben / dot / robison / at / adobe / dot / com.  You can also find me on Twitter (http://twitter.com/ben_rob).

Transactions

This whole process is state-less by definition.  This means that the protocol doesn’t have any concept of memory to remember something from one request to the next.  To compensate, both the client and the server have come up with a way to keep track of state between requests (cookies and sessions respectively).

The upshot of all of this is that the web is built upon a series of transactions of data, each one consisting of a request and response.  A user action in a browser, such as clicking a link, triggers a request to a server (please give me the contents of this page) and the server processes the request and sends a response (here are the contents of that page you asked for).

This is the way that web analytics data collection works as well.  Usually the sending of the request is handled by JavaScript which is triggered by the loading of a new page.  The request contains all the information about the page and visitor that you want to record.

The response is an image file (hence the name “image request”).  The only reason that anything comes back at all is because there must be a response.  That’s the way the web works.  No one really cares about the image itself, so we make it a 1×1 pixel transparent image that’s easy to hide away somewhere where no one will see it.

State

Abraham Maslow is perhaps best known for defining the hierarchy of needs.  He also coined a now-famous phrase that you’ve all heard before.  “If the only tool you have is a hammer, you tend to see every problem as a nail.”

State introduces a new aspect of web measurement.  Standard web measurement tells us how many things happened within a certain time frame. By contrast, state measurement tells us how many of something there are as of a particular point in time.  The funny part is that we, as humans, actually deal with states all the time without ever thinking about it, but it’s almost entirely foreign within the realm of web analytics.

The Difference

How much money did you make?  How much money do you have?  Two related, but very different questions.  The first is transaction-based asking how many dollars rolled in during a given period.  The second is state-based asking how many dollars are in your pocket?

You’re starting to hear the difference.  One is past-tense and one is now.  So what are the analytics equivalents?  Transaction-based would be something like how many visitors did we have last month?  Now let’s try the state-based version.  How many customers do we have?

Soul Searching

When was the last time you asked a question like that?  I’d wager most of you never have.  In 2.5 years as a member of Omniture Consulting, never once did a client ask me a question like this.  Why not?  Why don’t we ask questions like this?  Perhaps we’ve trained ourselves not to ask them because we don’t expect the answers to come from our analytics tools (but they can!).

I’m not debating the power and insight that rests in transactional data nor am I suggesting a replacement.  I’m stating that adding an understanding of status to your transactions can greatly enhance an understanding of your online presence.  Use transactional data for transactional questions and use status data for status questions.  With a little twist on standard implementation, SiteCatalyst can be used to help you keep track of your status.

Next time I’ll discuss in detail how we use status tracking within Test&Target to understand campaign usage,  but in the meantime, take a moment to think about how state applies to you and ask yourself a few questions you haven’t asked before. The how many new registrations did you get last month question can be enhanced by the how many registered users do you have question.  The percentage change in new user registrations question can be enhanced by the percentage change in the total size of the customer base question.  And also note that the status version of the question also accounts for attrition!  There are so many applications!

Photo Credits: A pocketful of change ca. 1970 by a.drian and Close Up of The Thinker by Brian Hillegas

Personalizing Your Data

All of the reports in SiteCatalyst correspond to a database table somewhere.  Some of these are populated out-of-the-box and some are left to the discretion of a company to decide what to use them for and how to collect the data.

Among these personalized variables are props, evars (also here), and events.  That provides a lot of room for customizing the data you collect and making sure that you’re meeting all the goals laid out in your measurement strategy.

When you organize your report suites into a hierarchy, you have to keep in mind that a single data point is likely going to end up in multiple locations.  A Video Start on the US site will also end up in the global site.  It sounds pretty straightforward, but there are some important effects that we’ll take advantage of.

1.  Some Of Your Data Is Only Important At The Top Level

There are some reports that are only important inside the upper levels of your hierarchy.  Inside the Omniture Suite for example, we capture Product in prop4/evar4.  Every time data is sent to the collection servers, prop4 and evar4 contain a value that simply identifies which product is being used: SiteCatalyst, Test&Target, Discover, etc.

Within an individual report suite, the value is limited, but at the global level, where all the data is, this is extremely valuable.  It provides me a single report that tells me how much usage each product is getting.  Interesting insights are available from watching this trend over time.

This concept will hold true regardless of whether you’ve organized your hierarchy by country, by product, by site section, or something else.  Some of your data is only important when you look from the top down.

2.  Some Of Your Data Is Only Important At The Bottom Level

We keep track of the usage of bid rules used for ad spend optimization, but SearchCenter is the only product that uses them.  An active campaign in Test&Target is its own event and isn’t repeated anywhere else throughout the suite.  We obviously want to keep track of these features, how people use them, and how effective they are.

We keep track of these kinds of things at the bottom level of the hierarchy, but we don’t need to expend the effort to measure them at the top level.  There is no added value in looking at this data at the top level, therefore, there is no need to track it at the top level. In fact there is no need to even turn these variables on at the top level.

Now here’s the neat trick.  Since we don’t roll these values up to the global level, it means that the lower levels can re-use the same variables to mean different things without corrupting any of your data.

3. Some Of Your Data Is Just Plain Important

This follows logically from points 1 and 2, but I thought I’d call it out just to be explicit.  Some of your data makes sense and provides value at the top, at the bottom, and everywhere in between.

Takeaways

When you determine that you need to measure a particular data point, you must decide where it has value, and at which level you need to keep track of it.

If a data point falls into category 1 (provides value at upper levels only), then you must ensure that every report suite underneath implements that data point consistently.  This means they must pass the same values (or kinds of values) as every other report suite, and they must remain constant over time.

If a data point falls into category 2 (provides value at lower levels only) then record it in a way that is consistent within that report suite, and do not even enable that variable at the upper levels.  This reduces risk of latency and also ensures that your users do not waste time trying to derive value from reports that have none.

Inside the Omniture Suite

As an (extremely simplistic) example, the diagram below shows a layout of our variables inside a report suite.  If it’s listed on a report suite, it’s being recorded there.  Orange text is unique to a given report suite and not recorded at a global level.  Faded text are the ones you would pay less attention to on a given level.

Category 1 - Inside the Omniture Suite, we have dedicated about half of our variables (props, evars, and events) that must be used consistently across all products.  There are global standards for implementing these variables so that the values mean the same thing and that they look the same when they come in.  Product Name is an example.

Not all 35 data points apply to every product, but they do apply to more than one.  If a particular data point does not apply to a product, we simply leave it alone.  Report Suite ID is one example.  We obviously want to record that for SiteCatalyst, Genesis, Discover, etc., but it just doesn’t apply to Test&Target or Insight. File Format is another example.  We want to know what formats are most popular when people download and schedule reports, but you can’t do that in Test&Target.  So Test&Target just doesn’t set that variable.

Category 2 - The variables that are not used in category 1 get used in category 2.  These are customized to the needs on an individual product and are not enabled at the upper level.  SearchCenter bid rules, Discover workspace segments, and Test&Target active campaigns are good examples.  Again, note that within this category, I can reuse the same variable to mean different things since they don’t roll up to the global level.

Category 3 - You can actually treat these as a subset of category 1 because the same rules apply.  They must be implemented consistently through space and time for all products.  Login Company is an example that provides value at the individual level and at the aggregate level.

It was Benjamin Franklin that said “An ounce of prevention is worth a pound of cure.”  A week or two ago I mentioned the measure-once-cut-twice philosophy that could be restated by a web analyst to say that “an ounce of planning is worth a pound of development.”  Or maybe it’s worth a pound of headache.  Maybe it’s worth more.

Data Organization is Important Too

You capture the data on your website or from your application, then it goes to Omniture for processing, but where exactly does it go?

All the data that Omniture receives and processes is stored in a Report Suite.  One of the bits of data that Omniture collects on every request is the report suite ID, which ultimately determines a data point’s final resting place after it is processed along with the rest of the click stream.  The size of the report suite determines the scope of all the data within in it.

Buckets and Buckets of Data

It may help to think of a report suite as a bucket that holds the data you’re interested in.  You might have a bucket for your US site, one for your French site, one for your German site etc.  The country manager responsible for the business in France is critically interested in every little detail of what’s happening on the French site (yes, I did the illustrations myself, how did you know?).

She’ll want to know how much revenue or how many leads are generated.  Which campaigns, keywords, and referring domains are generating those events.  Is the new microsite driving traffic to the multimedia?  Which of the homepage promotions are most popular?  Did they watch any videos or take the survey that we invited them to participate in?

She wants answers to each of these questions as it relates to the French site.  The US and Germany are peripheral because those sites fall outside the scope of her responsibility.  Data collected from these other countries is only an obstacle in her way.  We want to organize our data collection and processing so that she gets all of the data she needs and none of the data she doesn’t.  We need to organize our data in a way that France can be easily separated from the other countries.

Bigger Questions Need Bigger Buckets

In contrast to a country manager who wants to know every detail about an individual country, a CMO is interested in the big picture items across the whole globe and probably isn’t too concerned about an individual campaign running in Japan.  Trying to aggregate data from each individual country site is painful, time consuming, and introduces problems with data accuracy.  A CMO wants to look in a single location for data that spans the entire organization.  We need another bucket of data that has everything in it.

Architecture

When we talk about a Report Suite Architecture, we’re talking about the hierarchical layout of the report suites (or buckets) within a company.  It is the blueprint that tells Omniture how to put all your data together when it is received.  It determines how big each report suite is, where it fits in relation to the others, and which data goes into it.  You can organize a hierarchy around many dimensions, but the right one has to do with your organization and how your stakeholders think about their business.

Organization Techniques

In my experience, there are 3 primary lines along which an analytics program manager  will choose to organize report suites:

1. Geography
2. Site/Section
3. Product Line

If you think about how the chain of command runs in your organization, it’s likely that one of these three dimensions is a good fit already, and a logical choice.

You can use JavaScript to direct your data into 1 or many report suites by comma-separating report suite IDs in the s_account variable.  Another common technique is to let VISTA do the work by looking for particular values in the page URL or a country variable that you’re coding.

The Omniture Suite

Within the Omniture suite, we’ve laid out our report suites by Product Line.  Each individual product report suite rolls up into the Omniture Suite report suite (I’ve not pictured them all to save on space).

Product managers and engineers can get data about product and feature usage, product adoption, etc., the individual report suites, while the global level provides a holistic view of customers across the whole online marketing suite.  If you’ve seen the product wheel when you login into SiteCatalyst, that happens to be a pretty fair representation of how we’ve organized our report suites (raise your hand if you noticed that Insight was recently added to the wheel).

Architecting a hierarchy for your report suite helps you deliver the right information to the right people, without drowning them with the unnecessary pieces.  In my next post, we’ll take a deeper dive into the report suites themselves to discuss making the most of the individual data points.

As a general rule, if you can accomplish something with VISTA, you could achieve the same thing through server-side logic on your own back end or through JavaScript on the client side.  Depending on your particular business needs, VISTA can be a God-send, a requirement, a Band-Aid, or an unnecessary concern, but how do you know which?  I’ll start with some considerations and then share how I’m currently using VISTA in the Omniture Suite.  Considerations first:

1. Business Needs

There are two main kinds of VISTA rules, the standard rule and the database rule (referred to as VISTA and DB VISTA respectively).

The standard VISTA rule follows the general philosophy of “measure twice, cut once.”  With the help of your Account Manager or an Omniture Consultant, you’ll carefully scope the business need, define exactly what it is that you want the VISTA rule to do, and submit the request.

If your business question is dynamic in some way, meaning that it requires the ability for you to make regular updates to your VISTA rule, then you want to consider the database rule.  DB VISTA is just what it sounds like.  Omniture maintains a database and the VISTA rule will perform certain processing based on what’s in it.  In most cases, this database is custom to your needs, but in special cases, it can also be powered off of one of the Omniture-maintained databases (search engines, browsers, etc).

Why is this first in the list of considerations?  Odds are that given the time and development resources, you can probably duplicate a standard VISTA rule on your own.  DB VISTA rules tend toward more advanced business needs and the functionality may be more difficult to duplicate without having a negative impact in other areas (see # 6 below).  If your business needs require something in this territory, the DB VISTA solution might be the simpler route.

2. Timing

VISTA rules are built by Omniture’s Engineering Services team (now a division of Omniture Consulting).  Once scope has been agreed upon and the request has been made, VISTA rules have a 7-10 business day turnaround.  Depending on your organization’s development cycle, that might be ten times faster or slower than you can do it yourself. 

Some Omniture customers will even use a VISTA rule as a temporary solution until they can marshal the internal resources to get changes made to their own code.  If you’re in a rush, a VISTA rule might be just what you need.

3. Ease of Updates

No doubt you are already familiar with how easy it is to make updates to your own website, so let’s focus on the VISTA solutions.  VISTA rules are compiled, housed, and run on Omniture servers.  Engineering Services writes the code, posts the code, and you’ll never have to worry about any of the implementation details.  This has a few important ramifications.

If you’re using something like the Unified Sources VISTA rule, which depends on an Omniture-maintained lookup table, the updates will just happen for you and you won’t have to worry about re-implementing the Channel Manager plug-in when Microsoft launches a new search engine (you know who you are).

DB VISTA rules utilizing your own database are also fairly painless to update, but what about the standard VISTA rules?  This is why I advise you to measure twice and cut once.  If you need an update to a standard VISTA rule, then you’ll talk to your Account Manager or Omniture consultant to kick off the process and the 7-10 business day rule applies again.

4. Persistence/Cookies

VISTA rules run on every hit and are completely state-less meaning, that it doesn’t remember things from one page to another.  If the particular question you need an answer for involves remembering things across multiple pages or visits, then VISTA will not be able to completely meet your needs.

VISTA can still be a part of the solution if needed, but you may have to control the cookie/session logic and persistence on your own servers.  Then you can pass certain values to VISTA in a pre-defined way in order to trigger the desired processing.

5. Validation/Debugging

There are a number of standard VISTA rules that are quite popular for one reason or another.  For these rules, the implementation and validation process is pretty straightforward.  On the other hand, some are completely custom to a particular business need and from start to finish require custom scoping, development, testing, and validation.

The complexity increases if you have more than one VISTA rule affecting the same data points, and it can be challenging to trace these back to their source (since the code is not where you can see it – point #3).  If you’re seeing unexpected results in your dev suite while performing this validation, it’s likely that you’ll need help from ClientCare or your consultant to help you through this phase.

6. User Experience

Sometimes what you need to do isn’t particularly difficult or complex, but you’d like to avoid doing it in JavaScript because 1) it would add a lot of weight to your .js file, 2) it would be processor intensive, or 3) insert your own reason here.  Increasing load times and impacting the user experience for analytics sake is usually frowned on by most people I know, so if you have these concerns, look no further. 

VISTA processing happens asynchronously after the data is collected, so you don’t need to incur either of these costs at run-time.

VISTA in the Omniture Suite

So what do we use VISTA for in the Omniture Suite?  There are a few in place and a few more that I’ll be adding shortly.

• IP Exclusion: we move hits from internal IP addresses to a internal report suite (I’ll write another day about report suite architecture)
• Monitoring Agents: we move hits from Gomez, WASP, and other monitoring tools into a junk report suite
• Copying Props to Evars: it can easily be done in JavaScript, but it doing it with the VISTA engine helps keep the code cleaner and the length the request URLs under control
• Copying our Geosegmentation data into evars
• Time Parting, to keep track of time of day and day of week

Feel free to email me if you have questions about applying VISTA to your business needs.  If you think that a VISTA rule might be right for you, contact your Account Manager or your Omniture Consultant to get the ball rolling.

For those in the audience who’ve done this before, let us know what you’re using VISTA for.

Photo Credits: 1. Darrren Hester, 2. nicolasnova, 3. Jason Gulledge, 4. aresauburn, 5. Gerald Yuvallos, 6. Paul Veugen.

Now I get to take all that experience and internalize it, because I’ve just moved into a new position at Omniture.  For the purposes of this blog, I’ve got two main responsibilities.

Going Green

First, I’ve inherited - and now own - the Omniture implementation inside of the Omniture Suite, measuring how our customers are actually using our applications (very much like an Omniture customer would measure how their users are interacting with the website).  The current design was created during a time when Omniture’s products were very web-based.  It’s been extended a bit and we’ve got more mileage out of it than was ever intended, but it’s time to rethink a few things in an Omniture world that’s expanding beyond the web.  In effect, I’ve moved from the consulting role into the practioner/analyst role.  Let it not be said that Omniture won’t eat its own dog food.

Just like any implementation that many of you have undergone (or are perhaps working on now), this one can be complex.  There are multiple applications to measure (ten at last count), some are similar some are very different.  Some are web-based, some are not.  Some are managed from Omniture HQ, others are scattered around the world.  Beyond the applications, a few of them have add-ons.  Some of them have APIs.  Some of them allow you to schedule deliveries that occur at all times of the night without any interaction on the part of a visitor.  We need to measure all these things before we can truly understand how our products are being used.

Behavioral Metrics

Second, Omniture is the largest technology company focused on CMOs and marketers.  We’ve made a business out of getting the important information to decision makers.  I’ll be working to define behavioral metrics and integrate them with the flow of information.  By analyzing the way that Omniture customers are actually using our products, we’ll be able to improve those products.  Perhaps more importantly, we can help those customers to improve their usage of the products, helping them realize more value from the investments they’ve already made.

What This Blog Is

Some of the challenges we face are common to everyone while some are fairly unique to us.  On this blog, I’ll share with you some of the challenges we’re facing with implementation, reporting, analysis, and distribution.  I’ll discuss how we approach them and the solutions that we come up with in the hopes that my experiences can help you run your own analytics program.  There are many SiteCatalyst veterans out there and I hope that you will contribute your experiences, opinions, and expertise to the discussions as well.