QAGuild Weekly Article

"+title+"

Tue, 17 Aug 2010 05:41:27 -0700

Steve Miller
Pragmatic Software

Reducing Costs with Good Requirements and Code Reviews

Teams constrained by resources are always looking for a competitive advantage in reducing costs and improving software quality. This advantage can be achieved by implementing two best practices:

Writing good requirements

Providing peer code reviews

Studies have proven that the later a defect is discovered, the more it costs a team to fix. A study conducted by Dunn - "Software Defect Removal" - found that a defect that costs $1000 to fix when discovered during requirements analysis costs $25,000 to fix when left undiscovered until the quality assurance testing phase. Another study conducted by Jason Cohen, highlighted in his book - "Best Kept Secrets of Peer Code Review" showed that defects cost half as much to fix when found during code review vs. when discovered during quality assurance testing.

Defining Good Requirements

A good requirement contains these elements:

A narrative that explains what the requirement is going to accomplish

A prototype that depicts exactly what is expected (screen shots, report layouts, etc.)

A list of validation rules (e.g. allowable field sizes, allowed values, data validation, etc.)

A list of security rules (e.g. only administrators can access specific features)

A list of performance criteria (e.g. must be able to add a record in 3 seconds or less)

A list of audit rules (e.g. when adding or changing a record, store the history of the change)

Success criteria (e.g. objective criteria that can be evaluated to ensure this requirement meets the need)

Let's explore how taking this approach saves costs. Imagine we needed to develop a new screen to track our customer contacts and the screen entry needed to collect company name, contact name, email address and phone number. A poor requirement would simply describe the process (e.g. "create a data entry screen for collecting contacts") but would not include any of the other descriptive elements.

Without a prototype to guide him, the programmer might take liberties and design a screen that had a lot of ancillary fields (like birth date and marriage status) that may not be needed and may not include important fields (like email address and phone number). If no validation rules are specified, the programmers may not test a scenario where a new contact is added whose email address is the same as an existing contact -- causing a duplicate contact record. If no security rules are identified, the screen may allow anyone to delete a contact and this may cause your team to lose an important contact record. If no performance criteria are set, the team may not test a scenario where you have 100,000 contacts in your contact list and your new software may slow to a crawl. If no audit rules are in place and a person deletes a contact, you may never know who deleted an important contact and why.

With a good QA team, many of the issues above would shake out during the quality assurance phase. However, the cost of discovering these issues and re-working the code is dramatically higher when discovered at this time. If the requirements had been written using the detailed elements above, the programmer would not have taken liberties to add additional fields and leave out important ones, they would have stress-tested the solution with large number of contacts, they would have implemented validation logic to ensure all fields were validated, they would have added logic to prevent duplicate records and would have audited records when deleted.

Defining good requirements is equally crucial for Waterfall and Agile projects -- the difference is that Agile projects will break the requirements into smaller units of work that are described by User Stories (what a user wants to do in each case). In our example above, a Waterfall approach would define all of the elements of each requirement at the outset of the project. An Agile approach would break each of those elements into separate User Stories (a User Story for the prototype, a User Story for the Validation Rules, etc.) and would incrementally implement each item in an iterative fashion

It's also imperative to review requirements as a team (internal teams and client) once they are developed but before they are signed off. The review process lets extra eyes ensure that the elements above are addressed and that the requirement will meet the actual need.

Reducing Defects with Peer Code Reviews

As code is developed, it is important to review the code with your team. Below are a few items to look for in code reviews:

Ensure the code functionality meets the specifications in the requirement

Ensure each subroutine contains proper error trapping and handling

Inspect logic errors that can cause looping, memory leaks and performance issues

Inspect variable declarations to ensure that variables are cast properly to reduce boundary issues and invalid type casting

Review coding standard to ensure the code meets your internal standards, is well documented, and easy to maintain

The most efficient way to perform code reviews is to have team members inspect the code (called Peer Code Reviews) on a regular basis and openly discuss possible issues discovered during the code review. This activity can be done before checking the code in (called "pre commit") or after checking the code in (called "post commit"). The decision for pre or post commit is entirely up to your team and the way your team works best together.

By inspecting the code regularly, you can discover defects earlier in the development cycle which, like writing clear and detailed requirements, will reduce costs. Imagine a scenario where you reviewed the code and noticed that no logic had been added for preventing duplicate records from being added. By discovering the issue prior to testing, you eliminate costly QA time to discover and report the issue. And if QA doesn't find the issue, you will spend exponentially more fixing it once the software has shipped to customers.

From a technical perspective, imagine a scenario where you had some logic that reads data from a database using a record set, then executes a loop over and over again until all the data is processed. A common programming mistake is not checking for a "no data found" or "end of file" condition. A peer code review could quickly discover that the loop may be entered even if no data was loaded into the record set, which will only cause a failure under that condition. If this defect is not found until QA testing, your QA testers may see erratic behavior where it works sometimes (when data is found) but not other times (when no data is found). It may take your QA testers 2 to 3 days to discover the actual issue, so you have just added several days of effort to a problem that could easily have been discovered during a peer code review.

Reduce Project Costs with These Best Practices

Obviously, the sooner in your development cycle that you identify discrepancies in requirements and find defects, the less expensive the corrective measures will be. By writing good, detailed, and clear requirements at the outset of a project, and by having both team members and customers review them, you ensure that developers spend their time building the right thing the first time. By doing peer code reviews on your code before it goes to QA, you ensure that bugs are found immediately, while they are still easiest � and cheapest � to fix.

Can Tools Help?

Tools can help you with both of these issues. Tools allow you to keep all your requirements in a single place and allow your team to setup workflow to ensure that requirements are reviewed for best practices. Software Planner works equally well for Waterfall and Agile environments and allows teams to spot issues with requirements before it is too late and more costly to fix.

About the Author

Steve Miller is President and CEO of Pragmatic Software, responsible for formulating strategic goals for the company, and leading corporate initiatives. Pragmatic Software flagship product is Software Planner, an award winning application lifecycle management (ALM) tool that manages all phases of the software development life cycle.

"+title+"

Mon, 07 Dec 2009 02:01:59 -0800

Soon Hui
Esteem Innovation

As a Programmer, what were my mistakes?

You don't usually see people (including me!) talk about their mistakes openly. But I think it is good to think about the mistakes we made in the past, so that we don't commit the same errors in the future.

I've been a professional programmer for about 5 years now. Like anyone else, I made mistakes along my way. Usually I didn't recognize the wrong thing I did right away; I only knew about the mistakes after being exposed to the correct ways of doing things. Hopefully after reading this post you would draw something useful from it and won't make the mistakes (and pay the same price) as I did.

1. Not using a proper ORM

Data Access Layer code is always messy, tedious and boring. I remembered when I was first doing a simple internal bookkeeping application, I was horrified at the amount of code I had to write just to get the basic plumbing done. So I started to plug away the ADO.NET and manually coded up a home-brewed-with-a-very-specially-customized-to-specific-table-schema ORM to serve my purpose.

A few months later there were some changes in the business requirements, and that cascaded into changes in table schemas, which led to a modification of my ORM, which was so painful that I discarded it all together and opted for strongly typed dataset adapter

For a while this thing actually worked. But still, I wished that I used a proper ORM ( such as NHibernate) for the job. At least I don't have to worry about changing database vendor when my users grow in numbers.

2.Not learning generics soon enough.

I started my career as a programmer in .Net 1.1 days. The problem with .Net 1.1 is that it doesn't have generics support. As a result we couldn't have strongly typed list and could only be satisfied with the bland ArrayList. But using Arraylist, with casting and boxing all over your code is just painful to read and write. So we used CodeSmith to generate strongly typed collection list. But as the codebase grew those custom generated lists were becoming a monster on their own. Because I can modify the code easily, I often step in and change a method's behavior to suit my purpose, and that caused confusions and bugs later on.

I should have switched to .Net 2.0 and started to use generics as soon as it is available, instead of creating more and more custom collection lists that are simply unmaintainable.

3.Reinvent the wheel

New programmers always like to reinvent the wheel: "The current implementation wasn't good enough for me so I have to rewrite the whole thing from the scratch". I once thought about writing my own UI controls because Windows Forms UI controls were just too unsophisticated for my use.

As we all know there are a lot of excellent .Net UI control tools available; my GUI tools were not as good as those commercial ones, of course. I was just too naive then.

4.Too much documentation.

Code documentation is good, because it explains, in plain English, what your code is doing, right? Wrong

Documentation is often staled, outdated or downright wrong. I spent a lot of my time documenting my code ( XML documentation, remember?), only to find that I needed to update the documentation when I changed my code. Updating the code is a must, but updating the XML documentation isn't; it's a liability, it sucks time, and it serves no purpose. Eventually I ran out of patient changing the XML documentation and moved on to something else.

5.No automated build

Application deployment and packaging are comparatively easier than programming, and so I put a very low priority on that. Soon, I was receiving complaints from everyone the build wasn't working. "Prerequisites are missing, how to fix it?";"dlls are not updated, can you please send me a patch?";"Why the icons are running away?" and calls alike were reaching desk like avalanche.

At the end of the day I was worn out, not because of programming, but because of mind-numbing re-deployment and repackaging process. I could really "save" sometime from writing the automation scripts, but the time I wasted on fixing every error and on supporting other people was many times more than what I could "save". Your software should be built at one click, more than that is a waste.

6.Rely on visual inspection and debugging too much

It's so easy to come up a form and display your output. And Visual Studio is so powerful that one can easily step into the code and inspect the value on the fly. But debugger is harmful if you indulge into it too much. Imagine if your method will only get called after the application is up and running for 45 minutes, are you going to wait 45 minutes to reach that point and then only start debugging?

A better way is to break the application into sub-module that can be called independently. In this way you can just prepare the input that produces the faulty output and test it from there onwards.

7.No unit test

I used to think that my application is trivial, that it can be easily covered by manual testing. Unit testing is for something big, and sophisticated and not for me. The outcome of this was the application grew into a monster, with no separation of concern, hard-to-refactor and completely unmaintainable code base. There was a point when I was afraid to make even the slightest modification to my code because any changes may or may no result in breaking changes. There were a few times when a mysterious problem suddenly surfaced out of nowhere, and the root cause was a breaking change I introduced a few months ago. Working with this kind of legacy code is not just boring and straining, but mentally stressing as well.

But with unit tests, life improves dramatically. I wish I could have learn the art of unit testing, and practicing unit testing starting from day one. It's a shame that the school doesn't teach unit testing.

About the Author

Soon Hui is a .Net developer working in Esteem Innovation, a software company that creates desktop applications to help structural engineers in drafting, analysis, design and detailing

"+title+"

Mon, 30 Nov 2009 05:52:49 -0800

Pentest Limited
Pentest Limited

Testing Data

Does your organisation develop software in house? Are you responsible for testing applications? Are you a tester of applications? If so, where do you get your test data? You know, the data you put in the databases that allow you to test the applications, where does it come from?

If the answer is from the live databases then you could be in trouble. Yes, you personally.

Many organisations, both large and small, take cuts of live data to use for testing purposes. Indeed, this is often seen as a good way of generating the test data required to fully exercise the application code. But is it?

There�s no argument that it�s as close to �live� as possible because it is live, but do you do anything with the data before you place it in your development and test databases? Do you sanitise or obfuscate personal or sensitive fields? Do you restrict access to properly vetted and checked staff? If not you could be in breach of the law or regulatory conventions governing your organisation.

In the UK, personal information stored electronically comes under the Data Protection Act. Any such data collected can only be used for the purposes it was collected for. If �testing our application� is not included in the official purposes, then raw personal data taken from the live databases and used for testing is almost certainly done so in breach of the DPA.

In addition to the DPA, which covers personal data held on individuals, there are other good reasons for not taking live data, unaltered. These could be regulatory or compliance related, including FSA, PCI, or any other governing regulations.

But it�s not just about the rules and regulations. It�s just really bad practice. If all you�re doing is testing your application against known, good data, then you�re not properly exercising your code. You�re not performing thorough data and error handling tests. Part of application testing is ensuring the code handles the unexpected in a predictable and safe manner. If you don�t test with data that stretches your code, how do you know what your application will do should it come across such data?

So what should you do? How should you populate your test data?

This is where it gets interesting. These are good and valid questions, and there is no easy answer. The simple fact is that you are going to have to spend some time and effort on this problem. Test data needs to be created or generated and it needs to be good for testing. Taking a cut of live data and performing careful and thorough obfuscation is obviously one way of doing things. It needs to be done carefully though, as sensitive data may not be immediately apparent, for example, some free-form fields may contain sensitive data even after obfuscation. It�s also no good just populating the database with meaningless random data, as the testers may not be able to tell if they have the right results.

So it�s likely that the only way to achieve meaningless but realistic test data is to take the time (and money) to generate it, utilising relevant resources (DBAs, testers, developers, data owners) to ensure it covers the testing requirements. This test data could then be stored, added to, modified and reused throughout the lifecycle and lifetime of the application. You could even set up a function to centrally manage and distribute test data for all your applications and data users, ensuring high quality, useful and valid test data is available when needed.

There are a number of tools on the market which can aid with this process. A simple Google search for �database test data generator� pulls up dozens of suitable products out of about 1.7m hits, so there are resources available. Also you could visit www.grid-tools.com for Test Data generators

At the end of the day it comes down to a conscious decision by your organisation. You can either continue using live data for testing purposes in the hope that no security breach will occur, and that your breaking of the law or regulatory policies won�t be discovered, or you can stop cutting corners and take the time and effort to correct the situation.

Remember, ignorance of the law is not a defence under the law.

About the Author

Pentest Limited is an IT Security Company focused on providing independent Security Consultancy Services to major organisations across the UK, North America and Europe. The company maintains a vendor neutral position thereby providing truly independent advice to its clients whilst making sure that Pentest consultants keep abreast of all leading security products.Pentest have achieved ISO 9001 accreditation as part of their commitment to quality and standards.

"+title+"

Sun, 22 Nov 2009 22:02:30 -0800

Soon Hui
Esteem Innovation

Unit Testing, What a Pain!

Phil Haack told some of his pains in dealing with unit testing in the post Tell Me Your Unit Testing Pains. Here I would like to share some experience of my own.

I was doing desktop applications, CAD software like AutoCAD, so some of my experience may be different from those who do web applications. Unlike Phil Haack, I didn't mess around with ASP.NET, so I didn't have any issues in mocking the Framework. We needed to develop our own Framework on top of the .Net Framework. In retro respect I wished that there was a Framework, because at least an existing Framework would have been designed by many others who knew well about software development issues. Some of the unit testing pains, or should I say, a majority of the unit testing pains had to do with an improper design of our own framework.

I wasn't even thinking of unit testing when I first started my programming career. No one in my company practiced it, and even the most enlightened ones had had only some vague idea about it. We thought that unit testing was limited to methods or classes that were easy to test or had well-defined input/output. Static methods were obviously good candidates for unit testing. But Object Oriented Classes when all different types of instances interact together? No, not for unit testing.

And so I blithely developed my code, there were just pages and pages of these code. In the course of software development I cut corners, copy pasting similar logics, modifying method parameter calls without much thought, lumping different sets of data into a single class for convenient sake, premature optimization and practiced ( or committed?) other numerous software anti-patterns.

So in the end, at the maintenance stage, I looked at my code in awe. There simply was no way I could modify the code anymore without making subtle breaking changes. The code was highly coupled, lowly cohesive: exhibit A of How Not to Develop Software.

And I started to think of unit testing. But how to test spaghetti code that wasn't written with properly object oriented techniques and testability in mind?

My old code exhibited the following characteristics:

Free use of Singleton. In essence, you won't know what a class depended on unless you took a magnifying glass and took a good look of what was contained inside.

No separation of concern. Since we were not doing database, so we didn't use relational database such as SQL server etc. We used binary file to store the application data. Guess what, the logic of data access was tightly intertwined with the application logics, and the application logics were sprinkled in the midst of GUI code, right in the same class as the Form class and in the event delegate methods. Oh dear, try to test code that asked you to OK a dialog box out of nowhere? There was simply no way to get automated unit test.

Complicated dependencies between methods and classes. To properly test a method I needed to setup a lot of input objects that were difficult to setup. To make things worse, sometimes the whole purpose of setup input objects was to get only a single value or property from those objects. When you needed to write thousands lines of setup code just to test a single short method, you would lose all the interest in unit testing. There was simply no hope to test each method in isolation.

Don't know what to test! Yes, this was one of the hurdles I faced. As mentioned above, I was doing CAD software; the output could only be properly verified by looking at the screen and counting the objects, checking the colors etc. All the screen outputs were fuzzy; if they looked good, they were correct, if not, wrong. How to translate those requirements to precise logics and conditions?

So I concluded that the code was simply not testable and I threw away the existing code and wrote from scratch. The code is now in better form and there were some unit tests covering them.

Life had been better for me since then.

Excuses for Not Doing Unit Test

Although unit testing is now mainstream, but one can still avoid it if one is determined enough. Here are some of the excuses for not doing it that I have encountered over my professional life. Some are pretty easy to rebutted, some require a bit of thoughts to dispel, and some even genuine concerns.

See how many you can identify or rebutted!

We can use Debug.Assert to capture problem during runtime, and this is something unit test cannot capture. Unit Test is not powerful enough, so don't use.

Unit Test cannot capture all of the problems. For example, how you use unit test to test whether the pixel colors are rendered correctly on the screen? You can't right?

Very difficult to write test cases, too time consuming!

The test cases take a lot time to run! If we were to follow the whole TDD practice it would take us ages to finish our development.

Unit Test deals with a well-defined input/output, but our ins and outs are poorly defined. We are developing the next generation natural language search algorithm and there is no such thing as an "expected output", you can only say this output is better than next output. How to unit test?

Our inputs and outputs are too numerous, but unit test only works on small number of ins and outs. In order to setup a test case, we need to load tons and tons of information from database/text files and do some processing before getting to the logic class. Mocking those inputs are just plain not feasible. And checking the outputs are also not possible.

Unit test works best on individual components, but not the interaction of components

My code is jumbled up and it is impossible to separate into different components for proper unit testing

Your unit test thing is just the system test in disguised , since we are not testers and system testing properly belongs to the testers, so we don't need to do it

We have testers, so let them do all the testing, testing is none of our developer's business

Most of our problems cannot be captured by unit tests, it can only be captured in system testing or functional testing

We should have done this long time ago, but since we miss the bandwagon last time, so we can't do it now because we are already late into the development so it will take too much of our precious time. Sorry.

In a nutshell, unit test is not doable. You just don't understand my situation here; I can't explain it to you.

About the author

Soon Hui is a .Net developer working in Esteem Innovation, a software company that creates desktop applications to help structural engineers in drafting, analysis, design and detailing

"+title+"

Mon, 16 Nov 2009 00:42:55 -0800

Steve Miller
Pragmatic Software

Reduce Cost with Better Requirements & Reviews

Teams constrained by resources are always looking for a competitive advantage in reducing costs and improving software quality. This advantage can be achieved by implementing two best practices:

Writing good requirements

Providing peer code reviews

Studies have proven that the later a defect is discovered, the more it costs a team to fix. A study conducted by Dunn - "Software Defect Removal" - found that a defect that costs $1000 to fix when discovered during requirements analysis costs $25,000 to fix when left undiscovered until the quality assurance testing phase. Another study conducted by Jason Cohen, highlighted in his book - Best Kept Secrets of Peer Code Review" showed that defects cost half as much to fix when found during code review vs. when discovered during quality assurance testing.

Defining Good Requirements

A good requirement contains these elements

A narrative that explains what the requirement is going to accomplish

A prototype that depicts exactly what is expected (screen shots, report layouts, etc.)

A list of validation rules (e.g. allowable field sizes, allowed values, data validation, etc.)

A list of security rules (e.g. only administrators can access specific features)

A list of performance criteria (e.g. must be able to add a record in 3 seconds or less)

A list of audit rules (e.g. when adding or changing a record, store the history of the change)

Success criteria (e.g. objective criteria that can be evaluated to ensure this requirement meets the need)

Reducing Defects with Peer Code Reviews

As code is developed, it is important to review the code with your team. Below are a few items to look for in code reviews:.

Ensure the code functionality meets the specifications in the requirement

Ensure each subroutine contains proper error trapping and handling

Inspect logic errors that can cause looping, memory leaks and performance issues

Inspect variable declarations to ensure that variables are cast properly to reduce boundary issues and invalid type casting

Review coding standard to ensure the code meets your internal standards, is well documented, and easy to maintain

Reduce Project Costs with These Best Practices

About the author

Steve Miller is President and CEO of Pragmatic Software, responsible for formulating strategic goals for the company, and leading corporate initiatives. Pragmatic Software flagship product is Software Planner, an award winning application lifecycle management (ALM) tool that manages all phases of the software development life cycle.

"+title+"

Mon, 02 Nov 2009 00:39:32 -0800

Dr Mike Bartley
Test and Verification Solutions Ltd (TVS)

How to Build and Maintain a Successful Outsourced, Offshored Testing Partnership

Somebody recently asked me to review a course on �Building a Great Team� - where is the section on outsourcing? I asked. In my opinion, outsourcing should always be a consideration when trying to maximise the performance of a team. In this article I will consider this in the context of software testing. I will mainly consider software testing that is both offshored and outsourced1, although it should also be applicable to software development and to companies that just use outsourcing or offshoring. The article is based the successful application of outsourcing at ClearSpeed Technologies plc (which I�ve described elsewhere -see [1]) and in advising other companies on how to outsource their software development and testing.

Outsourcing = is the use of components or labor from external suppliers.
Offshoring = the relocation of some of a company's production, services or jobs overseas.

The objectives for offshoring are evolving. Initially companies looked for cost savings. Then companies looked externally for extra resource (for example during the Y2K crisis). Companies are now also looking for strategic benefits and better defined business impacts from their offshoring relationships. My aim in this article is to demonstrate how such benefits and impacts can be achieved with minimum risk.

To attain the strategic benefits you seek, you must first identify them. Let�s consider the benefits claimed for offshoring by managers of technology SMEs (Small/Medium Enterprises) in [2]:

To accelerate the maturation of new products: �The internal skills of rapidly developing products that meet market needs is augmented by offshored teams who have the patience, precision and perseverance to evolve young products into mature ones� (Jajiv Dholakia, Cenzic Inc.)
Agility and cost savings: �on average we are able to deliver projects at a much faster rate and save at least 30% cost to our clients� (Jing Liu, CEO, Enter Suite)
Better software through additional resources: �The additional talent and resource that we can carry allows us to develop better software than our competition can afford in the same period� (Scott Allan, Symbol technologies Inc.)
Concentrate on core competence: �Businesses should focus on clients and IP which satisfies their need. Everything else should be outsourced to those whose core competency is that business� (Bill Widmer, President and CEO, g8solutions)
Ability to undertake one-off projects: �we were entering a virtually untested market with a new product. The US team focused on the current roadmap while I was able to develop the new product at a rapid speed and at a low cost� (Gopan Madathil, Founder, TechCoire)

The strategic goals must also be agreed by a suitable forum. When I started on the offshoring route at ClearSpeed, I (as the Quality Manager) agreed the following prioritised objectives with the CFO and Engineering Manager.

To improve company focus.

To improve company resource flexibility and by doing so, reduce time to market.

Quality improvement.

Cost variablization2.

Cost reduction.

The above goals were shared with the offshore supplier. This gave us a common understanding which helped the day-to-day management of the projects. We also derived metrics to measure our progress against the goals. For example, we used Defect Detection Percentage (DDP) to measure the improvement in our testing (see [3] for an explanation of DDP). The time to market and the cost of testing relative to development were also tracked.

The next major decision regards on your road to offshoring is to decide what type or stage of testing to offshore. There are a number of considerations to take into account.

Amount of documentation: In order to test your software your offshore supplier will need to understand it. This will ultimately require good documentation.

Domain knowledge: How much domain knowledge will the tester need and how easy will it be to find or train testers with that knowledge?

Complexity of the environment: How complex is the test environment and how easy will it be for the offshore test organisation to re-create it?

Development process: Testing within the V model lends itself well to offshoring. Offshoring of agile development less so as it is based on nimble teams with good communication and rapid change.

Stability: How stable is the software that you are asking the supplier to test.

Level of interaction: To what extent will the offshored test team need to interact with the development team.

Level of independence: NASA uses an independent V&V team to improve the quality of their software (see [7]). An offshore test team allows you independence along three dimensions: managerial, financial and technical.

Taking a customer perspective: An offshore team can be used as a �friendly� customer, helping to identify all the problems that your real customers would otherwise find.

IP protection: This is always a major concern in outsourcing and the concern seems to deepen with offshoring.

Sign-off criteria and incoming acceptance: You will need to sign-off the work performed by your supplier through some sort of incoming Quality Assurance (yes � testing the tester!).

Now consider how the above affect your decision on what to outsource:

Unit testing:

This will probably require you to release source code which may lead to IP protection issues.

Unit code may not initially be stable and may not be well documented.

The offshore organisation can act as an independent V&V team.

The test environment for unit testing is not usually over-complex and the domain knowledge requirements tend to be lower.

Sign-off could be defined through structural coverage goals.

As an example, I successfully offshored the unit testing of a new library of memory management functions. The functions were developed using agile techniques in close collaboration with marketing and two lead customers and, once the specification was stable, we offshored the unit testing. We had an existing, trusted relationship with an offshore partner who first produced a test specification and then tests against that specification. Finally, they topped up their tests to ensure our structural coverage goals were met. Our incoming signoff consisted of signing off the test specification, reviewing a selection of actual tests against that specification and re-running the tests to check the coverage.

System testing:

In my experience by the time you reach system testing the software is more stable and the chances of some documentation are improved.

If you are following the V-model of development then the appropriate documentation may have been available for a while so that the offshore organisation could have already written test specifications reviewed by you.

The test environment and domain knowledge required can be complex.

The level of interaction should be reduced compared to earlier stages of testing.

The offshore team can perform the testing independently and can take a customer perspective in their testing.

IP protection issues can be avoided by releasing the binaries rather than source.

Testing the tester is harder as an objective measure of system test quality is more difficult - but it is not impossible. A system test specification and reviews of the test scripts for example. The test results should be also made available for audit.

Some of my most successful offshored testing has been in system testing at ClearSpeed. We had an automated test environment which gave very high unit and integration test coverage, and some system testing. We then offshored all of the system testing that was impossible or not cost-effective to automate. The offshore team was encouraged to act as a customer with no prior assumptions and not to use any work-a-rounds offered by �helpful� members of the development team!

There are numerous other types and phases of testing where the above considerations can be applied but space does not allow for here. For example: specialist testing (such as security testing) where the outsourcing arguments are very strong but expertise rather than price will be most important; compliance testing (where you are required to demonstrate conformance to a particular standard) where there is a strong argument for outsourcing to specialists in that standard; release or acceptance testing has similar arguments to system testing discussed above. And the list goes on�

You also need to identify your preferred outsourcing model. [5] describes three types of service deal related to the scope and complexity of the offshoring contract and your strategic objectives.

Efficiency: These deals tend to focus on cost efficiency, improved resource agility or greater organisational focus on core skills. The contract terms concentrate on service levels. So, for example, you may be outsourcing unit testing currently performed by developers to free them up for additional projects and the service level might be around test specification, structural coverage and time to completion.
Enhancement: These deals are more to do with process improvement. For example, you may outsource to a specialist test company to improve your security testing.
Transformation: These deals aim to improve your competitiveness through innovation in the relationship. So, for example your goal may be to reduce time to market or improve the quality of your software (through reduced DDP for example). Your outsource supplier would need to act in a consultancy role as well as a service supplier.
Obviously the above three imply an increasingly complex relationship with increasing levels of trust. [5] goes on to consider the appropriate commercial models for the deal and this is summarised in Table 1.

Table 1: Commercial models, service deals and risk

Commercial model	Description and discussion of risk;	Recipient Risk	Provider Risk
Cost-plus pricing	The recipient pays the providers costs plus a percentage. The provider is guaranteed cost recovery. The recipient is not guaranteed service levels.	Medium	Low
	Appropriate for efficiency deals.
Fee-for-service pricing	Price is based on the amount and/or quality of the work actually delivered. The provider�s costs are not covered unless service levels are met. The recipient costs are not entirely predictable and service levels are not guaranteed. Appropriate for efficiency and enhancement deals.	Medium	Medium
Fixed price	Defines a specific service level and a price for achieving it. This is higher risk for the provider. The recipient has lower risk as prices are capped. Appropriate for efficiency deals.	Low3	High
Shared-risk/reward pricing	This involves a flat rate with additional payments for achieving specified outcomes. Appropriate for enhancement and transformation deals.	High	High
Business outcome achievement	The provider only receives payment for achievement of specified business outcomes. The receiver has no guarantee that expected outcomes will be achieved. Appropriate for transformation deals.	Medium	High

You also need to consider your outsource location: on-shore, near-shore or offshore. This decision should be made in the context of your chosen service deal. I would also recommend considering multiple suppliers. In my experience this brings a number of advantages.

Using suppliers with different types of skills.

Using one supplier for an efficiency deal and another for an enhancement or transformation deal.

Allowing the suppliers to competitively tender for new work (rather than becoming increasingly dependent on a single supplier).

Now that you understand what you want to offshore and your preferred commercial model, the next consideration is your readiness for offshoring. A number of attempts at offshoring fail because an organisation has bad existing internal development practices: offshoring your chaos will just generate additional chaos! So, how do you measure your readiness? There are formal measurements such as CMMi (see [4]). Table 2 is taken from [8] and shows the national differences in CMMi take-up. It is also interesting to note from the table the differences in maturity from the countries more likely to be supplier to those more likely to be provider. In my experience providers tend to go for CMMi for a mixture of technical, management and marketing purposes.

Table 2: Number of Appraisals and Maturity Levels Reported to the SEI by Country (where total appraisals is above 20)

Country	Total App- raisals	Maturity Level Reported
Country	Total App- raisals	1	2	3	4	5
Argentina	47		31	10	2	3
Australia	29	1	8	4	2	4
Brazil	79		37	31	1	8
Canada	43		10	18	5	3
China	465	1	103	293	18	34
Egypt	27		12	11	2	2
France	112	4	67	34	1	2
Germany	51	7	27	7	1	1
India	323	1	11	127	22	151
Japan	220	16	64	88	13	15
Korea, Republic Of	107	1	31	48	11	7
Malaysia	42		15	24		3
Mexico	39	1	18	13	3	4
Spain	75	1	49	21	1	3
Taiwan	88		60	25		2
UK	71	3	36	24	1	2
US	1034	25	365	347	21	114

Most organisations will not want to gate their offshoring on such formal appraisals. In my experience, the minimum set of processes that your organisation will require to outsource software testing is:

Requirements management

Change tracking and management

Project planning, monitoring and control

Configuration management

Build, test and release automation

Validation and verification processes

You now need to select your partner. There is no magic in the selection process:

Identify potential suppliers.

Define your selection criteria.

Create a short-list.

Perform an in-depth appraisal of your short-list (including a visit if possible)

Contract negotiation.

Perform a pilot project with your selected supplier (or suppliers)

The major issues in the above list are identifying appropriate selection criteria and exchanging contracts. Below are some example selection criteria

People: Skills profile; Academic background of staff; Staff training statistics.

Retention: Staff attrition rates

Process: Process maturity; CMMi and other process maturity measures attained. You should consider how well matched the relative process maturity between your two organisations are. For example, do the outsource organisation processes require deliverables and maturity on your side that you cannot deliver on?

Quality: Quality processes employed; how well do they match your quality processes?

Project and risk management: What management processes do they follow? How well do they fit with your preferred management methodology?

Company profile: Financial stability; historic financial data strategic goals for the company.)

Technology: Domain knowledge/experience of the technologies employed by your company.

Cost: What are the typical pricing models? How are changes managed from a cost point-of-view? Recent cost change history.

If you are offshoring then your selection criteria will need to reflect circumstances in the providers� country. For example, if the destination is India then staff turnover or retention rate should be a major consideration.
Regarding contractual negotiations, I have found it helpful to have two documents: a Master Services Agreement (MSA) and a Service Level Agreement (SLA). The MSA covers the legal, financial and contractual issues (such as confidentiality, ownership of deliverables, assignment of rights, indemnification, warranties, liability, changes in personnel, subcontracting, termination, penalties for not meeting agreed service and delivery targets, etc.). The Service Level Agreement (SLA) should cover the service and support goals and is strongly affected by the type of service deal that you have (efficiency, enhancement or transformation). The MSA and SLA are complex legal documents which I will not attempt to cover in detail in this article.

You are finally ready to execute your offshored software testing project and this is where it starts to becomes difficult! If you have followed the process I have described above then your chances of your success are significantly improved. However, any offshored development is inherently risky. First there are the problems of managing a distributed team (communication issues, knowledge management, change management, etc). Then, according to Ralph Kliem in [7], in offshoring there are also �the unique challenges posed by geographical, cultural, and other differences�. There is not enough space in this article to go into details of how to manage offshored projects but I will highlight the main items to consider. Firstly, you need to distinguish management and governance. Management is about responsibility for specific decisions; Governance covers decision-making processes: the who, the how and the when. It is important to define the governance rules to allow your offshore team to know when they are empowered to make decisions and how to make good decisions.

Good governance involves at least the following

Established standard decision-making procedures.

Well-understood processes for handling exceptions.

Decisions made quickly at the correct level against well-defined criteria

Decisions get recorded.

And good governance is helped by the following

Having agreed strategic objectives.

Formal communication methods which record project status and decisions.

Controls & records to ensure governance procedures are followed.

Regular meeting of the stakeholder group.

Good management means the usual good software project management practices plus

Communication

Who can communicate with whom? And how often?

Formal vs. informal communication.

How does the communication get recorded?

What happens on a miscommunication?

Knowledge management

Getting up the initial learning curve and staying there through staffing changes

On-going knowledge management

What is the process for asking a question?

What is the process for recording an answer?

Audit trails

What trails do you need? Decision making, Execution history, Timesheets?

You need to save this data quickly and cheaply but what about access time?

How strict does your audit need to be? Are their any legal requirements?

What happens on a miscommunication?

Well-defined incoming QA procedures

In summary, there are many benefits to offshoring your software testing, especially if you aim to build a competence based rather than transactional outsourcing relationship. However, the risks are also high and in the above I have laid out a process to improve your chances of success.

Agree the strategic goals in an appropriate forum and share them with your provider.

Decide what type or stage of testing to offshore.

Identify your preferred outsourcing model and location.

Assess your readiness for outsourcing.

Select your partner or partners.

References

�How to Boost your Productivity through Outsourcing�, Mike Bartley, Software and Systems Quality Conference, London, Sept 2008

�Happy About� Outsourcing�, Mitchell Levy, January 2005

Identify your preferred outsourcing model and location.

�How to measure test effectiveness using DDP (Defect Detection Percentage)�, by Dorothy Graham et al, Grove Consultants (available from http://www.dorothygraham.co.uk/downloads/generalPdfs/DDP_Tutorial.pdf)

�Multisourcing: Moving Beyond Outsourcing to Achieve Growth and Agility�, Linda Cohen and Allie Young, Gartner, Inc.

�Managing the Risks Of Offshore IT Development Projects�, Ralph Kliem, Information Systems Management, 2004, Vol. 21 Issue 3

�The Role of Independent V&V in Upstream Development Processes�, Steve Easterbrook, NASA/WVU SW Research Lab (http://www.cs.toronto.edu/~sme/papers/1996/NASA-IVV-96-015.pdf )

�Process Maturity Profile�, SEI, 2008 (www.sei.cmu.edu/appraisal-program/profile/pdf/CMMI/2008MarCMMI.pdf)

About the author

Dr Mike Bartley gained a PhD in Logic at Bristol University and an MSc in Software Engineering and MBA with the Open University. More recently he has added ISEB Practitioner Testing Qualifications in both Test Management and Test Analysis. Mike has been involved in software and hardware development for 20 years, including outsourcing, and regularly writes articles and delivers conference papers. He recently established his own consultancy to help companies in product QA and outsourcing.

"+title+"

Mon, 26 Oct 2009 05:42:37 -0700

R. Rajan

CHANGE LEADERS

INTRODUCTION:

Successful change implementation should be one of the key objectives of all champions in any organization. It is not easy to reach the �Satisfied State of Successful Change Deployment� many times, as it is more of perception based and does not have very clear measure. Many changes start with a grand ribbon-cutting and disappear at the very early phase of the change management cycle. Gartner 2008 survey says; almost one in three project implementation fails. The reasons for a project failure could be many but the ultimate ownership is with the Top Management. The approach should be always �Top-Down�. This article will provide insight in to a common frame work for successful Change implementation. It is very important that more emphasis should be given to people for any change implementation as the benefit of the change must result in people using a much better systems that provides value to all the stake holders, with Customer being the center of the nucleus.

Why Change implementation fails?

Let us look at a very simple example of a change that we want to bring about in our daily life. The change is that we need to walk for 25 minutes daily for five days in a week and lose 150 calories of energy, to keep you fit. How would you go about �Creating the Sense of urgency�, which will make you to implement this change?

One will accept this change, fundamentally after getting answers to lot of questions. Why should we walk and strain our self? What will happen if I do not walk? Will the walking be instrumental in achieving all my goals? You can answer this question in multiple ways. A large collection of data showing percentage of people dies in heart attacks with varying age group might be required to accept that it�s worth taking efforts to walk. The need of reducing the fat by burning calories stored in the body must be triggered psychologically. You can have summary of data on people under obese category and difficulties they experience, to convince that this objective is very important.

You can ask a doctor to explain the need of walking daily. These are some of the ways of creating sense of urgency. You will have to prepare your system to take up this change for implementation. Remember that this is only a starting. You have not accomplished anything yet.

Few of the reasons for failure in implementing Changes, felt by many organizations are:

Failing to create a sense of urgency.

Allowing too much of complacency.

Failing to create a powerful guiding team.

Absence of a strong vision.

Not understanding the basic principle of communication.

Not able to create a short-term win.

Change Deployment Frame work:

With almost two decades of experience in change implementation across different industries and after conducting an extensive research on the published information available in internet, I was able to conclude that managers who are supposed to play �Change Leaders� role, do not follow a structured framework for change deployment. The term change deployment refers to any change undertaken to improve the existing state with the objective of delivering excellence. You can think of implementing a ERP system implementation, payroll system implementation, HR system implementation, VOIP communication system, Border Gateway protocol implementation, Lean six-sigma implementation, Marketing system implementation, Travel process enhancements, Testing process productivity improvement, Reuse management etc.,: You need to have a frame work which must help the organization to implement change.

I was inspired by the book �Leading Change� written by John P.Kotter for developing a comprehensive frame work for change implementation which can facilitate creating Change Leaders for contributing a very high value to the organizations they work with by achieving a very high success rate. The following are the pre-requisites for adapting this frame work.

Rule 1: Change is a Top- Down approach and Top Management needs to get involved.
Rule 2: Top management must create a very highly empowered team and believe that this team can win.
Rule 3: Managers need to understand that people are the most important asset of the organization and without involving them, change cannot be implemented.
Rule 4: Managers are responsible to create a Sense of urgency and sustain it until the change is disseminated in to the culture.
Rule 5: Managers should know that people will accept the change only when you show them why you
need it. To create an acceptance of the people, you need to follow a cycle �SEE-FEEL-CHANGE�.
Rule 6: As change leaders, you should be able to articulate a one minute vision of the Change.
Rule 7: The change should not be pursued further, if a short-term win cannot be created and all the stake holders accept it unanimously.

The successful change deployment framework is the one which will facilitate a very high level of people � process interaction. As an organization lot of time needs to be spent for preparing people for change and conduct marketing campaigns inside the organization to spread the need for change. We need to keep talking to people as the only thing that will insinuate in to their minds is the �Repetition�. Managers need to create a visual picture of the where do want to reach and what financial benefits will be achieved, after the change is implemented. To reach the end state of a change implementation, which is touching the heart of people so that the change becomes a daily routine, a lot of training and talking to people are required.

The 8 Stage Change Implementation Frame work

The frame work proposed in this article has eight stages as illustrated in the picture titled �Change Leaders Frame Work�.

Stage 1: Creating Sense of Urgency:

Managers are responsible for Pushing up the sense of urgency levels always to ensure that the change is implemented very effectively. The following are some of the ways by which the sense of urgency is pushed-up.

Create a crisis by explaining and detailing the financial loss.

Eliminate obvious examples of excess.

Set targets so high that cannot be met by conducting business-as-usual.

Evolve broader measures of business performance (no functional level performance goals measures).

Expose financial performance to more employees.

Insist people to talk regularly to unsatisfied customers, suppliers & disgruntled stake holders.

Bombard people with more information on feature opportunities and the current inability to pursue.

Sources of complacency

Stage 2: Create a Guiding Coalition:

It is the responsibility of Top Management to form a guiding coalition with appropriate positional power, empowerment to take decisions for implementing the change successfully.

Four key characteristics for effective guiding coalition.

Position Power
Expertise
Credibility
Leadership

Qualities to avoid

Egos & Snakes (people who can create enough mistrust to kill team work)

Building Coalition that can make change happen can be done through a very structured three step process.

Step 1.Find the right people:

With strong position power, broad expertise & high credibility.
With leadership & management skills.

Step 2. Create Trust:

Through carefully planned off-site events
With lots of talks and joint activities.

Step 3. Develop a common goal:

Sensible to the head
Appealing to the heart.

Stage 3: Developing a vision & Strategy:

This is an important stage wherein the Change Leader creates a visual picture of what would be the future, in financial terms after the implementation of the changes and document the vision in explicit terms which can be articulated within one minute by any team member. The visual picture is very important as it can reinforce quite strongly in to the minds of people and the buy-in becomes an easy task at a later point of time. The Vision thus articulated will meet the following characteristics.

Imaginable Desirable Feasible

Focused Flexible Communicable

The Change Leader then creates a strategy as shown in picture 3 �Role of a Leader and Management� (The Logic of how the vision could be achieved) and Budgets (plans converted in to financial projections and goals).

Role of a Leader and Management

Stage 4: Communicating the change vision:

Communicating the vision is an important stage as the change leaders will have to ensure that the vision reaches across a wider cross section of people, more importantly it creates an impact in the minds of people. We need to conduct campaigns within the organization and explain the importance of change with clear financial terms. Communication will be very successful only when it is repeated multiple times.

KEY ELEMENTS IN EFFECTIVE COMMUNICATION:

Simplicity: (do not use jargons)

Metaphor, analogy & Example: A verbal picture is worth than a thousand words.

Multiple forums.

Repetition: (Ideas sink in deeply only after it is heard many times).

Leadership by example

Give-and-take (Two way communication is always more powerful).

Stage 5: Empowering Employees for Broad based Action:

This is the phase where in the obstacles are dealt effectively that block action, especially disempowering bosses, lack of information, the wrong performance measurement, reward system and lack of self- confidence.

WHAT WORKS

Finding individuals with change experience who can bolster people�s self-confidence with

Recognition and reward systems that inspire promote optimism and build self-confidence.

Feed back that can help people make better vision-related decisions.

Re-tooling� disempowering managers by giving them new job that clearly show the need

WHAT DOES NOT WORK

Ignoring bosses who seriously disempowering their subordinates.

Trying to remove all the barriers at once.

Giving in to your own pessimism and fears.

Stage 6: Generating a Short-term win:

In this phase, the change leader will produce sufficient short-term wins, sufficiently fast, to energize the change helpers, enlighten the pessimists, defuse the cynics and build momentum for the effort.

The role of a short-term win:

Creating a short-term win is extremely important and any change effort without this will not sail through smoothly. The change leaders need to understand that the �short-term win� will facilitate buy-in from those who oppose the change.

The key roles of a short-term win are:

Reinforcement for the effort needed.

Opportunity to relax for few seconds & celebrate.

It is a test of vision against concrete conditions.

Will make blockers difficult.

Will retain the essential support of the bosses

Managers must be able to create a short term win which will be:

Visible.

Unambiguous.

Clearly related to the change effort.

The change that does not go through this face will be forced to be withdrawn.

Stage 7: Consolidate Gains and Produce More Change:

This phase will bring in more momentum to the change effort, using the short-term win gained in the previous phase. Since a visible short-term win was shown to many stake holders, now people who opposed for the change will also fall in your favor as you will get tremendous support from management. Every own will understand and know now that this change effort will bring in lot of benefits and visibility, thus will start providing their support whole-heartedly.

It is the time for you to take decision by removing all unwanted interfaces that will prevent you speeding up the change implementation process. Remember that you might be required to remove people who are not inclined to put their efforts to implement the change being pursued. You will add additional resources/budgets to ensure that the speed of implementation is stepped-up gradually.

As a change leader, you need to play a key role here by pushing-up the urgency levels in this phase so that the team does not become complacent.

Stage 8: Anchoring New Change in to the Culture:

This phase is a standardization phase and usually comes at last in any change implementation. The new process need to go and sink in to the minds of people so that it becomes a routine. To make this happen, the change leader needs to do a lot of talking to people, create a visual story about this change and benefits, conducting quizzes and workshops, designing a very strong and robust feedback mechanism for swift action and changing the people who are relentless.

Precautions:

Cultural factors come last, not first: Alterations in norms comes at the end of the transformation process.

Depends on results: New methods usually sink in to culture only after they work and superior to old method.

Requires a lot of talk: Without verbal instruction, people will be reluctant to new methods.

May involve turnover: Sometimes the only way to change culture is to change people.

Makes decision on succession is crucial: The old culture will reassert, if promotion processes are not changed to be compatible with the new practices.

Conclusion:

Organization need to mentor and develop more leaders to effective contribute to the bottom line and take the organization forward. Managers need to come forward without waiting for instructions and volunteer change initiatives, lead them successfully. The frame work for change deployment recommended in this article will work only if all the stages are meticulously followed, without skipping any stage.

About the author

R. Rajan M.S, MBA, MSC, CISA, SIX SIGMA BLACK BELT, LA ISO 9001, 27001 & QS 9000, LA & BCM. With 19 years of experience in Change Leadership Role from a wider cross section of industries such as Software Testing, IT Infra structure, IT Networking, Information Security, BPO, Portal and Manufacturing. Expertise includes Process Design, MBWA, BPR, Lean Six sigma, Process Automation, IT Network design, Information Security implementation, ISO 27001 Policy deployment, PCI Practices deployment, IS Audits, ISO 9001 Implementation and QS 9000 automotive standard implementation.

"+title+"

Sun, 18 Oct 2009 23:21:24 -0700

Soon Hui
Esteem Innovation

Why Developers should Do Testing

It suddenly occurs to my mind that asking developers to play tester's role once in a while is a good idea. Here are whys:

Enable the developers to gain system knowledge

Improve the developers writing skills

Improve software quality.

Make the developers better software engineer

Enable the developers to gain system knowledge

Typically on a large team, individual developers are each responsible for a component of the system. Testers, on the other hand, need to be at least rudimentary familiar with the whole system. Letting the developers test the system will get them to familiarize with the whole system, enabling them to see the big picture.

Improve the developers writing skills

Developers hate writing. Ask them to fill out form hurts them. Ask them to write report disgust them. Being a developer myself, I saw a lot of developers who have a hard time explaining his ideas in words.

But testers need to be reasonably good at writing, because the testers must be able to

Write down the steps to reproduce the problem

Explain what happened and what should happen.

Make sure that other people can follow their bug report

If the developers come up with a bug report that no one else can understand, they would be asked to clarify their meaning, and this actually trains them to be a better writer next time. Repeat this a few times you will find that their ability to write improves.

Instead of sending the developers to communication courses, ask them to fill in bug reports!

Improve software quality

The whole point of testing is to improve software quality. Testers do find bugs, but after finding bugs for so long their bug count will start to decrease because the software starts to get immune to their way of testing.

Getting the developers to do testing will provide a fresh approach to testing, and hence uncover more bugs. It is also a form of dogfooding. The more the developers use the software, the more they feel the pain of end users. In this way they can design better features, better UI next time. The software quality improves as a result.

Make the developers better software engineer!

Testing can develop a developer. A developer's main job is to construct, whereas a tester's main job is to destruct. Different mentality, which is why sometimes a good tester may not be a good developer, and vice versa. Developers have been concerning about constructing, constructing, constructing the code all the time. Sometimes this makes them oblivious to the edge cases and the holes in their program. After all, the developers are paid to make sure that their code is matched to the specification. But what about things that are not found in specs? What about the situations that no one has thought of? Good testers must be critical minded, or else they would miss a lot of edge cases. Won't it be good if the developers can be critical minded when constructing their programs instead of throwing their half-baked code over the wall for the testers to test?

Not only can that, testing job help developers to realize the value of unit testing. Get the developers to wear the tester's hat, and ask them to make sure that a particular brittle feature is always working, let them do the mundane job of testing a feature over and over again, until they realize that automated testing is the only way to go. Sounds like a good way to promote unit testing (although I haven't try it).

"+title+"

Mon, 12 Oct 2009 01:19:39 -0700

Waseem Ahmed
Standens Ltd

Learn �Proactive Approach� Which helps us in saving costs

In previous edition, we discussed about Lean Quality Manufacturing.

When I started writing this article, I thought to myself that I am not a novelist or a good story teller but all I knew was, it�s a good opportunity to convey you some of my thoughts through this article. Finally, I decided to write something that I have learnt in my career and always believe on it �Proactive Approach� which always helps us in saving costs. You can / may use some thoughts and ideas from this article and apply the theme to improve your processes and systems and save Standen�s money.

Transformation of Reactive thinking to Proactive thinking is as challenging as to bring revolution in a state / territory. I have a strong belief that deeper we get into thinking positively; more opportunities we will discover for improvements. In this issue I will show you some examples of reactive and proactive thinking (R- Reactive and P- Proactive)

R - PFMEA (Process Failure Mode Effect Analysis) is only focused as a document required for customer PPAP.
P - PFMEA is a risk assessment tool to improve manufacturing processes.
R � Lack of importance to voice of customer.
P - Every customer complaint is taken as an area for improvement. Focus is not only detection but also prevention of such problem recurrence in future. Decisions are now made using multiple brains from cross functional team
R - Doing things because the procedure asked for it
P - Review the procedures and align it with your current practices and eliminate non value added (NVA) tasks. (NVA activities are those tasks which are not adding any value to customer / paid by the customer)
R - Written procedures are just like lengthy story books and novels. Why, because, we want to document every single action that we perform in our procedures
P - Procedures should be user friendly and not to be written more than 1 or 2 pages otherwise, people will loose their interest in the subject. Flow chart is a good example of shrinking the matter of procedures
R - ISO / TS Auditors are taken as police officers and their focus is to find as many problems as they could.
P - Audits are not about fishing non-conformances but a method to check the compliance and effective implementation of Management Systems audits using process approach
R - Every thing is recorded on paper because it�s required by the auditors
P - Again, if it�s not adding any value, don�t just do it because it�s required by auditors. �Work with the system not for the system�
R - Every single activity of manufacturing process needs verification and approval by QA
P � Process capability studies should be done and their control charts to be monitored frequently. Once we have good process capability (Cpk >1.33), we can reduce verification / inspection activities and their frequencies. Until we achieve the desired situation QA has to play a vital proactive contribution in respect to finding issues, QA should work with production to resolve the issue not just recording it in a piece of paper and think the job is done.
R - Don�t worry about that customer because we don�t build lots for them
P - Customers have equal values. Their voice is important to their suppliers and most importantly, during the bad phase of economic crises these small customers become a mean of survival for us
R - This is not my job, go and talk to manufacturing
P - �Can Do Attitude� and positive team works resolves most of company�s issues
R � We have goals and objectives because it is required by ISO / TS standard.
P - Goals and objectives are the performance measurement of company�s progress. How do I know that my department is doing a good job? Answer: Through the performance measurements. These objectives are synchronized with company�s direction
R - Keep every single record for indefinite time period just because it�s a ISO / TS requirement
P - ISO / TS requirement is to identify the records you want to store and define the retention time for their disposal. Otherwise, if you keep it storing for indefinite period of time, then, very soon, you need more space for record storage
R - Found the Quality problem but still sending those parts to customer. Why, because customer will complain if they catch this issue. We will respond to such complaint that�s not a big deal.
P - If you find a problem in-house, don�t ship it to customer and fix the problem then and there. Also check the inventory to make sure it�s all good otherwise re-work on them
R - Do in-coming inspections for raw material and all components to verify if they are meeting our specs and requirements.
P - Trust your suppliers and develop strong relationships with them. Increase frequency of supplier assessments instead of doing full time incoming inspections. During production, operators But if we find any issues with supplier products we will charge them back with warranty claims. Let them be responsible for their products.
R - We need more inspectors for inspection activities
P - Inspectors / operators are human beings and they have potential to make mistakes. Rely on your process performance and make it robust. Apply every possible method to make it error proof for defects.
R � Long meetings indicate that we are busy
P - Make the best use of people�s time. Stay focused on the subject. Team facilitators to take control of such meetings and don�t let people go out of agenda. Time management is MUST to follow
R - Always complaining about less manpower.
P - LEAP (Lean Enterprise Assessment Program) is an effective tool to make the best use of current manpower and shop floor space. LEAP contributes to ROI and sales.
R - Based on experience, people jump to conclusion too fast for customer complaints
P - Using 8D �Problem Solving Activity� we eliminate the root causes, take corrective actions and actions to prevent its recurrence. Prevention always includes any possible mistake proofing activities

About the author

Waseem Ahmed has more than 12 years experience in Quality Engineering Arena. Waseem is presently working in Standens Ltd, a well known name in leaf spring manufacturers in North America as their QA Manager. His qualification includes ; Bachelors in Mechanical, ISO / TS Lead Auditor, ISO-14001 Lead Auditor, ASQ SSGB, ASQ CQA.

"+title+"

Thu, 01 Oct 2009 01:45:11 -0700

Waseem Ahmed
Standens Ltd

Layered Process Audit (LPA).

In this edition, we will discuss about Layered Process Audit (LPA).

Fundamental to improving process control, is verifying that critical process elements are compliant with requirements on an ongoing basis. During the past few years several automotive parts suppliers have implemented Layered Process Audits (LPA). LPAs have been mandated by Chrysler and strongly recommended by General Motors and Ford. Many OEMs and Tier 1 suppliers are considering directing their suppliers to implement Layered Process Audits. LPA is one of the most powerful strategies to take a good supplier and make them better; or to take a great supplier and keep their quality indices improving. Certainly, improvement in customer quality levels is important to the supplier; but for the supplier itself, this benefit is just the tip of the iceberg.

�Improvement in quality levels is important, but those benefits are just the tip of the iceberg.�

Layered Process Auditing is an ongoing chain of simple verification checks, which through observation, evaluation and conversations at the work stations, assure that key work steps are being performed properly.

LPAs for any given work station are performed by different layers of management and various staff on a set schedule. This ensures that each process is viewed by many sets of eyes and all levels of management. LPA helps eliminating human error and ensure that parts are produced right the first time. Since the checks are repeated daily, weekly, monthly and quarterly by all layers of management, it�s likely that process errors will be found sooner. LPA proactively minimizes process variation and the result will be evident in process, product and financial improvements, e.g., defective parts per million, control charts, productivity, overall equipment effectiveness, scrap and rework cost etc. I am confident that Standens will embrace LPAs and will find that the payback will be very significant.

LPA facilitates two-way communication between management and floor personnel. These interactions strengthen trust and demonstrate shared interest in work being done right. How can we meet the requirement of zero-defects? Simply with �zero� shipment of non-conforming products. How can we prevent shipping bad parts to customers? Need to develop a culture where every person is working towards �right the first time, every time.�

LPA is never being counted as a detection control but at the same time error proofing is not always shielded from variation and failure. These devices can be misaligned, damaged, mis -calibrated or even turned off. Also, human error can undo almost any system or safeguard.

Initially QA always voluntarily take ownership for LPA implementation and execution but delegates ownership to Manufacturing once it�s implemented across the shop-floor.

The focus in LPA is checking items related to known problems and cause factors linked to high risk problems. These are items that vary over time and lead to undesirable consequences. There might be some exceptions, but most LPA checks look at �inputs� to the process, that is: equipment settings, condition of tooling, craftsmanship and work sequence. Like other process audits, LPAs verify the details of how the process is performed.

LPA usually takes 10 to 15 minutes to complete an audit. Intent of LPA is to fix issues immediately.

�Who would you rather find an error � your supervisor or your customer?�

LPA show respect for operators by giving them feedback � feedback that they are complying or not complying. LPAs aren�t designed to catch workers making errors. The 15 minute window per shift allocated to LPA is way too small for that. What really changes employee behavior is when they do things right and are recognized for it. People do what gets measured; and employees respect what you inspect.

Improving processes will improve product Quality. Improving product Quality improves customer satisfaction. When customer starts building their confidence and good relationship with us, it will open new opportunities for new businesses.

Note: Next operation is line 1 for LPA implementation.

"+title+"

Sun, 20 Sep 2009 23:08:32 -0700

Pentest Limited

Database Security; why expensive tools may not be the whole answer

The world of Information Security is a fickle one, dominated by tools with fantastic capabilities or at least that what it says on the box. So it�s not surprising that when shifts occur within this industry, the tools and their associated claims are not far behind. The current shift is a move to the data, more specifically to the database and it is in this area that we see a rapidly growing selection of security tools. Most organisations have been aware for some time that the business risk posed by internal threats to sensitive data and supporting systems, in particular those posed by application and technical staff who have access to the most critical and sensitive corporate data, have been largely ignored.

The promise of compliance creates a compelling case for many of these tools (which vendor�s brochure does not mention SOX, PCI, HIPAA, etc.?). This is all the more understandable in the financial or any other highly regulated industry sector where reputations, bonuses and even careers can be lost by one failed audit.

The problem is that no matter how carefully you review, assess and then select a security tool, it is only part of the picture and if you don�t have the whole picture or if the tool does not easily fit into the picture you could be left with some significant gaps. Worst of all, you may not know what the gaps are or even that you have them.

So, are we saying that security tools are not fulfilling their purpose whilst being yet another drain on your ever decreasing budget? Absolutely not! The right tool is an invaluable part of most solutions and where they can deliver a measureable benefit within a defined budget and clear timescales they can prove to be a real life saver. The point is, where and when do they deliver this benefit.

The best way to answer this question is to decide where your security priorities lie. If intrusion detection, access reporting or break-glass access is your biggest priority then security tools will clearly go a very long way in assisting your understanding and mitigating risk, but this is rarely the case in a high compliance environment. Most organisations are comfortable with the proactive prevention of unauthorised access to data and of course the inevitable audits that follow. These security considerations call for at least as many procedural and organisational changes as technical changes and in these case tools will often satisfy no more than a fraction of your requirements.

Now that we have covered the bad news, its time for some optimism. Maintaining appropriate controls may seem a daunting task in light of the above but you should remember that you have already invested in the vast majority of the resources you need to secure your organisation, namely your existing service support and delivery organisation who already have the skills to provide the majority of the security controls you need. All that is needed is the framework that will satisfy the auditors and protect your data and systems. Could it be any easier?

The first step is deciding how to build your security framework. This is not as difficult as it might seem if you treat it like any other IT problem by using technology to meet the business requirement.

In this case the business requirement is defined by best security practice or your particular compliance rules. You can then assess the requirement and deliver the required standards as you would any other technical issue you have. A common approach to this is to employ experienced 3rd party consultancy or specialist security advisory services to help you build the framework always remembering that you are unlikely to be facing a situation that they have not encountered before.

So after all of those questions are answered, how do I know when security tools can help me? The simple answer is that once you have mapped all of your requirements and decided which of them you can satisfy, without security tools the requirements that are left drive your selection.

You can then use the following typical business cases for the use of 3rd party tools to refine your requirements. If any business case is a good fit with a requirement it becomes a further candidate for the use of a security tool:

The tool Delivers a Greater level of compliance

The tool Achieves segregation of duties via automation

The tool Delivers a better ROI case than a custom tool, process or procedure

The tool Delivers other significant benefits such as Management Information or Decision Support

The tool can be implemented within a required timescale that is not likely to be met by a custom tool or procedure

From the above business cases it should be clear that database security tools can provide a great many benefits and will have a role to deliver until the database vendors provide this functionality as standard (many are already striving to deliver these features but usually as add-on products such as Oracle�s Data Vault and Audit Vault).

The next step is to take the requirements identified above and ask Security tool vendors to demonstrate how their product can be used to fulfil those requirements, thereby removing some of the pain from selection process.

The key to meeting your security requirements is to define and balance them against your ability to define and deliver the security or compliance framework with your internal resources and then recognise the opportunities that the security tools offer to close the gap. This approach provides the following specific benefits:

Ensures the best possible use of tools reducing wasted licence, support and implementation costs

Uses the security requirements to define the correct scope and use of security tools

Aids in the tool selection process

Demonstrates the business case for any product expenditure

Assists in justification and negotiations at licence purchase and renewal time

Database Security Tools provide a specific and valuable benefit in database security but the hardest question may be why. Obviously �why� will define your priorities and we seem to have covered this but the last and most lingering question may unfortunately be out of your control. Do you secure for compliance or protection? If they were synonymous we would all have a single set of compliances rules and we clearly do not. The use of the correct policies aided by appropriate use of tools should deliver against your requirements but how can you define a requirement that equally satisfies the technically expert IT staff and the risk focussed auditor. Experience tells us that it is never that easy.

About the author

Pentest Limited is one of the foremost providers of IT security and penetration testing services in the UK and has a worldwide reputation for excellence. Each Pentest consultant has a deep and specialised knowledge in their respective fields, allowing Pentest to match your requirements as, when and where necessary. Pentest's specialist fields include database security, application testing and security, network security and wireless security.

"+title+"

Tue, 08 Sep 2009 23:56:13 -0700

Yaxiong Lin

Model-Based Testing and Its Tools (part 3 of 3)

In Part 1, I gave a brief introduction of MBT and different aspects in MBT. In Part 2, I reviewed and compared several MBT tools and discussed the architecture of TestOptimal and its modeling capability. In this final part, I will describe in more detailed about other features and capabilities TestOptimal supports.

State-of-the-Art Test Sequence Generators

TestOptimal offers 5 different test sequencers, they are:

randomWalk - based on Markov process to randomly traverse the model until a desired coverage is reached,

greedyWalk - similar to randomWalk except that it prefers the un-traversed transitions during the test sequence generation,

optimalSequence - generates the optimal test sequence to traverse all transitions in the model,

mCaseSerial - generates the test sequences to traverse all transitions and/or states in mCase in the order specified, and

mCaseOptimal - generates the optimal test sequence to traverse all transitions and/or states in mCase.

The transitions in mCase do not need to be adjacent to each other or in any order, TestOptimal automatically finds the appropriate transitions from the model to fill in the gap to ensure the complete test sequence is consecutive and valid.

Select different sequencer to achieve different effects to meet various testing needs. For example. you may want to use randomWalk to let it run until it finds a defect, or you may use one of mCase sequencers to test certain part of the model which has been recently changed without re-running the complete system testing. The ability to slice-and-dice the model to test different scenarios on the fly can help you reproduce the defect much quicker.

With the selected sequencer, TestOptimal can also generate a sequence graph from the model which can be used as a visual tool to validate the model.er.

Test Automation with XML Scripting

With the model created, the next step is to attach testing script to each transition to instruct TestOptimal test automation component what to do when the transition is traversed. This is done with mScrpt - an XML based scripting. Of course, you can also write java code as well.

The testing scripts are then executed through Selenium or HtmlUnit. Selenium tests your application in real browser (Internet Explorer, Firefox, Safari, etc.) while HtmlUnit tests your web application in a simulated browser with faster performance.

MBT Execution, Debugging and Monitoring

There are 3 ways to execute your model: dryRun, debug and run. DryRun just executes the model without test automation while debug and run do. With debug, you can set breakpoints at runtime and step through transitions and mScripts.

TestOptimal provides an extensive monitoring on MBT execution, including current coverage of the model, number of detects found and details of defects, number of traversals on each transition, average response time of each transition, estimated completion time, etc.

The execution results can be stored for comparison. TestOptimal uses the stored execution results to evaluate the performance of current execution as model is being executed.

You can assign a severity level to each defect and use the severity level to categorize the defects or to use it as the stop criteria.

Load and Stress Testing

With the built-in test automation, TestOptimal is capable of stress and load test your application by running the model concurrently on multiple sessions. With mCase, you can easily create a bunch scenarios and execute them against your application. As a result your application is exposed to a realistic simulation of production environment.

Licensing Editions

TestOptimal offers 5 editions to meet specific needs: BasicMBT, JavaMBT, WebMBT, Enterprise and Community. BasicMBT provides basic MBT modeling and test generation while JavaMBT and WebMBT include all functionalities in BasicMBT plus test automation. Enterprise edition offers advanced features like stress/load testing and custom plugin.

TestOptimal also offers Community edition with limited functionality free to MBT community for educational and non-commercial use. It can be downloaded at here..

I hope you are as convinced as I am on the benefits MBT can bring to the software testing [2] [3] [7] and these benefits can not be realized without a strong MBT community. If you need more practical how-to guidance, check out 6-part series tutorial Model-Based Testing HOWTO.

Tools :

Conformiq

MaTeLo

mbt.tigris.org

ModelJUnit

NModel

Reactis

Smartesting (formerly Leirios)

SpecExplorer

TestOptimal

About the author

Yaxiong Lin has over 20 years of experience in system modeling and software design, development and testing. He was the chief architect of several mission critical enterprise systems.He is the owner of open source project openOptima and is the chief architect of TestOptimal.

"+title+"

Mon, 31 Aug 2009 01:56:48 -0700

Yaxiong Lin

Model-Based Testing and Its Tools (part 2 of 3)

In Part 1, I gave a brief introduction of MBT and talked about different aspects in MBT. In this part, I will cover some of these tools and present a comparison matrix. To help you appreciate how far the tool can go to help you in your MBT project, I will focus on just one of these tools which I architected. If you need more detailed information about other tools, you may check the references section at the end of this article.

MBT Tools

There are quite a few tools and projects developed over the years. Unfortunately many of them have been either inactive or been morphed into other projects. I will only cover the active tools and projects here as of date of this article.

Conformiq - MBT modeling, test sequence generation with scripts generation in different languages.

MaTeLo - a statistical MBT test sequence generation using random Walk.

mbt.tigris.org - offers test sequence generation from imported models in graphML.

ModelJUnit - a Java library that extends JUnit to support MBT. It is used to unit test java classes using MBT. The model is created with java classes inheriting from the framework classes.

NModel - MBT modeling and test sequence generation using c#.

Reactis - offers random walk test sequence generation.

Smartesting (formerly Leirios) - offers MBT test sequence generation.

SpecExplorer - by Microsoft, a framework to develop MBT models in Spec# and Asml Test sequence generation using randomWalk and Chinese Postman algorithm.

TestOptimal - supports full MBT process with MBT modeling, 5 different test sequence generation algorithms, and test automation for web and java applications with plug-in architecture to support other types of applications.

MBT Tool Comparison Matrix
MBT Tool (alphabetically)	Licensing	Modeling	Sequencers	Test Automation
Conformiq	commercial	yes	yes	no
MaTeLo	commercial	no	yes	no
mbt.tigris.org	open source	no	randomWalk, A_star	no
ModelJUnit	open source	java	yes	java
NModel	open source	c#	yes	c#
Reactis	commercial	yes	randomWalk	no
Smartesting	commercial	no	yes	no
SpecExplorer	free	Spec#, Asml	randomWalk, CPP*	yes
TestOptimal	commercial, free Community Edition	yes	randomWalk, CPP*, mCase	yes

* Chinese Postman Problem algorithm.

Before you choose a tool for your MBT project, it is very important to know how the tool will fit into your development process and understand what the tool can and cannot do. There is nothing more frustrating than trying to make a tool do what it is not designed for.

In the following sections, I will discuss in greater details on one of these tools: TestOptimal to help you understand what to look for in the MBT tools.

TestOptimal Architecture

TestOptimal is built with a typical web client/server architecture. At its heart is the TestOptimal Server which can be deployed on Windows or Linux. You would usually run TestOptimal Server in your local machine while you are developing and when completed you deploy and execute the model on a dedicated server.

TestOptimal consists of modeling, test generation, test automation, execution and debugging, and statistical analysis of the execution results. It currently supports test automation for web and java applications but its flexible architecture will allow it to expand to other types of applications like Adobe Flex, and .NET applications.

You can access TestOptimal Server with a web browser from any location or use one of many supported interfaces to integrate TestOptimal with your IDE and existing testing framework / tools as illustrated in the architecture diagram above.

Supporting Full MBT Process

TestOptimal was built from ground up to support full MBT process starting at the modeling through test generation to test automation and execution result analysis. With TestOptimal, you can easily go back and forth between modeling, test sequence generation and test automation. Based on the result analysis, you can make changes to the model and immediately execute the model on your application right within the TestOptimal browser.

Modeling with TestOptimal

The model is created in the simple tree-view like model editor. It supports super state which is a state containing child states. Transitions are placed under the state. Each transition may be defined as optional or required with minimum number of times it must be traversed, a useful feature to devise a Data-Drive Testing. The model can then be viewed in a graph.

You may also import models in several formats: graphML, graphXML, SCXML, and XMI from Eclipse/RAD.

In Part 3, I will cover TestOptimal's 5 test sequence generators and its built-in test automation.

"+title+"

Tue, 25 Aug 2009 03:09:57 -0700

Yaxiong Lin

Model-Based Testing and Its Tools (part 1 of 3)

Model-Based Testing (MBT) is the newest development in the software testing industry. In this 3-part series, I will give a brief overview of MBT and provide comparisons of different tools from the perspective of UI interface testing with the intent to increase the awareness of MBT in the software testing community.

A Different Approach to Software Testing

Model-Based Testing (MBT) approaches software testing from a brand-new perspective that is quite different from the traditional software testing [1]. MBT is a methodology that is based on science with a clear objective to achieve the best test coverage of the software. The traditional "record / replay" and manually crafting test cases often do not provide sufficient coverage and are labor intensive to create resulting in a system that is very difficult to maintain especially in today's agile development process.

MBT provides an attractive alternative to address these challenges.

MBT Process

There are 5 distinctive phases in MBT process: modeling, test case (sequence) generation, test automation, test execution and test result analysis [2]. In practice this is often an iterative process. For example during test case generation or test automation phases, you might need to make changes to the model.

Having a tool that supports this iterative process is imperative for a successful MBT implementation.

MBT Modeling

The first phase in MBT process is modeling, an activity to create a behavior model from the requirements and specifications for the Application Under Test(AUT). The model is described with a Finite State Machine (FSM) diagram or State Diagram in UML

Using a web application as an example, a node (state) in the diagram usually represents a web page and the arrow (transition) represents a button or link which leads to another web page when clicked

In some cases it may be important to distinguish a web page under different context in which case you would use a collection of states to represent different contexts in a page. Flash movie model is one of such example.

When displayed in graph, the model becomes a visual representation of the requirements and specifications which is much easier to understand and validate and can be used as an effective communication tool.

Test Sequence Generation

The ability to generate test sequence from the application model is the single most important aspect in MBT. All MBT tools are equipped with at least one test sequence generator

Behind these test sequence generators there are different mathematical algorithms. These algorithms are the brains of test sequence generation. There are various types of algorithms ranging from as simple as random walk derived from Markov process to a fairly complex algorithm like the algorithm to solveChinese Postman Problem (aka Route Inspection Problem).

Often times you may want to test just certain parts of AUT or repeat certain parts of the model for Data-Driven Testing as a example. A good test sequence generator should allow users to change configurations to meet different testing needs.

Test Automation Integration

With the test sequence generated from the model, the next step is to turn them into executable. A typical test sequence may contain hundreds of thousands of test steps. Clearly you do not want to do this by hand.

At a minimum each MBT tool should generate the test script in certain format which can then be executed on a test automation tool. In fact this is what most of MBT tools do.

The problem with this approach is that no matter how good the vendor claims their tool can generate the script to run on someone else's tool, it is inevitable that you will end up making little change here and there to get it working on your test automation tool. As soon as you do that they become high maintenance artifacts.

There are MBT tools which offer built-in test automation giving you a nice seamless integration from modeling to test automation. This is certainly the tool you want for your project.

As we all know, a good tool enhances the process and enables us to become more productive. It is even more so with MBT. Before choosing an MBT tool, you will want to know how it will integrate with your development process and existing tool set. You will also want to have a good understanding of its capabilities and limitations.

In Part 2, I will review some of the tools and give a comparison matrix on them.

"+title+"

Fri, 21 Aug 2009 06:32:06 -0700

Soon Hui
Esteem Innovation

How to Evangelize Unit Testing

You can't force others to practice unit testing, especially if they are working on legacy applications. They will have a lot of excuses and true enough, doing unit testing on legacy code that have not been designed for testability is hard. But not doing any automated testing is tantamount to not paying your credit card bill. Sooner or later your debt will eat you up and you will go bankrupt. Or not, if you are lucky, you can pass your debt to either your descendants or successors and let them worry about the problem.

But sometimes you don't have the luxury (i.e., passing the mess around), worse still, you may need to anticipate the messy code problem because you may have to do the cleanup. Or you are such a purist that you can't see the whole application walking around without automated testing covering its ass, then my experience below maybe helpful to you.

Risk based testing is getting popular since it is aimed at optimization of time and resources available for testing without comprising on the quality of testing. However, most approaches for Risk Based Testing today, suffer from the following drawbacks:

First, a little background

We did our application in C#. Since the language is fairly modern, so it must follow that the software development style should be fairly modern, right? Unfortunately, no.

We didn't have unit testing until fairly late into the development. And so, I was in charge of brining the unit testing practice to the company because I was a fierce proponent of it.

One thing I can tell is that it's never easy to convince the developers to adopt new things unless you can show tangible benefits. But there is a chicken and egg problem . Since the benefits of unit testing is only apparent when you work it on long term, how can you prove to them that unit testing is a good thing? Unless you can take over their projects and do the test cases then it's a different story. But on a large scale commercial application this is simply not possible. Unit testing is a bit like religion; you can't taste the goodness unless and until you become a convert.

One may want to show that the code with unit testing coverage has the lowest bug count. But this number is not easy to get. And even if you can do that, so what? Using bug count to evaluate developers or to promote unit testing is no better than using bug count to evaluate testers. If you have very few bugs in your code, it could have meant that your code is still not tested, or that your work is so easy that there isn't much bugs lurking around, or that you are already a superior coder for your program is much better than the guy seating next to you, regardless of whether you do the unit tests or not.

The only way I can think of is to create integration test cases.

Well, it's not exactly integration tests. It's a hybrid between the integration tests and the unit tests, leveraging on the existing NUnitframework.

Sounds complicated?

It's not, they are just NUnit test cases that test large modules with different components interacting together. Since those test cases were not testing individual methods, so they were not exactly unit testing. Low level methods and classes were just too tightly coupled to be tested individually. So a compromise had to be made. Luckily we still had some high level design and low coupling between different modules, so creating NUnit test cases on module level were still feasible.

Since the developers hadn't bought in the idea, so I created the test cases for them.

This must be done carefully. The developers usually didn't like other people creating test cases for them because they would feel like they were not trusted. A political revolt might result if one did this briskly. Besides that, the code was also not structured for maintainability so you might need to refactor the code first before you could write tests. Generally, the developers' willingness to let you touch their code is proportional to the code quality. So the worse the code is, the more need there is to create tests for them. But the worse the code is, the harder it is-- either technically or politically--to test them.

This is a vicious cycle.

What I did was I appealed to the boss to lobby for automated regression tests. Luckily he fully understood the problem and gave me a go-ahead sign. I also reasoned to the developers and asked for their permission to

Refactor the code
Create test cases to test their code

At that point of time, the developers would normally agree to let you did the work, regardless of how grudging they were at the beginning. After all, they needed not to do extra work and since the boss had agreed on it, so there was no reason to complain.

After receiving the green lights to create integration tests, I understood that I must did this correctly, on the first attempt. There would be no second chance if I couldn't show the tangible benefits within a short time frame. Everyone was impatient to see the results. If they couldn't see the results in two months time, all the good will and enthusiasm would drop off exponentially and to pickup unit testing next time would be significantly harder.

Which was why it was necessary to go for the biggest bang for buck.

The next step I did, I strategized and look for the area that was most error-prone and easiest to create tests on. You wouldn't normally get both, but in my case, I was lucky to find the weakest spot and started to create tests on it.

And being the person who didn't write the code, one might not know what to test; after all, unit testing was supposed to be done by the developers who knew their code inside out.

So to work around the problem, I went through the bug tracker to look for relevant bugs. Since all cases had ( or were supposed to have) the reproduction steps, the observed outcome and the expected outcome, I knew how to reproduce the problem, what were the correct and incorrect results. And so, I knew what to test and how to test. Problem solved.

So, as test cases accumulated, the ability for the entire test suit to find bugs increased gradually. Seeing that the tests did catch bugs, the attitude towards unit testing changed gradually as well. After sometime I passed the whole test suit to the respective developers and asked them to maintain it themselves. Now they were willing to own the test suit.

I think there is an important point here. To help the developers to see the virtue of good software design, it is often necessary to let them realize the benefits of unit testing first before anything else. Too often the developers who write bad code will be reluctant to cleanup their design mess under various pretexts. But once their buy in unit testing, they will naturally design their code to be testable and will pickup good software design practices along the way.

No, we are still far from Test Driven Development (TDD), but at least the developers were more receptive to unit testing and started to create their own tests. Well, occasionally they might still forget to do it, but at that point, a little reminder could solve the problem. My personal experience with unit testing was that it was infectious; once I started doing it in one area of my code, I would like to apply it onto other areas. And with time I learn how to write test first, let the test drives the design!

I believe that gradually every developer would start to write test cases themselves automatically, without reminder once they got onto it.

QAGuild Weekly Article

"+title+"

Reducing Costs with Good Requirements and Code Reviews

Defining Good Requirements

Reducing Defects with Peer Code Reviews

Reduce Project Costs with These Best Practices

Can Tools Help?

"+title+"

As a Programmer, what were my mistakes?

1. Not using a proper ORM

2.Not learning generics soon enough.

3.Reinvent the wheel

4.Too much documentation.

5.No automated build

6.Rely on visual inspection and debugging too much

7.No unit test

"+title+"

Testing Data

"+title+"

Unit Testing, What a Pain!

Excuses for Not Doing Unit Test

"+title+"

Reduce Cost with Better Requirements & Reviews

"+title+"

How to Build and Maintain a Successful Outsourced, Offshored Testing Partnership

References

"+title+"

CHANGE LEADERS

INTRODUCTION:

Why Change implementation fails?

Change Deployment Frame work:

The 8 Stage Change Implementation Frame work

Stage 1: Creating Sense of Urgency:

Stage 2: Create a Guiding Coalition:

Stage 3: Developing a vision & Strategy:

Stage 4: Communicating the change vision:

Stage 5: Empowering Employees for Broad based Action:

Stage 6: Generating a Short-term win:

The role of a short-term win:

Stage 7: Consolidate Gains and Produce More Change:

Stage 8: Anchoring New Change in to the Culture:

Precautions:

Conclusion:

"+title+"

Why Developers should Do Testing

Enable the developers to gain system knowledge

Improve the developers writing skills

Improve software quality

Make the developers better software engineer!

"+title+"

Learn �Proactive Approach� Which helps us in saving costs

"+title+"

Layered Process Audit (LPA).

"+title+"

Database Security; why expensive tools may not be the whole answer

"+title+"

Model-Based Testing and Its Tools (part 3 of 3)

State-of-the-Art Test Sequence Generators

Test Automation with XML Scripting

MBT Execution, Debugging and Monitoring

Load and Stress Testing

Licensing Editions

Further Readings

Tools :

"+title+"

Model-Based Testing and Its Tools (part 2 of 3)

MBT Tools

TestOptimal Architecture

Supporting Full MBT Process

Modeling with TestOptimal

"+title+"

Model-Based Testing and Its Tools (part 1 of 3)

A Different Approach to Software Testing

MBT Process

MBT Modeling

Test Sequence Generation

Test Automation Integration

"+title+"

How to Evangelize Unit Testing

First, a little background