Software Testing - Tutorials, QTP, Manual Testing Automation Testing, Load Runner

Test Estimation Techniques

2009-10-31T10:27:00.001-07:00

.. Continuing the Test Management Series

I have received many questions/queries and problems on estimation test case preparation time, testing cycles and testing activities.

So on peoples request I am posting some Software Test Estimation tips. Here you go:

1. Take the estimates that the developers and testers give you, and double them. :)
Triple them if there is a lot of multi tasking/task switching going on.

In all seriousness, people don't actually work 8 hours a day- there's meetings to attend, and email to read and respond to, and interruptions. Account for this in your schedule by only scheduling people for 6 hrs/day.

No tool is going to do this for you. Estimating is a combination of comparing the work to be with work that's already been done, along with your best guess.

2. If a module does nothing but set a parameter to a Boolean, such as WINDOW_OPEN :== TRUE, it should take less than an hour to test it. If a module reads many schedules and contains many decision paths, it could take a week to test the module. So an estimating tool would have to take module complexity into account.

I have divided software modules into three categories: high, medium, and low complexity. Based on past experience, I have calculated average time to develop and test each type of module. Then I created an Excel spreadsheet that automatically calculates labour hour estimates based on the number of high, medium, and low complexity modules in the software project. It is also a handy tool for the software change board meetings, where the PM must explain the cost, schedule, and risk implications of every change accepted for incorporation into an on-going software project.

As the above first point states - developers are notoriously bad at estimating. Therefore, objective data should be used whenever possible.

3. Using a tool will not give the reasonable estimation for a project.
From my experience, firstly know the project criticality and then divide them into high level and low level functionalities and then estimate the time required.

For high level functionalities ---
Like where the data is entered and where we are getting the output.
Note:if there are any calculations included in it,then the estimation time will be more.

Low Level functionalities--- Like UI, cosmetic
This will not take much time and here the estimated time is low.

4. According to Roger Pressman, testing takes between 30% and 40% of the total project effort.
I've been guessing this effort to my test and it rarely fails.

5. Here is one approach, if helpful. Derive test cases based on negative and positive flows depending on importance of the functional flow. For example, some functional flows which have high business intensity may need more test cases (4 positive flows, 4 negative flows and 1 normal flow) per functional flow. And some may need less test cases. Again categorize test cases into high, medium and low level test cases, so that it is easy to allocate time per test case. Convert number of test cases into efforts (x number of test cases needs N amount of time).

We have to consider this for test case writing, execution, automation etc. Other than them, test strategy, test plan etc. would be standard efforts.

6. There are different ways for estimation and it depends upon the organization estimation techniques. Some organization follow SMC model, some follow Wide-delphi technique (as defined in 1st Point) + Use case estimation.

It is better to understand the requirements, discuss with your team, gain the information from them, time that requirement might take to test and apply that information for estimating using the SMC model or wide delphi technique. You will reach a certain number. Based on that you can add the PM effort + Risk Mitigation( Code delay, requirement changes) + Buffer.

This will help to reach to the close estimating figure and then treat this estimation as baseline for future estimation.

Your estimation also need to counter the experience of the Team members.

7. The very best estimates will be gained from historical data. Check for the last time you worked on a project of similar scope, architecture, and complexity, pull out the timesheet data, and use that as your estimate.

In my experience, the only way that testing could come out reliably to a set percentage of the overall project (whether 30%, 40%, or whatever), is by self-fulfilling prophecy. Testing stops simply because the deadline's been reached, the budget's exhausted, and other projects are waiting.

8. Also, for some projects, No Estimation method is accurate for testing (TP, FP, etc). All testing estimations are company and project specific.
Testing usually takes 35 - 40% of the entire project effort. 15-20 % variation is always.

Go Back to –> Software Test Management Suite

2. Software Testing Estimation Guidelines

Challenges in Agile Testing

2009-10-29T23:10:00.001-07:00

In this tough environment more and more companies are beginning to explore how "agile, with different flavors can help their companies deliver reliable application quickly and iteratively. The role of testing professionals increasingly important in the application of Agile Projects. Innovations are increasingly driven by the needs of society in general check. But the paper more difficult to change to agile development is the role of Tester. That is because the agile development contradicts plenty of things to plenty of testers have been taught is "best practice".

As the global crisis hit hard, more and more worries about how it will impact in the field of information technology. The companies to be more cautious and sometimes reluctant to give application business. Some customers of the withdrawal of its long-term projects, while others use the possibilities of renegotiating the contracts and lower price quote. Recent scandals also add fuel to the fire, and consequently the levels of customer satisfaction and plenty of projects will fall over expected and cost over expected. And all this happens while the client wants to accomplish better business processes for application vendors to increase customer satisfaction and save more funds.

So what are the challenges faced by testers when working with agile development teams and how they can meet the challenges?

In an agile development environment can not expect things to happen. You must be proactive. This is true for all team members, but is true for Testers. Members of the quality control team can not let anything get in the way of proof. Failure to follow the requirements they require to speak with the owner of the product for clarification. If you are unsure of how developers are going to implement the new code, they require to discuss how the code will execute with the developer. Must be self-directed experts in removing obstacles that prevent them from testing.

Recently, shipping through Application Testing Help I found an interesting article:

Tips to be more innovative in the era of Agile Testing \., written by JB Rajkumar - experienced trainer, quality manager, a frequent speaker for colleges, universities and international conferences.

In his long face Agile Testers of the problems when working with agile development team. And they must be able to apply root cause analysis when you find serious errors so that probably won't happen again. While Agile has different tastes, Scrum is a process for implementing Agile. A quantity of the rules scrum challenge to be followed by all individuals are:

- Get Number of hours of commitment Up Front
- Gather requirements / Estimates Up Front
- Introduction of the actual hours and estimated hours daily.
- Construction Daily
- Keep meetings short daily Scrum
- Code Inspections are critical

In order to meet the challenges, an agile testers require to be innovative with the tools they have. And here are some important keys to innovation:

- A lovely Agile Tester has to be creative in trying to cope up with the speed of development or release. For a tester, being creative is more important than being critical.
- Proof must be professional talent and strives more for learning and innovation of new ideas. Testers talent are seldom satisfied with what they have achieved and always strives to find errors unimaginable value and high priority.
- An Agile check should not be afraid to look at the code a developer and if necessary, and in extreme cases, to correct it.
- Must have a comprehensive view of customer expectations and delivering lovely product.
- Must be authorized to work in pairs. He will participate in pair programming so that the script shorter, better designs and finding more errors.
- Tester must be passionate and brings something distinctive that can be in terms of their innovative ideas or how to take the day to day.

Finally, Agile Tester must have multiple skills, and, Manual, functional performance tests of skills and soft skills such as leadership skills, communication skills etc, in this new world of professional testing focused on adding value to business and application development life cycle and act in a new and elevated position where the combination of structured-driven approach, creativity and ability to articulate and define quality criteria and testing new models will be critical for the success of the business application of technology.

(Source – Article by: Sergey Lesnikov, Kiev, Ukraine)

Related Posts:

Challenges in Software Testing

Software Requirement Specification document Review Guidelines and Checklists

2009-10-19T10:01:00.001-07:00

To write and prepare effective test cases, testers and QA engineers should review the software specs documents carefully and raise as much queries as they can.

The purpose of Software Requirement Specification Review is to uncover problems that are hidden within the specification document. These problems always lead the software to incorrect implementation. So following guidelines for a detailed specification review is suggested:

1. Always review specification document with the entire testing team. Discuss each point with team members.

2. While reviewing specification document, look carefully for vague/fuzzy terms like – “ordinarily, most, mostly, some, sometimes, often, and usually” and ask for clarification.

3. Many times it happens that list values are given but not completed. Look for terms: "etc., and so forth, and so on, such as." And be sure all the items/list values are understood.

4. When you are doing spec review, make sure stated ranges don’t contain unstated/implicit assumptions. For example: “The range of Number field is from 10 to 100.

But is it Real, Hex or Integer? Ask for Clarification.

5. Also take care of vague/fuzzy terms like - skipped, eliminated, handled, rejected, processed. These terms can be interpreted in many ways.

6. Take care of ambiguous pronouns like – “The ABC module communicates with the XYZ module and its control flag is set.” But whose control flag?

7. Whenever a scenario/condition is defined in paragraph, then draw a picture of that in ait to understand and try to find the expected result.

8. In the specification document, if a scenario is described which hold calculations, then work on its calculations with minimum two examples.

9. If any point of the specs is not clear then get your queries resolved from the Business Analyst or Product Manager as soon as possible.

10. If any mentioned scenario is complex then try to break it into points.

11. If there is any open issue (under discussion) in the specs (sometimes to be resolved by client), then keep track of those issues.

12. Always go thru the revision history carefully.

13. After the specs are sign off and finalized, if any change come, then see the impacted areas.

Also Read:

1. Testing the design and Requirements of a Software

2. Reviewing Test Cases

HP0 M15 Quality Center Certification Questions

2009-10-05T22:01:00.001-07:00

HP0 M15 Quality Center Certification sample Preparation Questions:

1) Quality Center is
- an ERP
- a standalone software
- a web based (client/server) tool
- a script

2) Traceability Alert is depicted with "@" Mark against a Test
- Yes
- No

3) For Test Plan Module , select the correct folder hierarchy
i. Subject - Tests - Test Step
ii. Subject - Test Steps - Step
iii. Requirements - Tests - Test Steps
iv. Requirements - Subject - Tests - Test Steps

4) Which Tab allows to configure conditional execution of Tests

Hint : Conditions like Test # 2 can be executed only if Test # 1 is completed

i. Execution Flow
ii. Design
iii. Attachments
iv. None of The Above

5) Test Cases created in Word / Excel can be uploaded in QC ?
i. Yes
ii. No

6) Types of Graphs available in QC
i. Reports
ii. Graphs
iii. Live Analysis Graphs
iv. All Of the Above

7) Follow Up Alert can be created on
i. A Test in Test Plan Tree
ii. A Test Instance in Execution Grid
iii. A Defect in Defects Grid
iv. All Of the Above

8) Favorites are almost always are High Priority Defects
Hint : User can mark a defect as favorite for easy tracking. A favorite can be PUBLIC or PRIVATE
i. Yes
ii. No

9) In QC, Requirements & Test Plan can be linked ?
i. Only Child Requirements
ii. Only Parent Requirements
iii. Both Child & Parent Requirements
iv. Neither Child nor Parent Requirements

10) In Test Lab Module , is it possible to store separate results for the SAME test case run at different dates and times ?
i. Yes
ii. No

11) Which of the Below is NOT a module in Quality Center
i. Requirements
ii. Business Component
iii. Defects
iv. Automation

12) Defect can be linked to
i. Tests & Test Steps
ii. Run & Run Steps
iii. Requirements
iv. All Of the Above

13) Test Cases can be added in Test Lab Module from
i. Requirements Module
ii. Defects Module
iii. Test Plan Module
iv. All Of the Above

14) Data is DYNAMICALLY updated in
i. Reports
ii. Graphs
iii. Live Analysis Graphs
iv. None Of the Above

15) Is is possible to link a single defect to multiple test cases ?
i. Yes
ii. No
iii. MAYBE. Depends on the TYPE of Defect

16) Which Tests can be scheduled to be run at a specific Date and Time ?
i. Manual Test Cases
ii. Automated Test Cases
iii. Both Manual & Automated Test Cases
iv. Neither Manual nor Automated Test Cases

17) In Test Lab Module
i. Tests are created
ii. Tests are executed
iii. Tests are created and executed

18) Results of an AUTOMATED test case are stored in "-------" module of Quality Center
i. Automation
ii. Test Plan
iii. Test Lab
iv. None of The Above

19) An Instance is
i. A copy of the Requirement
ii. A copy of Test Case
iii. A Copy of Defect
iv. A copy of Email

20) Quality Center is a
i. Defect Tracking Tool
ii. Requirements Management Tool
iii. Test Planning Tool
iv. Comprehensive Test Management Tool

Answers:

1) Quality Center is

Correct answer is: a web based (client/server) tool

2) Traceability Alert is depicted with "@" Mark against a Test

Correct answer is: NO

Traceability Alert is depicted with "!" Mark

3) For Test Plan Module , select the correct folder hierarchy

Correct answer is: Subject - Tests - Test Step

4) Which Tab allows to configure conditional execution of Tests

Correct answer is: Execution Flow

5) Test Cases created in Word / Excel can be uploaded in QC ?

Correct answer is: YES

6) Types of Graphs available in QC

Correct answer is: All Of the Above

7) Follow Up Alert can be created on

Correct answer is: All Of the Above

8) Favorites are almost always are High Priority Defects

Correct answer is: NO

Any priority defect can be marked a favorite.

9) In QC, Requirements & Test Plan can be linked ?

Correct answer is: Both Child & Parent Requirements

10) In Test Lab Module , is it possible to store separate results for the SAME test case run at different dates and times ?

Correct answer is: YES

11) Which of the Below is NOT a module in Quality Center

Correct answer is: Automation

12) Defect can be linked to

Correct answer is: All Of the Above

13) Test Cases can be added in Test Lab Module from

Correct answer is: Test Plan Module

14) Data is DYNAMICALLY updated in

Correct answer is: Live Analysis Graphs

15) Is is possible to link a single defect to multiple test cases ?

Correct answer is: YES

Any type of defect can be linked to multiple test cases.

16) Which Tests can be scheduled to be run at a specific Date and Time ?

Correct answer is: Both Manual & Automated Test Cases

17) In Test Lab Module

Correct answer is: Tests are executed

18) Results of an AUTOMATED test case are stored in "-------" module of Quality Center

Correct answer is: Test Lab

19) An Instance is

Correct answer is: A copy of Test Case

A Test Case copied from Test Plan Module to the Test Lab Module is called an instance of that Test Case

20) Quality Center is a

Correct answer is: Comprehensive Test Management Tool

Also Read:

Mercury Quality Center Interview Questions

QTP Certification HP0-M16 Practice Exam (Questions)

Things you might not know about Performance Testing

2009-10-04T06:55:00.001-07:00

Ten Things you might not know about Software Performance Testing:

by Dale Perry

Performance testing is fundamentally different from functional testing. In functional testing, we are interested in the user perspective: Does the software provide the required or requested functionality? In performance testing, we are interested in system characteristics. Many of these system elements will affect how the software is perceived but are indirectly related to functionality—e.g., a slow transaction will annoy the requester but the function will still work.
The role of the performance tester is similar to that of a consultant. A well-planned performance test requires the involvement of all key personnel—DBA, system administrator, network administrator, developer, etc. The tester understands the test issues and requires the support of others to comprehend the technical issues. It is a rare individual who fully understands all aspects of a system—application, network, database and file system, operating system, etc.
There is a critical difference between performance goals and business goals . Many people confuse the two, and this misunderstanding negatively impacts the viability of the performance test. The fundamental difference between the two is simple. If you cannot measure a system-based resource to assess the performance, it is not a performance goal—i.e., a goal is to improve user productivity. However, there is no system resource you can measure to prove this goal has been met. The system may be performing fine, but the user is just slow in using the application.
Performance testing requires extensive time and effort. To be successful, a performance test has to begin at the start of the project with planning, analysis, and design. This is especially true for rapid development projects (agile, etc.). The remaining six areas listed below take considerable time to analyze and plan the testing. Trying to attempt a first-time performance test one to two weeks (or days) before shipping is insane.
Performance testing requires an environment that resembles as closely as possible the actual target environment. Many aspects of the application and system will behave differently on varying-size environments. Many aspects of applications and systems are non-linear, so methods such as extrapolation are highly dangerous and typically invalid from the start. Extrapolation is a linear predictive model; systems tend to be nonlinear.
Load tools are the least important issue in performance testing. If you do not understand the key issues and aspects of the test, especially those listed below, a tool will give you numbers that have no real meaning. Tools provide answers; if you don’t know the question, what good is the answer? Additionally, there are three types of tools that must interoperate successfully: load generators, monitors, and response time tools (end to end). There are numerous issues that must be addressed for tools to be useful.
A successful performance test requires a proper load definition—an operation profile (OP). All too often I hear people refer to performance as a simple calculation: Load = Volume/Time. This is where many tests fail. Volume of what? And more importantly, over what time period? Key elements of an OP are the time period, the number and types of activities in each time sample, and the frequency and distribution of those activities. The interaction of events is often the cause of performance problems. Failure to have an accurate model of event (user) behaviour renders the test invalid.
There are many types of tests that we can run. There are tests to measure specific resources (CPU, memory, bandwidth, etc.), assess response time, and press the system to its limits (stress). The types of tests in which we are interested will be driven by our performance goals, possible environmental issues, etc. Just running a lot of events and measuring system elements is not performance testing.
Many tests fail before they begin because there are no measures defined to indicate what constitutes poor performance. What are poor response times, excess load, and limits on memory and CPU? These measures need to be understood at the beginning of the test so the tests can indicate where and when problems are occurring.
Tools provide answers. The issues noted above are essential to ensure that your tools provide useful information. Most load generation tools generate dozens of measures. These numbers make great graphs but are of little use if you don’t know what you are looking for. If you want to get maximum value from your tools, you need to address the previous nine items on this list. Tools are great, but a fool with a tool is still a fool.

2. 10 Things You Might Not Know About using Testing Standards

10 Things You Might Not Know About Sustainable Test-driven Development (TDD)

2009-10-04T02:54:00.001-07:00

About Test Driven Development:

TDD is basically for developers, not for testers.

How Test driven development works:

In test driven development, Developers write unit test scripts. Then they execute the scripts and fix the scripts.

Think, how difficult for developers to write unit test scripts for an application which is not developed yet.

10 Things You Might Not Know About Test Driven Development.

by Scott Bain

A test should be proven to fail for a well-understood, narrowly defined reason. If this is not proven, then it is possible that a test will never fail, and thus the behaviour it defines is not guaranteed.
A test should never fail for a reason other than one for which it was intended. When a test fails—yet the behaviour it was designed to cover is working properly—then the test misleads us as to the source of the failure. Often this is due to unexpected or undesirable coupling in the production code.
Multiple tests should not fail for the same reason. Multiple test failures for a single cause create maintenance problems when new features are added to the system, when existing features are removed, or when there are changes for performance, security, or any other reason. Agility requires that we embrace change, and large numbers of test failures due to a single change inhibit this.
Each test should make a unique behavioural distinction in the system that no other test makes. This will ensure adherence to points 1, 2, and 3.
Tests must be written within a well-understood scope. For example, if we are testing that an algorithm properly calculates tax, we do not need to write a test to ensure that the code being tested does not also reset the system clock, format the disk, or do anything else outside the scope of the problem. We test for errors, not malice.
A test tests all those things that are in scope but are not under the test’s control. Therefore, if we wish to test a single, narrowly defined thing, then everything else in scope must be brought under the test’s control. We use techniques such as mocking, shunting, endo-testing, dependency injection, etc., to accomplish this. Developers must be well trained in these techniques if the test suite is to remain maintainable as the project matures.
The role of a code coverage tool in TD is to verify, after a test is removed, that the system is indeed still fully covered. For example, when a test is found to duplicate the distinction of another test, one of them must be removed. If code coverage reveals an uncovered path, then either the test was not actually a duplicate or there is “dead” (unused) code in the system.
The specification of the tests is how we conduct analysis. The resulting test suite, along with the scenario descriptions that it satisfies, becomes the functional specification of the system. This has advantages over traditional specification documents in that the suite is written in concrete language, which is verifiable by the computer without human intervention.
The level of complexity in testing is an indicator of the quality of the design of the code being tested. For example, when test classes are significantly larger than the classes they test, this can indicate that the tested class does too many things. Or, a test that requires a large number of instances can indicate excessive or unwanted coupling in the system. Therefore, testability is a driver of good design.
Test suites must run fast so they can be run frequently without excessive cost. Therefore, all dependencies on the database, the GUI, the network, etc., that would make tests run slowly must be mocked for testing. These external dependencies should be tested, as well, but they can be tested by integration tests that are not run frequently and, therefore, are not expected to run as fast. (This article is from Better software Magazine - July-09 Issue)

Selecting Test cases for Regression Testing

2009-10-03T11:49:00.001-07:00

.. Continuing the Test Management Series

It was found from recent survey (from small and big Software testing companies) that good bunch of the defects reported by clients were due to last minute bug fixes creating side effects. And hence, selecting the test cases for regression testing is not easy. It is an art and one QA Lead/QA Manager should be perfect in this.

To select test cases for Regression Testing, QA Lead/QA Manager should know the following:

Knowledge on the bug fixes and how it affects the application.
In which area/functionality of application, the defects are frequently occurring.
Which area/functionality has undergone many/recent code changes?
Which features of the product/application are mandatory requirements of the client?
Which features of the product/application are Important to the customer?
In which functionality, the bugs are fixed in rush/hurry?

Selecting test cases for regression testing depends more on the criticality of bug fixes than the criticality of the defect itself. Fix of a minor bug can result in major side effect and a bug fix for an Extreme defect or a high severity defect can have no or a just a minor side effect. So the Test Lead/engineer or test Manager needs to balance these aspects for selecting the test cases and test scenarios for regression testing.

Solution: Make good relations with Development team lead/Development Technical manager. They can easily help QA team in identifying the above. A proper impact analysis should be done.

While selecting test cases and test scenarios for Regression Testing, we should not select only those test cases which fail while regular test cycles because those test cases and scenarios can have no or less relevance to the bug fixes. We need to select more positive test cases than negative test cases for final regression test cycle. It is also recommended and a best software testing practise that the regular test cycles (which are conducted before regression testing) should have right mix of both positive and negative test scenarios. Negative test scenarios are those test cases which are introduced with intention to break the application.

From a recent survey, it is found that several companies have "constant test cases set" for regression testing and they are executed irrespective of the number and type of bug fixes. Sometimes this approach may not find all bugs (side effects) due to the bug fixes in the application. Also in some cases it is observed that the effort spend on executing test cases for regression testing can be minimized if analysis or a study is done to find out what test cases are relevant for regression testing and what are not. This can be done by Impact Analysis.

It is a good approach to Planning Regression testing is from the beginning of project before the test cycles instead of hat you plan after the completion of regular test cycles. Best practise is –

Classify the test cases and test scenarios into various Priorities based on importance and customer usage. Here it is suggested the test cases be classified into three classes:

Priority 0 – In this category, those test cases falls which checks basic functionality and are executed for pre-system acceptance and when product goes thru major change. These test cases basically checks whether the application is stable enough for further testing or not. These are Sanity Test cases which delivers high project value to client and the entire software development/testing and quality assurance team.

Priority-1 – In this category, we can add test cases of those functionalities (which are very important to customer) in which we are getting major/critical bugs, the bugs of critical functionalities which are fixed in rush.

Priority-2 – These test cases deliver moderate project value. Executed part of Software Testing cycle and selected for regression testing on need basis.

There are various right approaches to regression testing which needs to be decided on "case to case" basis and we can prioritize the test cases:

Case 1: If the criticality and impact of the bug fixes are LOW, then it is enough a Software Tester selects a few test cases from Test Case Database (TCDB) and executes them. These test cases can fall under any Priority (Priority 0, Priority 1 or Priority 2).

Case 2: If the criticality and the impact of the bug fixes are Medium, then we need to execute all Priority 0 and Priority 1 test cases. If bug fixes need some additional test cases from Priority-2, then those test cases can also be selected and executed for regression testing. Selecting Priority-2 test cases in this case is desirable or optional but not must.

Case 3: If the criticality and impact of the bug fixes are High, then testing team need to execute all the Priority 0, Priority 1 and carefully selected Priority 2 test cases. Priority 2 test cases cannot be skipped in this case. So be careful while choosing the test cases.

Case 4: QA Lead or QA Manager can also go through the complete log of changes happened (can be obtained from Configuration Management Team) because of bug fixes and select the test cases to conduct regression testing. This is a detailed and sometimes complex process but can give very good results. Also don’t forget the One Hour Regression Test Strategy.

This is illustrated in the picture below;

Must Read:

1. The One Hour Regression Test.

2. Regression Testing and Its Best Practices.

3. Extended Random Regression Testing.

4. Difference between Regression Testing and Retesting.

Go back to Software Test Management Tutorials.

Test script defect – Test case defect and Test case Review

2009-09-14T23:13:00.001-07:00

1. What is a test script defect?

During a testing phase, it is not uncommon at all that testers face issues other than application failures. It is believed that during the first cycle of testing, about half the number of reported defects are related to test scripts; i.e. test script defects count about 50% of all the reported defects, while the other 50% are due to software failures and incorrect environment setup.

Test script defect is the discrepancies in the test cases which are prepared by Software testers.

2. What causes test script defects?

The root cause of the defects found in test scripts/test cases can be attributed to the following:

Not fully understanding the requirements or design or any other source documents that the test script is derived from and based on.
Designing test cases require thorough understanding of the application subject to test. Therefore, it is imperative that the test designers should have a clear understanding of the requirements and design flow documents, so they can write correct test cases.
Not working with the latest version of the base documents.
Proper change control and configuration management activities are the absolute necessary efforts in order to prevent the pitfalls of working with old or wrong versions of documents.
Not properly translating requirements and design flows in to test cases and breaking them down to test steps.
The same way a programmer needs to translate software requirement or software design into code, a tester must also be able to analyze a requirement and derive test cases from it. This in turn requires understandable and testable requirements.
Not realizing the person executing the test script could be someone from outside without knowledge of the application under test.
A lot of the time, when testers design test cases, they assume that the only people who will execute their scripts will be their team mates or peers and will have familiarity with the applications. Therefore, the steps are condensed or merged which could appear to be vague for someone less experienced with the applications and thus unable to follow the script to execute the tests.
Not proper use cases and specification documents.

3. Severities of Test Script Defects

Level 1: Issues in test script stopping the tester to carry out the execution.

A defect should be raised and logged and the changes and corrections to the test scripts must be made immediately during the execution phase and the test should be carried out with the new version of the test script.

Level 2: Issue in test script with a workaround, i.e. the tester can identify the issue and is able to continue testing with the workaround:
This is a moderate issue with a medium priority. Of course, if too many workarounds have to be made during the test execution phase, the priority for fixing the test script defects becomes high.

For example, an application requires username, password and a random generated key to verify user credentials. The script only asks username and password to be entered while the application is expecting the random number to be entered as well. The tester can enter the random number as well as the username and password and carryon with the rest of the script.

Level 3: Test script with cosmetic errors / suggestions
Spelling mistakes, missing step numbers or missing information on a section of the documents, e.g. reference to source documents are all minor issues and usually of a low priority.

4. How to prevent Test Script/Test Case Defects?

Before the testing phase begins, all test script documents (amongst other documents) should be subjected to formal reviews to prevent the above issues appearing during the formal testing phase. If at all possible, there should be a “dry-running” of the scripts before the formal test execution begins. This gives the testers a chance to raise any uncertainties or doubts about the nature of the scripts and to minimize the number of issues listed above.

Also testers writing the test scripts must have a thorough understanding of the applications and workflows in order to write effective test cases and to maximize the exposure of defects.

This is a modified article. Original article is by Amir.

Related Posts:

Peer Test Review

Top five suggestions in Software Testing

System Test Coverage Strategies

Prepare Test cases from Use Cases

Reviewing Test Cases

Bug triage Meeting Process

2009-09-12T08:17:00.001-07:00

.. Continuing the Test Management Series

Bug Triage Meeting -

A meeting held by the CFT**, QA Group, Test Manager, Quality Manager, Project Manager; Product Manager and Testing Leads for any project. The objective of the meeting is to prioritize and track the defects to be addressed, ensuring timely and accurate resolution. During iterative Quality Assurance testing, the QA department logs defects. In Bug Triage, the bugs are prioritized to determine when bug fixes are to be releases, the difficulty of the fix and the difficulty of retest.

[**CFT - Cross Functional Team - An inter-departmental team comprising a Cross-Functional Team Leader and team members representing a network of experienced and knowledgeable staff members. The Cross-Functional Team concept allows staffing levels for each development discipline to increase and decrease at the proper times during the development cycle.]

A bug triage meeting should be held regularly during the construction phase (or testing cycle) of a project. The Quality Assurance (QA) lead calls these meetings. The frequency and the number of occurrences will vary from project to project, but is typically based on the number of defects being reported, the overall project schedule, and the current status of the project (i.e. Red, Yellow, or Green Status).

Prior to the meeting, the QA lead will send out a bug report with the new defects reported in the current iteration. At the meeting, the CFT will reassess the severity and priority of each defect.

At the bug triage meeting, the CFT should also discuss the status of defects that were reported at the previous bug triage meeting.

At the bug triage meeting, each defect should be discussed, even those that are rated at a lower priority. The developer should present the level of complexity and the risk associated with fixing each defect. The CFT can then decide which defects should be addressed immediately or those that can wait for future release.

Triaging a bug involves:

1. Making sure the bug has enough information for the developers and makes sense

2. Making sure the bug is filed in the correct place

3. Making sure the bug has sensible "Severity" and "Priority" fields

DECIDING WHICH DEFECTS TO ADDRESS

To understand which defects to address during a project lifecycle, several factors should be considered:

• If the project is in the early iterations, it is feasible to address as many defects as possible, even the lower priority defects.

• If the project is near completion and its final stages of development and QA (i.e. less than 2 iterations left before completion), the CFT should concentrate on addressing only “A priority” defects or those that have low risk (i.e. the fix is not complex and re-testing is minimal).

BUG TRIAGE TEMPLATE

The QA lead will submit a bug report to the CFT prior to the scheduled Bug Triage meeting. This report will contain (but not limited to) the following fields and should be sorted by Fix Priority:

• ID

• Headline

• Date Reported

• Submitter

• Severity

• Fix Priority

• Owner

• Status

DEFECT TRACKING

During the Bug Triage Meeting, the QA engineer will be required to have running on a laptop or desktop. The QA engineer will make the changes / updates to each defect as it is being discussed. The “Comments” field is updated correctly. capture all changes to the defect entry.

At the conclusion of the meeting, the QA engineer will print a report from Defect Tracking System, capturing the updates. This report will serve as the meeting minutes.

ROLES & RESPONSIBILITIES of individuals in Bug Triage Meeting

1. Project Manager

- Assists in the prioritization of the defects

- Sends out meeting minutes when appropriate

- Tracks issues list

- Discusses the delivery date of next iteration to QA.

2. Product Manager

- Assists in the prioritization of the defects.

3. Test Lead (QA Lead)

- Calls the bug triage meeting

- Submits a defect report to the CFT, prior to the start of the meeting

- Assists in the prioritization/severity of the defects

- Assists in determining Root Cause of defect

- Manages defects in CQ

- Distributes updated defect report, capturing the notes from the bug triage meeting.

4. Development Lead and/or Developer

- Assists in the prioritization of the defects (don’t think development sets priority, they should set severity).

- Explains the level of complexity and the risk associated with each defect being presented at the bug triage meeting

- Assigns the defects to the appropriate developer

- Updates Resolution and development notes fields in CQ

- Assists in determining Root Cause of defect

- Discusses the delivery date of next iteration to QA.

5. User Transition Manager

- Ensures that appropriate User Representatives are invited to the bug triage meeting

- Assists in the prioritization of the defects

6. User Representative

- Assists in the prioritization of the defects

OUTPUTS

CFT is aligned on the severity and priority of defects discussed during the bug triage meeting.

EXIT CRITERIA of Bug Triage Meeting

QA lead will distribute the defect list from Defect Management System, capturing the updates.

METRICS –

In the end Bug Triage Metrics are prepared.

Hope this is helpful to all testing professionals in understanding the Bug Triage meeting process.

- Happy Testing

Go Back to –> Software Test Management Suite

2. Software Defect Management Process – Part 1

3. Software Defect Management Process – Part 2

4. Estimation Methods for Defect Estimation

5. Incident Management

Software Test Policy Document

2009-08-30T21:13:00.001-07:00

.. Continuing the Test Management Series

A Test Policy is a high level document and is at the top of the hierarchy of the Test Documentation structure. The purpose of the Test Policy document is to represent the testing philosophy of the company as a whole and to provide a direction which the testing department should adhere to and follow. It should apply to both new projects and maintenance work.

Setting an appropriate test policy by senior managers, provides a robust framework within which testing practitioners can then operate. This will help to ensure the maximization of the strategic value inherent in every project.

Contents of a Test Policy

1. Definition of Testing
Organizations need to be clear why they are testing. This will influence the remainder of the policy document and also the detailed testing techniques that are selected by test managers at the program and project level.
From the understanding of why testing is required it is possible to specify what the purpose of testing is within the organization. Without this fundamental linkage the test effort is destined to fail.

Example: “ensuring the software fulfills its requirements”

2. Description of the testing process
It is vital to establish a solid view towards the test process. We should address questions like, which phases and subtasks will the test process include. Which roles will be involved and the document structure associated with each tasks, as well as what test levels need to be considered.

Example: “all test plans are written in accordance with company policy”

3. Test Evaluation:
How are we going to evaluate the results of testing, what measures will we use to ensure test effectiveness in the project?

Example: “effect on business of finding a fault after its release”

4. Quality Level to be achieved:
Which quality criteria are going to be tested and which quality level is the system required to achieve prior to its release with regards to these criteria?

Example: “no outstanding high severity faults prior to products release”

5. Approach to Test Process Improvement
How often and when are we going to assess the usefulness of the current processes in place and what elements need improving and techniques that shall be used to improve the processes.

Example: “project review meetings to be held after project completion”

Go back to Software Test Management Tutorials.

Checklist for testing Web Applications

2009-08-30T10:42:00.001-07:00

This topic provides information about testing Website for usability, accessibility, credibility, and search engine readiness.

Part A:

1. General Web Site Usability

2. Usability of Web Page Contents

3. Accessibility Review
    a. Browsing with Different Browsing Technologies
    b. Usable and Useful Forms
    c. Accessible Data Tables
    d. Separating Structure and Style
    e. Special Controls
    f. Accessible PDF Documents
    g. Multimedia Content

4. Credibility Issues

5. Search Readiness Review
a. Structuring Pages for Search Engine Optimization

6. Accessibility Testing
a. Recommended Tools for Accessibility Testing
b. Practical Accessibility Test Plan

Part B:
1. Validation
    1. Validate the HTML
    2. Validate the CSS
    3. Check for broken links
2. Flexibility
    1. Try varying window sizes
    2. Try varying font sizes
3. Speed
    1. Access the site via a modem
    2. Check image size specifications
4. Accessibility
    1. Test accessibility
    2. View in text browser
5. Browser independence
    1. Try different browsers
    2. Check printed pages
    3. Switch Javascript off
    4. Switch plug-ins off
    5. Switch images off
6. Other checks
    1. Check non-reliance on mailto
    2. Check no orphan pages
    3. Check sensible page titles

Download the Web testing checklist Guide from here.

Note – This guide (in .pdf format) includes the details of Part A and Part B.

Below are some very basic Web Site Usability rules for newbies:

1. Is the Web site easy-to-use and useful?

2. The most important thing you can do with your Web site is to make it easy to use and useful. Do not distract users; they may not care about your cool technology as much as you do.

3. Does your Web site use a consistent layout on every page?

4. Page links to check: Make sure links to each page of the web site is working and link among the pages are working fine.

5. The relevance of inspection: Add and delete items from a dynamic web site and make sure this does not impact the other items present in the web page.

6. To check the function of the button is correct: If the update, cancel, delete, save and other functions are correct or not.

7. String length check: Put excess of characters a input box should support and make sure it is not taking that. Also make sure if the requirement is 256 characters then it is at least taking 256 characters.

8. With a consistent layout, visitors do not need to keep re-orienting themselves as they move from page to page.

9. Does your home page clearly identify the Web site's purpose?

10. Do not waste a visitor's time by not communicating the purpose of the Web site on the home page.

11. Does your Web site use familiar names and terms?

12. Character type checking: If special character is not allowed then give some special character, negative values and make sure proper error message has been thrown by the system.

13. Punctuation check: Make sure punctuation, space, enter key in the input field not giving any error (unit unless specified they are not allowed)

14. To deal with the Other language characters: If other languages are supported in the input box check whether the system can accept them and they are not garbled.

15. Check out the integrity of information: Update the information through web and then recall that information through search to make sure it is updated property in the back end database.

16. Check to retrieve the information if the names or IDs are case sensitive.

17. Check the deletion function: Delete something and try to retrieve that information. Make sure proper error message is thrown by the system.

18. Acronyms and jargon should not appear on Web pages without definitions or explanations.

19. Does your Web site handle acronyms and abbreviations properly?

20. Do not assume that users understand your abbreviations or acronyms. When using acronyms or abbreviations, you should explain the first occurrence on a page. Subsequent instances should use either the <acronym> or <abbr> tag.

21. Is the department's or customer's logo located in the header or banner of every page?

22. Submit the form multiple times: Fill up a form, submit it and then press browser back button. Try to resubmit the form. See how system behaves.

23. Search examination: Search function is one of the most important function to test. Check search for an item which exists. Check with invalid search criteria and see the error messages.

24. Tab location change: Check the tab location change is proper or not for a particular page.

25. From Download File Checker: Upload a proper file within the size limit and accepted format. Also check with large file, improper file format.

26. Required field validation: Omit some of the required fields and make sure system gives proper error messages.

27. Shortcut keys to check: whether to support the commonly used shortcut keys such as Ctrl + C Ctrl + V Backspace and so on

28. ENTER key to check: In the input directly after the ENTER key to see how the system reacts, whether throws some error.

29. Consistent use of logos helps establish a brand identity and a uniform "look and feel" for your Web site.

30. Does your site use consistent navigation?

31. Users will try to anticipate the location of certain items on your Web site. Make sure the navigation is easy to find and use so that visitors can easily get around your Web site. This is particularly important for visitors who use a screen reader.

32. Does your Web site reasonably limit the number of navigation items in a single group?

33. In general, the maximum recommended number of navigation items is nine.

34. Do all pages on the Web site contain navigational options?

35. Avoid orphaned pages, unless presented in a secondary pop-up browser window.

36. Does your Web site require horizontal scrolling?

37. Visitors can become easily annoyed with horizontal scrolling.

38. Is your Web site printer friendly?

39. It may need a different cascading style sheet (CSS) for printing.

40. Do your pages resize correctly?

41. Use relative sizes in your CSS. Use percentages to specify table widths. Use percentages, em, or the terms larger, smaller, etc. for font sizing.

42. Does your Web site display unnecessary windows or graphics?

43. These include windows or graphics that pop-up or pop-under.

44. Do your pages have a consistently quick download?

45. In general, a page should load in less than 10 seconds.

46. Does your Web site contain links to any necessary plug-ins?

Related Posts:

Selenium Interview Questions

2009-08-30T01:07:00.001-07:00

Posting the Interview questions of Selenium - Automated Software Testing Tool..

What does SIDE stand for?
What is the difference between an assert and a verify with Selenium commands?
What Selenese commands can be used to help debug a regexp?
What is one big difference between SilkTest and Selenium, excluding the price?
Which browsers can Selenium IDE be run in?
If a Selenium function requires a script argument, what would that argument look like in general terms?
If a Selenium function requires a pattern argument, what five prefixes might that argument have?
What is the regular expression sequence that loosely translates to "anything or nothing?"
What is the globbing sequence that loosely translates to "anything or nothing?
What does a character class for all alphabetic characters and digits look like in regular expressions?
What does a character class for all alphabetic characters and digits look like in globbing?
What must one set within SIDE in order to run a test from the beginning to a certain point within the test?
What does a right-pointing green triangle at the beginning of a command in SIDE indicate?
How does one get rid of the right-pointing green triangle?
How can one add vertical white space between sections of a single test?
What Selenium functionality uses wildcards?
Which wildcards does SIDE support?
What are the four types of regular expression quantifiers which we've studied?
What regular expression special character(s) means "any character?"
What distinguishes between an absolute and relative URL in SIDE?
How would one access a Selenium variable named "count" from within a JavaScript snippet?
What Selenese command can be used to display the value of a variable in the log file, which can be very valuable for debugging?
If one wanted to display the value of a variable named answer in the log file, what would the first argument to the previous command look like?
Where did the name "Selenium" come from?
Which Selenium command(s) simulates selecting a link?
Which two commands can be used to check that an alert with a particular message popped up?
What does a comment look like in Column view?
What does a comment look like in Source view?
What are Selenium tests normally named (as displayed at the top of each test when viewed from within a browser)?
What command simulates selecting the browser's Back button?
If the Test Case frame contains several test cases, how can one execute just the selected one of those test cases?
What globbing functionality is NOT supported by SIDE?
What is wrong with this character class range? [A-z]
What are four ways of specifying an uppercase or lowercase M in a Selenese pattern?
What does this regular expression match?
regexp:[1-9][0-9],[0-9]{3},[0-9]{3}
What are two ways to match an asterisk within a Selenese regexp?
What is the generic name for an argument (to a Selenese command) which starts with //?
What Selenese command is used to choose an item from a list?
How many matches exist for this pattern?
regexp:[13579][02468]
What is the oddity associated with testing an alert?
How can one get SIDE to always record an absolute URL for the open command's argument?
What Selenese command and argument can be used to transfer the value of a JavaScript variable into a SIDE variable?
How would one access the value of a SIDE variable named name from within a JavaScript snippet used as the argument to a Selenese command?
What is the name of the type of JavaScript entity represented by the last answer?
What string(s) does this regular expression match?
regexp:August|April 5, 1908
What Selenium regular expression pattern can be used instead of the glob below to produce the same results?
verifyTextPresent | glob:9512?
What Selenium globbing pattern can be used instead of the regexp below to produce the same results?
verifyTextPresent | regexp:Hush.*Charlotte

Related Posts:

Selenium Tutorials

Selenium for Functional testing of web applications

Selenium Tutorials - Discover the Software automation testing power of Selenium

2009-08-29T11:30:00.001-07:00

On the Readers request, we are posting the Selenium Automation Testing tool tutorials. We have complete 10 days tutorials, which helps you in mastering the Selenium tool. Tutorials also include detailed exercises and selenium script examples.

Here are the details:

Day 1: Day one discusses about the Selenium Tutorial Course Details of the next eight sessions, overview of the Selenium, installation of Selenium IDE (Integrated Development Environment). This class covers creating simple automation test scripts using Selenium IDE. Optionally you can install the useful Firefox Plug-ins which will be helpful for your testing. Also, you can create a test suite during this class.

Download the Selenium tutorial 1 for Software Automation Testing.

Password: osst

Day 2: Day two covers the Selenium IDE GUI (Graphical User Interface) features. The GUI Features are divided into two parts. The first part deals with simple features. The second part deals with advanced features. The day two session also focuses on giving high level overview of STLC (Software Testing Life Cycle).

Download the Selenium tutorial 2 for Software Automation Testing.

Password: osst

Day 3: Day three covers about Creating Selenium Test Suites, Selenium Test Runner, Selenium Concepts like Actions, Accessors and Asserts in detail and selenium IDE commands with examples.

Download the Selenium tutorial 3 for Software Automation Testing.

Password: osst

Day 4: Day four covers about Selenium commands, using Regular Expressions in Selenium and narrating test cases in HTML Selenes test cases. Provides an overview of OrangeHRM. Standard Test Case Templates. Students are urged to write their own test cases based on the sample test cases created from previous batches.

Download the Selenium tutorial 4 for Software Automation Testing.

Password: osst

Day 5: Day five covers writing test cases for Orange HRM. Provides an overview of Selenium Core. Installing Selenium Core. Running the test suite in different browsers like IE and Firefox. As an additional piece it discusses how to beatify your test case using CSS (Cascading Style Sheets).

Download the Selenium tutorial 5 for Software Automation Testing.

Password: osst

Day 6: Day six starts with installing Ruby. The Ruby class is divided into two parts, the one which covers the basics of the language. The one which covers the advanced concepts like methods, classes and modules. Some discussion about the books to read and/or buy is added as an additional piece of information.

Download the Selenium tutorial 6 for Software Automation Testing.

Password: osst

Day 7:

Day Seven divided into two parts. First part starts with advanced Selenium IDE concepts like Rollup and provides one sample excercise to cover the rollup feature.
Second part starts with installing Selenium RC. After installation, we'll explore converting an existing test case into Ruby format and running the test cases using RC. Provides overview of Selenium RC. Also, some refence materials provided for Selenium RC commands in different languages. The reference section provides useful tutorial links for different websites.

Download the Selenium tutorial 7 for Software Automation Testing.

Password: osst

Day 8:

Day Eight focuses on using Selenium RC to automate the OrangeHRM test cases. It takes existing 3 OrangeHRM Selenese Test Cases, converts them into Ruby format. Modify the test cases to run using Selenium RC without any problem. Create test suites using two differnt methods namely using "Ruby Require" method and "Ruby Test/Unit module" methods. Create a batch file to run automatically the test suite and to send the results to your e-mail id.
As an added bonus, if time permits, install MySQL Server and use Ruby to create table(s) and do the operations like Insert and Select.

Download the Selenium tutorial 8 for Software Automation Testing.

Password: osst

Day 9: Day ninth focuses on data driven testing on Selenium RC. The test cases focuses on couple of ways to get the data from different sources. The example test cases will focus on CSV (Comma Separated Values), XML and Random Values.

Download the Selenium tutorial 9 for Software Automation Testing.

Password: osst

Day 10: Day tenth class focuses on Selenium Grid.

Download the Selenium tutorial 10 for Software Automation Testing.

Password: osst

Selenium Interview Questions

Selenium for Functional testing of web applications

2009-08-29T10:58:00.001-07:00

Discover the automating power of Selenium

Functional testing and black box is a methodology used to test the behaviour that has an application from the viewpoint of their functions, validated for this purpose various aspects ranging from the aesthetics of the front end, the navigation within your pages between pages within the forms, the compliance of technical specifications associated with fields, buttons, bars, among other pages, entry permits and access to consultations and modifications, management of parameters, the management of the modules that constitute , and other conditions that make up the various "features" expected to provide the system to operate the end user as a normal and correct.

To meet this objective, the tester must choose a set of inputs under certain pre-defined within a certain context, and to check whether the outputs are correct or incorrect depending on the outcome defined in advance between the parties and / or techniques for the customer / supplier.

This form of testing an application is made "outside", that is why "black box testing" because the test covers the roads that follow the internal procedures of the program.

In connection with this test, although there are many tools now that this one out for various reasons will be published in future articles is: Selenium.

Selenium works directly on the web browser, its installation is simple and handling is so intuitive that allows quickly define and configure a test case, recording the journey in a page and then save the sequence of steps as a test script and and then play it when you want.

Selenium is an open-source tool that not only allows the testing of the system but also facilitates the acceptance testing of web applications.

Integrates with Firefox, and includes the ability to write the tests directly in Java, C #, Python and Ruby.

This solution has three basic tools to record a sequence of steps within a website, simulate the test with different browsers and automated test generation.

Selenium IDE is a plug-in for Firefox which allows you to record and execute scripts directly from your browser.

Selenium RC is a library and server written in Java that allows you to run scripts from local or remote through commands.

Grids Selenium: Selenium server allows multiple coordinate in order to run scripts on multiple platforms and devices at the same time.

Download - Selenium IDE is a plug-in for Firefox.

Useful links

Firefox Install Link

Firefox Addons Link

OpenQA WebSite

Software Test Management Suite

2009-08-19T23:04:00.001-07:00

Dear Readers,

We are continuously getting requests for Software Test Management tutorials.

So our team is in the process of compiling good test management tutorials which will be very helpful for you all. Our tutorials will give you control of and visibility into the entire software application testing process i.e. from planning to test-and-fix cycles to final decision. We will provide Software Test Management frameworks which enables you to manage the software testing process, whether your organization is small or large, centralized or distributed.

So here we go..

The Test manager must manage and implement or maintain an effective testing process. That involves creating a test infrastructure that supports robust communication and a cost-effective testing framework.

What the test manager is responsible for:

Planning, deploying and managing the testing effort for any given engagement/release.

Managing and growing testing assets required for meeting the testing mandate:

Retaining skilled testing personnel.

The test manager or lead must understand how testing fits into the organizational structure.

Tutorials on Test Management:

1. Test Estimation and Planning

Complete Tutorial on Test Plan.

Plan Your Software Test Activities

Software Testing Effort estimation

Measuring Test Effectiveness

Software Testing Strategy and Methodology

Traceability Matrix – In Detail

Software Defect Management Process – Part 1

Software Defect Management Process – Part 2

Software Testing Suite – A practical Approach (Contains Software testing framework also)

Software Test Policy Document

Software Bug Management Philosophy

Estimation Methods for Defect Estimation

Incident Management

Bug triage Meeting Process

Test Case Review Process. Defects in the Test cases.

Selecting Test cases for Regression Testing.

Note - This is not the end. We will post some more Software Test Management tutorials in coming days.

- Happy Testing

Team – One Stop Software Testing

Complete Tutorial on Test Plan

2009-08-19T18:33:00.000-07:00

From this, you can prepare testing plan for each type of testing like –
• Test Plan for Web Security Testing
• Test Plan for Functional Testing
• Test Plan for Web Performance Testing etc.

Test Plan Identifier
Each test plan and, more importantly, each version of a test plan should be assigned an identifier that is unique within the organization. Assuming the organization already has a documentation configuration management process (manual or automated) in place, the method for determining the ID should already have been determined. If such a process has yet to be implemented, then it may pay to spend a little time trying to improve this situation before generating additional documentation

Download the complete tutorial from here.

Password: osst

Also See Previous Test Estimation and Planning Topics:

1. Plan Your Software Testing Activity

2. What is Traceability Matrix from Software Testing perspective?

3. Software Testing Strategy and Methodology

Software Defect Management process - Part 2

2009-08-19T11:21:00.000-07:00

Previous Post:

Software Defect Management process - Part 1

Using defects to improve processes is not done by many organizations today, but it offers one of the greatest areas of payback. NASA emphasizes the point that any defect represents a weakness in the process. Seemingly unimportant defects are, from a process perspective, no different from critical defects. It is only the developer’s good luck that prevents a defect from causing a major failure. Even minor defects, therefore, represent an opportunity to learn how to improve the process and prevent potentially major failures. While the defect itself may not be a big deal, the fact that there was a defect is a big deal.

Based on the research team findings, this activity should include the following:
• Go back to the process that originated the defect to understand what caused the defect
• Go back to the verification and validation process, which should have caught the defect earlier. Not only can valuable insight be gained as to how to strengthen the review process, these steps make everyone involved in these activities take them more seriously.
This human factor dimension alone, according to some of the people the research team interviewed, can have a very large impact on the effectiveness of the review process.

NASA takes an additional step of asking the question: “If this defect could have gotten this far into the process before it was captured, what other defects may be present that have not been discovered?” Thus, not only is the process strengthened to prevent defects, it is strengthened to find defects which have been created but not yet discovered. This aggressiveness should be mandatory on life-critical systems.

Download the tutorial from here.

Password: osst

Previous Post:

Software Defect Management process - Part 1

Related Post: Software bug Management Philosophy

Software Defect Management process - Part 1

2009-08-19T11:16:00.000-07:00

Defect Management Process
The general principles of a Defect Management Process are as follows:
• The primary goal is to prevent defects. Where this is not possible or practical, the goals are to find the defect as quickly as possible and to minimize the impact of the defect.
• The defect management process, like the entire software development process, should be risk driven, i.e., strategies, priorities, and resources should be based on an assessment of the risk and the degree to which the expected impact of a risk can be reduced.
• Defect measurement should be integrated into the development process. Information on defects should be captured at the source as a natural by-product of doing the job. It should not be done after the fact by people unrelated to the project or system.

• As much as possible, the capture and analysis of the information should be automated. The QA analyst should look for trends and perform a root-cause analysis to identify special and common cause problems.
• Defect information should be used to improve the process. As imperfect or flawed processes cause most defects, processes may need to be altered to prevent defects.

Download the complete tutorial from here.

Password: osst

Continue Reading: Software Defect Management process - Part 2

Extended Random Regression Testing (ERRT)

2009-07-31T19:42:00.001-07:00

ERRT is a variation of a regression test, which consists of running standard tests from the test library in random order until the software under test fails. An important point to remember: the software under test has already passed successfully those tests in this build! That means that those tests add no more coverage as standard regression tests. ERRT is useful for system-level tests or some very specific unit tests. Typical defects found with this method include: timing problems, memory corruption, stack corruption, and memory leaks.

ERRT exposes problems that can’t be found with conventional test techniques. Troubleshooting such defects can be extremely difficult and very expensive.

Long Sequence Testing

Repeating test cases and critical operations over and over again during long sequence testing is one way to uncover those intermittent failures. Typically, automatically generated test cases are randomly selected from the test repository databank and executed over a very long time.

To test network-centric applications, high-volume long sequence testing (LST) is an efficient technique. McGee and Kaner explored it using what they call extended random regression (ERR) testing. A more promising method to test complex network-centric systems is using genetic algorithms coupled with high volume testing.
Genetic algorithms, in particular, provide a powerful search technique that is effective in very large search spaces, as represented by system environment attributes and input parameters in the testing arena.

Regression Testing and Its Best Practices.

Regression and Re-Testing.

The One Hour Regression Test.

Software Testing Suite – A Practical Approach

2009-07-12T09:34:00.001-07:00

Software Testing Practical Suite - Exclusive for the readers of One Stop Software Testing.

This testing suite explains the basics of practical software testing in plain English. Read these few pages. In less than 1 hour you'll be able to manage your own Test Project!

This contains the following:

1. Software testing e-book

2. Three essential testing templates for managing the testing activities.

The details of templates are explained in the e-book in various chapters. Go through the book from first chapter to last chapter and you are done.

Download the complete suite from HERE.

Open the file with Winzip.

Password of templates: osst

Password of e-book: osst

Templates are in.xls format. after entering the password osst, click on “Read Only” button.

- Happy Testing

Team – One Stop Software Testing

Unit Testing Best Practices

2009-06-25T18:34:00.001-07:00

In software development projects, unit testing is a software validation type where the coder verifies each units of source code versus its specification. A unit is nothing but a most atomic testable part of a developed application program. In procedural computer programming single program, function, procedure, etc. can be considered as unit. In object-oriented computer programming, the smallest unit is a method, that may lie in to a base/super class, abstract class or derived/child class.

As per Microsoft:

The primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as you expect. Each unit is tested separately before integrating them into modules to test the interfaces between modules. Unit testing has proven its value in that a large percentage of defects are identified during its use.

The most common approach to unit testing requires drivers and stubs to be written. The driver simulates a calling unit and the stub simulates a called unit. The investment of developer time in this activity sometimes results in demoting unit testing to a lower level of priority and that is almost always a mistake. Even though the drivers and stubs cost time and money, unit testing provides some undeniable advantages. It allows for automation of the testing process, reduces difficulties of discovering errors contained in more complex pieces of the application, and test coverage is often enhanced because attention is given to each unit.

Here are a few Best Practices:
1. The print out of a unit should reside on a single sheet of paper.
2. If there are more than 4 levels of nested unit, then maybe it should be split in two.
3. Whenever the cyclomatic complexness of a unit is greater than 10 then it's getting excessively complex.
4. Whenever it is content-coupled with another unit then it Is not a good unit.

The process is:
1. Prepare a test Plan of the unit tests.
2. Design the tests, the test environment, and operator instructions.
3. Identify and get the test data.
4. Code and run the tests.
5. Analyse the results.

1. Unit Testing vs System Testing

2. Test Levels

Tips for Software Testing Resume

2009-06-22T09:04:00.001-07:00

The purpose of this article to show you how to write a Software Testing Resume which get the most out of your qualifications. A good software tester resume helps you articulate your skills and opens the door for a job interview.
Lets Check the Sample Software Testing Resume below:

Always use MS Word to write your resume and a good font would be Verdana 10 or Arial 11.

Your Name
1234 Commonwealth Ave
Boston, USA-02134
1-234-567-0171 (Home)
1-234-567-0171 (Cell)
Youremail@gmail.com

Summary:

State this clearly in Bulleted points. This will make your achievements stand out:

Example:
• Has been working in the IT industry for more than eight and a half years and out of that over four years in SQA and testing arena.
• Has the ability and experience to understand the architecture and life cycle of a project in depth and would be an ideal candidate for a process oriented black box as well as white box testing and team management.
• Have in depth knowledge of processes and procedures needed in a professional performance management environment, well versed in testing methodologies as data driven tests, regression tests, code coverage, etc. and Mercury Interactive proprietary Test Scripting Language (TSL).

Skills/Tools:
The mentioned list below is quite comprehensive skill sets. But never list anything or too many thinks, particularly if you are writing fresher software testing resume with the hope that people wont able to catch it. Once you are hired, a falsehood on your resume can be grounds for termination. If your resume is examined as part of your promotion review, you could lose your job if someone finds a lie. Or, if your employer wants an excuse to fire you, he could investigate details on your resume with the hope of finding a lie.

Testing Tools: Win Runner 6.0/5.0, Load Runner 6.0/5.0, Silk Test 5.03, Silk Performer 3.5, Silk Test Radar 2.1, Test Track 5.0, Silk Test 5.03
Languages: Java, C++, C and Power Builder 4.0
Internet: ASP, Java Script, VB Script, HTML, XML and DHTML
GUI: Visual Basic 6.0/5.0/4.0/3.0, Oracle Forms 6i/4.5, Reports 6i/2.5, Crystal Reports 8.0/6.0
Statistical Package: SAS
Web Servers: IIS 5.0/4.0,Vignette Story Server, Web logic Server, SQL Server 7.0/6.0
Middleware: COM/DCOM, MTS
Trackers: PVCS Tracker, SQA Manager and Test Director
IDE’s: Visual Interdev, Home Site and Cold Fusion RDBMS: SQL Server 7.0/6.5, Oracle 8i/7.x/6.0, MS-Access
Operating Systems: Macintosh, Unix, Windows XP/2000/NT/98/95

Experience:

Company Name 1 and web site: It is a good practice to put the web site address so that people can find it quickly. Give the name of the latest company first.

May 2001 – Till date

Project # 1	Customization, development and maintenance of the web enabled fee based portfolio management solutions.
Clients	Give the name of the client
Duration	From September 2001.
Role	QA Lead
Technology	Development: ASP, JavaScript, HTML. Java, .NET, COM, SQL Server, etc. Testing: WinRunner, LoadRunner, Astra QuickTest, LoadTest, PVCS Tracker, Notify, Visual Source Safe, CSE HTML Validator, etc.
Description	Leading a team of 11 testers for various clients. Planning and implementation of tests for various projects at various stages. Generation of manual and automated test plans and test cases. Preparation of Automated Regression Test Suites for multiple projects. Execution of automated regression tests on a day-to-day basis. Assisting team members in generation of WinRunner, QuickTest scripts. Administration of on-site and local installations of PVCS Tracker for defect tracking. Performance testing using LoadRunner and Astra LoadTest. Reporting at multiple levels for multiple projects. Training members of QA Team in WinRunner, TSL, QuickTest and LoadTest. Assisting QA Consultants for the CMMI compliance activities including Gap Analysis, Internal Audit, etc.

Similarly put project # 2 and so on..

You can also use this format :
Company: Company name.
Web site: www.xyz.com
Role: Team Leader
Duration: January 1999 to March 2001.

Responsibilities:
• Leading teams of developers/designers/testers, system analysis, and quality assurance.
• Designing, Planning, Executing White Box and Black Box tests.
• Planning and scheduling multiple projects as well as overall administration and management.
• Development of various e-commerce web sites.
• Administration and Maintenance of Windows 2000 Advanced Server based production network and MS SQL Server based database.
• Maintenance of web sites including content management, online database administration using MS SQL Server Enterprise Manager and performance analysis.
• Remote administration of collocated web server, Windows 2000/IIS5, through PcAnywhere.
• Remote administration of LINUX/Apache server through SSH.

Education:

• Master in Computer Applications (MCA)
• PG Diploma in Computer Application (PGDCA)

Personal Information:
Age and Date of Birth:
Sex:
Marital status:
Nationality:
Place of Birth:
Religion:
Permanent Address:.

Passport Details:
Passport Number:
Date of Issue:
Date of Expiration:
Place of Issue:

Conclusion: Do not make your resume too long. On average try to keep it between 3 pages. Also if it is a Fresher Software Testing Resume then keep it in 1-2 pages only. If it is little more than one page then just try fit it into one page by doing little editing.

Happy Testing ツ

Measuring Software test effectiveness

2009-06-19T20:17:00.001-07:00

The main objectives of testing are to establish confidence and to find defects. This article describes some measures for test effectiveness.

Defect Detection Percentage (DDP)

DPP = Defects known by testing/Total Known Defects

Whenever a piece of software is written, defects are inserted during development. The more effective the testing is in finding those defects, the fewer will escape into operation. For example, if 100 defects have been built into the software, and our testing only finds 50, then its DDP is 50%. If we had found 80 defects, we would have a DDP of 80%; if we had found only 35, our DDP would only have been 35%. Thus it is the escapes, those defects that escaped detection, which determine the quality of the detection process. Although we may never know the complete total of defects inserted, this measure is a very useful one, both for monitoring the testing process and for predicting future effort.

The basic definition of the Defect Detection Percentage is the number of defects found by testing, divided by the total known defects. Note that the total known defects consists of the number of defects found by (this) testing plus the total number of defects found afterwards. The scope of the testing represented in this definition may be a test phase such as system testing or beta testing, testing for a specific functional area, testing of a given project, or any other testing aspect which it is useful to monitor.

The total known defects found so far is a number that can only increase as time goes on, so the DDP computed will always go down over time.

DDP at different test stages or application areas DDP can be measured at different stages of software development and for different types of testing.
• Unit testing. Because testing at this stage is usually fairly informal, the best option is for each individual developer to track his or her own DDP, as a way to improve personal professionalism, as recommended
by Humphreys (1997).
• Link, integration, or system testing. The point at which software is turned over to a more formal process is normally the earliest practical point to measure DDP.
• Different application areas, such as functional areas, major subsystems, or commercial products. The DDP would not be the same over all groups, as they may have different test objectives, but over time this can help the test manager to plan the best value testing within limited constraints.

• DDP of early defect detection activities such as early test design and reviews or inspections. Early test design (the V-model) can find defects when they are much cheaper to fix, as can reviews and inspections. Knowing the DDP of these early activities as well as the test execution DDP can help the test manager to find the most cost effective mix of defect detection activities.

Software Testing Effort Estimating

2009-06-19T19:14:00.001-07:00

Test Estimation Challenges

1. Successful test estimation is a challenge for most organizations because:

a. No standard Formulae/Methods for Test Estimation

b.    Test Effort Estimates also includes the Debugging Effort
c.    Difficult to attempt testing estimates without first having detailed information about a project
d.    Software Testing Myth that Testing can be performed at the End
e.    Difficult to attempt testing estimates without an understanding of what should be included in a 'testing' estimation for a project (functional testing? unit testing? reviews? inspections? load testing? security testing?)

Traditional Practices

1.    Current Test Planning Methods include
a.    Using Percentage of Development Effort
i.    Depends on the accuracy of the Development Effort
ii.    Does not account revisit of Development Effort
b.    Using Tester Developer Ratio
i.    May not be same for all types of Projects
ii.    Does not consider the size of the Project
c.    Using KLOC
i.    Does not consider Complexity, Criticality & Priority of the Project.

What’s the Best Approach to Test Estimation?

There is no simple answer for this. The 'best approach' is highly dependent on the particular organization and project and the experience of the personnel involved.

For example, given two software projects of similar complexity and size, the appropriate test effort for one project might be very large if it was for life-critical medical equipment software, but might be much smaller for the other project if it was for a low-cost computer game. A test estimation approach that only considered size and complexity might be appropriate for one project but not for the other.

Approaches to Test Estimation

Implicit Risk Context Approach:
Metrics-Based Approach:
Test Work Breakdown Approach:
Iterative Approach:
Percentage-of-Development Approach:
Test Estimation Process – A Practical Approach

1.    Combination of all the Approaches
2.    Considers Risk & Complexity Factors
3.    Based on Previous History i.e. Organization or Project Metrics
4.    Based on Work Breakdown Structure
5.    Based on Iterative Model

Elements of Test Estimation Process

1. Break sizing into smaller, easier to estimate tasks
a. Decompose the test project into phases:

i. System Test

ii. Unit Test

i. System Test Planning
ii. Test Execution

i. Executing a test scenario
ii. Writing a defect

2. Taking risk priority into account

3. Set up dependencies

Consider type of code (complex, reused, etc.)
Augment professional judgment and gut instinct with previous project data, industry metrics, and so forth.
Identify and, if possible, resolve discrepancies between the test subproject schedule and the project schedule.
Use the work-breakdown-structure and schedule to develop a budget. Extract from your work-breakdown-structure a complete list of resources. For each resource, determine the first and last day of assignment to the project.
If you have resources shared across multiple test projects within a given time period, understand the percentage allocation of each resource’s assignment to each project during various time periods.
Revisit the Estimation continuously in order to reflect any change in the Project Requirements or Schedule
Be Repeatable preferably Automat able

A Sample Estimation Process

Total work = Test case time + Defect time
Test Case Time = Test Case Development time + Test Case Execution Time
Defect Time = (Hours/Defect * # Defects)

Note: Consider the defect severities while arriving at Hours/Defect and also that Hours/Defect should take into account the Defect creation time, Debugging time and Defect Retesting time till the closure.

Test Case Time

Test Case Development Time = (Hours/Test case development* #Test cases)
Test Case Execution Time = (Hours/Test case Execution * #Test Cases)

Note: Consider the Risk, Complexity while arriving at the Hours/Test case Development and Hours/Test case Execution.

Previous Test Estimation and Planning Topics:

1. Complete Tutorial on Test Plan

2. What is Traceability Matrix from Software Testing perspective?

3. Software Testing Strategy and Methodology

4. Plan Your Software Testing Activities

Plan Your Software Testing Activity

2009-06-19T18:55:00.001-07:00

The success of testing activity generally depends on how well you plan for it. This article provides a few guidelines based on which you can plan your testing activity better. Remember the article is only a guideline to your planning process. You have to innovate and develop your own plan based on the application you are working on.

Build the Plan

1. Analyze the product.

What to Analyze

· Users (who they are and what they do)

· Operations (what it’s used for)

· Product Structure (code, files, etc.)

· Product Functions (what it does)

· Product Data (input, output, states, etc.)

· Platforms (external hardware and software)

Ways to Analyze

· Perform product/prototype walkthrough.

· Review product and project documentation.

· Interview designers and users.

· Compare w/similar products.

Possible Work Products

· Product coverage outline

· Annotated specifications

· Product Issue list

2. Analyze product risk.

What to Analyze

· Threats

· Product vulnerabilities

· Failure modes

· Victim impact

Ways to Analyze

· Review requirements and specifications.

· Review problem occurrences.

· Interview designers and users.

· Review product against risk heuristics and

quality criteria categories.

· Identify general fault/failure patterns.

Possible Work Products

· Component risk matrices

· Failure mode outline

3. Design test strategies.

General Strategies

· Domain testing (including boundaries)

· User testing

· Stress testing

· Regression testing

· Sequence testing

· State testing

· Specification-based testing

· Structural testing (e.g. unit testing)

Ways to Plan

· Match strategies to risks and product areas.

· Visualize specific and practical strategies.

· Look for automation opportunities.

· Prototype test probes and harnesses.

· Don’t over plan. Let testers use their brains.

Possible Work Products

· Itemized statement of each test strategy chosen and how

it will be applied.

· Risk/task matrix.

· List of issues or challenges inherent in the chosen

strategies.

· Advisory of poorly covered parts of the product.

· Test cases (if required)

4. Plan logistics.

Logistical Areas

· Test effort estimation and scheduling

· Testability engineering

· Test team staffing (right skills)

· Tester training and supervision

· Tester task assignments

· Product information gathering and

management

· Project meetings, communication, and

coordination

· Relations with all other project functions,

including development

· Test platform acquisition and configuration

Possible Work Products

· Issues list

· Project risk analysis

· Responsibility matrix

· Test schedule

· Agreements and protocols

· Test tools and automation

· Stubbing and simulation needs

· Test suite management and maintenance

· Build and transmittal protocol

· Test cycle administration

· Problem reporting system and protocol

· Test status reporting protocol

· Code freeze and incremental testing

· Pressure management in end game

· Sign-off protocol

· Evaluation of test effectiveness

5. Share the plan.

Ways to Share

· Engage designers and stakeholders in the

test planning process.

· Actively solicit opinions about the test plan.

· Do everything possible to help the

developers succeed.

· Help the developers understand how what

they do impacts testing.

· Talk to technical writers and technical

support people about sharing quality

information.

· Get designers and developers to review

and approve all reference materials.

· Record and reinforce agreements.

· Get people to review the plan in pieces.

· Improve reviewability by minimizing

unnecessary text in test plan documents.

Goals

· Common understanding of the test process.

· Common commitment to the test process.

· Reasonable participation in the test process.

· Management has reasonable expectations about the test process.

Previous Test Estimation and Planning Topics:

1. Complete Tutorial on Test Plan

2. What is Traceability Matrix from Software Testing perspective?

3. Software Testing Strategy and Methodology