Sahi - Web Automation and Testing Tool

Been a long year!

2011-12-28T19:56:00.000-08:00

2011 has been awesome for us. Tyto Software has grown and become a self sustaining unit with a team that has really come together. Sahi Pro is well received and we released V4.0 recently. We also added a lot of new customers; customers, who have expressed happiness at how our product works and how we support them. We added Flex capability, load testing, better reporting etc. to Sahi Pro, and the results have been very positive. A surprising hit feature was our Excel Framework, which saves a lot of time for users not very comfortable with programming.

While our support is generally well appreciated, we have been trying our best to keep improving on it. One sore point has been our lack of online payment processing methods, which we are trying to alleviate in January 2012.

Over all, we are geared well for 2012, having brought on a couple more to the team and put in processes which make the end user benefit a lot from using our product.

So where is Sahi Open Source? We have been very busy with the Pro version and we should be able to get a breather in January to port back fixes into Sahi OS.

Thank you all for your patronage, and wish you a very happy and prosperous New Year!

Sahi V3.5 2011-03-14 released

2011-03-14T05:47:00.000-07:00

Sahi V3.5 2011-03-14 is now available on SourceForge.

1) We have added support for HTML5 elements, so, for example,
<input type="number" id="myId"> is identified by Sahi as _numberbox("myId")

2) Proxy switching on IE has become a little better

3) _collect and _count have been added.

Eg.

_assertEqual(5, _count("_link", "View Details"));

and

var $els = _collect("_link", "/View/");
var $len = $els.length;
_assertEqual(5, $len);
_assertEqual("View Details", _getText($els[0]));

4) Ignorable ids made a user modifiable property in sahi.properties/userdata.properties. So if you have a new framework which generates ids in a definite pattern, you can ask Sahi to not use them as identifiers for accessors.
Look at recorder.ignorable_ids.pattern in sahi.properties

5) Checkbox simulation has fluctuated a bit in the last few builds of Chrome. Sahi now assumes Chrome 10+ as default. This is modifiable in sahi.properties

simulation.chrome.explicit_checkbox_radio_toggle=true

Change to false if the checkboxes seem to toggle back to original state.

Download the latest version from here: https://sourceforge.net/projects/sahi/files/
and do spread the good word. You can tweet about it, or like us on Facebook or write a blog post about it.

Wish you happy productive testing!

Stable accessors/locators using UI Relations

2011-02-27T19:15:00.001-08:00

Identification of elements in a web interface is one of the toughest challenges of software UI automation. First came accessing by location (x,y coordinates), which quickly faded away due to window resolutions, rendering differences etc. Then came the concept of identification using code structure. XPath became hugely popular;

Reasons being
1) Automation was only attempted by established developers and they understood XPaths
2) 5 years back web applications were simple enough to easily look at the DOM and identify XPaths.
3) Some (silly!) tools adopted and hugely promoted XPaths for want of innovation on that front.

But XPaths quickly became notorious for being unmaintainable and difficult to understand. If test automation was to spread to non-programmer testers, this would be a huge barrier. Tools then tried to move on to css-selectors, which is still complex for the target test automation teams.

So what could be a possible solution? The answer is in the User Interface.

What is often forgotten in the software industry, is that applications are built to satisfy business needs. So whatever the id, XPath or css-selector is, a textbox meant for "loan amount" will always be near some label saying "Loan Amount". An expand or collapse icon in a tree gets its meaning from the attached "User name" label for that node. These business requirements do not change as often as code or underlying HTML structure. An HTML table may get converted to an ext-js grid with scrollbars, but the cost will still be associated (and aligned) with a product and listed under the heading "Cost". Items related to each other would be shown inside a box with an appropriate heading.

How can we use this information to identify elements? Use UI relations for identification. Sahi, which has been a pioneer in web-automation innovation, handles it as below:

A textbox meant for "loan amount" becomes

_textbox(0, _near(_label("Loan Amount")))
or
browser.textbox(0).near(browser.label("Loan Amount"))

The cost becomes

_cell(0, _near(_div("My product one")), _under(_div("Cost")))

The expand collapse icon becomes

_image("/tree-node-icon/", _near(_span("Expand me")))

1) These accessors clearly communicate what the intention of the accessor is, in a straight forward way (within the constraints of programming language constructs).
2) These also take away the complexity in identifying and stabilizing XPaths, or understanding complex css-selectors, or writing custom javascript loops to accomplish the same thing.
3) They are quite stable. _near allows for more elements to come between two related elements, _under is spatially aligned. So you can move different elements around on the UI and still have your scripts working properly.

UI Relations are guaranteed to exist in any business application, irrespective of whether you have ids or not.

One argument that comes up is, what if there are two similar elements which satisfy the same condition. While it is still possible to index the similar accessors and uniquely identify them, it seldom happens in a real world application. If you have two textboxes near "Loan amount", it is going to confuse your end user. User-interfaces do not generally have such anomalies. That said, an added index to the accessor handles that scenario too, if it arises.

There is one other simplification that Sahi does in its accessor APIs. It does not explicitly state what the identifying property is. For example what could have been

_textbox(<where name="username">)

is simply given as

_textbox("username") in Sahi.

Why? Because there are very few cases where one textbox has name="username" and another has id="username". Such naming would obviously confuse the web developer forcing him to not do something like this.

Combined with UI relations, Sahi also takes care of automatically waiting for AJAX requests and page loads. This makes for robust scripts which hardly have unexpected failures. Other features like parallel playback of suites, automatic report generation, ant integration etc. make Sahi one of the most advanced web testing tools in the industry. If you have not tried it yet, download from http://sourceforge.net/projects/sahi/files/ now. You can also use Sahi from inside ThoughtWorks Studios Twist or PushToTest's TestMaker 6.

October 2010 Update

2010-11-08T05:10:00.000-08:00

Hope you had a happy Deepavali.

October continued to be busy. We worked towards the next open source release, with lots of fixes and enhancements. Release was made on 3rd Nov. 2010, just before Deepavali. Part of the focus was on better integration with test management tools. A detailed post on integration shall follow. The new build can be downloaded here: http://sourceforge.net/projects/sahi/files/sahi-v3/2010-11-03/sahi_20101103.zip/download

We did a 2 day online training of Sahi for Advance Internet Inc. We also delivered proof of concept of flex automation to a prospective customer in the banking domain. The flex solution does not need re-compilation of flex code, so may be suitable for environments where modifying code may not be possible.

We could not conduct a webinar last month, but we shall be conducting one on 16th Nov. 2010. Click here to join our webinar: https://www3.gotomeeting.com/register/301637230

We are streamlining the process of support and sales for Sahi Pro online, which should be done by the end of November. Meanwhile, if you have queries on Sahi Pro, please email us at support@sahi.co.in

I did get invited to GTAC on the second day, and managed to talk a little about Sahi's object recognition mechanism. But it would have been much better if I had had time for a full presentation on Sahi. We also had discussions with the WebDriver developers to see if we could adopt WebDriver underneath Sahi. Preliminary investigations reveal that we need to wait some time for WebDriver to mature before we can adopt it. Currently WebDriver supports only one instance of IE on a machine, which may work if you had a Google farm at your disposal, but will not work for Sahi's users who are used to running multiple IE instances simultaneously on a single machine.

September Update

2010-10-05T20:55:00.000-07:00

The past month has been very special for us. We delivered Sahi Pro to our customers, and without even publishing about it, we got many inquiries about Sahi Pro. This was something we had not expected, but would like more of.

Given the number of support queries coming in, we had to scale to better serve our customers. So Boopathi M joins our team as a support engineer. With a background in programming and teaching mathematics online, we look forward to utilizing his tutoring capabilities to solve technical issues in Sahi for our customers.

We also managed to have a presence in Java One, thanks to PushToTest's generous offer to all its partners to show case themselves. Given that Sahi has started playing a good role in TestMaker's web automation part, this turned out to be a good point of synergy.

I (Narayan) also participated as one of the panelists in the Birds Of a Feather session on open source testing tools, thanks again to Frank, CEO of PushToTest. (It is amazing what PushToTest is doing to promote open source products. If ThoughtWorks is the breeding ground for open source projects, I would say PushToTest is one of the few companies that helps them bootstrap and become commercially viable.) We realized Sahi is almost unheard of in the Java developer community! So we are coming up with ways to rectify that. If you have suggestions, do let us know.

This month was also great because we got lots and lots of praise for Sahi (read some testimonials). We had 4 instances of Sahi users reporting that they shifted from QTP to Sahi because QTP had problems identifying some web elements, which Sahi could easily identify. Some of them also had problems on QTP's Windows 7 support.

Selenium continues to annoy us with its presence. We again had 3 different discussions with users who said "We find Sahi to be much easier to use but developers/management (in the US) have heard only of Selenium." A stackoverflow question has considerably undermined Sahi's capabilities and slandered DP, one of Sahi's old and staunch supporters. It was frustrating to know we can't do much about it. Equally unhelpful is the fact that we will not be able to participate in GTAC 2010 because "the committee reviewed each application with special attention towards contributions to the professional community, among other aspects. We regret to inform you that you have not been short-listed to attend this year's conference." We would have loved some discussion with the WebDriver guys.

Our website was hacked twice last month and if you find google search results showing weird titles for Sahi searches, please let us know on the forums or email us.

We have started a facebook page for Sahi, so please do spread the word.

We now have integrations with Test Management tools like XQual, Testuff and Neuvosoft Test Manager, and we will continue to support more. Coming up is integration with QMetry

Things on the roadmap (Oct-Nov):
1) Flex/Flash support(Pro)
2) Simple loadtesting through HTMLUnit (Pro)
3) Integration APIs for Test Management tools (Open and Pro)

We continue with our integrations with PushToTest's TestMaker and Test Object Designer, and with ThoughtWorks Studios' Twist.

We will be starting webinars every two weeks as an introduction to Sahi. The first session is (tentatively) scheduled for 22 October 2010. If interested please email us at support@sahi.co.in

Thanks for reading through. If you have suggestions, queries or advice, please email us at feedback(at)sahi.co.in. Hope to see you next month!

Sahi V3 2010-06-10 released

2010-06-10T22:35:00.000-07:00

Sahi V3 (2010-06-10) is now available for download. https://sourceforge.net/projects/sahi/

There have been significant improvements and bugfixes in this release.

API _under ( http://sahi.co.in/w/browser-accessor-apis ) has been added to locate elements physically under another.
API _byXPath has been added to help users from Selenium and other tools move to Sahi.
API _row and _option have been modified to be in sync with other APIs. Existing scripts may need to be modified if you use these APIs.

Below is the changelog:

10 Jun 2010
-----------


* Bugfixes 
Fixed data truncation bug introduced in 30 Apr build 
Fixed getText bug which returned ab on FF and a b on IE for a<br>b 
Fixed for &nbsp; in select option text 
Fixed window.opener behaviour for link clicks 
Fixed parsing error for $a == $b 

* Features added 
Added Shiretoko as variant of Firefox 
_byXPath added. 
   For browsers without XPath support, 
   download the javascript file from http://coderepos.org/share/wiki/JavaScript-XPath 
   and save the contents in sahi/htdocs/spr /javascript-xpath.js 
_under added as a positional relation 
   Lets identify elements under another element eg. _cell(0, _under(_tableHeader("Status"))) 
5xx errors are displayed on screen too. 
Time taken by each test is displayed in logs. 
XHR redirects are excluded from injection 
Added Driver.setControllerMode. Takes sahi, java, ruby 
_option brought in sync with other APIs. 
   NOTE: older _option(selElement, value) will not work any more 
   Replace with _option(value, _in(selElement)) 
_row brought in sync with other APIs. 
   NOTE: older _row(tablElement, rowNumber) will not work any more 
   Replace with _row(rowNumber, _in(tablElement))

Ruby Sahi with Cucumber

2010-05-10T04:57:00.000-07:00

What is Cucumber?

Cucumber lets software development teams describe how software should behave in plain text. The text is written in a business-readable domain-specific language and serves as documentation, automated tests and development-aid - all rolled into one format.
- From http://cukes.info/

Follow the steps below to get started with Ruby Sahi and Cucumber.

Install Java

Install Ruby

Install cucumber:
```
gem install cucumber
```

Install Sahi proxy: Download Sahi from sourceforge and unzip to some location. (say D:\sahi)

Start Sahi:
```
cd D:\sahi\userdata\bin
start_sahi.bat
```

Install Sahi Ruby client:
```
gem install sahi
```

Create a file D:\test\login.feature, add the content below and save it.


Feature: Login
  In order to access the system
  As a user
  I want to be able to login
  
  Scenario: Login with valid credentials
    Given I am not logged in
    When I try to login with "test" and "secret"
    Then I should be logged in
    
  Scenario: Login with invalid credentials
    Given I am not logged in
    When I try to login with "test" and "wrongpassword"
    Then I should not be logged in
    And I should be shown error message "Invalid username or password"

Run this feature:
```
cd D:\test\
cucumber login.feature
```
There will be a lot of messages with hints on implementing the right steps.

Implement the steps:

Create a file D:\test\login.rb, add the content below and save it


require  'sahi'

def init_browser()
  #Use the correct paths from your system
  userdata_dir = "D:/sahi/userdata"
  browser_path = "C:\\Program Files\\Mozilla Firefox\\firefox.exe"
  browser_options = "-profile #{userdata_dir}/browser/ff/profiles/sahi0 -no-remote"
  return Sahi::Browser.new(browser_path, browser_options)
end

#open the browser at the start
browser = init_browser()
browser.open

#close the browser on exit
at_exit do
  browser.close
end

Given /^I am not logged in$/ do
  browser.navigate_to("http://sahi.co.in/demo/training/index.htm")
end

When /^I try to login with "([^\"]*)" and "([^\"]*)"$/ do |username, password|
  browser.textbox("user").value = username
  browser.password("password").value = password
  browser.submit("Login").click
end

Then /^I should be logged in$/ do
  if !browser.button("Logout").exists?()
    raise "Not logged in"
  end
end

Then /^I should not be logged in$/ do
  if !browser.submit("Login").exists?()
    raise "Logged in"
  end
end

Then /^I should be shown error message "([^\"]*)"$/ do |msg|
  value = browser.div("errorMessage").text() 
  if value != msg
    raise "Incorrect message: #{value}" 
  end
end

Run and watch the tests complete successfully
```
cd D:\test\
cucumber login.feature
```

Done!

Sahi API _under added

2010-05-08T20:51:00.000-07:00

Continuing with our tradition of innovation for simplicity, Tyto adds another wonderful API to Sahi.

NOTE: _under will be available in Sahi's next release

The problem:

Let us take the example of a dynamically generated grid. The example we use here is available at http://www.zkoss.org/zkdemo/userguide/#g7

We wish to assert the value of "Received" column for "Style Guide for ZK 3.5 released". If we bring up the controller and CTRL-hover over that element, what we see is

_div("2008/11/14 13:23:07")

which is not useful as a finder.

Looking at the alternatives listed, we notice that none of them can really help.

To fix this, we shall try using _near.

1) We put

_div(0, _near(_div("Style Guide for ZK 3.5 released")))

into the Evaluate Expression box, and click Highlight. This highlights the "Style Guide" element itself.

2) We experiment with the index passed as the first parameter, and using Highlight, pinpoint on the correct accessor as
_div(4, _near(_div("Style Guide for ZK 3.5 released")))

But this will not make a good accessor.
Why?
Because, this uses an index which seems like it would change when another column is added before the Received column.

What we really want, is that element, which is UNDER _div("Received").

Introducing the _under API

_under(element):

_under is a POSITIONAL marker. What it means is that it checks for coordinate based alignment under a particular element within a specific threshold.
So, in our case, it will look for a div which is roughly positioned underneath _div("Received")

Here is the final accessor:

_div(0, _near(_div("Style Guide for ZK 3.5 released")), _under(_div("Received")))

Note how this accessor is independent of the order of the rows and the columns.

And then the testers lived happily ever after ...

Notes:

This grid is not a simple table but actually composed of 2 tables, one for the header and one for the contents. So we could have approaced this problem using _cell(_table(2), "Style Guide for ZK 3.5 released", 4), but again the 4 would trip us later if the order/number of columns changes.

_under(el) can be passed as a last parameter to any Sahi API.

NOTE: _under will be available in Sahi's next release

Choosing the right web automation tool or web testing tool

2010-05-03T21:29:00.000-07:00

Web automation is a little trickier than most other automation because there are many combinations of browsers and operating systems and they are fast evolving too.

What do you look for before you choose a tool for web automation?

The answer may actually depend on what your organization specializes in. If you are a product company which ships applications meant only for Internet Explorer, you need not consider multi-browser support or Linux support. But if you develop outward facing web applications, you may need to test the application on multiple browsers.

Here are some factors you should consider before choosing a testing tool.

The tool:

Should be techincally sound
1. Should be able to identify elements/record on all browsers.
2. Should handle complexities like HTTPS, Frames, IFrames, AJAX, dynamic ids.
3. Should not need tinkering with source code of tool
4. Should not require hard coded waits

Should save time and effort for teams
1. Ramp up time should be minimal. Users should get productive within an hour.
2. Complexities like AJAX, dynamic ids, object identification etc. should be handled by the tool instead of passing it on to testers. *
3. Should run reliably across browsers and operating systems to reduce re-runs and debugging effort.
4. Should not be dependent on knowledge of various other tools/technologies.
5. Should need minimal maintenance of scripts/code

Should work with existing teams instead of requiring a drastic overhaul
1. Should not require your teams to change from testers to developer testers, but let them easily pick up some scripting knowledge and get functional.
2. Should not require expertise in various peripheral technologies like Java, Junit, TestNG, XPath, Firebug, Browser DOM etc. to just get started.

Should require minimal stakeholders
1. Should not need developer involvement for modification to application in the name of “testability”. Dynamic ids, elements without ids, etc. should be handled well by the tool. *

Should be easy to scale testing teams
1. Should be easy to hire and add more members to your testing team. This requires the tool to be simple to use.
2. Should be able to move the teams across projects and products. This means that the tool needs to be sound enough to work with various technologies and frameworks.

Should have authoritative support available

Should be cost effective. The following need to be considered:
1. Cost of acquiring the tool
2. Cost of employing capable testers who can use the tool
3. Cost of maintaining test infrastructure
4. Cost of authoritative support
Too often, especially with open source tools, the amount of money wasted in man hours due to limitations of the tool, incompatibility with existing expertise of team, lack of support etc. far outweighs the cost of acquiring alternative commercial tools. (Developers and testers with not much business experience invariably think that their time is not a cost to their company, and do not mind spending a week on an effort which should have lasted a day, thus wasting 25% of a month’s salary for a tool which may cost 10%)

* It is possible to just use the Sahi Controller and identify various elements reliably. Because tools like Selenium cannot record across frames, iframes, the tester is forced to learn to use Firebug to figure out what ID or XPath to use, add a line of selectFrame etc. These are very tool specific. While learning to use Firebug is an awesome skill to have, it should not be required at each step of the automation process. Adding conditional waits with knowledge of DOM is an unnecessary effort put on the tester, which can be handled by intelligent tools. Same goes for making developers add custom id generators for handling dynamic ids.

Sahi vs. Selenium

2010-04-24T20:58:00.000-07:00

Though Sahi is aimed at non-programmer testers and Selenium at programmers, we get a lot of queries on their differences. Here is a brief document which compares Selenium and Sahi.

To those who read this:

If you are a Selenium fan, be open minded and verify the claims for yourself or contact us for clarification. And again, please be open minded. You may save a lot of time for yourself and your team.
This document is biased towards Sahi because we built Sahi. We believe what we state is true, but if you have evidence to prove otherwise, please do contact us at support@sahi.co.in.
When we say "Not sure" or "?" it means we do not have enough information because of lack of research on our part. Please verify for yourself.
If you want a comparison between other tools and Sahi, we would be happy to discuss it.
If you are another tool developer/supporter, please let us know how it compares and let the world benefit from alternatives.

Sahi vs. Selenium: Comparison document

Recorder

Selenium	Sahi
Works only on Firefox	Works on all browsers (IE, FF, Chrome, Safari, Opera)
Has trouble recording IFrames, Frames and popup windows	Can record on IFrames, Frames and popup windows
For Frames and IFrames, need to explicitly selectFrame	Implicit smart identification of elements even across Frames and IFrames
Uses XPath for identification of elements if id or name not present	Uses various algorithms to uniquely identify elements in a simple human recognizable way

Programming Language support

Selenium	Sahi
Java, Ruby, Perl, Python, C# (and may be more).	Sahi Script, Java, Ruby Sahi Script has the syntax of javascript but can interoperate with any Java code directly from script. The Java/Ruby drivers are available since Sahi V3
Needs language bridges for each new feature. For example, needs java bridge to invoke Flash via ExternalAPI.	Sahi Script can directly invoke anything exposed by javascript.

Ease of use

Selenium	Sahi
Easy to start with because of Selenium IDE which is a firefox plugin. Estimated start time less than 5 minutes	More difficult than Selenium to start because it needs installation of Sahi. Estimated start time 10-30 minutes, depending on Java installation etc.
Deep learning curve when the need is felt to move from Selenium IDE to Selenium RC.	There is only one mode of operation for Sahi. Extremely simple to learn and use for testers
Knowledge of programming language required	Can achieve most automation with just functions and variables. Sahi has inbuilt APIs for most other complex tasks like data driven testing
Needs JUnit style of programming	Can choose your own style
Uses XPath based identification for elements in complex html structures or those with dynamic ids. css selectors and javascript may also be used.	Has nearness APIs like _in and _near which can help show nearness of elements. Eg. _image(“delete.gif”, _near(“username 4”))
Needs waits for making AJAX work	No waits needed in 90% cases
Supports parallel execution	Inbuilt parallel execution. Needs only one parameter change

Stability of scripts and ease of maintenance

	Selenium	Sahi
Smart DOM Relations resilient to UI changes	No Dependent on XPath Difficult for testers to understand and debug	Yes Does not use XPaths. Uses _near and _in
Implicit waits for page load and AJAX: 1) Saves time 2) Keeps scripts simple 3) Reduces random failures	No Explicit waits needed.	Yes
Ease of adoption by a team of testers	Needs testers to know TestNG/Junit, XPaths, HTML structures, Frames IFrame knowledge, Javascript for AJAX conditional waits	Sahi abstracts out all these for the tester.

Dependency on other tools

Selenium – Java (Others need something similar)	Sahi
Needs JUnit (and optionally eclipse) to run tests	No additional tools required. Tests run from the Sahi Controller/command line/ant
Non persistent reporting. Needs TestNG or something similar for that	Persistent HTML reporting which can be shared via URL or file

Stability of product and number of releases

Selenium	Sahi
Started 2004(?) in ThoughtWorks	Started 2005 in ThoughtWorks
Version 1 took 5 years, Version 2 planned mid-2010. Moving away from original architecture to WebDriver based architecture	Current release: Version 3 Number of stable releases in 2009: 7

Footprint

Selenium	Sahi
RC: 10.5 MB, Grid 15 MB	less than 2 .5 MB with source
Not sure	Runtime ~ 50MB for 3 parallel threads

Reporting

Selenium	Sahi
Needs external tools to create readable reports	Inbuilt HTML reports with click through to relevant portion of script

Others

	Selenium	Sahi
Build tool integration (ant, batch files)	Yes	Yes
Multiple OS support	Yes	Yes
Version Controllable Scripts/Code	Yes	Yes
HTTPS support/redirects	Not sure	Yes
401 Authentication, Windows/NTLM Authentication dialogs	Not sure	Yes
External proxy tunneling	Yes	Yes
In built APIs for data driven testing	No	Yes
Works only with browsers	Yes	Yes
Needs privileged modes on browsers for operation. (Privileged is bad)	Yes	No
Extensible on future browsers	Depends on finding a way to use privileged mode on that browser	Yes. Very little dependency on type of browser.
Editor support	Has good editors in various languages	Editor support for javascript is not as good as for Java.

Support available

	Selenium	Sahi
Free support via Forums	Yes	Yes
Paid support available	Yes	Yes
Authoritative training available	?	Yes

Presentation at XP Goa day

2010-04-20T07:40:00.000-07:00

Narayan Raman presented on Functional Testing of Web Applications using Sahi at the XP Goa Day in Goa University.

Goa XP day

View more presentations from narayanraman.

As part of a presentation , we did a small demo on record and playback of a script using Sahi, then refactored the code to be maintainable.

The site under test is available here: http://sahi.co.in/demo/training
The first cut from the recorder came out to be this:


_setValue(_textbox("user"), "test");
_setValue(_password("password"), "secret");
_click(_submit("Login"));
_setValue(_textbox("q"), "2");
_setValue(_textbox("q[1]"), "1");
_setValue(_textbox("q[2]"), "1");
_click(_button("Add"));
_assertExists(_textbox("total"));
_assert(_isVisible(_textbox("total")));
_assertEqual("1150", _textbox("total").value);
_click(_button("Logout"));

This was then refactored into 2 scripts, one containing functions and the other invoking it:


// goa3_included.sah
function login($username, $password){
 _setValue(_textbox("user"), $username);
 _setValue(_password("password"), $password);
 _click(_submit("Login"));
}

function addBooks($numJava, $numRuby, $numPython){
 _setValue(_textbox("q"), $numJava);
 _setValue(_textbox("q[1]"), $numRuby);
 _setValue(_textbox("q[2]"), $numPython);
 _click(_button("Add"));
}

function verifyTotal($total){
 _assertEqual($total, _textbox("total").value); 
}

function logout(){
 _click(_button("Logout"));
}


// goa3.sah
_include("goa3_included.sah");

login("test", "secret");
addBooks(2, 1, 1);
verifyTotal(1150);
logout();

The next step was to modify


function addBooks($numJava, $numRuby, $numPython){
 _setValue(_textbox("q"), $numJava);
 _setValue(_textbox("q[1]"), $numRuby);
 _setValue(_textbox("q[2]"), $numPython);
 _click(_button("Add"));
}

such that identifiers "q", "q[1]" and "q[2]" become more meaningful and are independent of their order. Using the _near API, the function becomes:


function addBooks($numJava, $numRuby, $numPython){
 _setValue(_textbox("q", _near(_cell("Core Java"))), $numJava);
 _setValue(_textbox("q", _near(_cell("Ruby for Rails"))), $numRuby);
 _setValue(_textbox("q", _near(_cell("Python Cookbook"))), $numPython);
 _click(_button("Add"));
}

We then data drive the whole test by wrapping the various steps into a single function "addAndVerify", build a 2 dimensional array of values and then invoke "addAndVerify" for each row of values using _dataDrive


// club the functionality into a single function
function addAndVerify($numJava, $numRuby, $numPython, $total){
 login("test", "secret");
 addBooks($numJava, $numRuby, $numPython);
 verifyTotal($total);
 logout();
}

// build a 2D array
var $data = [
[2, 1, 1, 1150],
[3, 2, 1, 1650],
[1, 1, 1, 850]
]

// automatically invoke addAndVerify for each row in $data. 
_dataDrive(addAndVerify, $data);

We concluded the talk with an enthusiastic Q & A session.
Thank you Goa University for being a great host!

ThoughtWorks Studios' Twist 2.0 with Sahi

2010-03-02T19:06:00.000-08:00

ThoughtWorks announces Twist 2.0 availability from 31st March 2010:
http://www.prnewswire.com/news-releases/thoughtworks-studios-new-twist-20-provides-collaborative-agile-test-management-85809582.html

Tyto Software has been collaborating with ThoughtWorks Studios to integrate Sahi with Twist and results will be visible in Twist 2.0.

"Twist 2.0 has added Sahi as an additional option for web testing. The main benefit of Sahi is that it abstracts out most difficulties that testers face while automating web applications. Its features include an excellent recorder, platform and browser independence, no XPaths, no waits and multi-threaded playback. In addition, it allows you to identify UI components within the application as you record test scenarios." Announcing Twist 2.0: Available for download on March 31

Sahi automating ZK 3.6 Sample Application - Demo video

2010-02-24T05:26:00.000-08:00

Here is a brief video which shows how Sahi can be used to automate ZK based applications. Notice how Sahi uses neither XPaths nor explicit waits, and works with the same script on both Firefox and IE.

ZK testing with Sahi

2010-02-14T09:21:00.000-08:00

Joseph Neuhaus has written a detailed article on testing ZK applications with Sahi. The article explains how to run Sahi tests headless on a linux machine.

Excerpts from the well written, thorough article:

If you have attempted to create browser-based functional tests with Selenium, or load tests using Grinder, then you will marvel at the simplicity of Sahi.

The Problem
Testing ZK Web Applications can be a challenge with Selenium and Grinder because ZK dynamically generates element IDs, and these testing frameworks identify elements within the rendered Web Page using these IDs. Therefore, once a session has been recorded, it cannot be replayed because the element IDs will change the next time the Web Application is launched. To address this, ZK provides a hook so you can generate your own IDs. This is a great feature to be sure; however, now you're not testing the application that will be promoted into production. Also, keeping track of "special" test builds of your application increases work and complexity. To make matters worse, some testing frameworks, such as Selenium, require you to install browser plugins to create the recorded browser sessions used for playback. When browser updates occur, you can't upgrade until the plugins are updated too. More moving parts means more issues maintaining the Test Environment. There must be a better way, and there is. It's called Sahi.

The Solution
Sahi can record and playback sessions directly on your production ZK Web Application without having to use a custom ID generator. It requires no browser plugins to create recorded sessions to be used for playback. Sahi is pure Java, so it integrates nicely with ANT. The Sahi scripting language is simple and elegant, so there's no need to break out the XPATH documentation to get your tests running. You can even run your Sahi tests headless using Firefox in an X-Window virtual frame buffer on Linux. In headless mode you can run real browser-based tests on your continuous integration machines without having to be logged in. If you're testing on windows, you can run your tests on IE, Firefox, Chrome, Safari, and Opera. Despite some idiosyncrasies Sahi seems magical at times, but more importantly it makes testing productive - so productive it's almost fun!

The full article is available here: http://docs.zkoss.org/wiki/Making_ZK_Functional_Tests_With_Sahi

Joe Neuhaus has over twenty years of experience in software development, systems design, and technical management roles.

Narayan Raman wins Safari Books Online's Coder Challenge for Sahi

2010-02-10T20:26:00.000-08:00

Narayan Raman from Tyto Software won Safari Books Online's Coder Challenge for his contribution to Sahi.

What followed was a day of good fun, interactions with the other winners, wine tasting at Kendell Jackson and Korbel wineries, a pleasant dinner and lots of discussions with Tim O'Reilly, O'Reilly Media, Paige Mazzoni, VP marketing, Safari Books Online, and CJ Rayhill, senior VP, product management and technology, Safari Books Online.

Narayan Raman and the other winners also won a three month subscription to Safari Books Online.

Narayan Raman, Ashley Aberneithy, Tim O'Reilly, Aral Balkan and Arturo Fernandez-Sanchez

CJ Rayhill, Narayan Raman, Ashley Aberneithy, Tim O'Reilly, Aral Balkan, Arturo Fernandez-Sanchez and Paige Mazzoni

Press release: http://www.businesswire.com/portal/site/home/permalink/?ndmViewId=news_view&newsId=20100202005735&newsLang=en

Thank you Safari Books Online for the award and the excellent hospitality.

More pictures on Flickr

Sahi in "Nagios - The Practice Book: Open Source Monitoring in the Enterprise"

2010-01-07T00:00:00.000-08:00

There is a fairly detailed section on monitoring web applications using Nagios and Sahi in Gerhard Lausser's new book (in German): "Nagios - The Practice Book: Open Source Monitoring in the Enterprise", ISBN: 978-3-8273-2800-7

http://www.amazon.de/Nagios-Praxisbuch-Open-Source-Monitoring-Unternehmen/dp/3827328004

Web automation does not need XPaths

2010-01-05T09:17:00.000-08:00

Really.

Learn this web automation nursery rhyme today!

XPaths are evil,
XPaths are fickle,
Developers touch code,
And the testers are in a pickle!

Have you ever used XPaths and found that it needs non-trivial amount of effort in maintenance?

Especially testers, who do not have the time or energy to get XPaths right, stay away from XPaths. Use ids or names or any other attribute the web element provides.

Sahi, since it is aimed at testers rather than developers, does not encourage use of XPath, which means that you can use it if you want, but the controller will never show you XPaths.

So how does Sahi handle something like this?

My name		Edit
Your name		Edit
His name		Edit

Simple, Sahi uses the _in and _near APIs.

So to access the edit link of Your name, use

_link("Edit", _near(_cell("Your name")))

Like wise

_link("Edit", _near(_cell("My name")))
_link("Edit", _near(_cell("His name")))

Visit us again or subscribe to this blog for more tips on web automation ...

Sahi Nightly Release 2009-07-15

2009-07-17T03:43:00.000-07:00

A new nightly build is available at https://sourceforge.net/projects/sahi/files/
This fixes a file upload issue and adds support for 401 Authentication and HTTPS Client Certificates.

* Features added
Support for 401 authentication. A dialog box is shown on the browser for authentication.
Support for HTTPS client certificates. Look at ssl.client.* settings in sahi.properties
_hidden(identifier) added
_byClassName(className, tagName [, domRelation]) added

* Bugfixes
checkbox onchange triggered for IE.
Content-Length removed from parts of MultiPartRequest

Parsing XML in Sahi scripts

2009-06-26T02:22:00.000-07:00

Sahi uses Rhino as its javascript engine and Rhino has excellent support for handling XML.

Below is a script which reads and asserts XML nodes and attributes. The example has been picked from http://www.ibm.com/developerworks/webservices/library/ws-ajax1/ so that it is easy to experiment with the ibm examples in this script.


var xmlStr = '' +
'<people>' +
'   <person gender="male">' +
'      <name>Ant</name>' +
'      <hair>Shaggy</hair>' +
'      <eyes>Blue</eyes>' +
'      <height measure="metric">176</height>' +
'   </person>' +
'   <person gender="male">' +
'      <name>Paul</name>' +
'      <hair>Spiky</hair>' +
'      <eyes>Grey</eyes>' +
'      <height measure="metric">178</height>' +
'   </person>' +
'</people>';

var $x = new XML(xmlStr);
_assertEqual("Ant", $x.person[0].name.toString());
_assertEqual("Grey", $x.person[1].eyes.toString());

for each (var $p in $x.person){
    var $measure = $p.height.@measure.toString();
    _assert($measure == "metric");
    _assert($p.height > 170);
}

Two points to note:

1) All nodes that you access are of type xml. You will need to use toString() on them before you assert them.

2) Using @ from inside a Browser Action Function (like _click, _assert etc.) causes the script to fail because of a parsing error in Sahi's code. So first assign it to a variable and then use it, like it has been used for $measure. This bug will be fixed in the coming release.

There is a lot more that can be done with the XML object. Have a look at these links:

http://www.ibm.com/developerworks/webservices/library/ws-ajax1/

http://www.xml.com/pub/a/2007/11/28/introducing-e4x.html

Configuring Eclipse for Sahi

2009-06-23T13:14:00.000-07:00

Eclipse's JSDT plugin provides syntax highlighting and verification for Javascript. It can be configured to work well with Sahi scripts too. This video goes through the different steps to configure Eclipse to work with Sahi scripts.

JSDT looks at all functions in the given source folder and can list them in content-assist. Taking advantage of this, we use a dummy definitions file called apis.sah which has all the Sahi APIs. This file, apis.sah, can be downloaded from here and needs to copied into the "scripts" folder.

JSDT is a part of the Web Platforms Toolkit.

It can be installed as a plugin or can be downloaded as a single bundle in the "Eclipse IDE for Java EE Developers (163 MB)" (For windows: eclipse-jee-ganymede-SR2-win32.zip). This is downloadable from http://www.eclipse.org/downloads/

Increasing interaction with Sahi users

2009-06-04T21:59:00.000-07:00

Sahi has been around for about 4 years now. It started in 2005 and was released before or just around the time Selenium RC's first cut was released. But there has been a huge gap in the visibility of Sahi and Selenium. While Selenium seems to be everywhere, Sahi is hardly heard of. After a lot of conversations with different people, this is what we learnt.

1) Sahi is aimed at testers and not developers. Its focus on strong record and playback and on simple scripting, is primarily meant for testers in the Indian industry. These are people who are good at manual testing and automation with tools like QTP, but are not from a programming background. Selenium and Watir on the other hand are aimed at testers and developers who are fine with programming. Unfortunately for Sahi, testers in India who use Sahi, do not blog or voice their opinions on the internet, or contribute back in forums. (Most of those who do, are good technical testers.) That meant that there was no viral aspect to the spread of Sahi.

2) Very little emphasis was made on Sahi's website or in the program to collect user data. No testimonials, no newsletters, which meant that people were not informed of developments and good user stories to boost their confidence. While the program continued to evolve with releases every two months, few people really saw the progress.

3) Being aimed at testers and using javascript for scripting, and because of our own lack of savvy in marketing to the internet, Sahi could never become sexy to the blogging bunch. Who would notice when there was Selenium and Ruby to talk about, which could get you more hits and popularity?

While the first and third problem cannot be solved easily, the second is quite addressable. A new section for testimonials and user stories has been added to Sahi Forums. This blog you are reading is being updated with new developments. Updates and news of Sahi can be followed via twitter at _sahi. Presentations on Sahi will be available here

If you are a Sahi user, tweet with #sahi so others may know. If you have a user story do post on the forums.

HTTPS Problem Resolution: Unable to tunnel through proxy

2009-06-03T04:46:00.000-07:00

Sahi had been using its own custom implementation of proxy tunnelling till a few months back. Owing to a lot of demand for some features, we moved to Java's httpsurlconnection which supported tunnelling through a corporate proxy with authentication.

But unfortunately a bug in Java's httpsurlconnection was tripping up a few users on some https sites. One case was the failure of websites using login via SiteMinder.

The exception thrown was

Unable to tunnel through proxy. Proxy returns "HTTP/1.1 400 Bad Request"

After some research we figured that it was due to this bug 6687282

Switching to the latest java 1.6.0_14 fixed this issue for us.

Sahi V2 Release 2009-05-21 is now available!

2009-05-21T08:41:00.000-07:00

Sahi V2 2009-05-21 is a stable build.
Please use and give feedback to make next release more stable. Download here

* Features added
Optimizations to increase speed.
Optimized listing of log files.
Added better error messages for el = null errors.
Added tests for failing features.

Added _parentNode(el, tagName, occurrence),
Added _parentCell(el, occurrence)
Added _parentRow(el, occurrence)
Added _parentTable(el, occurrence)
Added _cell("cellText")
Added _in() and modified other accessors to take inElement as parameter

saveAs looks at Content-Disposition: attachment to download files
Added tests around accessor identification
Test and highlight on the Recorder tab work on selected text
Temp files saved via "Save As" have session id prefixed to prevent overwrite.

_near
_rte for richtext editors based on iframe
Reset button added to Playback. Does not refresh page.
_scriptPath added
_type added to type partial content without blur or onchange.
moved to chunked encoding

Added element.visibility_check.strict = false to sahi.properties. This is the default now.
Added download.download_if_contentdisposition_is_attachment = false to sahi.properties.
Set this to true to automatically download files with header Content-Disposition:attachment.
Causes problems with blogger.com profile images.

Stubs for Sahi's APIs and basic objects like window and document added.
This allows variable declarations like
var $ln = _link("Form Test");
_click($ln);
Added xhr.wait_when_ready_state_3 to sahi.properties.
Set to false for cases if some XHRs never reach readyState 4. Needed for gmail.

* Bugfixes

Playback to recorder tab switch when moving between multiple domains during playback fixed.
Messages displayed multiple times during retry of step removed. Shows only once.
Fixed NaN error in _logException and _logExceptionAsFailure.
Fixed Content-Type related errors for state.js.
Fixed order of response headers (SignIn Cookie problem).
Log viewer shows file not found message and does not throw exception on console.

Autosuggest on Controller Accessor field does not disappear to the right.
APIs dropdown only lists public APIs of Sahi
_imageSubmitButton indexes fixed. (Treats alt and title equivalently)
saveAs NPE fixed

Fixed traffic logging. It had stopped working after moving to streams.

Divs are identified as _div instead of _spandiv
Spans are identified as _span instead of _spandiv

Sahi Forums updated

2009-05-20T01:17:00.000-07:00

We have upgraded the PunBB version of Sahi Forums. Kudos to the PunBB folks for creating such a nice light weight forum software.

The new Sahi forums have a few nice features, namely:

Tagging topics.
Sending private messages.
Ability to mark a topic "Closed".
Private forums for paid customers.
Adding attachments (for paid customers).

Have a look at the new forums at http://sahi.co.in/forums

Comments and suggestions are welcome.

Sahi V2 Nightly Unstable Build 2009-04-23 Released

2009-04-22T23:03:00.000-07:00

Sahi V2 Nightly Unstable Build 2009-04-23 has been released. (Download)

This build has a few significant improvements. It now uses Rhino 1.6R2 as its Javascript engine.

NOTE that this and further builds need Java 1.5 or greater.

* API _near, similar to _in has been added. Any element can be found relative to another by using _near.
Eg.


_checkbox(0, _near(_span("user name 1")))
_link("delete", _near(_span("user name 1")));

* All Sahi accessor API calls can be set to variables now.
Eg.


_click(_link("click me"));

can be written now as


$ln = _link("click me");
_click($ln);

* Check for visibility of elements is now controlled via element.visibility_check.strict property in sahi.properties. It is set to false by default.