Web applications testing and Selenium blog by Deep Shift Labs http://www.deepshiftlabs.com/sel_blog Thu, 25 Apr 2013 05:54:59 +0000 en-us hourly 1 http://wordpress.org/?v=3.4.1 Virtualizer toolbox – grow file system on Fedora on VMWare http://feedproxy.google.com/~r/WebApplicationsTestingAndSeleniumBlogByDeepShiftLabs/~3/UZxkVd8UXe4/ http://www.deepshiftlabs.com/sel_blog/?p=2925&lang=en-us#comments Thu, 25 Apr 2013 05:54:59 +0000 Igor http://www.deepshiftlabs.com/sel_blog/?p=2925 In this post I would like to show how to increase file system space on installed Fedora VMWare instance. You might need it if you use VMWare (and we know many people do) to spin VMs for Selenium testing.

In this particular case I use I installed Fedora and allocated 10GB but later I started using this VM for quite large databases and free space quickly disappeared.

View Code SH
# df -k
Filesystem                    1K-blocks    Used Available Use% Mounted on
devtmpfs                         498132       0    498132   0% /dev
tmpfs                            508060       0    508060   0% /dev/shm
tmpfs                            508060     772    507288   1% /run
/dev/mapper/vg_fedora-lv_root   7641992 6590912    656228  91% /
tmpfs                            508060       0    508060   0% /sys/fs/cgroup
tmpfs                            508060       0    508060   0% /media
/dev/sda1                        487652   84379    377673  19% /boot
View Code SH
# fdisk -l
 
Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders, total 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e3a8c
 
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048    20971519     9972736   8e  Linux LVM
 
 
Disk /dev/mapper/vg_fedora-lv_swap: 2113 MB, 2113929216 bytes
255 heads, 63 sectors/track, 257 cylinders, total 4128768 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
 
 
Disk /dev/mapper/vg_fedora-lv_root: 8086 MB, 8086618112 bytes
255 heads, 63 sectors/track, 983 cylinders, total 15794176 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

To grow disk space I went to VMWare console and added one more disk as shown below:
 
 
Adding a disk to existing VMWare VM
 
 
Now we can see a new disk in “fdisk -l” command output.

View Code SH
Disk /dev/sdb: 21.5 GB, 21474836480 bytes
255 heads, 63 sectors/track, 2610 cylinders, total 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Now we create partition sdb1 covering all new disk using “fdisk /dev/sdb”.

View Code SH
Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p):
Using default response p
Partition number (1-4, default 1):
Using default value 1
First sector (2048-41943039, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-41943039, default 41943039):
Using default value 41943039
Partition 1 of type Linux and of size 20 GiB is set
 
Command (m for help): v
Remaining 2047 unallocated 512-byte sectors
 
Command (m for help): w
The partition table has been altered!

Finally we just expand our volume group and file system:

View Code SH
pvcreate /dev/sdb1
vgextend vg_fedora /dev/sdb1
lvextend -l +100%FREE /dev/mapper/vg_fedora-lv_root
resize2fs /dev/mapper/vg_fedora-lv_root
]]> http://www.deepshiftlabs.com/sel_blog/?feed=rss2&p=2925&lang=en-us 0 http://www.deepshiftlabs.com/sel_blog/?p=2925&lang=en-us Functional monitoring at Yandex (translation) http://feedproxy.google.com/~r/WebApplicationsTestingAndSeleniumBlogByDeepShiftLabs/~3/rnCzdfG_atA/ http://www.deepshiftlabs.com/sel_blog/?p=2886&lang=en-us#comments Tue, 19 Mar 2013 01:26:24 +0000 Igor http://www.deepshiftlabs.com/sel_blog/?p=2886 Functional monitoring at Yandex

We recently launched functional monitoring support in Nerrvana and we think it will be interesting for our readers to learn why Yandex (taking fourth place behind Google, China’s Baidu and Yahoo, according to ComScore) uses functional monitoring.

Post is translated with Mikhail Levin’s permission and its original (in Russian) is located on Habrahabr. Yandex also shared technical details of their system at Yet Another Conference 2012 – video and slides are available here (but also in Russian).

Here is it:

“Do you monitor your services in production? Whose responsibility it is in your company?

When we think of monitoring; server side developers, system administrators and DBAs often come to mind. They must watch the data processing queues and free disk space for the availability of individual hosts and their load.

Such monitoring really gives a lot of information about the service, but does not always show how the service works for an end user. Therefore, in addition to system monitoring, we, at Yandex, have created a functional monitoring system, tracking the state of the service from final interfaces – through the way the app looks and works in a browser, or how it works at the API level.

What is functionality monitoring in our understanding? To understand this better, let’s look at how things progressed for us.
It all began, of course, with regression auto testing. These auto-tests were also launched after release to test the service under real conditions. The fact that running regression tests in production sometimes finds bugs puzzled us.

What is it and why do we need it?

Why are functional tests, written for regression testing, and completing with no failures in testing, often fail in production?

We have identified a few reasons:

• Differences between the configuration of the test and production environments.
• Problems with the internal or external data suppliers.
• Hardware problems affecting the functionality.
• Problems that appear over time and/or at a specific workload.

If such tests can find problems, we decided that we need to try to running them regularly in production and monitor services’ status.
Let’s take a closer look at the problems functional monitoring tests can help us with.

Data providers

A good example of a page which depends on the data providers is Yandex’s home page.
Weather and News, Events and TV shows, even a photo of the day with search suggestions is the data from internal and external data providers.

For example, in Arkhangelsk block Posters once looked like this:
 
 
Yandex home page - Missing 'Portal' section in one region
 
 
While in Murmansk everything looked alright.
 
 
Yandex home page - no problem in 'Portal' section in another region
 
 
This is because the data supplier has not sent data for Arkhangelsk (or import has not updated on our side). Sometimes the problem is a one-off and, in some cases, KPIs can be formulated based on the percentage of available data and their freshness.

Hardware problems

In our services, fault tolerance and performance plays an important role. Therefore, the team created a service with a distributed architecture and load balancing mechanisms. Failure of the individual pieces of hardware, as a rule, does not affect the user, but major problems with the data centers or routing between them are sometimes visible in the end user interfaces.

To trace the connection between hardware problems and functionality is exactly the thing functional monitoring help us with to complement the system monitoring.

For example, in Yandex.Direct we had a situation when a slowly “dying” server caused a gradual degradation of service, making it unavailable for some regions. Functional monitoring in this case served as a trigger for an emergency investigation and detection of the root of the problem.

Another interesting example are drills held in our company. During the exercises, one of the data centers is intentionally disconnected to make sure that this does not affect the health of the services we provide. It is also to train the staff to minimize time to identify possible problems and fix them. Data center failure does not harm services due to their tolerance, but functional monitoring in this situation helps to monitor systems’ behavior.

Service degradation over time

The use of an applications in production sometimes creates unforeseen situations. Causes of the problem may be a combination of volume, duration and type of load, or, for example, the accumulation of system errors, not identified in the testing phase. Setup and configuration errors of the infrastructure can cause problems, leading to service degradation of the system, or its failure.

If such problems cannot be identified in the testing phase, it is necessary to quickly identify them when they occur in production. Here the system and functional monitoring can complement each other to find problems and report them.

So functional monitoring is functional auto-tests, “sharpened” to search for specific problems, and continuously run in production.

What’s inside?

There is a second component of the monitoring functionality – the way you process results.

The large flow of results that come from constantly running production tests must be aggregated and filtered. A system must promptly report any problems and at the same time minimise false positives. Also, there is the problem of integration of information from the functional monitoring into a single system performing health evaluation service which combines all monitoring results.

To avoid false positives, our system, built on the Apache Camel framework, allows the aggregation of several sequential results from one test into a single event. For example, you can configure filtering 3 out of 5, that will notify about a problem only if the test produced an error three times in five consecutive runs (you can specify, for example, filtering, 2 out of 2, or remove the filter – 1 out of 1). The frequency of test runs is also important in order for the filters you set to make sense.

Since we have many different services, our results are consumed by different departments: some failures are sent to testers, some reports contain data which managers are interested in, some results are sent and integrated into the overall monitoring system.

Recipe

The idea of functionality monitoring is very simple, and such monitoring can be very effective for your business.

To ‘cook’ functional monitoring:

1. Assess which of your services fail in production and why.
2. Write (or select from existing) auto-tests for this bit of functionality.
3. Run these tests in production as often as you need and your monitoring system can cope with.
4. Process and notify the results, compare with other sources of information looking after the same system.

PS: For quite a long time we wanted to find out to what extent the idea of functional monitoring is spread and how it is used in other companies. Some people confirm that they use it, some want to implement once they hear the idea and some think that such monitoring is unnecessary, given the system is monitored in a ‘classical’ way.

How do you monitor the status of your production services, what tools do you use and how they are assembled?”

]]> http://www.deepshiftlabs.com/sel_blog/?feed=rss2&p=2886&lang=en-us 0 http://www.deepshiftlabs.com/sel_blog/?p=2886&lang=en-us New languages, browsers and Selenium version http://feedproxy.google.com/~r/WebApplicationsTestingAndSeleniumBlogByDeepShiftLabs/~3/4qhTSY2iYB8/ http://www.deepshiftlabs.com/sel_blog/?p=2867&lang=en-us#comments Mon, 18 Feb 2013 10:39:32 +0000 Igor http://www.deepshiftlabs.com/sel_blog/?p=2867
Nerrvana now supports – PHP 5.4.1 (with PHPUnit 3.6.10), Java 1.7.0, Python 2.7.3 (with nose 1.1.2, Selenium 1.0.3), Perl 5.14.2 and Ruby 1.9.3p194.

Selenium was updated to 2.29.0. We added support for Firefox 18, Chrome 24, Opera 12.14 on Windows XP and Windows 7 and Internet Explorer 10 on Windows 7 only.

Here you can see a full list of platforms supported today by Nerrvana.

]]> http://www.deepshiftlabs.com/sel_blog/?feed=rss2&p=2867&lang=en-us 2 http://www.deepshiftlabs.com/sel_blog/?p=2867&lang=en-us Nerrvana now supports Windows 7 http://feedproxy.google.com/~r/WebApplicationsTestingAndSeleniumBlogByDeepShiftLabs/~3/xmF8uwC5ekc/ http://www.deepshiftlabs.com/sel_blog/?p=2844&lang=en-us#comments Mon, 26 Nov 2012 23:28:58 +0000 Igor http://www.deepshiftlabs.com/sel_blog/?p=2844
We are glad to inform our clients about Windows 7 Selenium testing support in Nerrvana. A full list of available platforms is shown below (Nerrvana UI screenshot):

Nerrvana now supports Windows 7

Our plans – we moved from CentOS to Fedora which will allow us to expand a list of browsers on Linux. Adding support for launching Selenium tests written in Python and Ruby. Windows 8 is next.

]]> http://www.deepshiftlabs.com/sel_blog/?feed=rss2&p=2844&lang=en-us 0 http://www.deepshiftlabs.com/sel_blog/?p=2844&lang=en-us Using Nerrvana – final notes http://feedproxy.google.com/~r/WebApplicationsTestingAndSeleniumBlogByDeepShiftLabs/~3/0dfRLfaCcOI/ http://www.deepshiftlabs.com/sel_blog/?p=2628&lang=en-us#comments Thu, 22 Nov 2012 10:18:53 +0000 Igor http://www.deepshiftlabs.com/sel_blog/?p=2628 Using Nerrvana - final notes

Testing the application with two database types – configuration files ‘as is’ – future plans

Part 1 – Using Nerrvana – our setup
Part 2 – Using Nerrvana – SVN hooks to Jenkins
Part 3 – Using Nerrvana – deployment & Jenkins (part 1)
Part 4 – Using Nerrvana – deployment & Jenkins (part 2)
Part 5 – Using Nerrvana – Jenkins setup for Selenium testing
Part 6 – Using Nerrvana – final notes – this post

At this point, we’ve talked about how we test our application with Jenkins and Nerrvana. In our recount, some details have been omitted in order not to complicate the essence of the process.

It’s now time to show the real configuration files.

Here’s the script publish.sh. As you can see there is not one section, as shown earlier, but two. One deals with unpacking and setting up the Answers application on deployment host to work with MySQL, the other with PostgreSQL.

View Code BASH
#!/bin/sh
 
WWW_MYSQL_DIR='/var/www/answers/answers_mysql'
WWW_PGSQL_DIR='/var/www/answers/answers_postgres'
DB_NAME='answers'
 
rm -Rf $WWW_PGSQL_DIR/*
unzip -o prj.zip -d $WWW_PGSQL_DIR
mv $WWW_PGSQL_DIR/config/Settings.class.php.pgsql $WWW_PGSQL_DIR/config/Settings.class.php
chmod -R 777 $WWW_PGSQL_DIR
psql -U postgres -f $WWW_PGSQL_DIR/install/crt-pgsql.sql
psql -U postgres -f $WWW_PGSQL_DIR/install/postgres.sql $DB_NAME
 
rm -Rf $WWW_MYSQL_DIR/*
unzip -o prj.zip -d $WWW_MYSQL_DIR
mv $WWW_MYSQL_DIR/config/Settings.class.php.mysql $WWW_MYSQL_DIR/config/Settings.class.php
chmod -R 777 $WWW_MYSQL_DIR
mysql -v -u root  < $WWW_MYSQL_DIR/install/crt-mysql.sql
mysql -v -u root  < $WWW_MYSQL_DIR/install/mysql.sql $DB_NAME

Here is our build.xml. In fact, the same as shown previously, but with similar actions to MySQL, for PostgreSQL (lines 13-14, 22-23 and 35-36).

View Code XML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98

<?xml version="1.0" encoding="UTF-8"?>
<project name="${env.JOB_NAME}" default="build" basedir=".">
  <property environment="env"/>
  <property name="tests" value="${env.WORKSPACE}/tests"/>
  <property name="ci" value="${env.WORKSPACE}/ci"/>
  <property name="code" value="${env.WORKSPACE}/code"/>
 
 
  <target name="make_settings">
      <exec executable="sed" output="${code}/config/Settings.class.php.mysql">
        <arg 
            line="-f ${ci}/deployment/config/mysql/settings-mysql.sed ${code}/config/Settings.class.template.php"
        />
      </exec>
      <exec executable="sed" output="${code}/config/Settings.class.php.pgsql">
        <arg 
            line="-f ${ci}/deployment/config/pgsql/settings-pgsql.sed ${code}/config/Settings.class.template.php"
        />
      </exec>
    </target>
 
  <target name="make_sql">
    <exec executable="sed" output="${code}/install/mysql.sql">
      <arg 
           line="-f ${ci}/deployment/config/mysql/install-mysql.sed ${code}/install/mysql.sql"
      />
    </exec>
    <exec executable="sed" output="${code}/install/postgres.sql">
      <arg
          line="-f ${ci}/deployment/config/pgsql/install-pgsql.sed ${code}/install/postgres.sql" 
      />
    </exec>
  </target>
 
  <target name="move_files" depends="make_settings,make_sql">
    <!-- Move authentication files to WAUT root -->
    <move file="${ci}/deployment/misc/login.php" tofile="${code}/login.php" />
    <move file="${ci}/deployment/misc/logout.php" tofile="${code}/logout.php" />
 
    <!-- Move MySQL db creation files to install folder -->
    <move
         file="${ci}/deployment/config/mysql/crt-mysql.sql"
         tofile="${code}/install/crt-mysql.sql"
    />
 
    <!-- Move PGSQL db creation files to install folder -->
    <move 
         file="${ci}/deployment/config/pgsql/crt-pgsql.sql" 
         tofile="${code}/install/crt-pgsql.sql" 
    />
 
    <!-- Move publish script to the workspace root, so it will be beside prj.zip -->
    <move 
         file="${ci}/deployment/publish-over-ssh/publish.sh"
         tofile="${env.WORKSPACE}/publish.sh"
    /> 
  </target>
 
  <target name="save_commi
 
build-mysql.xml в свою очередь внутри себя делает:
 
t_info" depends="move_files">
      <!-- Temp fix for SVN client and SVN plugin incompatibility -->
      <exec dir="${env.WORKSPACE}" executable="svn" >
          <arg 
              line="--username jenkins --password 123456 upgrade"
          />
      </exec>
      <exec dir="${env.WORKSPACE}" executable="svn"  output="${env.WORKSPACE}/version.txt">  
          <arg line="--username jenkins --password 123456 log -r ${env.SVN_REVISION}" />
      </exec>
      <exec dir="${env.WORKSPACE}" executable="svn">  
          <arg line="--username jenkins --password 123456 log -r ${env.SVN_REVISION}" />
      </exec>          
  </target>	
 
  <target name="svn" depends="save_commit_info">
      <java
           dir="."
           jar="${ci}/deployment/scm-decorator.jar"
           fork="true"
           failonerror="true"
      >
 
          <arg value="svn"/>
          <arg value="${env.WORKSPACE}/version.txt"/> 
      </java>    	
  </target>
 
  <target name="zip" depends="svn" description="Compress project files">
      <jar destfile="${env.WORKSPACE}/prj.zip"
            basedir="${env.WORKSPACE}/code"
            excludes=".svn" />
  </target>
 
  <target name="build" depends="zip" />
</project>

The Jenkins job configuration page contains one extra Nerrvana plugin step. The idea is to test the application with MySQL, and then with PostgreSQL. What are the differences between them?

First, they use different Test Run names as well as executable files.

The names should be different. Here are the test results after the build completion, when an application is tested in such a way. Names of Test Run’s clearly tell what databases we tested on.

I did not mention how tests know what URL to knock. The URL is set directly in the tests Ant file. We just haven’t touched this topic. There are two Ant files. Each of them sets a URL to the appropriate virtual host for Answers working with MySQL and PostgreSQL respectively.

That is, for example, xbuild-mysql.sh used to run tests for an application working with MySQL. Its contents is:

ant -f build-mysql.xml all > build.log 2 >& 1

build-mysql.xml does this:

View Code XML
<copy file="${resources}/config-mysql.xml" tofile="${output}/config.xml"/>

And finally in the config-mysql.xml file we see what tests will run and what URL will be used.

View Code XML
<properties>
    <entry key="suite-title">Answers Test Suite</entry>
    <entry key="tests-disabled">
        com.deepshiftlabs.testui.AnswersStartyCo. TestLogin,
        com.deepshiftlabs.testui.AnswersStartyCo.TestAddRemoveQuestionByAnonymous,
        com.deepshiftlabs.testui.AnswersStartyCo.TestAddRemoveQuestionByUser,
        com.deepshiftlabs.testui.AnswersStartyCo.TestSearch,
        com.deepshiftlabs.testui.AnswersStartyCo.TestReply,
        com.deepshiftlabs.testui.AnswersStartyCo.TestFilterSort,
        com.deepshiftlabs.testui.AnswersStartyCo.TestTags,
        com.deepshiftlabs.testui.AnswersStartyCo.TestSmoke,
    </entry>
 
    <entry key="website">http://answers.dslabs.lan/mysql</entry>

You can see that all the tests are currently disabled except TestLogin.

The second difference between the two Nerrvana plugin calls in our job is that in the first call to the plugin we synchronize tests with Nerrvana, and we skip it in the second one, since it’s the same code. This saves us some time. Also you can see that we switched off message analyzer in the first step of testing the application with MySQL, and enabled it in the second. Thus, we let tests complete on both hosts and then analyze messages and set build status.

The report prepared by the plugin is based on messages received from both steps and contains two sections.

Finally, under the spoiler you will find an example of a console log of this double plugin invocation execution (I shortened some verbose outputs).

View Code TXT
Started by an SCM change
Building in workspace /var/lib/jenkins/jobs/Answers/workspace
Cleaning local Directory .
Checking out http://192.168.3.97/repos/answers/trunk at revision 146
A         tests
----------- SOME LINES REMOVED ---------------------------------
A         ci/toolchain/build-toolchain.xml
At revision 146
[deployment] $ ant -file build.xml build
Buildfile: build.xml
 
make_settings:
 
make_sql:
 
move_files:
     [move] Moving 1 file to /var/lib/jenkins/jobs/Answers/workspace/code
     [move] Moving 1 file to /var/lib/jenkins/jobs/Answers/workspace/code
     [move] Moving 1 file to /var/lib/jenkins/jobs/Answers/workspace/code/install
     [move] Moving 1 file to /var/lib/jenkins/jobs/Answers/workspace/code/install
     [move] Moving 1 file to /var/lib/jenkins/jobs/Answers/workspace
 
save_commit_info:
     [exec] Upgraded '.'
----------- SOME LINES REMOVED ---------------------------------
     [exec] Upgraded 'ci/deployment/misc'
     [exec] Upgraded 'ci/toolchain'
     [exec] ------------------------------------------------------------------------
     [exec] r146 | igork | 2012-11-14 08:22:44 +0000 (Wed, 14 Nov 2012) | 1 line
     [exec] 
     [exec] "Fixed" robots.txt to initiate Jenkins build
     [exec] ------------------------------------------------------------------------
 
svn:
 
zip:
      [jar] Building jar: /var/lib/jenkins/jobs/Answers/workspace/prj.zip
 
build:
 
BUILD SUCCESSFUL
Total time: 0 seconds
SSH: Connecting from host [phpbb.deepshiftlabs.com]
SSH: Connecting with configuration [Answers test server] ...
SSH: EXEC: STDOUT/STDERR from command [chmod +x publish.sh
./publish.sh] ...
Archive:  prj.zip
   creating: /var/www/answers/answers_postgres/META-INF/
  inflating: /var/www/answers/answers_postgres/META-INF/MANIFEST.MF  
   creating: /var/www/answers/answers_postgres/config/
   creating: /var/www/answers/answers_postgres/include/
----------- SOME LINES REMOVED ---------------------------------
   creating: /var/www/answers/answers_postgres/utils/
  inflating: /var/www/answers/answers_postgres/ask.php  
----------- SOME LINES REMOVED ---------------------------------
  inflating: /var/www/answers/answers_postgres/utils/DBUtils.class.php  
  inflating: /var/www/answers/answers_postgres/utils/Utils.class.php  
  inflating: /var/www/answers/answers_postgres/utils/init.php  
DROP DATABASE
CREATE DATABASE
REVOKE
DROP ROLE
You are now connected to database "answers".
psql:/var/www/answers/answers_postgres/install/crt-pgsql.sql:7: NOTICE:  table "USERS_SOURCE_TABLE" does not exist, skipping
DROP TABLE
psql:/var/www/answers/answers_postgres/install/crt-pgsql.sql:17: NOTICE:  CREATE TABLE / PRIMARY KEY will create implicit index "users_source_table_pkey" for table "users_source_table"
CREATE TABLE
INSERT 0 1
INSERT 0 1
----------- SOME LINES REMOVED ---------------------------------
GRANT
Archive:  prj.zip
   creating: /var/www/answers/answers_mysql/META-INF/
  inflating: /var/www/answers/answers_mysql/META-INF/MANIFEST.MF  
   creating: /var/www/answers/answers_mysql/config/
   creating: /var/www/answers/answers_mysql/include/
----------- SOME LINES REMOVED ---------------------------------
   creating: /var/www/answers/answers_mysql/utils/
  inflating: /var/www/answers/answers_mysql/ask.php  
----------- SOME LINES REMOVED ---------------------------------
  inflating: /var/www/answers/answers_mysql/utils/DBUtils.class.php  
  inflating: /var/www/answers/answers_mysql/utils/Utils.class.php  
  inflating: /var/www/answers/answers_mysql/utils/init.php  
--------------
drop database answers
--------------
 
--------------
create database answers
--------------
 
--------------
DROP TABLE if exists USERS_SOURCE_TABLE
--------------
 
--------------
CREATE TABLE `USERS_SOURCE_TABLE` (
  USERS_ID_COLUMN int(11),
  username varchar(50),
  USERS_DISPLAY_NAME_COLUMN varchar(50),
  USERS_EMAIL_COLUMN varchar(50),
  CONSTRAINT USERS_SOURCE_TABLE_pkey PRIMARY KEY (USERS_ID_COLUMN)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
--------------
 
----------- SOME LINES REMOVED ---------------------------------
 
--------------
CREATE VIEW `USERS_VIEW_NAME` AS
  SELECT
    `USERS_ID_COLUMN` AS `USERS_ID_COLUMN`,
    `USERS_DISPLAY_NAME_COLUMN` AS `USERS_DISPLAY_NAME_COLUMN`,
    `USERS_EMAIL_COLUMN` AS `USERS_EMAIL_COLUMN`
  FROM
    `USERS_SOURCE_TABLE`
--------------
 
--------------
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES_PREFIX_users TO 'ANSWERS_DB_USER'@localhost
--------------
 
--------------
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES_PREFIX_answers TO 'ANSWERS_DB_USER'@localhost
--------------
 
--------------
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES_PREFIX_questions TO 'ANSWERS_DB_USER'@localhost
--------------
 
--------------
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES_PREFIX_tags TO 'ANSWERS_DB_USER'@localhost
--------------
 
--------------
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES_PREFIX_tags_to_quest TO 'ANSWERS_DB_USER'@localhost
--------------
 
--------------
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES_PREFIX_subscribers TO 'ANSWERS_DB_USER'@localhost
--------------
 
--------------
GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES_PREFIX_votes_a TO 'ANSWERS_DB_USER'@localhost
--------------
 
--------------
GRANT SELECT ON USERS_VIEW_NAME TO 'ANSWERS_DB_USER'@localhost
--------------
 
SSH: EXEC: completed after 1,003 ms
SSH: Disconnecting configuration [Answers test server] ...
SSH: Transferred 2 file(s)
Build step 'Send files or execute commands over SSH' changed build result to SUCCESS
2012-11-14 08:22:49 
 
---BEGIN PLUGIN EXECUTION---
2012-11-14 08:22:49 ---INITIALIZATION STARTED---
2012-11-14 08:22:49 
---BEGIN PLUGIN SETTINGS---
	Nerrvana HTTP address: https://api.nerrvana.com
	Nerrvana API key: 2bf7c81-a031f-1ad2-fd3a6-f59b9e0668e
	Secret key:  wctfFwelygx3tXS4TasrsOS4oXV7YadcPppvEnx55WG7qPk6jrAHjJi1TRoLlhrarIlabte4H9zWROXVkLnXto2LlVC47EXx91Uu
	Space ID: 4144
	Space: Answers
	Space path[FTPS folder]: Answers/_files
	Selenium nodes per platform: 1
	Test run name: Answers TRUNK MySQL
	Nerrvana platforms:
		Opera 12.02 (WinXP)
		Firefox 15.0 (WinXP)
	Executable file: xbuild-mysql.sh
	Nerrvana FTPS address: ftp.nerrvana.com
	Nerrvana FTPS user: demo154
	Nerrvana FTPS pass: dem0l54
	Workspace folder: ./tests
	Max execution time: 3600
	Poll period: 20
	Parse user messages mode[results analyzer]: OFF
-----END PLUGIN SETTINGS---
2012-11-14 08:22:49 -----INITIALIZATION COMPLETED---
 
2012-11-14 08:22:49 Generated test run name: Answers TRUNK MySQL build #55
2012-11-14 08:22:49 Generated test run description:
Created by Nerrvana-Jenkins plugin
Revision: 146
Committer: igork
Date: 2012-11-14 08:22:44 +0000 (Wed, 14 Nov 2012)
"Fixed" robots.txt to initiate Jenkins build
 
2012-11-14 08:22:49 ---BEGIN UPLOADING TESTS TO NERRVANA FTPS---
[workspace] $ lftp -f upload-build-55-1352863447229969289.ftp
2012-11-14 08:23:22 -----END UPLOADING TESTS TO NERRVANA FTPS
 
2012-11-14 08:23:25 Creating and starting test run via Nerrvana HTTP API call...done.
2012-11-14 08:23:25 New test run ID#952.
2012-11-14 08:23:25 New execution ID#5769.
2012-11-14 08:23:25 ---BEGIN NERRVANA POLLING CYCLE (waiting for tests to complete)
2012-11-14 08:23:47 	Current execution status: run
 
2012-11-14 08:24:09 	Current execution status: run
 
----------- SOME LINES REMOVED ---------------------------------
 
2012-11-14 08:26:42 	Current execution status: run
 
2012-11-14 08:27:04 	Current execution status: ok
 
2012-11-14 08:27:04 -----END NERRVANA POLLING CYCLE---
2012-11-14 08:27:04 Saving execution results into /var/lib/jenkins/jobs/Answers/builds/2012-11-14_08-22-45/results.xml...
2012-11-14 08:27:04 Done.
2012-11-14 08:27:04 
-----END PLUGIN EXECUTION---
 
 
2012-11-14 08:27:04 
 
---BEGIN PLUGIN EXECUTION---
2012-11-14 08:27:04 ---INITIALIZATION STARTED---
2012-11-14 08:27:04 
---BEGIN PLUGIN SETTINGS---
	Nerrvana HTTP address: https://api.nerrvana.com
	Nerrvana API key: 2bf7c81-a031f-1ad2-fd3a6-f59b9e0668e
	Secret key:  wctfFwelygx3tXS4TasrsOS4oXV7YadcPppvEnx55WG7qPk6jrAHjJi1TRoLlhrarIlabte4H9zWROXVkLnXto2LlVC47EXx91Uu
	Space ID: 4144
	Space: Answers
	Space path[FTPS folder]: Answers/_files
	Selenium nodes per platform: 1
	Test run name: Answers TRUNK PostgreSQL
	Nerrvana platforms:
		Opera 12.02 (WinXP)
		Firefox 15.0 (WinXP)
	Executable file: xbuild-pgsql.sh
	Nerrvana FTPS address: ftp.nerrvana.com
	Nerrvana FTPS user: demo154
	Nerrvana FTPS pass: dem0l54
	Workspace folder: ./tests
	Max execution time: 3600
	Poll period: 20
	Parse user messages mode[results analyzer]: ON
	User message threshold: ERROR
-----END PLUGIN SETTINGS---
2012-11-14 08:27:04 -----INITIALIZATION COMPLETED---
 
2012-11-14 08:27:04 Generated test run name: Answers TRUNK PostgreSQL build #55
2012-11-14 08:27:04 Generated test run description:
Created by Nerrvana-Jenkins plugin
Revision: 146
Committer: igork
Date: 2012-11-14 08:22:44 +0000 (Wed, 14 Nov 2012)
"Fixed" robots.txt to initiate Jenkins build
 
2012-11-14 08:27:04 ---TESTS UPLOAD SKIPPED---
2012-11-14 08:27:06 Creating and starting test run via Nerrvana HTTP API call...done.
2012-11-14 08:27:06 New test run ID#953.
2012-11-14 08:27:06 New execution ID#5770.
2012-11-14 08:27:06 ---BEGIN NERRVANA POLLING CYCLE (waiting for tests to complete)
2012-11-14 08:27:28 	Current execution status: run
 
2012-11-14 08:27:49 	Current execution status: run
 
----------- SOME LINES REMOVED ---------------------------------
 
2012-11-14 08:30:03 	Current execution status: run
 
2012-11-14 08:30:26 	Current execution status: ok
 
2012-11-14 08:30:26 -----END NERRVANA POLLING CYCLE---
2012-11-14 08:30:26 Saving execution results into /var/lib/jenkins/jobs/Answers/builds/2012-11-14_08-22-45/results.xml...
2012-11-14 08:30:26 Done.
2012-11-14 08:30:26 ---BEGIN TEST EXECUTION RESULTS---
2012-11-14 08:30:26 At least 11 message(s) from Nerrvana side reach(es) or exceed(s) threshold level (ERROR).
Message analyzer marks execution as failure.
2012-11-14 08:30:26 -----END TEST EXECUTION RESULTS---
2012-11-14 08:30:26 
-----END PLUGIN EXECUTION---
 
 
Build step 'Nerrvana plug-in' marked build as failure
Finished: FAILURE

As you can see, the Selenium testing process took 8 minutes (8:22:49 – 8:30:26). Everything that happens before it is the deployment and takes only a few seconds. In this case, we use two platforms, and Answers was tested with MySQL and PostgreSQL sequentially.

We have 8 tests in total and each does a lot of checks, testing a particular piece of Answers’ functionality – search, tags, adding questions and answers, deleting and editing of questions and answers by the authors, moderators and administrators and so on. We can run tests in 4 threads with 2 tests in each thread. By using 4 threads and adding 4 more platforms we remain roughly within the same 8 minutes. By using 8 threads (one test per thread), we can add more platforms or reduce testing time.

Let’s look in more detail how these 8 minutes were spent in the logs that I have brought.

Initialization and the first Test Run name and description generation – 1 second.

View Code TXT
2012-11-14 08:22:49 ---INITIALIZATION STARTED---
----------- SOME LINES REMOVED ---------------------------------
2012-11-14 08:22:49 -----INITIALIZATION COMPLETED---
2012-11-14 08:22:49 Generated test run name: Answers TRUNK MySQL build #55
2012-11-14 08:22:49 Generated test run description:
Created by Nerrvana-Jenkins plugin
Revision: 146
Committer: igork
Date: 2012-11-14 08:22:44 +0000 (Wed, 14 Nov 2012)
"Fixed" robots.txt to initiate Jenkins build

Syncing Selenium tests between Jenkins and Nerrvana – 33 seconds.

View Code TXT
2012-11-14 08:22:49 ---BEGIN UPLOADING TESTS TO NERRVANA FTPS---
[workspace] $ lftp -f upload-build-55-1352863447229969289.ftp
2012-11-14 08:23:22 -----END UPLOADING TESTS TO NERRVANA FTPS

Creating the Test Runa in Nerrvana and launching it via API – 3 seconds.

View Code TXT
2012-11-14 08:23:25 Creating and starting test run via Nerrvana HTTP API call...done.
2012-11-14 08:23:25 New test run ID#952.
2012-11-14 08:23:25 New execution ID#5769.

Running tests – 3 minutes 39 seconds.

View Code TXT
2012-11-14 08:23:25 ---BEGIN NERRVANA POLLING CYCLE (waiting for tests to complete)
2012-11-14 08:23:47 	Current execution status: run
----------- SOME LINES REMOVED ---------------------------------
2012-11-14 08:27:04 	Current execution status: ok
2012-11-14 08:27:04 -----END NERRVANA POLLING CYCLE---
2012-11-14 08:27:04 Saving execution results into /var/lib/jenkins/jobs/Answers/builds/2012-11-14_08-22-45/results.xml...
2012-11-14 08:27:04 Done.
2012-11-14 08:27:04 
-----END PLUGIN EXECUTION---

The second Nerrvana plugin step is launched.

Initialization and the second Test Run name and description generation – 1 second.

View Code TXT
---BEGIN PLUGIN EXECUTION---
2012-11-14 08:27:04 ---INITIALIZATION STARTED---
2012-11-14 08:27:04 
2012-11-14 08:27:04 -----INITIALIZATION COMPLETED---
2012-11-14 08:27:04 Generated test run name: Answers TRUNK PostgreSQL build #55
2012-11-14 08:27:04 Generated test run description:
Created by Nerrvana-Jenkins plugin
Revision: 146
Committer: igork
Date: 2012-11-14 08:22:44 +0000 (Wed, 14 Nov 2012)
"Fixed" robots.txt to initiate Jenkins build

We do not sync Selenium tests with Nerrvana as they were synced during the previous step.

View Code TXT
2012-11-14 08:27:04 ---TESTS UPLOAD SKIPPED---

Creating the Test Run in Nerrvana and launching it via API – 2 seconds.

View Code TXT
2012-11-14 08:27:06 Creating and starting test run via Nerrvana HTTP API call...done.
2012-11-14 08:27:06 New test run ID#953.
2012-11-14 08:27:06 New execution ID#5770.

Running tests – 3 minutes 20 seconds.

View Code TXT
2012-11-14 08:27:06 ---BEGIN NERRVANA POLLING CYCLE (waiting for tests to complete)
2012-11-14 08:27:28 	Current execution status: run
----------- SOME LINES REMOVED ---------------------------------
2012-11-14 08:30:26 	Current execution status: ok
2012-11-14 08:30:26 -----END NERRVANA POLLING CYCLE---

Results parsing and report preparation – 1 second.

View Code TXT
2012-11-14 08:30:26 Saving execution results into /var/lib/jenkins/jobs/Answers/builds/2012-11-14_08-22-45/results.xml...
2012-11-14 08:30:26 Done.
2012-11-14 08:30:26 ---BEGIN TEST EXECUTION RESULTS---
2012-11-14 08:30:26 At least 11 message(s) from Nerrvana side reach(es) or exceed(s) threshold level (ERROR).
Message analyzer marks execution as failure.
2012-11-14 08:30:26 -----END TEST EXECUTION RESULTS---
2012-11-14 08:30:26

Thus, we see that most of the time was spent on testing. Everything else, including the synchronization of tests between Jenkins and Nerrvana (remember that LFTP uses the ‘mirror’ command and only updates changed files) is about 45 seconds. These figures are for the configuration shown in the very first post – deployment and jenkins hosts are on my home network as VMWare virtual machines, I’m connected to the Internet with ADSL2 +, I am in Sydney and Nerrvana is in the USA. No super speed here or close physical proximity.

How to reduce the test execution time twofold in our case?

We can just test our application on MySQL and PostgreSQL not sequentially, but simultaneously. But in this case, as shown in the image below in our Jenkins Dashboard we will see not one job, but three (one upstream and two downstream)

or four (two upstream and two downstream).

Moreover, a report produced by the Nerrvana plugin and shown in the picture above, will be located inside two different jobs, that will complicate and slow down its analysis for us. Another hitch will be the timing of tests. As the two Nerrvana plugins will be running in parallel, they will both have to do synchronization. It will add erxtra33 seconds, but will reduce the test time in half. That is, we get the total time around 4.5 minutes.

We run tests in parallel using our framework which utilizes TestNG in a single job, as you may have guessed. However when we will implement reports aggregation in our framework (more about it below) we possibly move on to parallel testing with different databases. We most likely will add support for MS SQL in Answers soon and will have to test on three different database types. In this case, we can forget about the short report Nerrvana plugin creates in Jenkins, and use a single aggregate report from our framework.

The main disadvantage now is the need to view as many reports as platforms we tested our application on. Nerrvana plugin reports are already aggregated but they give very basic information. Sometimes it is enough, and sometimes it’s not. Therefore, we will improve our framework to generate the results in XML format. This information will be uploaded back to Jenkins and then our new generator/aggregator will use XMLs to create a single report. We have described previously how such reports will look. We are happy to share the details of the implementation when we get to it.

Schematically, it will work like this:

Even if you do not need to test different settings as we do with different databases and multiple Nerrvana plugin steps, you will still be able to aggregate all platforms into one report inside a single build by adding extra an aggregation step after Selenium testing.

Well, the last thing I would like to do, is to get the web server’s logs and PHP error logs (we use PHP) and integrate them to our Selenium reports. Then it would be easy to move from errors in the logs to a place in Selenium report and vice versa. Also we found this article and want to try to get code coverage data after Selenium testing to see which areas were not touched by tests. If it works it will make sense to integrate this info into our reports.

This ends the story of web applications Selenium testing in Deep Shift Labs with Jenkins and Nerrvana. We hope that you find it useful.

Our next blog story will be about exporting Selenium test results to HP Quality Center (HP QC) using the Bumblebee web service from Agiletestware.

]]> http://www.deepshiftlabs.com/sel_blog/?feed=rss2&p=2628&lang=en-us 0 http://www.deepshiftlabs.com/sel_blog/?p=2628&lang=en-us Using Nerrvana – Jenkins setup for Selenium testing http://feedproxy.google.com/~r/WebApplicationsTestingAndSeleniumBlogByDeepShiftLabs/~3/ALm0TOZpnBQ/ http://www.deepshiftlabs.com/sel_blog/?p=2257&lang=en-us#comments Fri, 16 Nov 2012 01:26:54 +0000 Igor http://www.deepshiftlabs.com/sel_blog/?p=2257 Using Nerrvana - Jenkins setup for Selenium testing

Creating the Nerrvana plugin configuration
file – walking through it – running
our tests – checking test results

Part 1 – Using Nerrvana – our setup
Part 2 – Using Nerrvana – SVN hooks to Jenkins
Part 3 – Using Nerrvana – deployment & Jenkins (part 1)
Part 4 – Using Nerrvana – deployment & Jenkins (part 2)
Part 5 – Using Nerrvana – Jenkins setup for Selenium testing – this post

In the previous post, we have automated our web application deployment process and are now ready to continue improving our build and include Selenium testing into it. At this point Jenkins is able to react to the commit, prepare and install our application on the deployment host. We also learned how to extract commit information from SVN, parse and save it to version.txt file.

Today we will launch Selenium tests in Nerrvana with our Jenkins plugin. The Nerrvana plugin is available at http://your_jenkins_instance/pluginManager/available. The plugin uses LFTP to synchronize tests between Jenkins and Nerrvana. So, please install it (yum install lftp) on the same server running Jenkins.
We recommend that you run your tests from Nerrvana’s UI and verify that Nerrvana can run them before proceeding. The process for running tests manually is described on the ‘Get Started‘ page.

To create a configuration file for the Nerrvana plugin, go to the page ‘Add new test run’ or ‘Edit test run’. As shown below, select the desired Space (A), enter the Test Run name (B), and select an executable file (C). Note: Try not to use a long name for the test run since the plugin will add a Jenkins build number to it. We recommend that you leave the Description field (D) blank and allow the plugin to add commit information to it dynamically. Select desired platforms (E) and, if your tests can run in parallel, the number of nodes per platform (F). Nerrvana is now able to create a configuration file (G), where you will need to set a few parameters before using it in Jenkins. Please save the file.

Generate Nerrvana Jenkins config in UI

Section ‘api-params’ contains your Nerrvana API access keys. They are added automatically when you generate the configuration file. They are also available on the Settings page, where you can change them, if necessary.

View Code XML
<api-params>
        <!-- Address of the Nerrvana gateway. -->
        <gateway>https://api.nerrvana.com</gateway>
        <!-- User-specific key which identifies user on Nerrvana side. -->
        <!-- Available as an 'API public key' on Settings page 
            (https://cloud.nerrvana.com/user/editAccount) 
            in Nerrvana. -->
        <apikey>a6c171d-20eb5-61e3-60991-8a1e523ce</apikey>
        <!-- This key is used by the Nerrvana plug-in to create a checksum of API 
            call parameters to ensure their consistency. -->
        <!-- Available as an 'API private key' on Settings page in Nerrvana. -->
        <secretkey>z3rCWlLdAP35eCCYAmQZ2kPw9X0LByrcc3XGh</secretkey>
    </api-params>

Name of the Test Run is in the elemnt ‘test-run-name’. The plugin will add the Jenkins build number, thus generating a new name for each new Test Run.

View Code XML
<!-- Parameters related to Nerrvana-driven Selenium tests. -->
    <!-- Test Run name template, Jenkins build number will be added to the 
        end automatically. -->
    <test-run-name>Answers TRUNK MySQL build #51</test-run-name>

The Test Run description is defined by elemnt ‘test-run-descr’. You can leave it blank or add, for example, ‘Generated by Jenkins plugin’.

View Code XML
<!-- Test Run description. All Test Runs created by this Jenkins build 
        step will have this description. -->
    <test-run-descr></test-run-descr>

Comments for the element ‘test-run-descr-file’ explain how to use a file to add a description dynamically with each job build. We will change the file name to version.txt, because such a file is generated in the deployment part of our build. You can create your own file and include any information you want.

View Code XML
<!-- Content of this file, if not empty, will be added to a description,
        defined by ‘test-run-descr’ parameter above.
        File path is relative to the Jenkins job workspace.
        During deployment phase you can extract revision number, commiter name and a
        commit message from your version control system, put them into this file and
        use them as a description.
        You can extract and parse SVN information into info.txt file with our little
        tool - https://github.com/deepshiftlabs/nerrvana-plugin-for-jenkins-ci
        You can read how we do it with SVN in our blog. -->
    <test-run-descr-file>version.txt</test-run-descr-file>

Here’s how a Test Run created by the plugin will look if:

- Test-run-name – ‘Answers TRUNK MySQL’

- Test-run-descr – ‘Created by Nerrvana-Jenkins job’

- The contents of ‘test-run-descr-file’:

Revision: 142
Commiter: igork
Date: 2012-10-17 07:39:20 +0000 (Wed, 17 Oct 2012)

Dynamic description

The Jenkins build number (‘build # 51′) was added to the Test Run name. The description is a combination of ‘test-run-descr’ and the content of a file specified by the content ‘test-run-descr-file’ parameter.

‘executable-file’ parameter defines an executable file we selected – ‘xbuild-mysql.sh’.

View Code XML
<!-- Which executable file Nerrvana should use to start tests. -->
    <executable-file>xbuild-mysql.sh</executable-file>

This file should be able to launch your tests. For example:

ant -f build-mysql.xml all > build.log 2 >& 1

Here, under the spoiler, you can see our build-mysql.xml, but I will not talk about it today. If you have any questions – we are happy to answer.

View Code XML
<project name="dslabs-test" default="" basedir=".">
  <!-- BEGIN versions -->
    <property name="title"          value="Tests of http://answers.starty.co"/>
    <property name="version"          value="0.02.000"/>
  <!-- END versions -->
 
  <!-- BEGIN build properties -->
    <property name="build.compiler"       value="javac1.6"/>
    <property name="libdir"               value="${basedir}/lib"/>
    <property name="output"               value="${basedir}/output"/>
    <property name="sources"              value="${basedir}/src"/>
    <property name="resources"            value="${basedir}/res"/>
    <property name="dist"                 value="${basedir}/dist"/>
    <property name="logdir"               value="${basedir}/log"/>
    <property name="bindir"               value="${basedir}/bin"/>
    <property name="logs"            	value="${basedir}/logs"/>
    <property name="test.failonerror"     value="true"/>
    <property name="compile.debug"        value="true"/>
    <property name="compile.debuglevel"   value="lines,vars,source"/>
    <property name="compile.deprecation"  value="true"/>
  <!-- END build properties -->
 
    <taskdef resource="testngtasks" classpath="${libdir}/testng-5.8-jdk15.jar"/>
 
    <path id="classpath">
        <fileset dir="${libdir}">
            <include name="*.jar"/>
        </fileset>
    </path>
 
    <target name="clean">
        <delete dir="${output}"/>
        <delete dir="${logs}"/>
    </target>
 
    <target name="prepare" depends="clean">
        <mkdir dir="${output}"/>
        <mkdir dir="${logs}"/>
    <!-- !!!-->
        <mkdir dir="${logs}/html"/>
        <copy todir="${logs}/html/js">
            <fileset dir="${resources}/js" />
        </copy>
        <copy todir="${logs}/html/img">
            <fileset dir="${resources}/img" />
        </copy>
        <copy todir="${logs}/html/css">
            <fileset dir="${resources}/css" />
        </copy>
 
        <copy file="${resources}/log4j.xml" todir="${output}"/>
        <copy file="${resources}/testng.xml" todir="${output}"/>
        <copy file="${resources}/config-mysql.xml" tofile="${output}/config.xml"/>
    </target>
 
    <target name="bld" depends="prepare">
        <javac
            srcdir="${sources}"
            destdir="${output}"
            excludes=""
            encoding="UTF8"
            debug="${compile.debug}"
            deprecation="${compile.deprecation}"
            optimize="${compile.optimize}"
            debuglevel="${compile.debuglevel}">
            <classpath refid="classpath" />
        </javac>
    </target>
 
    <path id="tests_classpath">
        <fileset dir="${libdir}">
            <include name="*.jar"/>
        </fileset>
        <pathelement location="${output}"/>
    </path>
 
    <target name="run" description="${title}">
        <java classpathref="tests_classpath"
            fork="true"
            dir="${output}"
            classname="org.testng.TestNG"
            failonerror="false">
            <arg value="-d" />
            <arg value="${logs}/testng_reports" />
            <arg value="testng.xml"/>
        </java>
<!--delete dir="${output}"/-->
    </target>
 
  <!-- build all -->
    <target name="all" depends="bld,run" description="Clean, build, and deploy project"/>
</project>

Section ‘platforms’ lists the platforms on which the tests will run. All supported platforms are included, with the selected ones uncommented.

View Code XML
<!-- List of platforms to run tests against for this config. -->
    <platforms>
        <!-- List of available platforms. Uncomment to use. -->
        <!--platform>
            <code>centos_58_firefox_36</code>
            <name>Firefox 3.6 (CentOS)</name>
        </platform-->
        <!--platform>
            <code>winxp_sp3_firefox_110</code>
            <name>Firefox 11.0 (WinXP)</name>
        </platform-->
        <!--platform>
            <code>winxp_sp3_firefox_150</code>
            <name>Firefox 15.0 (WinXP)</name>
        </platform-->
        <!--platform>
            <code>winxp_sp3_firefox_36</code>
            <name>Firefox 3.6 (WinXP)</name>
        </platform-->
        <!--platform>
            <code>winxp_sp3_ie_8</code>
            <name>IE 8 (WinXP)</name>
        </platform-->
        <!--platform>
            <code>winxp_sp3_opera_1162</code>
            <name>Opera 11.62 (WinXP)</name>
        </platform-->
        <!--platform>
            <code>winxp_sp3_opera_1202</code>
            <name>Opera 12.02 (WinXP)</name>
        </platform-->
        <!--platform>
            <code>winxp_sp3_safari_515</code>
            <name>Safari 5.1.5 (WinXP)</name>
        </platform-->
        <!--platform>
            <code>winxp_sp3_safari_517</code>
            <name>Safari 5.1.7 (WinXP)</name>
        </platform-->
 
        <platform>
            <code>winxp_sp3_chrome_2001132</code>
            <name>Chrome 20.0.1132 (WinXP)</name>
        </platform>
        <platform>
            <code>winxp_sp3_chrome_2101180</code>
            <name>Chrome 21.0.1180 (WinXP)</name>
        </platform>
    </platforms>

Next element ‘nodes-count’ allows you to configure the plugin to run tests in parallel. More information about parallel execution is available here. If your tests will not work in parallel, change the number of nodes here to 1.

View Code XML
<!-- How many Selenium nodes should be used for each platform. -->
    <nodes-count>2</nodes-count>

The next parameter ‘folder-with-tests’ points to the location of the tests in the Jenkins workspace.

View Code XML
<!-- Parameters related to the transfer of the tests from Jenkins 
        to Nerrvana. -->
    <!-- Folder in the workspace of Jenkins job where Selenium tests will be 
        located. It is assumed that the SCM build step, which always occurs 
        BEFORE other steps, will put tests there. -->
    <folder-with-tests>.</folder-with-tests>

Our Selenium tests are in the folder ‘tests’ in the same repository. The structure of the project was shown in the post “Using Nerrvana – deployment & Jenkins (part 1)”. As our tests are in the ‘tests’ folder, we set it this way:

View Code XML
<folder-with-tests>./tests</folder-with-tests>

Space ID, Space name and FTPS folder were specified previously during configuration file generation and should be left as they are in the element ‘space’.

View Code XML
<!-- Nerrvana space previously created by you through the Nerrvana UI. -->
    <space>
        <id>4144</id>
        <name>Answers</name>
        <ftp-path>Answers/_files</ftp-path>
    </space>

Section ‘ftp’ defines FTPS access credentials. It is necessary to add the password to your Nerrvana account.

View Code XML
<!-- Address and credentials of the Nerrvana FTPS connection. Note that 
        a system running Jenkins should have LFTP application installed. -->
    <ftp>
        <server>ftp.nerrvana.com</server>
        <!-- Your username -->
        <username>demo123</username>
        <!-- Replace this value with your password!! -->
        <password>[p a s s w o r d]</password>
    </ftp>

The parameter ‘skip-tests-sync’ allows you to speed up testing by skipping test synchronization when the Nerrvana plugin is called multiple times in a single build. Now we’ll leave the default value – false, as we call the plugin once. In the next post I will show how it is used by us when we call it twice to test our Answers application with MySQL, and then with PostgreSQL.

View Code XML
<!-- You can skip test sync when you call the Nerrvana plugin many times 
        in a single job and tests were synced in a previous step. -->
    <skip-tests-sync>false</skip-tests-sync>

The parameters shown below are relevant only if you are using messages, which Nerrvana can analyze. There is a detailed description of these parameters in the plugin configuration file.

View Code XML
<!-- Job status will be set to FAILED if test execution in Nerrvana is 
        unsuccessful (tests did not sync, executable file did not start your 
        tests, etc).
            If tests were executed successfully, you have a few options:
        1. Mark build as successful in Jenkins and analyse generated reports 
        for problems your tests discovered.
        2. If you want to mark a build as FAILED based on errors your tests 
        found, you can:
            (a) Add additional step, load results, parse them to analyse errors 
            your tests discovered, and mark build as FAILED based on it. Keep 
            this parameter as 'false' in this case.
            (b) Use Nerrvana "Messages" and let the Nerrvana plugin analyse 
            results, and change build status to FAILED based on it. Keep this 
            parameter as 'true' in this case and use additional parameter 
            'message-threshold' to add additional logic.
            If you use messages and invoke the Nerrvana plug-in several times 
            in the same build, keep this option as 'false' for all invocations 
            except the last one to let the build complete all tests and analyse 
            errors at the very end. -->
    <use-messages-to-set-build-status>false</use-messages-to-set-build-status>
 
    <!-- Defines the level at which a FAILED status is generated. If this 
        value is set to 'WARN', for example, and your tests generate one or more 
        'WARN' or higher severity messages (ERROR or FATAL), Nerrvana execution 
        status, and Jenkins build, will be FAILED. For the full list of levels, 
        visit http://www.nerrvana.com/docs/using-messages page. -->
    <message-threshold>ERROR</message-threshold>

As our tests generate messages, which Nerrvana can analyze, we set the parameters as follows:

View Code XML
<use-messages-to-set-build-status>true</use-messages-to-set-build-status>
    <message-threshold>ERROR</message-threshold>

In this case, the plugin will not only receive and store messages polling Nerrvana while tests are running, but also analyze whether there are messages with the level ERROR or higher (FATAL). If any are found, the build status will be set as FAILED in Jenkins.

If you do not use Nerrvana messages and still want to set the status of the Jenkins build based on test results (rather than whether the run was completed successfully), you can add an extra step in your build after the Nerrvana plugin. Get test results from Nerrvana via FTPS, analyze them, and change the status of the Jenkins build.

The parameter ‘max-execution-time’ will stop tests if, for example, they go into an infinite loop. If you run tests using the Nerrvana UI, you can set the real value with some extra time added.

View Code XML
<!-- Maximum execution time (in seconds). Defines how long the Nerrvana 
        plug-in will wait for tests to complete. Start by setting to a large 
        value and adjust accordingly after a few runs.-->
    <max-execution-time>3600</max-execution-time>

The last parameter ‘poll-period’ specifies how often the Nerrvana plugin will poll API about the status of test execution.

View Code XML
<!-- How often the Nerrvana plug-in will update test execution status 
        from Nerrvana (in seconds). -->
    <poll-period>20</poll-period>

The flowchart of the plugin, especially in the part relating to polling and setting the build’s status, depending on these parameters, shown in greater detail in the documentation.

Now we only need to add a step to our build …

Add Nerrvana plugin step

… and copy the contents of the configuration file into the plugin configuration window.

Add Nerrvana plugin step - just copy/paste

You can save changes in a Jenkins job, run the build, and watch your tests running in the Nerrvana cloud. First, I launched a single test on a single platform. In the UI you can see how the plugin creates a Test Run and it is immediately started.

Nerrvana Test Run started by Jenkins

The Jenkins build number was added to the Test Run name and the description contains information we specified in the plugin configuration with version.txt file content, which was previously parsed by our decorator. We can see that the test has been completed successfully with no errors found – we cannot see any messages.

Nerrvana Test Run completed

From Nerrvana we can open (arrow in the picture above) the report our test generated.

Our Selenium report for one test

In the first launch, I turned off all the tests, except one – a login test in order to quickly make sure that everything works. Now I’ll enable all the tests, commit and Jenkins will create and start a new Test Run. Oh, and I will also add one more platform to it.

Below, under the spoiler, is an example of a Jenkins build console log, which continues the console log we publicised in a previous post about deployment. My comments follow it.

View Code TXT
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

---BEGIN PLUGIN EXECUTION---
2012-10-17 07:39:24 ---INITIALIZATION STARTED---
2012-10-17 07:39:24
---BEGIN PLUGIN SETTINGS---
   Nerrvana HTTP address: https://api.nerrvana.com
   Nerrvana API key: a6c171d-20eb5-61e3-60991-8a1e523ce
   Secret key: z3rCWlLdAP35eCCYAmQZ2kPw9X0LByrcc
   Space ID: 4144
   Space: Answers
   Space path[FTPS folder]: Answers/_files
   Selenium nodes per platform: 1
   Test run name: Answers TRUNK
   Nerrvana platforms:
      Opera 12.02 (WinXP)
      Firefox 15.0 (WinXP)
   Executable file: xbuild-mysql.sh
   Nerrvana FTPS address: ftp.nerrvana.com
   Nerrvana FTPS user: demo123
   Nerrvana FTPS pass: dem0I23
   Workspace folder: ./tests
   Max execution time: 3600
   Poll period: 20
   Parse user messages mode[results analyzer]: ON
   User message threshold: ERROR
-----END PLUGIN SETTINGS---
2012-10-17 07:39:24 -----INITIALIZATION COMPLETED---
 
2012-10-17 07:39:24 Generated test run name: Answers TRUNK MySQL build #51
2012-10-17 07:39:24 Generated test run description:
Created by Nerrvana-Jenkins plugin
Revision: 142
Committer: igork
Date: 2012-10-17 07:39:20 +0000 (Wed, 17 Oct 2012)
Launching all tests on two platforms
 
2012-10-17 07:39:24 ---BEGIN UPLOADING TESTS TO NERRVANA FTPS---
[workspace] $ lftp -f upload-build-51-8414449724631519802.ftp
 
2012-10-17 07:40:16 -----END UPLOADING TESTS TO NERRVANA FTPS
 
2012-10-17 07:40:18 Creating and starting test run 
                    via Nerrvana HTTP API call...done.
2012-10-17 07:40:18 New test run ID#884.
2012-10-17 07:40:18 New execution ID#5486.
2012-10-17 07:40:18 ---BEGIN NERRVANA POLLING CYCLE 
                       (waiting for tests to complete)
 
2012-10-17 07:40:40    Current execution status: run
 
2012-10-17 07:41:02    Current execution status: run
 
------ Some оutput removed ---------------------------------
 
2012-10-17 07:44:17    Current execution status: run
 
2012-10-17 07:44:38    Current execution status: ok
 
2012-10-17 07:44:38 -----END NERRVANA POLLING CYCLE---
2012-10-17 07:44:38 Saving execution results into results.xml...
2012-10-17 07:44:38 Done.
2012-10-17 07:44:38 ---BEGIN TEST EXECUTION RESULTS---
2012-10-17 07:44:38 At least 43 message(s) from Nerrvana side reach(es) 
                    or exceed(s) threshold level (ERROR).
Message analyzer marks execution as failure.
2012-10-17 07:44:38 -----END TEST EXECUTION RESULTS---
2012-10-17 07:44:38
-----END PLUGIN EXECUTION---
 
Build step 'Nerrvana plug-in' marked build as failure
Finished: FAILURE

In the first 26 lines, the plugin outputs the parameters parsed from its configuration file.

In lines 28-34, the plugin creates a new test run name and description.

In lines 36-39, you see the synchronization process between tests in the Jenkins workspace and tests stored in the Nerrvana space used for this job.

In lines 41-44, the plugin creates a new Test Run in Nerrvana, launches it and gets a Test Run ID and unique Test Run execution ID back from API. This will be used to monitor test status by the plugin later. The same Test Run can be launched several times, but each run will have its own unique internal execution ID. At this stage, the Test Run is visible in the Nerrvana UI. “Answers TRUNK MySQL build # 51″ Test Run is created and starts running.

Nerrvana Test Run started by Jenkins - all tests and two platforms

In lines 45-58, the plugin polls test status every 20 seconds (polling parameter), while reading Nerrvana messages, which become available in real time during tests’ execution.

Once the tests are completed (lines 59-60), the plugin analyzes the errors (lines 61-63) and on the basis of the analysis sets the build status (lines 64-70). If you’re not using Nerrvana messages, the build would be marked as successful.

Using Nerrvana’s UI we can see how long it took for all tests to complete.

Nerrvana Test Run completed - all tests and two platforms

This is what messages look like in the UI, that the plugin reads and analyzes.

Nerrvana messages can be analyzed by plugin

We see that the reports generated by our framework contain more tests than the previous run (left panel).

Our Selenium report for all test

Another advantage of using Nerrvana messages is that the plugin can create simple reports (1). Reports show the status of the build and message threshold level used by the plugin (2) and the description of the Test Run (3). In our example, the description contains information about the commit that triggered the tests. Platform names (4) are linked to the Nerrvana FTPS server, which will contain detailed reports generated by the tests. Finally, clicking on the last column (5) expands the table and shows the messages that your tests sent to Nerrvana, which were received and collected by the Jenkins plugin.

Very often, this provides enough information for a developer to understand where the errors are, especially when he or she also maintains tests :)

Nerrvana plugin report in Jenkins

We have described the whole process of setting up and running tests. However, as you know, we test our application with two databases, and therefore we use a more complicated configuration. We deploy twice, and call the Nerrvana plugin two times in the same Jenkins job.

We will talk about this in a future post. We also ponder how the process can be improved.

]]> http://www.deepshiftlabs.com/sel_blog/?feed=rss2&p=2257&lang=en-us 0 http://www.deepshiftlabs.com/sel_blog/?p=2257&lang=en-us Running Selenium IDE tests without IDE http://feedproxy.google.com/~r/WebApplicationsTestingAndSeleniumBlogByDeepShiftLabs/~3/W-5W02VYxms/ http://www.deepshiftlabs.com/sel_blog/?p=2626&lang=en-us#comments Wed, 14 Nov 2012 02:27:49 +0000 Igor http://www.deepshiftlabs.com/sel_blog/?p=2626 Running Selenium IDE without IDE

We are glad to introduce the Selenium IDE formatter for Nerrvana.

What is Nerrvana? Nerrvana is a cloud service which allows you to run functional and cross browser Selenium tests on demand.

We hope it will help those who use the Selenium IDE to create and run tests or monitoring scripts in our cloud, and help you to transition from recording tests in the IDE to writing much more powerful and easy to maintain tests in Java.

Use 720p for the best quality

If there will be a demand from our clients we will extend the framework to create reports in the same format for tests generated by the Nerrvana Selenium IDE formatter.

Even though Selenium IDE tests have some limits it won’t compromise the quality of the reports you see and will minimise the time you spend to analyse them.

So please register your free account and give it a try.

]]> http://www.deepshiftlabs.com/sel_blog/?feed=rss2&p=2626&lang=en-us 0 http://www.deepshiftlabs.com/sel_blog/?p=2626&lang=en-us Nerrvana Jenkins plugin released http://feedproxy.google.com/~r/WebApplicationsTestingAndSeleniumBlogByDeepShiftLabs/~3/O6hNDvhTkdU/ http://www.deepshiftlabs.com/sel_blog/?p=2730&lang=en-us#comments Mon, 12 Nov 2012 01:52:14 +0000 Igor http://www.deepshiftlabs.com/sel_blog/?p=2730 Nerrvana Jenkins plugin released

We are glad to release Jenkins plugin for Nerrvana. We are using final version for a couple of months now and completely happy with it. Please visit documentation page for details.

In the next article here we will share how do we use Nerrvana Jenkins plugin. Stay tuned!

What is next?

Selenium IDE formatter for Nerrvana. Yes, it will allow you to export Selenium IDE tests in a format and structure which is fully launchable in Nerrvana as is.

]]> http://www.deepshiftlabs.com/sel_blog/?feed=rss2&p=2730&lang=en-us 0 http://www.deepshiftlabs.com/sel_blog/?p=2730&lang=en-us Using Nerrvana – deployment & Jenkins (part 2) http://feedproxy.google.com/~r/WebApplicationsTestingAndSeleniumBlogByDeepShiftLabs/~3/Ja2mjCSNxPs/ http://www.deepshiftlabs.com/sel_blog/?p=2343&lang=en-us#comments Wed, 24 Oct 2012 07:32:32 +0000 Igor http://www.deepshiftlabs.com/sel_blog/?p=2343 Using Nerrvana - deployment & Jenkins (part 2)

Jenkins job creation – our web
application deployment in details – deployment
for Selenium testing completed

Part 1 – Using Nerrvana – our setup
Part 2 – Using Nerrvana – SVN hooks to Jenkins
Part 3 – Using Nerrvana – deployment & Jenkins (part 1)
Part 4 – Using Nerrvana – deployment & Jenkins (part 2) – this post

In the previous post we have finished configuring Jenkins and now we can create a Jenkins job to test our application.

Open the page http://your_jenkins_host/view/All/newJob and type in the job name. Since our application is Answers, we use it as our job name.

In practice, it is convenient to add branch information to your job name. For example, job ‘Answers TRUNK’ tests TRUNK, job ‘Answers Release 1.4′ tests Release 1.4.

New Jenkins job

Down in the section “Advanced Project Options” press the button “Advanced …” and set “Quiet period” to 0 seconds. We know that many people do builds a few times a day. In this case, you will configure this option differently. We build for each commit, and therefore we do not need to wait for 5 seconds. Jenkins help for this step describes the practical cases, when this option is required.

New Jenkins job - Advanced project options

Now we go to the section ‘Source Code Management’ and select SVN, which we use.

New Jenkins job - Source Code Management

Jenkins will ask you to enter SVN user details. Choose ‘Check out strategy’. We use – ‘Always check out a fresh copy’.

New Jenkins job - Check out strategy

The next section is ‘Build Triggers’. Use ‘Poll SCM’ option. In our case, the ‘push’ trigger, which Jenkins help advises to use, is the SVN hook that we added earlier. If you open context help for the option ‘Build periodically’, you will see the following remark: “When people first start continuous integration, they are often so used to the idea of regularly scheduled builds like nightly/weekly that they use this feature. However, the point of continuous integration is to start a build as soon as a change is made, to provide a quick feedback to the change.” This is why our builds start immediately after each commit.

New Jenkins job - Build Triggers

Skip the next section ‘Build Environment’ and go to the ‘Build’ section. Push the button ‘Add build step’ and then choose the ‘Invoke Ant’.

New Jenkins job - Add Build Step

As you can see, the Ant configuration file we use here is also in SVN. Upon completion of the checkout on the previous step, it falls into the right place at the right time. The purpose of this step is to prepare the code extracted from the version control system to be transferred to the deployment host where deployment will be completed. We want to do the maximum possible operations on jenkins host.

New Jenkins job - Invoke Ant

Next I’ll tell you what Ant does. Here’s what our build.xml is:

View Code XML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

<?xml version="1.0" encoding="UTF-8"?>
<project name="${env.JOB_NAME}" default="build" basedir=".">
    <property environment="env"/>
    <property name="tests" value="${env.WORKSPACE}/tests"/>
    <property name="ci" value="${env.WORKSPACE}/ci"/>
    <property name="code" value="${env.WORKSPACE}/code"/>
 
    <target name="make_settings">
        <exec executable="sed" output="${code}/config/Settings.class.php.mysql">
            <arg line="-f ${ci}/deployment/config/mysql/settings-mysql.sed ${code}/config/Settings.class.template.php" />
        </exec>
    </target>
 
    <target name="make_sql">
        <exec executable="sed" output="${code}/install/mysql.sql">
            <arg line="-f ${ci}/deployment/config/mysql/install-mysql.sed ${code}/install/mysql.sql" />
        </exec>
    </target>
 
    <target name="move_files" depends="make_settings,make_sql">
        <!-- Move authentication files to WAUT root -->
       <move file="${ci}/deployment/misc/login.php" tofile="${code}/login.php" />
        <move file="${ci}/deployment/misc/logout.php" tofile="${code}/logout.php" />
 
        <!-- Move MySQL db creation files to install folder -->
        <move file="${ci}/deployment/config/mysql/crt-mysql.sql" tofile="${code}/install/crt-mysql.sql" />
 
        <!-- Move publish script to the workspace root, so it will be beside prj.zip -->
       <move file="${ci}/deployment/publish-over-ssh/publish.sh" tofile="${env.WORKSPACE}/publish.sh" />
    </target>
 
    <target name="save_commit_info" depends="move_files">
        <exec dir="${env.WORKSPACE}" executable="svn" >
                    <arg line="--username jenkins --password 123456 upgrade" />
        </exec>    
      <exec dir="${env.WORKSPACE}" executable="svn"  output="${env.WORKSPACE}/version.txt">  
          <arg line="--username jenkins --password 123456 log -r ${env.SVN_REVISION}" />
      </exec>
      <exec dir="${env.WORKSPACE}" executable="svn">  
          <arg line="--username jenkins --password 123456 log -r ${env.SVN_REVISION}" />
      </exec>
 
    </target>   
 
    <target name="svn" depends="save_commit_info">
       <java dir="." jar="${ci}/deployment/scm-decorator.jar" fork="true" failonerror="true">
         <arg value="svn"/>
         <arg value="${env.WORKSPACE}/version.txt"/> 
       </java>       
    </target>
 
    <target name="zip" depends="svn" description="Compress project files">
        <jar destfile="${env.WORKSPACE}/prj.zip"
            basedir="${env.WORKSPACE}/code"
            excludes=".svn" />
    </target>
 
   <target name="build" depends="zip" />
</project>

All other operations are performed on the deployment host after the results of this step are sent to it using the ‘Send files or execute commands over SSH’ build step.

I hope my explanation will help you create Ant build better, smarter and faster.

First of all, I want to say that if you get stuck with paths – paste code below into your Ant file and use it as a target – you will see in the Jenkins console output what paths Jenkins operates while executing your Ant file.

View Code XML
<target name="test">
    <echo message="Jenkins workspace: ${env.WORKSPACE}"/>
    <echo message="Job directory: ${env.WORKSPACE}/../../${env.JOB_NAME}"/>
    <echo message="Build data: ${env.WORKSPACE}/../../${env.JOB_NAME}/builds/${env.BUILD_ID}"/>
</target>

In lines 4-6, we give short names (aliases) to often used directories to make fewer mistakes down below in our commands and make the file easier to read.

View Code XML
<property name="tests" value="${env.WORKSPACE}/tests"/>
    <property name="ci" value="${env.WORKSPACE}/ci"/>
    <property name="code" value="${env.WORKSPACE}/code"/>

Next (lines 8-12), our task is to create a configuration file for the application. As we wrote earlier, our application is downloadable and is installed with the included installation script. Here we just use ‘sed’ to convert the template Settings.class.template used in the installation script into the application configuration file Settings.class.php.

View Code XML
<target name="make_settings">
        <exec executable="sed" output="${code}/config/Settings.class.php.mysql">
            <arg line="-f ${ci}/deployment/config/mysql/settings-mysql.sed ${code}/config/Settings.class.template.php" />
        </exec>
    </target>

I will not show the files in their entirety, but just a few lines from a template file – Settings.class.template, a few lines of settings-mysql.sed file and the same rows in the newly created configuration file Settings.class.php.

View Code PHP
// Database IP or hostname (use colon, if runs on non-standard port: 127.0.0.1:563)
    const DB_HOST = "";
 
    // Username to connect to DB
    const DB_USER = "";
View Code BASH
s/const DB_USER = "";/const DB_USER = "ANSWERS_DB_USER";/g
s/const DB_HOST = "";/const DB_HOST = "localhost";/g
View Code PHP
// Database IP or hostname (use colon, if runs on non-standard port: 127.0.0.1:563)
    const DB_HOST = "localhost";
 
    // Username to connect to DB
    const DB_USER = "ANSWERS_DB_USER";

You can see that in the file names and paths we have ‘mysql’ That’s because we are also testing our application with PostgreSQL. I removed everything about PostgreSQL from this post and, as promised, will show the real configuration files in the final post. I do not like to be overloaded with too much information when someone is explaining something new to me, and I decided to show you the same courtesy.

As a result of this step, we created a configuration file for the application to work with MySQL – Settings.class.php.mysql, which is stored in the same folder with Settings.class.template.php – ${code}/config .

In lines 14-18, we create the necessary SQL file, which when executed on the deployment host, will recreate the MySQL database for our application.

View Code XML
<target name="make_sql">
        <exec executable="sed" output="${code}/install/mysql.sql">
            <arg line="-f ${ci}/deployment/config/mysql/install-mysql.sed ${code}/install/mysql.sql" />
        </exec>
    </target>

Again, the Answers installer uses a prepared for the installation script mysql.sql file. Based on user input, installer replaces, for example, the names of the tables, but we can use it unchanged. A few changes which are necessary are made using ‘sed’. In this case, we modify the original file ${code}/install/mysql.sql.

View Code BASH
s/USERS_TABLE/USERS_VIEW_NAME/g
s/USE ANSWERS_DATABASE/USE answers/g

Next – target “move_files” (lines 20-30). As you will recall, our application is embedded and does not have its own login page. However, for testing we need login and logout pages, which we have created. They are in the ${ci}/deployment/misc/ folder and now it’s time to move them to the root of the application – $ {code}/, as these pages will be used by Selenium tests (lines 21-23).

View Code XML
<target name="move_files" depends="make_settings,make_sql">
        <!-- Move authentication files to WAUT root -->
       <move file="${ci}/deployment/misc/login.php" tofile="${code}/login.php" />
        <move file="${ci}/deployment/misc/logout.php" tofile="${code}/logout.php" />
 
        <!-- Move MySQL db creation files to install folder -->
        <move file="${ci}/deployment/config/mysql/crt-mysql.sql" tofile="${code}/install/crt-mysql.sql" />
 
        <!-- Move publish script to the workspace root, so it will be beside prj.zip -->
       <move file="${ci}/deployment/publish-over-ssh/publish.sh" tofile="${env.WORKSPACE}/publish.sh" />
    </target>

In lines 25-26, we move another SQL file – crt-mysql.sql, from ${ci}/deployment/config/mysql/ to $ {code}/install/, since we send to the deployment host only the content of ${code}/ folder.

What is in the file crt-mysql.sql file? Here again, because of the embeddable nature of our application that creates its tables in the database of your main application, we need at least to (re)create a basic application tables (users’ accounts). After that we would be able to execute the file mysql.sql, we have prepared earlier which will embed Answers data structures. That is the role of the script – rebuild the database, create the “main” application and its users, and delete the database user, the previous deployment run created. That’s how it looks without cuts:

View Code SQL
DROP DATABASE answers;
CREATE DATABASE answers;
use answers;
 
DROP TABLE if exists USERS_SOURCE_TABLE;
CREATE TABLE `USERS_SOURCE_TABLE` (
  USERS_ID_COLUMN int(11),
  username varchar(50),
  USERS_DISPLAY_NAME_COLUMN varchar(50),
  USERS_EMAIL_COLUMN varchar(50),
  CONSTRAINT USERS_SOURCE_TABLE_pkey PRIMARY KEY (USERS_ID_COLUMN)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
 
INSERT INTO USERS_SOURCE_TABLE VALUES(1,'answ_test_admin@dslabs.lan','Arnie - Admin','answ_test_admin@dslabs.lan');
INSERT INTO USERS_SOURCE_TABLE VALUES(2,'answ_test_user@dslabs.lan','Ivan Danko - Captain','answ_test_user@dslabs.lan');
INSERT INTO USERS_SOURCE_TABLE VALUES(3,'answ_test_staff@dslabs.lan','Lee - Guru(Staff)','answ_test_staff@dslabs.lan');
 
DROP USER 'ANSWERS_DB_USER'@localhost;
FLUSH PRIVILEGES;

In conclusion of this part in lines 28-29, we move publsih.sh script to the root of our Jenkins job workspace. The script knows what to do with packed into zip file content of /code, which we have prepared. Publish.sh content will be discussed when we describe the step ‘Send files or execute commands over SSH’.

In lines 32-43, we put information about the commit which triggered our Jenkins build into a text file.

View Code XML
<target name="save_commit_info" depends="move_files">
        <exec dir="${env.WORKSPACE}" executable="svn" >
                    <arg line="--username jenkins --password 123456 upgrade" />
        </exec>    
      <exec dir="${env.WORKSPACE}" executable="svn"  output="${env.WORKSPACE}/version.txt">  
          <arg line="--username jenkins --password 123456 log -r ${env.SVN_REVISION}" />
      </exec>
      <exec dir="${env.WORKSPACE}" executable="svn">  
          <arg line="--username jenkins --password 123456 log -r ${env.SVN_REVISION}" />
      </exec>                  
    </target>

In lines 33-35, I have a workaround for the problem when your SVN server is a higher version than the one supported by the Jenkins SVN plugin. It can happen. In this case the command ‘svn info’ errors with version mismatch and asks to upgrade the working copy, which I do before running ‘svn info’. Lines 39-41 are only to output information we saved into the text file and also to the Jenkins console log. We will use the information saved into this file in a future step to convert it to a more pleasant format and add it later in the Nerrvana test run’s description.

Now we are using a utility which is written by us, and is available on GitHub, which takes the file shown below as an input and …

View Code BASH
#cat version.txt
------------------------------------------------------------------------
r69 | igork | 2012-10-03 07:38:38 +0000 (Wed, 03 Oct 2012) | 1 line
 
Fixed bug #342
------------------------------------------------------------------------

… converts it into this file:

View Code BASH
#cat version.txt
Revision: 69
Committer: igork
Date: 2012-10-03 07:38:38 +0000 (Wed, 03 Oct 2012)
Fixed bug #342

You can extend the code by adding support for a version control system that you use, if it will be useful to you.

That’s how conversion is done with Ant:

View Code XML
<target name="svn" depends="save_commit_info">
       <java dir="." jar="${ci}/deployment/scm-decorator.jar" fork="true" failonerror="true">
         <arg value="svn"/>
         <arg value="${env.WORKSPACE}/version.txt"/> 
       </java>       
    </target>

As you can see we added the parameter expecting someone else, or ourselves, to expand for Git later.

Finally, we zip the folder /code/ in lines 51-55 to file prj.zip.

View Code XML
<target name="zip" depends="svn" description="Compress project files">
        <jar destfile="${env.WORKSPACE}/prj.zip"
            basedir="${env.WORKSPACE}/code"
            excludes=".svn" />
    </target>

Go to the next build step (and last one for today) – transfer prepared files and do final steps of deployment on deployment host.

New Jenkins job - Send files or execute commands over SSH

You can see that we are transferring to the deployment host (determined by the first parameter – ‘SSH Server’ and we created it, when configured Jenkins – in this post) two files prj.zip and publish.sh. We also instruct Jenkins to run publish.sh after the transfer:

chmod +x publish.sh
./publish.sh

Now is the time to see what publish.sh script does.

View Code BASH
#!/bin/sh
 
WWW_MYSQL_DIR='/var/www/answers/answers_mysql'
DB_NAME='answers'
 
rm -Rf $WWW_MYSQL_DIR/*
unzip -o prj.zip -d $WWW_MYSQL_DIR
mv $WWW_MYSQL_DIR/config/Settings.class.php.mysql $WWW_MYSQL_DIR/config/Settings.class.php
chmod -R 777 $WWW_MYSQL_DIR
mysql -v -u root  < $WWW_MYSQL_DIR/install/crt-mysql.sql
mysql -v -u root  < $WWW_MYSQL_DIR/install/mysql.sql $DB_NAME

Cleans Apache virtual host folder:

View Code BASH
rm -Rf $WWW_MYSQL_DIR/*

Decompresses Answers into it:

View Code BASH
unzip -o prj.zip -d $WWW_MYSQL_DIR

Renames prepared Answers configuration file to be used with MySQL database:

View Code BASH
mv $WWW_MYSQL_DIR/config/Settings.class.php.mysql $WWW_MYSQL_DIR/config/Settings.class.php

Changes permissions (I admit, we could be a little bit more conservative here):

View Code BASH
chmod -R 777 $WWW_MYSQL_DIR

Creates a ‘main’ application: (-v is used to be able to see verbose output from SQL script inside Jenkins console log):

View Code BASH
mysql -v -u root < $WWW_MYSQL_DIR/install/crt-mysql.sql

Integrates Answers into the ‘main’ application:

View Code BASH
mysql -v -u root < $WWW_MYSQL_DIR/install/mysql.sql $DB_NAME

We are done! We do a commit and see the build’s console logs in Jenkins until we have no errors and have a running Answers application on http://deployment_host/answers_mysql.

As you can see all the files used by Jenkins are also in the version control system. This allows us to not use custom workspaces in Jenkins (we started with them), which would require Jenkins pre-configuration as well as remembering to update files in our version control system and in Jenkins. All files fall into the right places after Jenkins updates the source code in the workspace after a commit. Also, with this approach, we can use the environment variables Jenkins sets in our configuration files.

Under the spoiler below you can find a sample console log with my comments. I removed some long outputs and marked it.

View Code TXT
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77

Started by an SCM change
Building in workspace /var/lib/jenkins/jobs/Answers_deployment_only/workspace
Cleaning local Directory .
Checking out http://192.168.3.97/repos/answers/trunk at revision 140
A         tests
------ Output removed - comment by Igor --------------------------------- 
AU        ci/deployment/scm-decorator.jar
A         ci/deployment/build.xml
A         ci/toolchain
A         ci/toolchain/build-toolchain.xml
At revision 140
[deployment] $ ant -file build.xml build
Buildfile: build.xml
 
make_settings:
 
make_sql:
 
move_files:
     [move] Moving 1 file to /var/lib/jenkins/jobs/Answers_deployment_only/workspace/code
     [move] Moving 1 file to /var/lib/jenkins/jobs/Answers_deployment_only/workspace/code/install
     [move] Moving 1 file to /var/lib/jenkins/jobs/Answers_deployment_only/workspace
 
save_commit_info:
     [exec] Upgraded '.'
------ Output removed - comment by Igor --------------------------------- 
     [exec] Upgraded 'ci/deployment/misc'
     [exec] Upgraded 'ci/toolchain'
     [exec] ------------------------------------------------------------------------
     [exec] r140 | igork | 2012-10-08 11:46:01 +0000 (Mon, 08 Oct 2012) | 1 line
     [exec] 
     [exec] Test 2345
     [exec] ------------------------------------------------------------------------
 
svn:
 
zip:
      [jar] Building jar: /var/lib/jenkins/jobs/Answers_deployment_only/workspace/prj.zip
 
build:
 
BUILD SUCCESSFUL
Total time: 0 seconds
SSH: Connecting from host [mantis.deepshiftlabs.com]
SSH: Connecting with configuration [Answers test server] ...
SSH: EXEC: STDOUT/STDERR from command [chmod +x publish.sh
./publish.sh] ...
Archive:  prj.zip
   creating: /var/www/answers/answers_mysql/META-INF/
------ Output removed - comment by Igor ---------------------------------   
  inflating: /var/www/answers/answers_mysql/utils/Utils.class.php  
  inflating: /var/www/answers/answers_mysql/utils/init.php  
--------------
drop database answers
--------------
 
--------------
create database answers
--------------
------ Output removed - comment by Igor --------------------------------- 
--------------
GRANT SELECT ON USERS_VIEW_NAME TO 'ANSWERS_DB_USER'@localhost
--------------
 
SSH: EXEC: completed after 1,003 ms
SSH: Disconnecting configuration [Answers test server] ...
SSH: Transferred 2 file(s)
Build step 'Send files or execute commands over SSH' changed build result to SUCCESS
Finished: SUCCESS
 
1-11 - Jenkins responded to a commit and checked out a fresh copy
12-40 - build.xml is running with target the ‘build’
24-28 - 'svn upgrade' is updating a working copy (workaround I described)
29-33 – our SCM decorator parses ‘svn log’ output
44-46 - the results were sent to the deployment host and publish.sh is executed
49-52 - prj.zip unpacked to the root of the Apache’s virtual host
53-63 - publish.sh executes SQL script

In our next post we will extend Jenkins job we created today and launch Selenium tests in Nerrvana.

]]> http://www.deepshiftlabs.com/sel_blog/?feed=rss2&p=2343&lang=en-us 0 http://www.deepshiftlabs.com/sel_blog/?p=2343&lang=en-us Virtualizer toolbox – install Win 7 on Xen http://feedproxy.google.com/~r/WebApplicationsTestingAndSeleniumBlogByDeepShiftLabs/~3/84l80P4Tzfs/ http://www.deepshiftlabs.com/sel_blog/?p=2340&lang=en-us#comments Fri, 12 Oct 2012 04:13:00 +0000 Igor http://www.deepshiftlabs.com/sel_blog/?p=2340 Virtualizer toolbox - install Win 7 on Xen

This post continues our two recent posts where we explained how to install Xen and minimise Windows 7 to make it ready for Selenium testing. We cut out as many components and services as possible, because we are going to only use browsers.

As we have Xen installed and the Windows 7 install image – today, we will complete our story and show you how to install Windows 7 on Xen. This same process can be used to install standard Windows 7 but you will need to allocate more space (file or logical volume) and more RAM.

In our Xen installation post we already created a logical volume …

#lvcreate –size 7G –name win7 vg0

… launched a VNC server, connected to CentOS with VNC Viewer, launched Virtual Machine Manager and, finally, pressed the ‘New’ button.

From this point forward everything is very simple.

Step 1 - Virtual Machine Creation

Step 2 - Virtual Machine Name

Step 3 - Virtualization Method

Step 4 - Installation Method

This file is the Windows 7 install image that we created. If you use the original install disk, you will need to make the ISO file and put it on your Linux file system.

Step 5 - Installation Media

We install it on the logical volume we created earlier.

Step 6 - Storage

Step 7 - Network

Step 8 - Memory and CPU allocation

Step 9 - Finish Virtual Machine Creation

Installation begins …

Win 7 install is running now

]]> http://www.deepshiftlabs.com/sel_blog/?feed=rss2&p=2340&lang=en-us 0 http://www.deepshiftlabs.com/sel_blog/?p=2340&lang=en-us