Create Parameterized Test Service with Jenkins

Introduction

This article describe how to create a Jenkins Project that run parameterized test case with specified browsers/cases parameter.

In this way, anyone who use Win 8.1 and can access the Jenkins web page can simply download win_folder, run *.bat to start selenium standalone, then run Jenkins Job and see the test cases run on hes/her screen.

NOTE: The method only works with Remote WebDriver, Jenkins has problem to run local driver directly.

Pre-Request

Use Remote Webdriver in Selenium

Create Parameterized Test Case

Install Jenkins on Ubuntu 14.04

Steps

There are quite a few steps,

0. update mode of log file

Change mode of SeleniumTest/log/test.log so Jenkins can access it.

# go to SeleniumTest/log
cd log
# chmod so Jenkins can write to test.log
chmod 666 test.log

1. Create custom test task

To run rake test with specified file set, we need to write a custom task,
run

rails g task test

to generate task file then modify it as

test.rake


2. Create Jenkins Job

Please refer to
online demo

and
content of jenkins project

3. Run Jenkins Job

Assume I use win 8.1, have installed required browsers, downloaded the win_folder and run the *.bat as needed, then I can run Jenkins job as below:

(please ignore the music, just for replce my original audio whitch contains lots of noise)


online demo

In the demo above, parameterized_with_env_test.rb will run on all specified browsers, remote_webdriver_test.rb will run on browsers written in test case.

References

How to generate rake task

Rake Task to Run a Custom Group of Tests

Rake::TestTask Document


Running selenium tests in ruby with Jenkins

Create Parameterized Test Case

Introduction

This article describe how to create a parameterized test case in Rails / Minitest, so the test case can be used as a service with Jenkins via web page interface later.

Pre-Request

Use Remote Webdriver in Selenium

How to do

Create a testWithEnvParam method in test_helper, it receives an instance of test case, parse given 'browsers' parameters to an array, make a driver with respect to each element of the array then call 'doTest' method of the given test with the driver.

For more information, please refer to the two files below.

Added / Updated Files

Added parameterized_with_env_test.rb

Modified test_helper.rb

References

How to pass parameters to MiniTest::Unit::TestCase suite?

How do I convert a String object into a Hash object?

Calling a Function From a String With the Function's Name in Ruby

Take Screenshot with Ruby Selenium

Simple Note

Needed for visual testing so wrote a sample for later reference.

Refer to save_screenshot_test.rb


Reference

How to take a screenshot with Ruby's Selenium Web Driver

Test Multiple Version of Firefox in Selenium

Description

This article describe how to test multiple version of firefox with Selenium and Firefox Portable.

Demo Video

http://screencast.com/t/NggoPkcusBn

Pre-Request

Rails, minitest and Selenium

Use Remote Webdriver in Selenium

Steps

1. Download firefox portable as needed

For Linux:

PortableLinuxApps


For Windows

Mozilla Firefox, Portable Edition

You can also find other versions at SourceForge

2. Change mode / install

In Ubuntu, you need to run

chomd +x firefox31.0-64bit

to make it executable

In Windows, you need to double-click it to install (extract) it.

3. Write test case and test

For local driver, just update firefox path by

Selenium::WebDriver::Firefox.path = "path/to/firefox/binary"

as needed.

For remote driver, you can specify firefox binary path by

-Dwebdriver.firefox.bin="path/to/firefox/binary"

when run selenium-server-standalone.jar

Added FirefoxPortable version 38 to startup.bat, version 39 to startOpera.bat

For more information, please refer to
 test case and startup.bat.

Added / Updated Files

Added firefox_portable_test.rb


Added vendor/portable_app

Updated startup.bat and startOpera.bat



References

Selenium RubyBindings

how to use different version of firefox using webdriver?

How can you detect the version of a browser?

Add Opera Driver to Selenium

Introduction

Current implementation of OperaDriver supports only 12.x and older versions, to work on opera versions > 12.X, Operachromiumdriver has been developed.

This article describe how to use Operachromiumdriver with Selenium in Ruby / Rails.

Pre-request

Add Chrome Driver to Selenium

Steps

In short, just do what you have done in Add Chrome Driver to Selenium , just replace chrome driver with opera driver and probably need to specify more params.

Similar for remote driver.

0. Install Opera

1. Download and extract operachromiumdriver from operachromiumdriver/releases

2. Change mode
Probably need to run command below after extracted opera driver since it is not executable by default

# make opera driver executable
chmod +x operadriver

3. Write another start script for opera on remote Windows
Just change

-Dwebdriver.chrome.driver

to the path of opera driver, and also use different port so can run it with chrome one.

For more information, please refer to startOpera.bat, you can run it by double-clicking start.bat / startOpera.bat on Windows, currently it uses 64 bit opera driver.

4. Write test case

Refer to add_opera_driver_test.rb

5. Test it

Added / Modified Files

add_opera_driver_test.rb

Added vendor/win_folder

References

How to use OperaChromiumDriver for opera version >12.X

Selenium and new OperaChromiumDriver

Remote WebDriver ignore certificate errors for Chrome

how to run a selenium-server-standalone?

operachromiumdriver/docs/desktop.md

Install Jenkins on Ubuntu 14.04

Introduction

To make test case more easy to use, you can use Jenkins to manage the test tasks. This article describe how to install jenkins on Ubuntu 14.04.

Steps

Install Oracle JDK (or any other JDK/JRE you want)

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer

Install jenkins

wget -q -O - https://jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins

(Optional) Change Jenkins port

Jenkins will use port 8080 by default, change port as below if needed

# assume change port from 8080 to 9999
# you can also edit /etc/default/jenkins manually
sudo sed -i 's/HTTP_PORT=8080/HTTP_PORT=9999/g' /etc/default/jenkins
# restart jenkins with new port setting
sudo service jenkins restart

Now you can link to jenkins at localhost and start to use it:

http://localhost:8080/

References

How To Install Java on Ubuntu with Apt-Get

Installing Jenkins on Ubuntu

how to change port number for Jenkins installation In Ubuntu 12.04

Frequently Used Selenium Functions

Simple Note

Just created a simple html page and test case to memo frequently used selenium functions, including click element, get or set value/attributes/css class, run javascript, do ajax call, handle alertbox/confirmbox and iframe.

(Basically, you can do most of things with driver.execute_script if you are familiar with JavaScript)

For more information, please refer to selenium_methods_test.rb below.

Added / Updated Files

test_helper.rb

selenium_methods_test.rb

vendor / html / *

Use Remote Webdriver in Selenium

Introduction

This article describe how to use Remote WebDriver to run Web Driver on remote machine.

Note:
Environments:
Test Machine -  Ubuntu 14.04, rails 4.1
Remote Server - Windows 8.1, IE11

Pre-Request

Make sure you have set up Selenium correctly
Rails, minitest and Selenium

Steps

On the remote server (e.g., windows 8.1 NB)

0. make sure Java JRE is installed

1. Download selenium-server-standalone-2.46.0.jar from
http://selenium-release.storage.googleapis.com/index.html
selenium-server-standalone-2.46.0.jar

2. Download and extract chrome driver and IE driver
chrome driver
http://chromedriver.storage.googleapis.com/index.html
chromedriver_win32.zip

IE driver
http://selenium-release.storage.googleapis.com/index.html
IEDriverServer*.zip

Note:

Basically you should choose 32 bit or 64 bit version of IE Driver according to your OS version, but you can try to switch version if you find send_keys pretty slow.

3. Open console, run selenium-server-standalone-2.46.0.jar by

# update path/filename as needed
java -jar -Dwebdriver.ie.driver="D:/things/Downloads/IEDriverServer.exe" -Dwebdriver.chrome.driver="D:\things\Downloads\chromedriver.exe" D:\things\Downloads\selenium-server-standalone-2.46.0.jar

4. Make sure that Internet Options → Security has the same Protected Mode setting (on or off, it doesn't matter as long as it is the same value) for all zones.

On the testing machine (Ubuntu 14.04 this case)

1. Add remote browsers to test_helper.rb

# Capabilities for IE driver
iecaps = Selenium::WebDriver::Remote::Capabilities.internet_explorer
# need to ignore zoom or it will
# complain zoom is 250% but it should be 100%
# on IE11/Windows 8.1
iecaps["ignoreZoomSetting"] = true;
# ...
@@remoteBrowsers = [
  { "url" => "http://192.168.31.169:4444/wd/hub",
      "desired_capabilities" =>  iecaps},
  # assume also want to test remote chrome
  { "url" => "http://192.168.31.169:4444/wd/hub",
      "desired_capabilities" =>  :chrome },
]
# ...
def self.remoteBrowsers
  @@remoteBrowsers
end

2. Generate test case

bin/rails generate integration_test remote_webdriver

3. Write test case

require "test_helper"

class RemoteWebdriverTest < ActionDispatch::IntegrationTest
  def test_sanity
    ActiveSupport::TestCase.remoteBrowsers.each do |remoteBrowser|
      driver = Selenium::WebDriver.for(:remote, :url => remoteBrowser["url"], :desired_capabilities => remoteBrowser["desired_capabilities"])
      doTest(driver)
    end
  end
  def doTest (driver)
    driver.navigate.to "http://google.com"
    element = driver.find_element(:name, 'q')
    element.send_keys "Hello WebDriver!"
    element.submit

    puts driver.title

    sleep 2
    # quit driver, close browser window
    driver.quit
  end
end

4. Run test case

rake test test/integration/remote_webdriver_test.rb

References

Selenium wiki RubyBindings

InternetExplorerDriver Zoom Level Error - Stack Overflow

How to set path for IE driver for a remote machine while using remotewebdriver from the code

Updated / Added Files

test_helper.rb

 remote_webdriver_test.rb

Add Chrome Driver to Selenium

Introduction

This article describe how to add chromedriver and run test case to test multiple browsers in Selenium.

Note:
Environment: Ubuntu 14.04, Rails 4.1

Pre-Request

Make sure you have set up Selenium correctly
Rails, minitest and Selenium

Steps:

1. Download and install google chrome

2. Download and extract chromedriver from
http://chromedriver.storage.googleapis.com/index.html

3. Create a softlink to the extract chromedriver

# create soft link
# change /your/home/ as needed
sudo ln -s /your/home/Downloads/chromedriver /usr/bin/chromedriver

4. edit SeleniumTest/test/test_helper.rb
 to declare browser array

# other code
@@browsers = [:firefox, :chrome]
# other code
def self.browsers
    @@browsers
end

5. generate a new test case

# generate test case
bin/rails generate integration_test add_chrome_driver

6. write test case to test firefox and chrome in one method

require "test_helper"

class AddChromeDriverTest < ActionDispatch::IntegrationTest
  def test_sanity
    ActiveSupport::TestCase.browsers.each do |browser|
     driver = Selenium::WebDriver.for browser
     driver.navigate.to "http://google.com"

     element = driver.find_element(:name, 'q')
     element.send_keys "Hello WebDriver!"
     element.submit

     puts driver.title

     sleep 2
     # quit driver, close browser window
     driver.quit
    end
  end
end

7. then run test cases

# run test cases
rake test

References:

(get correct path from)
https://github.com/watir/watirbook/blob/master/manuscript/installation/ubuntu.md#chrome

(choose soft link instead of move file because)
https://code.google.com/p/selenium/wiki/ChromeDriver


Added / Updated Files:

SeleniumTest/test/test_helper.rb

SeleniumTest/test/integration/add_chrome_driver_test.rb

Rails, minitest and Selenium

Introduction

Assume we will follow TDD (huh?) so build testing environment first.

This article describe how to setup and run selenium with minitest in Rails 4.

Pre-request

Make sure you have installed Rails 4, see
Install Rails 4 in Ubuntu 14.04

Steps

Open Terminal (Ctrl + Alt + T)

execute

# create testing project
rails new SeleniumTest
# go into project folder
cd SeleniumTest

Then edit SeleniumTest/Gemfile, append

gem 'selenium-webdriver'
gem 'minitest-rails'

save Gemfile, go back to terminal and run

# install for selenium-webdriver and minitest-raills
bin/bundle install
# create first test case
bin/rails generate integration_test check_selenium_work

edit generated test case
test/integration/check_selenium_work_test.rb
as below

require "test_helper"
require "selenium-webdriver"

class CheckSeleniumWorkTest < ActionDispatch::IntegrationTest
  def test_sanity
    driver = Selenium::WebDriver.for :firefox
    driver.navigate.to "http://google.com"

    element = driver.find_element(:name, 'q')
    element.send_keys "Hello WebDriver!"
    element.submit

    puts driver.title

    # do not quit driver so we can see search result
    # driver.quit
  end
end

save test case, then run

rake test

in terminal,

you should see firefox navigate to google and
search "Hello WebDriver!"

References

https://code.google.com/p/selenium/wiki/RubyBindings
https://github.com/blowmage/minitest-rails

Sample Project

Rails Selenium Test