Selenium with Rails -- The Working Case

Introduction

Just want to share my current working case, put all previous parts together, and more.

Online Demo

https://www.youtube.com/watch?v=arSE3p7fako

As you can see no diff image at second time since skipRealDiff is 'true'.

Pre-Request

Create Parameterized Test Service with Jenkins

Visual Testing with Selenium in Rails

Manage Image Files from Selenium Visual Testing

Steps to Apply This Project

1. Clone the WorkinCase/SeleniumTest

then you should be able to run

# run all test cases
rake test:fileset

or

# run all cases that quoted by single quote and separated by comma
cases="['simple_test']" rake test:fileset

in terminal

2. Create Jenkins Job with the content of Jenkins Job Settings at

README.rdoc


basically two parts, parameters and shell, then you can run it with Jenkins as the demo video shown above.

3. Regarding the Rails Web App, run it as your current user if you always run test cases from termnal, or run it by Jenkins as described in Manage Image Files from Selenium Visual Testing
if you usually run test cases with Jenkins

Modify test_helper.rb#L234 and Line 248 to your rails app url to change the url of output message


Different in Details

Add one more page '/vt/cases' to list all integration test cases (see routes.rb#L2 )

The working case postpone image processing by overriding MiniTest::CompositeReporter (see test_helper.rb#L200 ) so no need to read/write file.

Still override Rake::Task to rescue from rake aborted (see test.rake#L4 )
so Jenkins job will not be interrupted (since some operation needed after task finished)

Take care of Errors / Exceptions and Image diff to report summary of failed cases, see test_helper.rb
Line 35, 41, 240 ~ 256 and the if - fi block of Jenkins shell script

Take care of permission, see visual_testing_controller.rb#L66 , test_helper.rb#L259 and chmod command of Jenkins shell script

Postpone Image Processing to All Test Cases Finished in Rails Selenium Visual Testing

Introduction

As we already known we cah do Visual Testing with Selenium by using Chunky PNG, but Chunky PNG may spend much time to decode a large PNG image and make selenium timeout. This article describe how to postpone the image processing after all test cases done.

NOTE: The content is more complex and contains something with respect to my real work, so I put it at a different branch that will not be merged back to master.

NOTE 2: This article is exactly just a simple note, please refer to the code and git commit diff if needed.

Pre-Request

Visual Testing with Selenium in Rails

How To Do

There are two major issues need to be solved.

1. How to do something after all test cases are finished?

2. How to pass date from test cases to task?

The first one can be done by overriding invoke method of Rake::Task, please refer to the updated test.rake file.


The second one can be done via read/write a file or standalone caching server since the task and test cases are under different processes. Regarding how to read/write hash from/to file, please refer to test.rake#L21 and visual_testing_test.rb#L37.

Also override Selenium::WebDriver::Driver to store ENV params in visual_testing_helper.rb so can separate folder for different remote driver.
, and move ActiveSupport::VisualTestingHelper to visual_testing_helper.rb then require it from test.rake and test_helper.rb.


To make the base image can be loaded faster for future testing, I convert it from png to rgba at test.rake#L28, this is a mistake since the code should be moved to ActiveSupport::VisualTestingHelper.convert_rgba.


References

adding attributes to a Ruby object dynamically


How does one - without inheritance - override a class method and call the original from within the new method?


Why does dumping and loading a Hash using Marshal in Ruby throw a FormatError?


wvanbergen/chunky_png/wiki#encoding-options

Visual Testing with Selenium in Rails

Pre-Request

Image Compare in Ruby

Simple Note





Gemfile

Added gem 'chunky_png'
(so need to run 'bundle install')



test_helper.rb

require two more modules 'fileutils' and 'chunky_png'

Added class ActiveSupport::VisualTestingHelper for visual testing



Added test case 'visual_testing_test.rb'



The screenshots will be stored in folder visualTesting
each test case has its own folder



Almost done the Selenium part, need to build Web App to work with it (e.g., confirm and select new base image via webpage) later.

Make Test Case as Assistant

Introduction

Assume you use TDD, write test case first, and now you need to start to produce the minimum amount of code to pass that test.

This article describe how to make a test case as an assistant that can keep updating current issue when you are writing code, so you do not need to run test case by hand periodically to check current status.

Result

Please refer to the online demo:
ruby selenium auto retest

Pre-Request

Actually not really needed, just related for this article,
can work with local driver from terminal well.

Use Remote Webdriver in Selenium

Create Parameterized Test Service with Jenkins

Steps

1. Create an error filter

As an assistant, I wish it can report summarized error message in stead of full backtrace, so I write a filter method in test helper:

def self.displayError (error, file)
  puts "Error: " + error.message
  errArray = error.backtrace.map{ |x|
    x.match(/^(.+?):(\d+)(|:in `(.+)')$/); 
    [$1,$2,$4]
  }
  errArray.each do |err|
    if err[0].end_with? File.basename(file)
      puts "Line: "+err[1]+", "+err[2]+", File: "+File.basename(file)
      puts ""
    end
  end
end

2. Make it can retest automatically if failed.

So it will do test again and again until everything works fine, the code will as below:

reTest = true
# ...
while reTest do
  # clear reTest
  reTest = false
  # do some action and quit
  begin
    # ...
  rescue => error
    reTest = true
    # ...
  ensure
    driver.quit unless reTest
  end
end

Done, now you can run the test case then start to write your code, you will keep receiving current status from the test case.



Added/Updated Files


Modified test_helper.rb


Added auto_retest_if_fail_test.rb

Added auto_redo_test.html

Added ubuntu_folder
So can use remote webdriver for 127.0.0.1



References

Ruby - get the file being executed

Catching line numbers in ruby exceptions

Create Parameterized Test Service with Jenkins

Introduction

This article describe how to create a Jenkins Project that run parameterized test case with specified browsers/cases parameter.

In this way, anyone who use Win 8.1 and can access the Jenkins web page can simply download win_folder, run *.bat to start selenium standalone, then run Jenkins Job and see the test cases run on hes/her screen.

NOTE: The method only works with Remote WebDriver, Jenkins has problem to run local driver directly.

Pre-Request

Use Remote Webdriver in Selenium

Create Parameterized Test Case

Install Jenkins on Ubuntu 14.04

Steps

There are quite a few steps,

0. update mode of log file

Change mode of SeleniumTest/log/test.log so Jenkins can access it.

# go to SeleniumTest/log
cd log
# chmod so Jenkins can write to test.log
chmod 666 test.log

1. Create custom test task

To run rake test with specified file set, we need to write a custom task,
run

rails g task test

to generate task file then modify it as

test.rake


2. Create Jenkins Job

Please refer to
online demo

and
content of jenkins project

3. Run Jenkins Job

Assume I use win 8.1, have installed required browsers, downloaded the win_folder and run the *.bat as needed, then I can run Jenkins job as below:

(please ignore the music, just for replce my original audio whitch contains lots of noise)


online demo

In the demo above, parameterized_with_env_test.rb will run on all specified browsers, remote_webdriver_test.rb will run on browsers written in test case.

References

How to generate rake task

Rake Task to Run a Custom Group of Tests

Rake::TestTask Document


Running selenium tests in ruby with Jenkins

Create Parameterized Test Case

Introduction

This article describe how to create a parameterized test case in Rails / Minitest, so the test case can be used as a service with Jenkins via web page interface later.

Pre-Request

Use Remote Webdriver in Selenium

How to do

Create a testWithEnvParam method in test_helper, it receives an instance of test case, parse given 'browsers' parameters to an array, make a driver with respect to each element of the array then call 'doTest' method of the given test with the driver.

For more information, please refer to the two files below.

Added / Updated Files

Added parameterized_with_env_test.rb

Modified test_helper.rb

References

How to pass parameters to MiniTest::Unit::TestCase suite?

How do I convert a String object into a Hash object?

Calling a Function From a String With the Function's Name in Ruby

Take Screenshot with Ruby Selenium

Simple Note

Needed for visual testing so wrote a sample for later reference.

Refer to save_screenshot_test.rb


Reference

How to take a screenshot with Ruby's Selenium Web Driver

Test Multiple Version of Firefox in Selenium

Description

This article describe how to test multiple version of firefox with Selenium and Firefox Portable.

Demo Video

http://screencast.com/t/NggoPkcusBn

Pre-Request

Rails, minitest and Selenium

Use Remote Webdriver in Selenium

Steps

1. Download firefox portable as needed

For Linux:

PortableLinuxApps


For Windows

Mozilla Firefox, Portable Edition

You can also find other versions at SourceForge

2. Change mode / install

In Ubuntu, you need to run

chomd +x firefox31.0-64bit

to make it executable

In Windows, you need to double-click it to install (extract) it.

3. Write test case and test

For local driver, just update firefox path by

Selenium::WebDriver::Firefox.path = "path/to/firefox/binary"

as needed.

For remote driver, you can specify firefox binary path by

-Dwebdriver.firefox.bin="path/to/firefox/binary"

when run selenium-server-standalone.jar

Added FirefoxPortable version 38 to startup.bat, version 39 to startOpera.bat

For more information, please refer to
 test case and startup.bat.

Added / Updated Files

Added firefox_portable_test.rb


Added vendor/portable_app

Updated startup.bat and startOpera.bat



References

Selenium RubyBindings

how to use different version of firefox using webdriver?

How can you detect the version of a browser?

Add Opera Driver to Selenium

Introduction

Current implementation of OperaDriver supports only 12.x and older versions, to work on opera versions > 12.X, Operachromiumdriver has been developed.

This article describe how to use Operachromiumdriver with Selenium in Ruby / Rails.

Pre-request

Add Chrome Driver to Selenium

Steps

In short, just do what you have done in Add Chrome Driver to Selenium , just replace chrome driver with opera driver and probably need to specify more params.

Similar for remote driver.

0. Install Opera

1. Download and extract operachromiumdriver from operachromiumdriver/releases

2. Change mode
Probably need to run command below after extracted opera driver since it is not executable by default

# make opera driver executable
chmod +x operadriver

3. Write another start script for opera on remote Windows
Just change

-Dwebdriver.chrome.driver

to the path of opera driver, and also use different port so can run it with chrome one.

For more information, please refer to startOpera.bat, you can run it by double-clicking start.bat / startOpera.bat on Windows, currently it uses 64 bit opera driver.

4. Write test case

Refer to add_opera_driver_test.rb

5. Test it

Added / Modified Files

add_opera_driver_test.rb

Added vendor/win_folder

References

How to use OperaChromiumDriver for opera version >12.X

Selenium and new OperaChromiumDriver

Remote WebDriver ignore certificate errors for Chrome

how to run a selenium-server-standalone?

operachromiumdriver/docs/desktop.md

Frequently Used Selenium Functions

Simple Note

Just created a simple html page and test case to memo frequently used selenium functions, including click element, get or set value/attributes/css class, run javascript, do ajax call, handle alertbox/confirmbox and iframe.

(Basically, you can do most of things with driver.execute_script if you are familiar with JavaScript)

For more information, please refer to selenium_methods_test.rb below.

Added / Updated Files

test_helper.rb

selenium_methods_test.rb

vendor / html / *

Use Remote Webdriver in Selenium

Introduction

This article describe how to use Remote WebDriver to run Web Driver on remote machine.

Note:
Environments:
Test Machine -  Ubuntu 14.04, rails 4.1
Remote Server - Windows 8.1, IE11

Pre-Request

Make sure you have set up Selenium correctly
Rails, minitest and Selenium

Steps

On the remote server (e.g., windows 8.1 NB)

0. make sure Java JRE is installed

1. Download selenium-server-standalone-2.46.0.jar from
http://selenium-release.storage.googleapis.com/index.html
selenium-server-standalone-2.46.0.jar

2. Download and extract chrome driver and IE driver
chrome driver
http://chromedriver.storage.googleapis.com/index.html
chromedriver_win32.zip

IE driver
http://selenium-release.storage.googleapis.com/index.html
IEDriverServer*.zip

Note:

Basically you should choose 32 bit or 64 bit version of IE Driver according to your OS version, but you can try to switch version if you find send_keys pretty slow.

3. Open console, run selenium-server-standalone-2.46.0.jar by

# update path/filename as needed
java -jar -Dwebdriver.ie.driver="D:/things/Downloads/IEDriverServer.exe" -Dwebdriver.chrome.driver="D:\things\Downloads\chromedriver.exe" D:\things\Downloads\selenium-server-standalone-2.46.0.jar

4. Make sure that Internet Options → Security has the same Protected Mode setting (on or off, it doesn't matter as long as it is the same value) for all zones.

On the testing machine (Ubuntu 14.04 this case)

1. Add remote browsers to test_helper.rb

# Capabilities for IE driver
iecaps = Selenium::WebDriver::Remote::Capabilities.internet_explorer
# need to ignore zoom or it will
# complain zoom is 250% but it should be 100%
# on IE11/Windows 8.1
iecaps["ignoreZoomSetting"] = true;
# ...
@@remoteBrowsers = [
  { "url" => "http://192.168.31.169:4444/wd/hub",
      "desired_capabilities" =>  iecaps},
  # assume also want to test remote chrome
  { "url" => "http://192.168.31.169:4444/wd/hub",
      "desired_capabilities" =>  :chrome },
]
# ...
def self.remoteBrowsers
  @@remoteBrowsers
end

2. Generate test case

bin/rails generate integration_test remote_webdriver

3. Write test case

require "test_helper"

class RemoteWebdriverTest < ActionDispatch::IntegrationTest
  def test_sanity
    ActiveSupport::TestCase.remoteBrowsers.each do |remoteBrowser|
      driver = Selenium::WebDriver.for(:remote, :url => remoteBrowser["url"], :desired_capabilities => remoteBrowser["desired_capabilities"])
      doTest(driver)
    end
  end
  def doTest (driver)
    driver.navigate.to "http://google.com"
    element = driver.find_element(:name, 'q')
    element.send_keys "Hello WebDriver!"
    element.submit

    puts driver.title

    sleep 2
    # quit driver, close browser window
    driver.quit
  end
end

4. Run test case

rake test test/integration/remote_webdriver_test.rb

References

Selenium wiki RubyBindings

InternetExplorerDriver Zoom Level Error - Stack Overflow

How to set path for IE driver for a remote machine while using remotewebdriver from the code

Updated / Added Files

test_helper.rb

 remote_webdriver_test.rb

Add Chrome Driver to Selenium

Introduction

This article describe how to add chromedriver and run test case to test multiple browsers in Selenium.

Note:
Environment: Ubuntu 14.04, Rails 4.1

Pre-Request

Make sure you have set up Selenium correctly
Rails, minitest and Selenium

Steps:

1. Download and install google chrome

2. Download and extract chromedriver from
http://chromedriver.storage.googleapis.com/index.html

3. Create a softlink to the extract chromedriver

# create soft link
# change /your/home/ as needed
sudo ln -s /your/home/Downloads/chromedriver /usr/bin/chromedriver

4. edit SeleniumTest/test/test_helper.rb
 to declare browser array

# other code
@@browsers = [:firefox, :chrome]
# other code
def self.browsers
    @@browsers
end

5. generate a new test case

# generate test case
bin/rails generate integration_test add_chrome_driver

6. write test case to test firefox and chrome in one method

require "test_helper"

class AddChromeDriverTest < ActionDispatch::IntegrationTest
  def test_sanity
    ActiveSupport::TestCase.browsers.each do |browser|
     driver = Selenium::WebDriver.for browser
     driver.navigate.to "http://google.com"

     element = driver.find_element(:name, 'q')
     element.send_keys "Hello WebDriver!"
     element.submit

     puts driver.title

     sleep 2
     # quit driver, close browser window
     driver.quit
    end
  end
end

7. then run test cases

# run test cases
rake test

References:

(get correct path from)
https://github.com/watir/watirbook/blob/master/manuscript/installation/ubuntu.md#chrome

(choose soft link instead of move file because)
https://code.google.com/p/selenium/wiki/ChromeDriver


Added / Updated Files:

SeleniumTest/test/test_helper.rb

SeleniumTest/test/integration/add_chrome_driver_test.rb

Rails, minitest and Selenium

Introduction

Assume we will follow TDD (huh?) so build testing environment first.

This article describe how to setup and run selenium with minitest in Rails 4.

Pre-request

Make sure you have installed Rails 4, see
Install Rails 4 in Ubuntu 14.04

Steps

Open Terminal (Ctrl + Alt + T)

execute

# create testing project
rails new SeleniumTest
# go into project folder
cd SeleniumTest

Then edit SeleniumTest/Gemfile, append

gem 'selenium-webdriver'
gem 'minitest-rails'

save Gemfile, go back to terminal and run

# install for selenium-webdriver and minitest-raills
bin/bundle install
# create first test case
bin/rails generate integration_test check_selenium_work

edit generated test case
test/integration/check_selenium_work_test.rb
as below

require "test_helper"
require "selenium-webdriver"

class CheckSeleniumWorkTest < ActionDispatch::IntegrationTest
  def test_sanity
    driver = Selenium::WebDriver.for :firefox
    driver.navigate.to "http://google.com"

    element = driver.find_element(:name, 'q')
    element.send_keys "Hello WebDriver!"
    element.submit

    puts driver.title

    # do not quit driver so we can see search result
    # driver.quit
  end
end

save test case, then run

rake test

in terminal,

you should see firefox navigate to google and
search "Hello WebDriver!"

References

https://code.google.com/p/selenium/wiki/RubyBindings
https://github.com/blowmage/minitest-rails

Sample Project

Rails Selenium Test