Selenium with Rails -- The Working Case

Introduction

Just want to share my current working case, put all previous parts together, and more.

Online Demo

https://www.youtube.com/watch?v=arSE3p7fako

As you can see no diff image at second time since skipRealDiff is 'true'.

Pre-Request

Create Parameterized Test Service with Jenkins

Visual Testing with Selenium in Rails

Manage Image Files from Selenium Visual Testing

Steps to Apply This Project

1. Clone the WorkinCase/SeleniumTest

then you should be able to run

# run all test cases
rake test:fileset

or

# run all cases that quoted by single quote and separated by comma
cases="['simple_test']" rake test:fileset

in terminal

2. Create Jenkins Job with the content of Jenkins Job Settings at

README.rdoc


basically two parts, parameters and shell, then you can run it with Jenkins as the demo video shown above.

3. Regarding the Rails Web App, run it as your current user if you always run test cases from termnal, or run it by Jenkins as described in Manage Image Files from Selenium Visual Testing
if you usually run test cases with Jenkins

Modify test_helper.rb#L234 and Line 248 to your rails app url to change the url of output message


Different in Details

Add one more page '/vt/cases' to list all integration test cases (see routes.rb#L2 )

The working case postpone image processing by overriding MiniTest::CompositeReporter (see test_helper.rb#L200 ) so no need to read/write file.

Still override Rake::Task to rescue from rake aborted (see test.rake#L4 )
so Jenkins job will not be interrupted (since some operation needed after task finished)

Take care of Errors / Exceptions and Image diff to report summary of failed cases, see test_helper.rb
Line 35, 41, 240 ~ 256 and the if - fi block of Jenkins shell script

Take care of permission, see visual_testing_controller.rb#L66 , test_helper.rb#L259 and chmod command of Jenkins shell script

Manage Image Files from Selenium Visual Testing

Simple Note

Now I can do Visual Testing with Selenium, but manage image files between folders such like check base image, delete base image, apply new image as base image, or view base/new/diff image at a time usually annoyed me.

To solve this issue, I have worte some simple web pages to manage the image files

Pre-Request

Visual Testing with Selenium in Rails

Online Demo

https://www.youtube.com/watch?v=RSC4LNMQ5_0

(One thing is missing, you can hold ctrl key and click image to open it)

NOTE: There are some permission issue if you run test case by Jenkins job, to solve those issue, you can try:

1. Start rails app by user jenkins :

# cd to rails project folder first

# make rails project folder can be
# accessed by jenkins
# use 777 or any other proper value
sudo chmod 777 -R .

# switch user to jenkins
sudo su jenkins

# start rails app by user jenkins
rails s

or

2. Make jenkins can run command as root if needed, please refer to How to run a script as root in Jenkins?

In this case, you will need to manage security with jenkins properly, see Standard Security Setup

Related Files

routes.rb

Define some route rules, including get folder content, get png file and list images for specific test case.

visual_testing_controller.rb

Controller to do the backend works, including list folder content, return png file and return images for specific test case.

application.html.erb

Just slightly modified to load jQuery.

show.html.erb

Simply list folder content.

listimgs.html.erb

Most complex one (with some ugly JavaScript / HTML) , list all images and provide some control.

Postpone Image Processing to All Test Cases Finished in Rails Selenium Visual Testing

Introduction

As we already known we cah do Visual Testing with Selenium by using Chunky PNG, but Chunky PNG may spend much time to decode a large PNG image and make selenium timeout. This article describe how to postpone the image processing after all test cases done.

NOTE: The content is more complex and contains something with respect to my real work, so I put it at a different branch that will not be merged back to master.

NOTE 2: This article is exactly just a simple note, please refer to the code and git commit diff if needed.

Pre-Request

Visual Testing with Selenium in Rails

How To Do

There are two major issues need to be solved.

1. How to do something after all test cases are finished?

2. How to pass date from test cases to task?

The first one can be done by overriding invoke method of Rake::Task, please refer to the updated test.rake file.


The second one can be done via read/write a file or standalone caching server since the task and test cases are under different processes. Regarding how to read/write hash from/to file, please refer to test.rake#L21 and visual_testing_test.rb#L37.

Also override Selenium::WebDriver::Driver to store ENV params in visual_testing_helper.rb so can separate folder for different remote driver.
, and move ActiveSupport::VisualTestingHelper to visual_testing_helper.rb then require it from test.rake and test_helper.rb.


To make the base image can be loaded faster for future testing, I convert it from png to rgba at test.rake#L28, this is a mistake since the code should be moved to ActiveSupport::VisualTestingHelper.convert_rgba.


References

adding attributes to a Ruby object dynamically


How does one - without inheritance - override a class method and call the original from within the new method?


Why does dumping and loading a Hash using Marshal in Ruby throw a FormatError?


wvanbergen/chunky_png/wiki#encoding-options

Visual Testing with Selenium in Rails

Pre-Request

Image Compare in Ruby

Simple Note





Gemfile

Added gem 'chunky_png'
(so need to run 'bundle install')



test_helper.rb

require two more modules 'fileutils' and 'chunky_png'

Added class ActiveSupport::VisualTestingHelper for visual testing



Added test case 'visual_testing_test.rb'



The screenshots will be stored in folder visualTesting
each test case has its own folder



Almost done the Selenium part, need to build Web App to work with it (e.g., confirm and select new base image via webpage) later.

Image Compare in Ruby

Simple Note

Image compare util that will be used in my Selenium Visual Testing later.

Steps

Add chunky_png to Gemfile

gem 'chunky_png'

then run (in terminal)

bundle install

Test Images:

test1.png

test2.png

Compare Scripts:

method_from_web.rb (Copied from Reference)


compare_multi_blocks.rb (Modified from above)


Run .rb files from terminal as below

ruby path/to/compare_multi_blocks.rb

Compare Result

diff.png (by method_from_web.rb)


diff2.png (by compare_multi_blocks.rb)


Reference

Comparing images and creating image diffs

Make Test Case as Assistant

Introduction

Assume you use TDD, write test case first, and now you need to start to produce the minimum amount of code to pass that test.

This article describe how to make a test case as an assistant that can keep updating current issue when you are writing code, so you do not need to run test case by hand periodically to check current status.

Result

Please refer to the online demo:
ruby selenium auto retest

Pre-Request

Actually not really needed, just related for this article,
can work with local driver from terminal well.

Use Remote Webdriver in Selenium

Create Parameterized Test Service with Jenkins

Steps

1. Create an error filter

As an assistant, I wish it can report summarized error message in stead of full backtrace, so I write a filter method in test helper:

def self.displayError (error, file)
  puts "Error: " + error.message
  errArray = error.backtrace.map{ |x|
    x.match(/^(.+?):(\d+)(|:in `(.+)')$/); 
    [$1,$2,$4]
  }
  errArray.each do |err|
    if err[0].end_with? File.basename(file)
      puts "Line: "+err[1]+", "+err[2]+", File: "+File.basename(file)
      puts ""
    end
  end
end

2. Make it can retest automatically if failed.

So it will do test again and again until everything works fine, the code will as below:

reTest = true
# ...
while reTest do
  # clear reTest
  reTest = false
  # do some action and quit
  begin
    # ...
  rescue => error
    reTest = true
    # ...
  ensure
    driver.quit unless reTest
  end
end

Done, now you can run the test case then start to write your code, you will keep receiving current status from the test case.



Added/Updated Files


Modified test_helper.rb


Added auto_retest_if_fail_test.rb

Added auto_redo_test.html

Added ubuntu_folder
So can use remote webdriver for 127.0.0.1



References

Ruby - get the file being executed

Catching line numbers in ruby exceptions

Install and Configure Gitlab for Local Network in Ubuntu

Introduction

Gitlab is a Web Application for Git Version Control System, written in Rails.
This article describe how to install Gitlab on Ubuntu 14.04 and config it to work with local network.

Pre-Request


A working Ubuntu installation, e.g.

Install Ubuntu 14.04 on VirtualBox

Steps

Install Gitlab CE

sudo apt-get update

# from official site
# 1. Install and configure the necessary dependencies
sudo apt-get install curl openssh-server ca-certificates postfix

# 2. Add the GitLab package server and install the package
curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash
sudo apt-get install gitlab-ce

# E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

sudo apt-get update
sudo apt-get install gitlab-ce

# E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

sudo apt-get install gitlab-ce --fix-missing

# E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

sudo apt-get update --fix-missing
sudo apt-get install gitlab-ce

# Finally it works...
# it will tell you where you can link to after reconfigure

I don't know which step fixed the problem, probably just the network issue?

Configure and Start Gitlab

# 3. Configure and start GitLab

sudo gitlab-ctl reconfigure

Then link to the url provided by "# 2. Add the GitLab package server and install the package", for me its "http://benbai-virtualbox/"

use the name/pass from official site to login (its root/5iveL!fe now)


Change myhostname and set rewrite rule for sender

After gitlab launched, I tried to sign up a new account but has problem to receive confirmation email
the message below is found in my /var/log/mail.log

... Our system has detected that this message is 550-5.7.1 likely unsolicited mail... Our system has detected that this message is likely unsolicited mail. To reduce the amount of spam sent to Gmail, this message has been blocked. For more information, review this article.

what I did to solve this:

edit /etc/postfix/main.cf

# change myhostname:

# this is raquired to escape from gmail 550 5.7.1

myhostname = yahoo.com.tw

# add sender_canonical_maps:

# specify the file that contains rewrite rule for sender

sender_canonical_maps = hash:/etc/postfix/canonical

create /etc/postfix/canonical and set rewrite rule as below

gitlab@benbai-VirtualBox    your_acc@yahoo.com.tw

then create the postfix db and restart postfix

# create the postfix db
sudo postmap /etc/postfix/canonical
# restart postfix

sudo postfix stop

sudo postfix start

finally I can receive the confirmation email, but it is still placed in 'spam' box...

Change Address for Gitlab

Another problem is the link of confirmation email is http://benbai-virtualbox but I want it be real IP address.

I solve this as below:

edit /etc/gitlab/gitlab.rb
modify external_url

# someone said this is the correct place

# but it doesn't work for confirmation however

external_url 'http://192.168.31.122'

edit /var/opt/gitlab/gitlab-rails/etc/gitlab.yml
modify host

# this works for the confirmation mail

host: 192.168.31.122

then restart Gitlab

# restart gitlab, you can also try simply 'sudo gitlab-ctl restart'

sudo gitlab-ctl stop

sudo gitlab-ctl start

Finally everything works well.

References

Official Download Page

SMTP Error Reference

Why has Gmail blocked my messages?

How to rewrite outgoing address in Postfix

How to change URL of a working GitLab install?