Selenium Auto Exec Server(AES)

Japanese / English

Selenium Auto Exec Server(AES) is a tool to do a continuous regression test by Selenium.

Selenium AES aims help of the test in the project.

If Selenium AES is used, the test of the HTML form of Selenium is executed at the time decided every day, and the result can be transmitted with mail.

Various functions to help the regression test that uses Selenium are offered.


Selenium AES enhances Selenium RC and is made.

Moreover, the one named Selenium HTMLSuite Extension is opened to the public as a tool that improves convenience to HTMLSuite of Selenium RC, and Selenium AES uses this library.
(Selenium HTMLSuite Extension was made during the development of Selenium AES.)


Selenium AES has the following functions.

Regular execution of test

The test can be executed automatically at the time decided every day.

With Subversion

After the test case is acquired from the repository of Subversion, the test can be executed when beginning to test.

As a result, time when the test case is arranged in the machine for the test can be saved.

Result Mail Sending

The test result can be transmitted as Mail(SMTP).

Test result is attached to this mail.

Test situation confirmation on a browser

The test situation can be confirmed on a browser.

Moreover, the execution of the test can be directed on a browser.

Test execution API

It is also possible to operate in cooperation with other CI servers because there is API to direct the execution of the test with HTTP.

The plug-in is prepared for Hudson.

Jar that becomes a client to execute API of Selenium AES from the command line etc. is being offered, too.

Multiple HTMLSuite can be specified

Corresponding HTMLSuite to a specific file pattern ("suite" or "Suite" is included in the file name) in a specified directory can be collectively executed.

Automatic construction of HTMLSuite

HTMLSuite need not be made.

HTMLSuite to execute those test cases can be generated with making the test case by a specific file pattern ("case" or "Case" is included in the file name) automatically, and it execute it.

Multiple browser execution

It can be specified that the same test case is executed by two or more browsers.


Operating environment

Java Runtime Environment (JRE1.5 or more) is required.

I think that OS operates by Windows, Unix/Linux, and Mac OS X as well as Selenium RC.
(It is a confirmed operation in Windows XP and Mac OS X. )

Download

selenium-autoexec-server-1.2-dist.zip ver 1.2 (2011/08/02)


The source code under development can be obtained in the following.

selenium-extension - Google Code

License

Apache License, Version 2.0

How to use

The downloaded zip file is defrosted.

(In the defrosted state, the test that uses Firefox and IE to http://www.google.com/ as a sample can be executed. )

Adjusting the configuration file

Configuration file (setting.properties) is adjusted responding to the environment used.

The content of the configuration file is as follows.

Item Required Content
browser Required An execution browser of the test is specified. The plural can be specified by switching off the comma district.
etc: *iexplore,*firefox,*opera,*safari
startURL Required URL that begins testing is specified.
generateSuite - "false" is specified when not doing "true" when automatic of HTMLSuite generating it. (Default is "true". )
The file that attaches "Case" or "case" in the directory specified with suite is judged to be HTML of the test case, and the HTMLSuite file is made.
Please refer to the column of "suiteDir" for details.
suiteDir - The directory of test case and HTMLSuite are specified (The relative path is OK).

The test case and HTMLSuite executed by setting generateSuite change.
[The directory name is specified. (The test case is in a specified directory. ) : generateSuite=true or non set.]
HTML of the test case is retrieved from a specified directory, HTMLSuite is made for this directory with the file "generatedTestSuite.html", and it executes it.
[The directory name is specified. (The test case is not in a specified directory.) : generateSuite=true or non set.]
HTMLSuite of each directory under the control of a specified directory is made when there is no test case in a specified directory.
The file name of HTMLSuite becomes <directory name>.html. It is convenient to make HTMLSuite separately by the function.
[File name specification : generateSuite=true or non set.]
HTMLSuite is generated with the specified file name automatically.
The test case becomes an object in the same directory as specified HTMLSuite.
[generateSuite=false]
HTMLSuite is retrieved ("Suite" or "Suite" is included in the file name), and executed in the directory.
suiteRepo - It is repository (Subversion) at the destination of the acquisition of the test case and HTMLSuite.
It arranges it in the directory from the specified repository from which the file is specified with the import doing and suiteDir.
suiteRepoUsername - It is an user-name of repository (Subversion) at the acquisition destination.
suiteRepoPassword - It is a password of repository (Subversion) at the acquisition destination.
permanentResult - "true" is leave test result.
When the test result is referred back, it is necessary to note it because a past test result cannot be referred when "false" is specified (The test result is always overwrited).
resultDir - The directory that outputs the test result is specified.
The file of the test result is output under the control of a specified directory as "<test execution time *>/<browser name>/<HTMLSuite name>_result.html."
(* The test execution time is not included in passing for permanentResult=false. It will always be output to the same directory.)
It operates assuming that "Result" is specified when omitting it.
autoExecTime - execution time of day.
The test is executed automatically at the time specified here every day.
It specifies it with "hh:mm". (example: 1:00 and 14:30)
port - The port number in which the Selenium server is started is specified.
Default is 4444.
timeout - The timeout period (second) when HTMLSuite is executed is specified.
Default is 1800 seconds.
singleWindow - When "true" is specified, the test is executed in single window.
debug - When "true" is specified, a detailed log (debugging log) is output.
log - To output the log to the file, the file name is specified.
userExtensions - User extension file of Selenium described by JavaScript.
proxyHost - The host of the proxy used is specified.
proxyPort - The port number of the proxy used is specified.
avoidProxy - When "true" is specified, requests other than "/selenium-server" come not to be received with the Selenium server.
firefoxProfileTemplate - The profile directory used with Firefox is specified.
mail.host - It is a host name with the mail server used when the test result is transmitted.
It is transmitted with SMTP.

The setting example concerning mail is described later, and give that to me in the reference in detail, please.
mail.port - It is a port number of the mail server used when the test result is transmitted.
default is 25(standard port of SMTP).
mail.ssl - When SSL is used by Mail Sending, "true" is specified.
mail.username - It is an user-name used for the authenticate at Mail Sending.
mail.password - It is a password used for the authenticate at Mail Sending.
mail.to - It is a destination address of the test result mail.
mail.from - It is a from address of the test result mail.
mail.subject - It is a subject of test result mail format.
The variable named ${result} can be used. ("passed" and in case of failure "failed" when test result is normality)
Default is the following formats.
Selenium Test Report [${result}]
mail.debug - When "true" is specified, the DEBUG log of Mail Sending is displayed in the console.
beforeCommand - The command executed before it begins to test is specified.
afterCommand - The command executed after the test ends is specified.
trustAllSSLCertificates - When "true" is specified, all SSL certificates are trusted.
If this option is not specified when warning goes out by the SSL certificate that the server to be examined offers, it becomes an error.

Arrangement of test case and HTMLSuite

When the test case and HTMLSuite are acquired from the repository of Subversion, information on the repository is set to suiteRepo, suiteRepoUsername, and suiteRepoPassword of the configuration file.

# Repository URL where test case and HTMLSuite are arranged
suiteRepo=http://192.168.0.1/svn/trunk/selenium/
# user/password to access the above-mentioned repository(When the attestation is unnecessary, you may not set anything. )
suiteRepoUsername=devuser
suiteRepoPassword=devpass

The file of latest revision is acquired from a specified repository when the test is executed by setting it like the above-mentioned, and import is done to the directory specified for suiteDir of the configuration file.


The test case is arranged in a specified directory by hand power when not cooperating with Subversion.

In the defrosted state, the test case of the sample is arranged. Please delete it when actually using it by the project.

Start of Selenium AES

Start Selenium AES.

"start.bat" is executed for Windows. "start.sh" is executed for Linux.

If "Start Selenium Auto Exec Server" is displayed in the console, it is start completion.

The execution environment of Java (JRE1.5 or more) is needed for the execution of Selenium AES.

Execution of test

The test can be executed as follows by various methods.

(A) Regular execution of test

It can be set that it executes it at the time decided every day.

The time that wants to be executed is specified for autoExecTime of the configuration file.

autoExecTime=1:30

It is executed at 1:30 every day when setting it as stated above.

(B) Execution on a browser

It accesses the machine that starts Selenium AES from a browser, and the test can be executed on a browser.

URL is "http://<Machine that starts Selenium AES>:<Port number specified with "port" of configuration file>/selenium-server/autoexec/"

The following are URL when it refers by a browser on the machine that starts Selenium AES (Specification is not in port of the configuration file (Become 4444 of the defaults)).

http://localhost:4444/selenium-server/autoexec/

top page

When the "Run Test" button is pressed on this screen, the test is executed. (Additionally, the content that can be confirmed on the screen is described later. )

(C) Hudson and cooperation

To start Selenium AES from Hudson that is the CI tool, the plug-in is being offered.

The plug-in can be installed from the plug-in update center of Hudson.


The execution of Selenium AES can be added by using the plug-in as Bild procedure.

And, it comes to be able to refer to the execution result of Selenium AES on Hudson.

Hudson

(D) Executes from the command line by using jar for the client.

The jar file for the client is bundled. (selenium-autoexec-client-*.jar)

The test can be executed from the command line by using this.

"http://<Machine that starts Selenium AES>:<Port number specified with "port" of configuration file>/selenium-server/autoexec/" specified for a parameter.

java -jar selenium-autoexec-client-1.2.jar http://localhost:4444/selenium-server/autoexec/command/

Only when the execution of the test is instructed without waiting for the result of the test, "-a" is specified for a parameter.

java -jar selenium-autoexec-client-1.2.jar -a http://localhost:4444/selenium-server/autoexec/command/

(E) Executes as JUnit.

Class (com.enjoyxstudy.selenium.autoexec.client.SeleniumAutoExecRemoteTest) that executes it as a test case of JUnit is included in jar file (selenium-autoexec-client-*.jar) for the client.

The test can be executed by using this as one of the test cases.

Bildfile to execute this with Ant is described to the following as an example.

<?xml version="1.0" encoding="UTF-8"?>
<project basedir="." default="junit-test" name="selenium-autoexec-client">
  <path id="classpath">
    <pathelement location="selenium-autoexec-client-1.2.jar"/>
  </path>

  <target name="junit-test">
    <mkdir dir="junit-dist"/>
    <junit fork="yes" printsummary="withOutAndErr">
      <!--<sysproperty key="seleniumaes.async" value="true"/>-->
      <sysproperty key="seleniumaes.url" value="http://localhost:4444/selenium-server/autoexec/command/"/>
      <formatter type="plain"/>
      <test name="com.enjoyxstudy.selenium.autoexec.client.SeleniumAutoExecRemoteTest" todir="junit-dist"/>
      <classpath refid="classpath"/>
    </junit>
  </target>

</project>

(F) Execution from Web API

The execution of the test can be directed by the HTTP request by using API.

All these API is used excluding regular execution and it operates it.


URL is "http://<Machine that starts Selenium AES>:<Port number specified with "port" of configuration file>/selenium-server/autoexec/command/run".

The following are URL when it refers by a browser on the machine that starts Selenium AES (Specification is not in port of the configuration file (Become 4444 of the defaults)).

http://localhost:4444/selenium-server/autoexec/command/run

The following test results are returned as body of the response.

result: passed
number of cases: passed: 2 / failed: 0 / total: 2
start time: 2008-11-08 22:11:15
end time  : 2008-11-08 22:12:02
----------------------------------------------
generatedTestSuite.html: *firefox: passed
generatedTestSuite.html: *iexplore: passed
----------------------------------------------

first line is "result: passed" for test passed. "result: failed" for test failed.

"type=json" is applied to query, the result is returned by the JSON form. (Undergone plastic operation on the following as seen easily. )

{
  "result" : "passed",
  "totalCount" : 2,
  "passedCount" : 2,
  "failedCount" : 0,
  "startTime" : "2008-11-08 22:40:42",
  "endTime" : "2008-11-08 22:41:04",
  "suites" : [
    {
      "suiteName" : "generatedTestSuite.html",
      "resultPath" : "/selenium-server/autoexec/result/20081108224042/firefox/generatedTestSuite_result.html",
      "browser" : "*firefox",
      "status" : "passed"
    },
    {
      "suiteName" : "generatedTestSuite.html",
      "resultPath" : "/selenium-server/autoexec/result/20081108224042/iexplore/generatedTestSuite_result.html",
      "browser" : "*iexplore",
      "status" : "passed"
    }
  ]
}

Only when the execution of the test is instructed without waiting for the result of the test, it specifies "/selenium-server/autoexec/command/run/async" for URL.

http://localhost:4444/selenium-server/autoexec/command/run/async

"result: success" is returned as body of the response at the acceptance completion.

Information from the browser

The execution situation and the test result can be referred to on a browser.

URL is "http://<Machine that starts Selenium AES>:<Port number specified with "port" of configuration file>/selenium-server/autoexec/".

The following are URL when it refers by a browser on the machine that starts Selenium AES (Specification is not in port of the configuration file (Become 4444 of the defaults)).

http://localhost:4444/selenium-server/autoexec/

top page

The content is updated as for this screen every 30 seconds.

The content of each display item is as follows.

Item Content
status It is a state of Selenium AES.
  • idle: idle
  • run: The test is being executed.
The "Run Test" button can be pushed for idle. The test can be executed by the button pressing.
result It is an execution result of the test.
  • passed
  • falied
number of cases It is a detail of the test result.
The number of HTMLSuite of each results and the total of HTMLSuite are displayed.
start time It is a start time of the test.
end time It is a end time of the test.

It is a content of the item of each HTMLSuite.

Item Content
suite It is HTMLSuite file name.
It changes to a detailed screen of the test result when the link is clicked.
browser It is an execution browser name.
status It is a state of HTMLSuite.
  • wait
  • run
  • passed
  • failed

Example of filling in configuration file for result Mail Sending

The item described to the configuration file is different according to the setting of the mail server used.

The setting example is described to the following.

(A) SMTP

# server name
mail.host=smtp.enjoyxstudy.com

# The following are set, except when the port number is 25. 
#mail.port=925

# to address
mail.to=dev-ml@sample.enjoyxstudy.com
# from address
mail.from=report@sample.enjoyxstudy.com

(B) SMTP-AUTH

The authentication information is specified with "mail.username" and "mail.password".

# server name
mail.host=smtp.enjoyxstudy.com

# The following are set, except when the port number is 25. 
#mail.port=925

# authentication user name
mail.username=userhoge
# authentication password
mail.password=paspaspas
              
# to address
mail.to=dev-ml@sample.enjoyxstudy.com
# from address
mail.from=report@sample.enjoyxstudy.com

(C) SMTP-AUTH over SSL

It comes to use SSL by setting "true" to "mail.ssl".

# server name
mail.host=smtp.enjoyxstudy.com

# server port number
mail.port=465

# use SSL
mail.ssl=true

# authentication user name
mail.username=userhoge
# authentication password
mail.password=paspaspas
              
# to address
mail.to=dev-ml@sample.enjoyxstudy.com
# from address
mail.from=report@sample.enjoyxstudy.com