blog

Let's talk about testing

Web testing with Cucumber

This entry is part 1 of 10 in the series Test Automation

Agile testing methods

One of the keys to a successful agile software project is considering testing from the beginning. While in some projects, especially web projects, testing is sometimes an afterthought, one member of the team clicking away on the site for a while before deployment to production, which is not really a sustainable way of operation. At least not in a website that is used for something else then blogging. There’s nothing wrong with blogging, mind you, it’s just that the software is typically “ready” for use and people just change the content or posts. One way of achieving this goal is to use test driven development (TDD) or behavior driven development (BDD).

Test driven development

The word “driven” here refers to the way of writing the tests first, then the code to make the tests pass. Test driven development is more of a developer oriented approach, a more low level way of doing things.

Behaviour driven development

In BDD on the other hand, the tests are described in a natural language, which makes the tests more accessible to people outside of development or testing teams. It can describe the functionality as specifications do.

Cucumber

One implementation of BDD is Cucumber. Cucumber is a Ruby based test tool for BDD, Cucumber.  With Cucumber, the user describes the behavior of the system with natural language with some specific keywords. The process starts with creating a feature file, which explains a feature of the system and some scenarios of different test situations. As Cucumber doesn’t know how to interpret the features by itself, the next step is to create step definitions explaining it what to do when finding that step in one of the scenarios. The step definitions are written in Ruby. This gives much flexibility on how the test steps are executed. It can use and already has step definitions for Webrat, an acceptance testing tool for Ruby, which simulates a browser without Javascript support. But it can also be combined with a web automation framework like Watir to implement browser based web automation tests. Watir is what I’m going to be using in this example. This is not the traditional way of using Cucumber but works quite well anyway.

Feature

Here is an example of a simple feature description from a Cucumber feature file:

Feature: Website Navigation
  In order to navigate website pages
  I need to be able to click on menu item

The feature is an explanation of what we want to accomplish. It doesn’t really do much by itself but works are a specification for the system and the test.

Scenario

Now this feature needs to be tested with several scenarios. For example, user can be on a different page in the beginning of the test or the user can click on a different link. One scenario is shown below. It describes an operation, clicking on a menu item on our website:

Scenario: Going to page Services
  Given that I am on spriteCloud Home
  When I click on link Services
  Then the page title should be "spriteCloud - Services"

Step definitions

This is the part that we need to explain to Cucumber so that it knows what to do. In this case, we are using Watir with the Watir-webdriver gem. We also use RSpec gem to be able to use the should operation.

require 'spec'
require "watir-webdriver"

Then we define some constants, one for the site address and another one for referring to the browser. Note the syntax for starting an instance of Firefox with webdriver.

SITE = "www.spritecloud.com"
BROWSER = Watir::Browser.start(SITE, :firefox)
PAGES = {
  "spriteCloud Home" => "http://www.spritecloud.com/"
}

The hash PAGES is not really necessary in this case, but can be useful in a case when there is more data and pages involved. It is used for looking up a page URL based on page name in the scenarios.

Now that we covered the setup part, we need to start with the steps included in the scenario. The keyword “Given” is about setting up the test. We need to be on the home page in the beginning of the test.

Given /^that I am on (.*)$/ do |page|
  BROWSER.goto(PAGES[page])
end

Notice that the step syntax is formatted as a regular expression. Given is a keyword but everything else is basically just something that Cucumber can recognize as being part of your test. Let’s break that part down a bit:

  • Sentence starts with “Given” keyword. This is mandatory
  • What follows is a space and the words “that I am on”
  • Another regular expression (.*), which matches everything from “on ” until the end of the line “storing” it for later use for the variable page

Using the capturing group (.*) is important because it allows us to write more generic steps, where for example, the page where we are or the link that we are following is defined in the scenario, not in the step. This again allows for the less technical person to write tests without having to write Ruby code. It is of course also better programming practice not to write the same things multiple times and just changing a value.

Next keyword, “When” refers to an operation we do on the page. In this case, we want to click a link on the page. A link can be found with several different methods in Watir, here we use the :text method, which basically clicks on the link with the text we specify in the scenario.

When /^I click on link (.*)$/ do |link|
  BROWSER.link(:text, link).click
end

The last keyword in this example, “Then” refers to the result we are expecting. For the purposes of this example, we check that the page title is correct, i.e. the same as in the test scenario definition.
Then /^the page title should be "([^"]*)"$/ do |title|
  BROWSER.title.eql?(title).should == true
end

Test Execution

This type of test can be run either during the implementation phase, which is the original purpose or later on when the site is fully ready. Even if the test was created during the implementation phase, it should still be run with every new release that goes out. It can even be used as a part of a stand-alone test set for the site.

Part of the power of Cucumber is the nicely formatted output displaying immediately the successful and the failed tests and test steps. Successful tests are colored green on the terminal and failed one, you guessed it, red.

mbp-2:features marko$ cucumber .
Feature: Website Navigation
    In order to navigate website pages
    I need to be able to click on navigation

  Scenario: Going to page Services                        
    Given that I am on spriteCloud Home                   
    When I click on link Services                        
    Then the page title should be "spriteCloud - Services"

1 scenario (1 passed)
3 steps (3 passed)
0m1.261s

Summary

This post only scratches the surface of what can be done with either Watir or Cucumber. They are both powerful test tools by themselves and combining them can be the basis for a successful test automation project. It all depends on the needs in your project.

Interested in test automation and cucumber training? Let us know!


  • Sam

    * corrections

    require ‘spec’ # should be require ‘rspec’

    PAGES = {
    “spriteCloud Home” = “http://www.spritecloud.com/”
    }

    # should be

    PAGES = {
    “spriteCloud Home” => “http://www.spritecloud.com/”
    }

  • Joe Beale

    Your hash example has an equal sign with no right arrow.
    PAGES = {
    “spriteCloud Home” = “http://www.spritecloud.com/”
    }

    Inside of the braces, the equals sign should be followed by a right arrow in order for that code to work.
    PAGES = {
    “spriteCloud Home” => “http://www.spritecloud.com/”
    }

    • Thank you for the feedback Joe, has been updated!

  • Pingback: Detecting javascript errors using JSErrorCollector in firefox with cucumber and selenium or watir-webdriver()

  • Good post!
    Just a small fine tuning; the expresion:
    BROWSER.title.eql?(title).should == true

    …can be shortened and simplified using:
    BROWSER.title.should include(title)

  • shama

    hi
    I need to integrate cucumber tool with vsts 2010. Will anyone help me to do this set up.

  • stephen

    Nice, logical post to step through not only what Cucumber is but how it is implemented

    • spriteCloud

      Thanks! It remains one of our most visited posts 🙂

  • Karol

    Hi I am using Cucumber wit capybara and Selenium Grid 2.0
    Does anyone know how can I run test on one remote node but on more than one browser ? I could easily do that in C# in visual studion byt just setting up as many browsers as I wanted but I am completely new to Ruby, Cucumber and Capybara and I have no idea how do I do that. Do I need to modify my env.rb file ?

    my current env.rb file is below

    require ‘capybara’
    require ‘capybara/cucumber’
    require ‘rspec’
    require ‘selenium/webdriver’

    caps = Selenium::WebDriver::Remote::Capabilities.firefox

    caps.platform = :MAC

    Capybara.default_driver = :selenium
    Capybara.register_driver :selenium do |app|
    Capybara::Selenium::Driver.new(app,
    :browser => :remote,
    :url => “http://localhost:4444/wd/hub”,
    :desired_capabilities => caps)

    end

    Thanks

  • laxman

    hi ,
    does any one know what type of testing we can perform using cucumber testing tool ?
    thanks

    • spriteCloud

      You can perfom any kind of functional testing.

  • Pingback: Mobile test automation in 3 simple steps()

  • anil

    Hi every one,I am new to start Ruby on rails development,
    What tools are used to development AND testing tools?
    can you tell me the Names on that one

    Plz help me

  • ATester

    So I’ve been involved in manual testing for the last 15 years…and because of that timeframe most of what I’ve done is in a classic waterfall environment.

    Adapting to Agile/Scrum has been difficult, to say the least…because it typically flies in the face of everything I know about testing. No documents? No long term planning? No TIME?

    I’ve used tools like WinRunner to automate tests using a simple recording method (Similar to Selenium) where I can get some results…but the use of these tools has been clunky at best. Now with Cucumber it’s almost seeming that the trend is to change your testers into developers with a knowledge of how to code? Then remove the need or even desire to have classic testers in a development team?

    Excuse me, but that’s not nearly as easy as you’d like to think it is.

    Your examples above don’t address this possibility of someone who still has a lot to provide to a development group, but was never someone with great knowledge of coding.

    Should I be worried that this type of software development trend is going to put me out of a job because I can’t understand how to use a tool like Cucumber?

    And if I have to spend all of my time coding test scenarios in Cucumber, how is that really saving me time as opposed to writing them up in a spreadsheet in plain english that I can just give ANYONE (testing background or not, development background or not) to execute?

    • spriteCloud

      I don’t think that worrying about this trend is necessary or useful. You are absolutely correct that automating test cases isn’t a silver bullet to solve all development problems.

      I would view it not as a worrying trend, but as an opportunity: you will see a natural split emerge between activities surrounding the definition of a good test case, and activities surrounding the execution of existing test cases.

      Only the latter can be automated. The opportunity lies in contributing to the former! Developers who can automate test cases are fairly easy to find; people with the skillset to establish good test cases are much rarer.

  • Mark

    Wow….that seems like a lot of effort and unnecessary complexity in order to create what is essentially a weak test case.

    I accept that for the purpose of demonstration your example was deliberately simple but the whole approach seems sketchy to me.

    Complexity is the rub….unknown requirements, unknown dependencies, unintended consequences, race conditions and last but but least…the passing of time.

    Trying to reduce requirements to binary functions cannot capture the full and necessary complexity of many systems.

    This might work for something simple I do believe it will scale well with complexity.

  • Joe

    Mark,
    I am using Cucumber for all kinds of tests, from web automation to running batch jobs through a scheduler to hitting a database to comparing files and creating reports. If you can document the manual test, then you can automate it.

  • surekha

    Hi Joe,
    I am new to Cucumber… I want to learn this… Will you provide online classes? or please guide me if any one is giving trainings on this tool…

  • sanjeeb ray

    Hi Joe,
    I am new to Cucumber… I want to learn how to find element for mouse hover action . pls help

    • I’m not entirely sure what you mean. If you want to fire a hover event for an element, try e.g.:

      browser.element(:how => what).hover

  • vipul

    hi

    i was trying to develop a ruby code to Log some data into successive cells of a column in an Excel Sheet.

    Kindly guide me on this.

    • spriteCloud

      Hi,

      the topic is *web* testing with cucumber. If your application doesn’t have a web interface, then some of the things here simply don’t apply.

      You can probably use cucumber to drive the tests, but anything to do with watir described here will need to be replaced with something that drives Excel.

      Instead of using watir, you might want to try out https://github.com/jarmo/rautomation

      Hope that helps!

  • Peter

    Hi Marko,

    We are interested in the 2 days hands on training of staff about test automation using the open source frameworks: Selenium / Cucumber / Ruby posted at http://www.spritecloud.com/testing/automation/. My team is in US. Do you offer online training via WebEx?
    Please direct me to your point person for the training.
    Thanks,
    Peter

    • Hi,

      there’s no reason we can’t offer training via WebEx, but you should really contact info@spritecloud.com for any of this. Blog comments aren’t monitored by the people in charge of making this happen. I’ve forwarded your request, though, so no need to do it this time 🙂

      Thanks,
      Jens

      • Peter

        Thank you Jens.

  • Rahul

    It seems like half of this post is missing. How do you actually use the above? On one hand you it is extremely simplistic in what it is trying to do, and the other, it assumes the user knows how to use Cucumber. What directories/files should be created for the above? What needs to be installed? All of this information would be helpful

    • Thank you for your feedback. You’re right, that this post assumes basic cucumber knowledge. It is mainly about integrating Watir with cucumber.

      But it and other posts here are also out of date. We are preparing a new series of posts over the next few weeks containing all the details. Until then, we can only recommend looking to https://cukes.info for basic usage instructions.

  • Brian

    Hi Mark

    Can you suggest an approach to testing an existing web application system, which has a working back end, and a new front end being engineered.

    Tests are not yet automated, however all scenarios, test steps and data have been specified, and the original front end has been tested manually.

    We are considering learning how to implement Cucumber, but are also wondering about using Selenium and recording the tests, for the existing system.

    Our questions are:
    1 – Which approach (record using Selenium or create new Cucumber) will be fastest to provide the tests for the front end (we already have the exact steps and data available).

    2 – Is there a downside to either approach?

    3 – Am I right in assuming that for new features, we would be best to use proper BDD approach with Cucumber?

    Looking forward to your opinion.

    Thanks

    Brian

    • Hi Brian,

      that mostly depends whether you record Selenium tests or not, and whether you’ve got a multi-functional team or not.

      Selenium record and playback is of course the most convenient method for creating test suites, but maintenance may be time consuming when the test target changes. Also, tweaks to the test suite will require a new recording.

      If either of that is a problem, it’s better to code test suites using either Selenium or cucumber. When you’re using cucumber, you’re also going to use Selenium to drive the tests – for example, using watir.

      What cucumber adds to Selenium is the ability to write non-technical, text-only test suites, and have someone else automate these test cases. That way, you have an interface between technical and non-technical stakeholders. It also greatly helps in documenting the *purpose* of each test scenario, as opposed to merely what it *does*.

      Because of those benefits, we’d generally recommend cucumber and watir.

      • Brian

        Hi

        Thanks for your reply. Very helpful. We should be able to convert our tests into Cucumber “speak” pretty easily.

        As I mentioned before, we will continue to create new features for the product. This leads me to my next question….

        Can you recommend a good way to record our stories / features and the associated Gherkin style test cases which would be Cucumberised.?

        We use Confluence wiki and Jira and have in the past used Excel to specify test cases and data.

        As it’s my role to make a decision going forward in how we specify / capture tests, etc, I would be keen to hear of any templates or forms / formats that you have experienced, so that we can make use of best practice as you see it.

        Once again, I’m looking forward to your reply.

        Thanks

        Brian

        • Right now, I don’t have a good recommendation for cucumberising selenium recordings, no.

          We’ll probably build that eventually anyway, if nobody beats us to it. If you have a strong enough interest in this that it translates into wanting to co-operate here, send me an email: jens@spritecloud.com

  • bhuvan.sundar

    Hi All
    I am new to Watir tool.I am using Notepad++ for developing code.Here using Puts statements.Where can I see the output
    Please let me know which is the good editor for Watir.
    Appreciate if you could clear my queries.
    Thanks,
    Bhuvan

  • bhuvan.sundar

    Hi Joe Beale,

    I am executing below code in Rubymine. Getting below errors.Please help on this.

    Feature: Amazon Search functionality

    Scenario: User can search for a specific item on Amazon
    Given a User goes to Amazon
    When they search for “pens”
    Then amazon should return results for “pens”

    When(/^they search for “([^”]*)”$/) do |arg|
    @b.text_field(:id=>”twotabsearchtextbox”).set “#{arg}”
    @b.send_keys :return
    end

    Then(/^amazon should return results for “([^”]*)”$/) do |arg|
    @b.div(:id=>”results_2″).wait_until_present
    output1=@b.div(:id=>”resultsCol”).text.include? “#{arg}”
    if output1==true then
    puts “pass”
    Else
    puts “fail”
    end
    end

    Execution Errors:

    C:Ruby22binruby.exe -EUTF-8 -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) C:Ruby22bin/cucumber C:/Users/admin/RubymineProjects/Sample/features/sam1 –format Teamcity::Cucumber::Formatter –expand –name “^User can search for a specific item on Amazon$” –color -r features
    Testing started at 21:32 …
    wrong number of arguments (1 for 3..4)
    Error creating formatter: Teamcity::Cucumber::Formatter (ArgumentError)
    C:/Program Files (x86)/RubyMine 7.1.4/rb/testing/patch/bdd/teamcity/cucumber/formatter_03103.rb:22:in `initialize’
    C:/Ruby22/lib/ruby/gems/2.2.0/gems/cucumber-2.1.0/lib/cucumber/runtime.rb:209:in `new’
    C:/Ruby22/lib/ruby/gems/2.2.0/gems/cucumber-2.1.0/lib/cucumber/runtime.rb:209:in `create_formatter’
    C:/Ruby22/lib/ruby/gems/2.2.0/gems/cucumber-2.1.0/lib/cucumber/runtime.rb:202:in `block in formatters’
    C:/Ruby22/lib/ruby/gems/2.2.0/gems/cucumber-2.1.0/lib/cucumber/configuration.rb:176:in `block in formatter_factories’
    C:/Ruby22/lib/ruby/gems/2.2.0/gems/cucumber-2.1.0/lib/cucumber/configuration.rb:171:in `map’
    C:/Ruby22/lib/ruby/gems/2.2.0/gems/cucumber-2.1.0/lib/cucumber/configuration.rb:171:in `formatter_factories’
    C:/Ruby22/lib/ruby/gems/2.2.0/gems/cucumber-2.1.0/lib/cucumber/runtime.rb:201:in `formatters’
    C:/Ruby22/lib/ruby/gems/2.2.0/gems/cucumber-2.1.0/lib/cucumber/runtime.rb:183:in `report’
    C:/Ruby22/lib/ruby/gems/2.2.0/gems/cucumber-2.1.0/lib/cucumber/runtime.rb:67:in `run!’
    C:/Ruby22/lib/ruby/gems/2.2.0/gems/cucumber-2.1.0/lib/cucumber/cli/main.rb:32:in `execute!’
    C:/Ruby22/lib/ruby/gems/2.2.0/gems/cucumber-2.1.0/bin/cucumber:8:in `’
    C:/Ruby22/bin/cucumber:23:in `load’
    C:/Ruby22/bin/cucumber:23:in `’
    -e:1:in `load’
    -e:1:in `’

  • CHACKRAVARTHI J

    Hi,

    I am always getting the below error when I try to run the Ruby script using Cucumber syntax.

    `Cucumber::Ast` no longer exists. These classes have moved into the `Cucumber::Core::Ast` namespace, but may not have the same API.

    Error creating formatter: Teamcity::Cucumber::Formatter (RuntimeError)

    Ruby Version : Ruby 2.2.4 (x64)
    Cucumber Version : Cucumber 2.2.3
    IDE : JetBrains RubyMine 5.4
    OS: Windows 7 64 bit

    Please let me know if you need other information. Ima stuck with thi issue for the past 1 week.

    Thanks
    Chackravarthi

  • narasimha

    package cucumber.examples.java.calculator;

    import cucumber.api.CucumberOptions;
    import cucumber.api.testng.TestNGCucumberRunner;
    import cucumber.api.testng.CucumberFeatureWrapper;
    import org.testng.annotations.AfterClass;
    import org.testng.annotations.BeforeClass;
    import org.testng.annotations.DataProvider;
    import org.testng.annotations.Test;

    /**
    * An example of using TestNG when the test class does not inherit from
    * AbstractTestNGCucumberTests but still executes each feature as a separate
    * TestNG test.
    */
    @CucumberOptions(format = “json:target/cucumber-report-feature-composite.json”)
    public class RunCukesByFeatureAndCompositionTest extends RunCukesByCompositionBase {
    private TestNGCucumberRunner testNGCucumberRunner;

    @BeforeClass(alwaysRun = true)
    public void setUpClass() throws Exception {
    testNGCucumberRunner = new TestNGCucumberRunner(this.getClass());
    }

    @Test(groups = “cucumber”, description = “Runs Cucumber Feature”, dataProvider = “features”)
    public void feature(CucumberFeatureWrapper cucumberFeature) {
    testNGCucumberRunner.runCucumber(cucumberFeature.getCucumberFeature());
    }

    @DataProvider
    public Object[][] features() {
    return testNGCucumberRunner.provideFeatures();
    }

    @AfterClass(alwaysRun = true)
    public void tearDownClass() throws Exception {
    testNGCucumberRunner.finish();
    }
    }

    can any can explain what this program will do line by line?

Reputation. Meet spriteCloud

Find out today why startups, SMBs, enterprises, brands, digital agencies, e-commerce, and mobile clients turn to spriteCloud to help improve their customer experiences. And their reputation. With complete range of QA services, we provide a full service that includes test planning, functional testing, test automation, performance testing, consultancy, mobile testing, and security testing. We even have a test lab — open to all our clients to use — with a full range of devices and platforms.

Discover how our process can boost your reputation.