How to use Cucumber for GUI component testing and why you should not do this

Sometimes you have to deal with all of these:
  • user stories which are nothing else but simple requirements for some input fields; 
  • requirement to write e2e tests, where e2e is understood as "everything we can do using GUI";
  • phrase "we should use Cucumber because... just because this is the way we are used to do". 
We've started like this in one of our projects. I'm really happy finally we managed to get rid of "Cucumber for everything" requirement and are not using it for GUI component testing anymore. But we used to do this and here's some bad experience.

Imagine you have a "user story" for a input field called "Customer Reference", which looks something like this (in fact, it was more complex, but the main idea remains the same):

As a user I want to be able to enter customer reference number.
Acceptance criteria:
- number may contain alphanumeric symbols (including spaces)
- number can be up to 20 symbols length

Well, our Cucumber scenario might look like this:

        
Scenario Outline: Valid text input is available up to 20 symbols
    Given I am at the "Page1" page

    When I enter customer reference "<input reference>"

    Then I see input fields and values as listed
        | name                      | value               |
        | Customer Reference Number | <display reference> |

    Examples: For Page1 page
        | input reference                   | display reference      |
        | ref                               | ref                    |
        | ref with spaces                   | ref with spaces        |
        | 12356                             | 12356                  |
        | abcdefghij1234567890              | abcdefghij1234567890   |
        | abcdefghij1234567890_moreThan20   | abcdefghij1234567890   |

At first it might look OK. For a single page component testing this works rather well.

Now imagine we have reusable components and for some reason (e.g., customer requirement :)) we must run all components tests on each page they are used. Even more - we must have option to run tests only for selected pages.

What we can do with Cucumber tests? We can use tags and multiple example tables for scenario outline. To do this we need to add page as a variable to example tables and to create example tables for each page:

        
Scenario Outline: Valid text input is available up to 20 symbols
    Given I am at the "<page>" page

    When I enter customer reference "<input reference>"

    Then I see input fields and values as listed
        | name                      | value               |
        | Customer Reference Number | <display reference> |

    @Page1
    Examples: For Page1 page
        | page  | input reference                   | display reference      |
        | Page1 | ref                               | ref                    |
        | Page1 | ref with spaces                   | ref with spaces        |
        | Page1 | 12356                             | 12356                  |
        | Page1 | abcdefghij1234567890              | abcdefghij1234567890   |
        | Page1 | abcdefghij1234567890_moreThan20   | abcdefghij1234567890   |

    @Page2
    Examples: For Page2 page
        | page  | input reference                   | display reference      |
        | Page2 | ref                               | ref                    |
        | Page2 | ref with spaces                   | ref with spaces        |
        | Page2 | 12356                             | 12356                  |
        | Page2 | abcdefghij1234567890              | abcdefghij1234567890   |
        | Page2 | abcdefghij1234567890_moreThan20   | abcdefghij1234567890   |

Note that line 2 now contains reference to page variable in each of the example tables. Page1 and Page2 are just human names of page objects, which contain mapping with real URLs.

So, this works. We can run tests only for selected pages using Cucumber tags. What's so bad in all of this? Well, imagine that you have 20 pages with same components. So you copy/paste example tables 20 times. Later - imagine requirements change and you have to update ALL those copy/pastes. Usually you'll have more than one reusable component...

Finally for really complex components (e.g., date picker from calendar) you might need to check some rules "calendar does not allow to select day X if we have condition Y". So you might end with writing parameters in example tables, and parameters definetely have nothing to do with some "user value". Image below shows how it might look like.


So believe me, you don't want to use Cucumber for GUI component testing :)

What we are using now? Protractor + Jasmine for component tests, Protractor + Cucumber for real e2e tests.

Comments

Popular posts from this blog

Migrate Protractor E2E tests to async / await

Allure reporter, Gradle and JUnit5

Protractor, Jasmine and promises for non-browser functions