CSS Regression Testing With Selenium and Hermione.js

Table of contents

Introduction

We all know that CSS often breaks as all our styles live in the global namespace across the whole page. There are tools to limit this risk such as CSS modules or Styles Components (for React JS) but these ones force you to follow their standards even if you don’t like them.

You may also be interested in tools such as “Pixel Perfect” but these ones force you to manually compare each section, which is definitely not going to help you in the long term.

Fortunately, we may automate this process! The Internet lacks resources on this but they are there and this article is going to be a summary of the most important things concerning the configuration of Hermione.js.

What actually is regression testing?

Regression testing means testing existing software applications to make sure that a change or addition hasn’t broken any existing functionality. Regression testing is the process of testing changes to computer programs to make sure that the older programming still works with the new changes. Regression testing is a normal part of the program development process.

What is Selenium?

Selenium is a portable framework for testing web applications. It also provides a test domain-specific language (Selenese) to write tests in a number of popular programming languages. The tests can then run against most modern web browsers. Selenium deploys on Windows, Linux, and macOS platforms.

It is open-source software, released under the Apache 2.0 license: web developers can download and use it without charge. Testing in Selenium may be performed in parallel, meaning less waiting time.

What is Hermione.js and how can it help me?

Hermione.js is a “browser test runner based on mocha and wdio”. It opens the desired browser and performs actions defined in the test suites. Tests run in parallel so the waiting process is short.

Hermione has a very feature-rich API to cover all your needs. You may, among others, type your desired resolution to capture the screen at.

But what’s most important for us – we may take a reference screenshot of a page and then compare this screenshot each time we make a change in UI to make sure that we touched only what was intended.

Testing with Hermione.js is a plain process with minimal configuration.

CSS regression testing with Selenium and Hermione.js demo

Let’s say we have the following dummy page:

See how to do regression testing with Selenium and Hermione.js

Let’s pretend we want to make each option 10% taller. We write some simply transform and check what it looks like:

Regression testing with Selenium and Hermione.js.

The change is very subtle. Each option is a pixel or so taller so at first everything looks cool. Let’s run Hermione just to make sure:

CSS regression testing with Selenium demo.

Oops! It looks like we accidentally also upscaled the description on the right!

See what the difference between the original and changed image looks like?

Testing with Selenium and Hermione.js, you can spot unwanted visual changes immediately before they hit your users, making both devs and businesses happy.

When you run a diff and the only differences are the intended ones, then you may save the image as new references using hermione –update-refs.

How to set up Hermione.js

Step 1

You will need “selenium-standalone”.

Step 2

After you install Selenium Standalone, type selenium-standalone start in your command line.

Step 3

Install Hermione globally -> npm install -g hermione.

Step 4

Install “HTML reporter” – a Hermione plugin for reporting differences between two images -> npm install html-reporter.

Step 5

Create a Hermione configuration file called .hermione.conf.js in, for example, the following manner:

module.exports = {
sets: {
desktop: {
files: 'tests/common'
}
},

baseUrl: 'http://localhost:3000',
gridUrl: 'http://localhost:4444/wd/hub',
compositeImage: true,

browsers: {
chromeXL: {
desiredCapabilities: {
browserName: 'chrome',
chromeOptions: {
args: ['--headless'],
},
},
windowSize: '1300x900',
},
chromeMobile: {
desiredCapabilities: {
browserName: 'chrome',
chromeOptions: {
args: ['--headless'],
},
},
windowSize: {
width: 414,
height: 800
},
},
safariXL: {
desiredCapabilities: {
browserName: 'safari',
},
windowSize: '1300x900'
},
safariMobile: {
desiredCapabilities: {
browserName: 'safari',
},
windowSize: {
width: 414,
height: 700
}
}
},

plugins: {
'html-reporter/hermione': {
enabled: true,
path: 'my/hermione-reports',
defaultView: 'all',
baseHost: 'http://localhost:3000'
}
}
};

Note: We are running headless Chrome here. It’s much faster and better for the developer.

You may read more about headless Chrome here: https://developers.google.com/web/updates/2017/04/headless-chrome

Step 6

Create a simple test in tests/common such as skills.js and fill it with your test suite, e.g.:

it('creates a screenshot of the skills page', function() {
return this.browser
.url('/skills')
.assertView('Whole page', '#root')
});

Note: assertView is the most important method here. It’s used to compare images.

Step 7

If it is the first time you take a screenshot, type hermione –update-refs in your terminal window.

Step 8

Each time you make any visual change type hermione.

Step 9

If any of your tests do not pass, it will be reported with an original, changed and diff images in:

file:///Users/<USER>/path/to/project/my/hermione-reports/index.html

You will see the above path in the terminal if any of your tests fail.

Final notes

As you can see, setting up regression testing tools isn’t difficult. Using Selenium + Hermione.js can boost your productivity and eliminate (or at least heavily reduce) visual regression bugs. If you find your team struggles with constant CSS issues, Hermione.js is definitely a good idea.