Visual regression Test in CypressIO

A visual regression test is done whenever we update our application. This test is used to compare our previous version of the application with the updated version with the help of images. We can check whether the updated version has affected some previous version functions.

Now let us learn how to do a visual regression test in CypressIO. For this let us start from scratch with the Cypress.

Create a folder for visual regression testing on your PC and open that folder in the VS Code. Now you have to install the cypress and set the base for the cypress in the VSCode. Follow the below steps to install and to do a Visual regression test in Cypress.

In the terminal of the VSCode type the below command to create package.json.

npm init

Give the below command in the terminal to install both cypress and image snapshot plugin. The image snapshot plugin is installed to capture the snapshots.

npm install cypress cypress-image-snapshot

Give the below command to install prettier

npm install prettier

Create a file named .prettierrc in the node modules of VS Code and give the below setting.

{
  "semi": false,
  "singleQuote": true,
  "useTabs": true,
  "tabWidth": 2,
}

If you want, you can redefine the npx cypress run command or npx cypress opencommand of the cypress as given below in the package.json.

{
  "name": "cy-visual_test",
  "version": "1.0.0",
  "description": "Visual regression testing using cypress",
  "main": "index.js",
  "scripts": {
    "cy:open": "cypress open",
    "cy:run-tests": "cypress run"
  },
  "keywords": [
    "cypress",
    "visual",
    "testing",
    "snapshots"
  ],
  "author": "Tharani",
  "license": "ISC",
  "dependencies": {
    "cypress": "^5.5.0",
    "cypress-image-snapshot": "^3.1.1",
    "prettier": "^2.1.2"
  }
}

In the cypress.json give the below configuration:

{
    "video":false,
    "defaultCommandTimeout":5000,
    "pageLoadTimeout": 10000,
    "waitForAnimations":true
}

In the plugin/index.js file give the below plugin configuration

const { addMatchImageSnapshotPlugin } = require ("cypress-image-snapshot/plugin")
module.exports = (on, config) => {
  addMatchImageSnapshotPlugin(on,config)
}

Add a custom command in the support/command.js, so that whenever we call the matchImageSnapshot() method below function will be performed.

import { addMatchImageSnapshotCommand} from "cypress-image-snapshot/command"
addMatchImageSnapshotCommand({
    failureTreshold: 0.00,
    failureTresholdType:'percent',
    customDiffConfif:{treshold:0.0},
    capture: 'viewport'
})
Cypress.Commands.add("setResolution",(size)=>{
    if(Cypress._.isArray(size)){
        cy.viewport(size[0], size[1])
    }
    else{
        cy.viewport(size)
    }
})

Example program: In the below program we are visiting the website and capturing a screenshot of the website.

describe('Visual Regression', () => {
    it('My 1st visual regression test', () => {
        //load website
        cy.visit("https://example.com")
        cy.matchImageSnapshot()
    })
})

When you run the program for the first time you may see the below output.
visualsregression-cypressio

A screenshot of the test with the name of the respective describe block and it block is saved in the snapshots folder.

This snapshot serves as the base snapshot because this snapshot is compared with the other snapshots.

When you open the file under the snapshot the UI will be as the below image.
first-output-cypress

When you run the same program but by visiting different websites, in the output, it throws an error even though the program is correct.

describe('Visual Regression', () => {
    it('My 1st visual regression test', () => {
        //load website
        cy.visit("https://chercher.tech/")
        //compare snapshots
        cy.matchImageSnapshot()
    })
})

Even though the program is correct, an error is thrown in the output because we have changed the website and the describe block and it block description remains the same.

So, when the program is executed another snapshot file is created with the same name as the previous snapshot. The snapshot with the same name is created because the second program also has the same describe block and it block description as of the previous one.

The picture is compared against one another and hence when we execute the program for the second time, It throws an error and the test fails.
visualregression-error-cypressio

The image is stored with the same name as the previous one. Since it is a failure snapshot, it is saved with the extension diff.png
vsregression-error-output-cypress

When we open the My 1st visual regression test diff.png snapshot the below image is got as the output which shows the difference between the two images.
difference-screenshot-cypress

Updating Base snapshot in Visual Regression testing of CypressIO

Consider a scenario where you have changed the UI of your webpage. Without updating the base snapshot when you execute the test it throws an error.

So to avoid that you have to update the base snapshot by giving the below configuration in the scripts of the package.json file.

"cy:update-snapshots":"cypress run --env updateSnapshots=true"

Package.json

"scripts": {
    "cy:open": "cypress open",
    "cy:run-tests": "cypress run",
    "cy:update-snapshots":"cypress run --env updateSnapshots=true"
  },

Now execute the test by giving the below command.

 npm run cy:update-snapshots

Output : Now the program executes without throwing any error by updating the base snapshot.
base-snapshot-updating-cypress

Now when you open the snapshot which has the My 1st visual regression test snap.png, you can notice that the snapshot of the website has been changed which means the base snapshot is updated.
after-updating-baseshot-cypress

Changing the Viewports in Visual Regression testing of CypressIO

In the below program we are going to take snapshots of the webpage with different resolutions and sizes.

Example program :

const pages= ["http://example.com/",""]
const sizes =["iphone-6","ipad-2",[1200,800]]
describe('Visual Regression',() =>{
    sizes.forEach(size => {
        pages.forEach(page => {
            it('Should match ${page} in resolution ${size}', () =>{
                cy.setResolution(size)
                cy.visit(page)
                cy.matchImageSnapshot()
            })
        })
    })
})

Output: Three images with different viewport sizes are taken and saved.
array-endpoint-size-output-cypress

Single element screenshot in Visual regression testing of CypressIO

In the below program we are going to take a screenshot of a specific element, which is called a Single element Screenshot.

Example program :

//URL you are visiting 
const pages= ["http://example.com/"]
//Viewport
const sizes =["iphone-6","ipad-2",[1200,800]]
describe('Visual Regression',() =>{
    sizes.forEach(size => {
        pages.forEach(page => {
            it('Should match ${page} in resolution ${size}', () =>{
                let currentTime = new Date(Date.UTC(2020,1,1)).getDate()
                cy.clock(currentTime)
                cy.setResolution(size)
                cy.visit(page)
            })
        })
    })
})
describe('Single element snapshot',() =>{
    it("Should match a single element on the page",() =>{
        cy.visit("http://example.com/")
        cy.get("h1").matchImageSnapshot()
    })
})

Execute the program by giving the below command

npm run cy:run-tests

Output:
single-element-screenshot-output-cypress

When you open the relevant snapshot file the below image is displayed.
single-element-screenshot-cypress

We saw at first when you try to run the program with the same describe and it block description with different looks of a page it throws an error.

So to avoid visual regression comparison, we can execute the program with the below command,

npx cypress run --env failOnSnapshotDiff=false

This method is not recommended while performing product testing. But it can be used for debugging purposes.

Locally overriding date, time and setting the viewport in Visual regression testing of CypressIO

In the example program, the below line locally overrides the date and sets the date to the system date.

let currentTime = new Date(Date.UTC(2020,1,1)).getDate()

We can set the viewport by giving the below line.

const sizes =["iphone-6","ipad-2",[1200,800]]

Example program:

//Changed the visited URL page
const pages=["https://chercher.tech/"]
const sizes =["iphone-6","ipad-2",[1200,800]]
describe('Visual Regression',() =>{
    sizes.forEach(size => {
        pages.forEach(page => {
            it('Should match ${page} in resolution ${size}', () =>{
                let currentTime = new Date(Date.UTC(2020,1,1)).getDate()
                cy.clock(currentTime)
                cy.setResolution(size)
                cy.visit(page)
            })
        })
    })
})

Output :
vsregression-error-modified-cypress

There is another way to set the viewport. In the below program we are going to take a snapshot of the webpage with the different viewports.

Example program :

describe("Visual regression -Login page", () =>{
    before(function(){
        cy.visit("http://zero.webappsecurity.com/index.html")
        cy.get("#signin_button").click()
        cy.get("#user_login").type("test")
        cy.get("#user_password").type("test")
        cy.get("#user_remember_me").click()
})
it("Desktop Layout" ,()=>{
    cy.setResolution([1280,720])
    cy.matchImageSnapshot()
})
it("Tablet layout",() => {
    cy.setResolution("ipad-2")
    cy.matchImageSnapshot()
})
it("Mobile layout",() => {
    cy.setResolution("iphone-6")
    cy.matchImageSnapshot()
}) 
})

Execute the specific file by giving the below command

npx cypress run --spec="cypress\integration\login.spec.js"

Output:
e2e-vsregression-output-cypressio

The relevant snapshots with the respective viewport names are saved in the snapshots/login.spec.js
e2e-vsregression-picture-output-cypressio

Visual Testing of tables in CypressIO

By taking the snapshot of the tables we will get to know when the data change occurs. So that we can easily keep a track of the data in the table. Even when data is lost we can check the snapshot and modify the table again.

Example program :

describe("Visual regression -Datapage", () =>{
    before(function(){
        cy.visit("http://zero.webappsecurity.com/index.html")
        cy.get("#signin_button").click()
        cy.get("#user_login").type("username")
        cy.get("#user_password").type("password")
        cy.get("#user_remember_me").click()
      cy.get('input[name="submit"]').click()
})
it("should load data tables",() => {
    cy.get("#account_activity_tab").click()
})
it("Data Table snapshot",()=>{
    cy.matchImageSnapshot()
})
})

Output :
data-tables-cucumber-output-cypress

Snapshot: You can find the snapshot in the below-highlighted file.
e2e-vsregression-datatables-cypress

When you click the snapshot you will see the below image.
data-table-output-vsregression-cypressio

0 results
Comment / Suggestion Section
Point our Mistakes and Post Your Suggestions