Visual Validation & Tools

Visual Validation Testing

Visual Validation testing is a quality assurance activity that is meant to verify that UI appears correctly to users. Many people confuse that with traditional, functional testing tools like Selenium and QTP/UFT which were designed to help you test the functionality of your application through the UI.

When we're talking about visual testing we want to make sure that the UI itself looks right to the user and that each UI element appears in the right colour, shape, position and size.

We also want to ensure that it doesn't hide or overlap any other UI elements. In fact, many of these types of tests are so difficult to automate that they end up being manual tests.

Top Visual Validation Tools for Software Testers

Two main approaches used for visual validation testing:

1. Code Based Approach
2. Configuration Based Approach

Code Based Approach

1. PhantomCSS (UJs CapserJs) - takes screenshots captured by CasperJS and compares them to baseline images using Resemble.js to test for rgb pixel differences. PhantomCSS then generates image diffs to help you find the cause.  

2. FBSnapshotTestCase (Uses XCTest) - uses a “snapshot test case” that takes a configured UIView or CALayer and uses the renderInContext: method to get an image snapshot of its contents. It compares this snapshot to a “reference image” stored in your source code repository and fails the test if the two images don't match.

3. Gemini (Uses JS DSL) - is a utility for regression testing the visual appearance of web pages.

4. Needle (Uses Python) - is a tool for testing visuals with Selenium and nose.

It checks that visuals (CSS/fonts/images/SVG/etc.) render correctly by taking screenshots of portions of a website and comparing them against known good screenshots. It also provides tools for testing calculated CSS values and the position of HTML elements. 

5.  Rspec Page Regression  (Uses Capibara) - is an RSpec plugin that makes it easy to headlessly regression test your web application pages to make sure the pages continue to look the way you expect them to look, taking into account HTML, CSS, and JavaScript. It provides an RSpec matcher that compares the test snapshot to an expected image, and facilitates management of the images.

6. Pix-Diff  (Uses JS and Protractor) - is a lightweight Protractor plugin for image comparisons on (mobile/desktop) browsers and hybrid app screens or screen regions.

7. Selenium Visual Diff  (Uses Java and WD) - the goal is a better integration of screenshots taking in maven executed selenium2 functional tests, storing and versioning of screenshots to get a report of visual differences between two application versions.

8. Vizregress  (Uses .NET and WD) - is used to spot visual regression in your web application by comparing screen-shots against an approved set of screen-shots. This was created as a  research project where they used TDD (Test Driven Development) to support visual regression testing. They also used SpecFlow as their BDD (Behavior Driven Development) tool to perform browser based tests.

9. VisualCeption (Uses PHP and CodeCeption) - uses a combination of the “make a screenshot” feature in webdriver, imagick and jquery to compare visual elements on a website.

10. Specter (Uses a JS DSL) - is a Node.js wrapper for PhantomJS. PhantomJS is a headless browser which enables developers to load pages that rely on Javascript. Specter was made to easily access these dynamically and asynchronously created HTML pages using Node.js.

Configuration Based Approach

1. BaskstopJS - automates visual regression testing of your responsive web UI by comparing DOM screenshots over time. A good place to get started with BackstopJs would be css-tricks article on automation CSS regression test.

2. dpxdtMake –Depicted - makes continuous deployment safe by comparing before and after webpage screenshots for each release. Depicted shows when any visual, perceptual differences are found. This is the ultimate, automated end-to-end test.

3. CSSCritic - lightweight tool for regression testing of Cascading Style Sheets. 

4. Grunt Photobox - a plugin to prevent your project of broken layout via screenshot photo sessions of your site.

5. VIFF - helps you to find visual differences between web pages in different environments (developing, staging, production) and browsers.

6. GreenOnion - helps solve the issue of having to catch regression bugs and solve the testing of HTML and CSS issue of  having to be such a back and forth between designers and devs. It also help you to follow TDD and BDD against the presentation layer? GreenOnion helps you get the same results on testing front-end styling that you've enjoyed in your unit and integration tests up to now.

GreenOnion records ‘skins', which are snapshots of the current state of a view (or any page that a browser can navigate to). The first time that it is run on a view, it saves the skin; the second time that it is run, it will compare the first skin to the fresh (new) skin. You can either compare with a diff image that is automatically created, or by an outputted percentage of difference.

7. Kobold - is a visual regression testing framework, comparing screenshots from multiple builds.

8. Wraith-Selenium - uses either PhantomJS or SlimerJS to create screen-shots of different environments and then creates a diff of the two images, the affected areas are highlighted in blue.

9. CSS Visual Test - helps you to test your front end components visually. Currently supports components built with react and commonjs / browserify.

10. Snap And Compare - is an easy to use screenshot comparison tool used for CSS Visual Regression Testing built using NodeJS.

11. Grunt-Vig is a simple visual regression testing tool using phantomJS 2