Skip navigation

Create a test suite

A test suite is a collection of test scenarios that are grouped together based on your decision. You have the flexibility to determine which scenarios should be included in the same test suite. For example, a test suite can encompass test cases for the entire Home page, including different blocks, or focus solely on the Header block with various states (e.g., closed/opened menu).

In one of my projects, the objective was to assist customers in migrating multiple websites from an old platform to a newer one. To ensure that everything was functioning correctly, I created dedicated test suites for each website. This allowed the QA engineer to test any individual website as needed.

At a minimum, a test suite contains a list of URLs to be tested. For example:

scenarios:
  - url: https://alloy.eshn.dev/pages/home.html
  - url: https://alloy.eshn.dev/pages/service.html
  - url: https://alloy.eshn.dev/pages/portfolio.html
  - url: https://alloy.eshn.dev/pages/contact.html

The purpose of these tests is to load the specified URLs and capture screenshots. For more complex test cases, you can utilize a combination of the properties provided below.

Properties

Required properties

scenarios

Type: ScenarioModel[].

List of test scenarios. Refer to create a scenario for more information.

Optional properties

asyncCaptureLimit

Type: number.
Default value: 5.

This parameter determines the maximum number of screens captured in parallel during testing. Increasing this value can accelerate the execution of your test suite. However, it's important to note that if your network connections or your website's capacity to handle concurrent requests is limited, raising this limit too high could result in less stable test results.

asyncCompareLimit

Type: number.
Default value: 50.

This parameter dictates the maximum number of screens that can be compared in parallel. Unlike asyncCaptureLimit, this value isn't dependent on your network connections or your website's capacity. Instead, it's contingent on your computer's hardware specifications. If your computer boasts robust hardware capabilities, increasing this value will result in quicker comparisons between reference and test results.

browser

Type: string.
Default value: chromium.
Allowed values: chromium, firefox, webkit.

The browser used for the test.

debug

Type: boolean.
Default value: false.

Should the tool displays the Chrome window as tests are running. This will allow you to visually see your app state at the time of your test.

Example:

debug: true
cssOverridePath

Type: string.

CSS override path.
Default value: data/_override.css

hideSelectors

Type: string[].
Default value: [].

Array of selectors set to visibility: hidden.
This property will affects all the scenarios within the same test suite.

misMatchThreshold

Type: number.
Default value: 0.1.
Min value: 0.
Max value: 100.

This parameter represents the percentage of permissible differing pixels for a test to pass. A lower value ensures more precise test results. The choice of a suitable value depends entirely on your project's specific requirements.

postInteractionWait

Type: number.
Default value: 1000.

Wait for X miliseconds prior to screen shot.

removeSelectors

Type: string[].
Default value: [].

Array of selectors set to display: none.
This property will affects all the scenarios within the same test suite.

urlReplacements

Type: ReplacementModel[].
Default value: [].

Find/replace for the URLs in test command.

Example:

# Below settings have no effect when running under `ref` command.
# When running using `test` command,
# replace the URL from `https://legacy1.domain.com/` to `https://new1.domain.com/`,
# and from `https://legacy2.domain.com/` to `https://new2.domain.com/`
urlReplacements:
  - ref: https://legacy1.domain.com/
    test: https://new1.domain.com/
  - ref: https://legacy2.domain.com/
    test: https://new2.domain.com/
useCssOverride

Type: boolean.
Default value: true.

Should the scenarios use CSS override.

viewportNames

Type: string | string[].
Default value: all available viewports.

Choose the viewports you want applied for the test suite.

Sample:

viewportNames:
  - tablet
  - desktop