Testing - Get Started

Prerequisites

Caution

Please follow the instructions on this page exactly. We’ve observed that testers sometime use other tools or IDEs (e.g., Visual Studio, IntelliJ IDEA) not specified here. While these alternatives may work, many advanced features, such as auto-suggest, linting, and error detection,... will not be available, making your test cases more prone to errors. In such cases, support from library authors will be limited.

Before you begin, make sure you have the following programs installed on your system:

Node.js - 18.x, 20.x or higher
Visual Studio Code - Latest version

Additionally, ensure you have the following Visual Studio Code extensions installed:

ESLint - Microsoft
Path Intellisense - Christian Kohler
Prettier - Code formatter - Prettier
After installation, set Prettier as the default formatter for your project.
Regressify - Tuyen Pham
YAML - Red Hat
After installation, go to Settings ⟶ YAML Store ⟶ Disable.

Initial Setup

If this is your first time working with Regressify, follow these steps (only required once). Please note that the process may take 10 minutes or more depending on your internet connection.

(Windows only) Ensure that you have enabled script execution in PowerShell by running the following command in Admin mode:
```
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned
```
Install the playwright testing framework:
```
npx playwright install
```
Install the regressify npm package globally:
```
npm i -g regressify
```

Installation

Create a new folder for the regression test.
Navigate into to the newly created folder, init the project executing the following command in the terminal:
```
npm init --yes
```
Create the default test suites:
```
regressify init
```
(Optionally) Initialize git in your folder:
Ensure you have Git installed locally.
```
git init
git add .
git commit -m "Initial commit"
```

Creating your first Test Suite

Creating a Test Suite.

Create a test suite in the visual_tests folder. Each test suite should adhere to the naming convention: <name>.tests.yaml.
For instance, you can create a file named visual_tests/alloy.tests.yaml with the following content:
```
scenarios:
  - url: https://alloy.eshn.dev/pages/home.html
  - url: https://alloy.eshn.dev/pages/service.html
  - url: https://alloy.eshn.dev/pages/portfolio.html
  - url: https://alloy.eshn.dev/pages/contact/default.html
```
For more information on available options within a test suite, refer to the Create a test suite and Create a scenario documentation.
Taking a Snapshot of the Website.

To capture a snapshot of the website at the current time, use the following command:
```
regressify ref --test-suite alloy
```
Please note that the ref command only captures the snapshot but does not open any report in the browsers.

Notes
If you run the ref command for the first time, it will always fail. The ref command compares the result with a previously approved result, which doesn't exist on the first run, causing the failure. This behavior is expected.
Approving the Snapshot.

To approve the captured snapshot, run the following command:
```
regressify approve --test-suite alloy
```
Testing for Visual Changes.

Suppose the development team has just deployed new code to the site, and you want to ensure that nothing is breaking visually or that all visual changes are as intended. Use the following command to perform the test:
```
regressify test --test-suite alloy
```
After the command finishes, it will open the test report in your default browser, showing the results.

If the test results appear to be correct, run the approve command again as mentioned above to set the new reference. However, if any issues are identified, inform the development team so that they can fix the problems and then re-run the testing step.
Regression Testing after New Deployment.

Whenever there is a new deployment, you can re-execute the test and approve it again to perform regression testing on the site visually.