Get Started

Prerequisites

Caution

Please follow the instructions on this page exactly. We’ve observed that testers sometime use other tools or IDEs (e.g., Visual Studio, IntelliJ IDEA) not specified here. While these alternatives may work, many advanced features, such as auto-suggest, linting, and error detection,... will not be available, making your test cases more prone to errors. In such cases, support from library authors will be limited or unavailable.

Before you begin, make sure you have the following programs installed on your system:

Node.js - 18.x, 20.x or higher
Visual Studio Code - Latest version

Additionally, ensure you have the following Visual Studio Code extensions installed:

ESLint - Microsoft
Path Intellisense - Christian Kohler
Prettier - Code formatter - Prettier
After installation, set Prettier as the default formatter for your project.
YAML - Red Hat
After installation, go to Settings ⟶ YAML Store ⟶ Disable.

Installation

Clone the regression-test repository by running the following command in the terminal:
```
git clone https://github.com/precise-alloy/regression-test.git
```
Install the required node modules by executing the following command in the terminal:
```
npm install
```

Creating your first Test Suite

Creating a Test Suite.

Create a test suite in the data folder. Each test suite should adhere to the naming convention: <name>.tests.yaml.
For instance, you can create a file named data/alloy.tests.yaml with the following content:
```
scenarios:
   - url: https://alloy.eshn.dev/pages/home.html
   - url: https://alloy.eshn.dev/pages/service.html
   - url: https://alloy.eshn.dev/pages/portfolio.html
   - url: https://alloy.eshn.dev/pages/contact.html
```
For more information on available options within a test suite, refer to the Create a test suite and Create a scenario documentation.
Taking a Snapshot of the Website.

To capture a snapshot of the website at the current time, use the following command:
```
npm run ref -- --test-suite alloy
```
Please note that the ref command only captures the snapshot but does not open any report in the browsers.

Notes
If you run the ref command for the first time, it will always fail. The ref command compares the result with a previously saved result, which doesn't exist on the first run, causing the failure. This behavior is expected.
Approving the Snapshot.

To approve the captured snapshot, run the following command:
```
npm run approve -- --test-suite alloy
```
Testing for Visual Changes.

Suppose the development team has just deployed new code to the site, and you want to ensure that nothing is breaking visually or that all visual changes are as intended. Use the following command to perform the test:
```
npm run test -- --test-suite alloy
```
After the command finishes, it will open the test report in your default browser, showing the results.

If the test results appear to be correct, run the approve command again as mentioned above to set the new reference. However, if any issues are identified, inform the development team so that they can fix the problems and then re-run the testing step.
Regression Testing after New Deployment.

Whenever there is a new deployment, you can re-execute the test and approve it again to perform regression testing on the site visually.