Sample: Frequently Changed Data

In this tutorial, we will explore the concept of frequently changed data and how to handle it in an automation test tool. We will discuss a use case scenario and provide guidelines for creating a test suite to effectively test such data.

Use case

Consider the following sample chart:

If you reload this page, you will notice that it displays completely different data. This makes it challenging to perform visual regression testing using an automation tool. Although the chart in this page is just a sample, in real-world scenarios, there are various instances of frequently changed data, such as an auto-play video, auto-slide carousel, ads, exchange rates, stock index, weather forecast, and more.

Approach

To address scenarios involving frequently changed data, I suggest a combination of manual and automated testing. Follow the steps below:

Manual Testing

A QA engineer should perform an initial manual test on the chart (or any part with frequently changed data). This allows for verification and validation against the expected data.

Automated Testing

After the manual test is completed successfully, the remaining parts of the page can be tested using an automation tool.

However, the section with frequently changed data should be hidden from view during the automated tests. This ensures that test results remain consistent and unaffected by dynamic data changes.

Test suite

To create the test suite for frequently changed data, follow these steps:

1. Create a file named data/frequently-changed-data.tests.yaml in your test project.

2. Add the following content to the file:

   scenarios:
     # Scenario:
     # Testing the page without the chart
     - url: https://tuyen.blog/optimizely-cms/testing/frequently-changed-data-test-sample/
       label: Test page without the chart - 1st way
       hideSelectors:
         - '#my-chart'
   
     # Scenario:
     # Testing the page without the chart
     # Note that this scenario is identical with the previous one,
     # using a different syntax
     - url: https://tuyen.blog/optimizely-cms/testing/frequently-changed-data-test-sample/
       label: Test page without the chart - 2nd way
       actions:
         - hide: '#my-chart'
   
     # Scenario:
     # Testing the page WITH the chart
     # NOTE: This case will fails
     - url: https://tuyen.blog/optimizely-cms/testing/frequently-changed-data-test-sample/
       label: Test page with the random chart
   

Please note that the last scenario in the test suite will intentionally fail. You can visually inspect the results after running the test to identify the reason for the failure.

Verify

Execute the following command to verify the test suite:

bun run ref -- --test-suite frequently-changed-data
bun run approve -- --test-suite frequently-changed-data
bun run test -- --test-suite frequently-changed-data

Conclusion

By combining manual testing and automation techniques, you can effectively handle scenarios involving frequently changed data in your test suites. This approach ensures reliable and accurate test results while accommodating dynamic data changes.