Why SafeTest?
SafeTest offers a comprehensive solution for testing both applications and components. It does this by injecting hooks into your app during setup, so tests run smoothly without affecting normal usage. This means tests load only when needed, keeping things efficient.
SafeTest main goals include:
- Simplifying UI testing for React applications.
- Offering powerful features such as overrides and reporting capabilities.
- Ensuring test reliability and consistency.
- Facilitating testing in corporate environments with complex authentication requirements.
Using other UI tools with SafeTest such as Playwright gives SafeTest powerful features, such as deep linking to specific tests and video recording. It's user-friendly too, seamlessly fitting into your existing development setup.
While integration tests excel in speed and ease of setup, they lack the ability to simulate real application behavior and struggle with complex UI interactions.
On the other hand, E2E tests, conducted with tools like Cypress and Playwright, provide a realistic testing environment but face difficulties in isolating components and setting up test fixtures.
SafeTest bridges this gap by combining the strengths of both approaches. It allows for easy test setup, seamless driving of tests, and isolation of components, all while enabling comprehensive application testing, including click-ability of elements, screenshot testing, video recording, and trace viewing. By offering the best of both worlds, SafeTest enhances testing efficiency and confidence in the reliability of web applications.
To illustrate, let's consider a typical scenario involving a user authentication flow. Integration tests may verify the functionality of individual components, such as login forms, but might not adequately simulate scenarios like network delays or authentication failures. On the other hand, E2E tests excel at capturing these real-world scenarios but may struggle with isolating specific components for testing or simulating complex user interactions.
This is where SafeTest comes into play. By combining the strengths of both integration and E2E testing, SafeTest offers a comprehensive testing solution that addresses these challenges.
SafeTest vs Market Leaders: Cypress and Playwright
In order to see how SafeTest stacks up against other UI tooling available in the market, below we’ve compared SafeTest to industry favourites Cypress and Playwright:
Playwright:
Approach: Remotely controls a browser to visit a URL and interact with the page.
Strengths:
- Provides control over the page for thorough testing.
- Supports video recording, trace viewing, and pause page functionality.
Challenges:
- Difficulty in making calls to alternative API endpoints without custom network layer API rewrite rules.
- Limited ability to assert on spies/mocks or execute code within the app.
Cypress:
Approach: Focuses on integration testing, operating by remotely controlling a browser.
Strengths:
- Offers solutions for end-to-end component testing.
- Deep linking and two-way communication between the browser and test context.
Challenges:
- May struggle with complex enterprise applications, especially those with OAuth or a complex build pipeline.
- Updates to TypeScript usage could break tests until the Cypress team updates their runner.
SafeTest:
Approach: Utilizes a novel approach to UI testing, injecting hooks during the application bootstrapping stage.
Strengths:
- Leverages Playwright for ideal browser control in tests.
- Allows deep linking to specific tests without a node test server.
- Two-way communication between the browser and test context.
- Video recording, trace viewing, and pause page functionality for testing.
Features:
- Familiar testing syntax with hooks, making it user-friendly.
- Overrides for modifying values during tests, e.g., API responses or app-level values.
- Powerful reporting with automatic linking of video replays and Playwright trace viewer.
- Supports various frameworks beyond React, including Vue, Svelte, Angular, NextJS, and Gatsby.
Conclusion:
Playwright and Cypress: Good for traditional integration testing but have limitations.
SafeTest: Addresses challenges by introducing a unique approach, providing control, flexibility, and advanced features. SafeTest aims to revolutionize UI testing by combining the strengths of traditional approaches and addressing their limitations, making it a powerful choice for comprehensive and flexible UI testing.
SafeTest’s Architecture
Installation of SafeTest
In this section, we'll walk you through the steps to install SafeTest in your project.
Step-by-Step Installation Guide
Follow these simple steps to install SafeTest in your project:
1- Install Node.js and npm:
If you haven't already, you need to install Node.js and npm on your system.
2- Install Safetest as a dependency in your project:
- Navigate to Your Project Directory: Use the cd command to change directory to your project's root folder.
- Run npm Install Command: Once you're in your project's directory, execute the following command:
With npm:
npm install --save-dev safetest
With yarn:
yarn add --dev safetest
- After running the command to install safeTest as dependency in your project, you will find node_module folder, package-lock.json and package.json created in your project directory.
3- Add run command to package.json scripts:
Add the following line to your package.json scripts:
"scripts": {
"safetest": "OPT_URL=${TARGET_URL:-http://localhost:3000} react-scripts --inspect test --runInBand --testMatch '**/src/*.safetest.{j,t}s{,x}' --setupFilesAfterEnv ./setup-safetest.tsx",
"safetest:ci": "rm -f artifacts.json && OPT_URL=${DEPLOYED_URL} OPT_CI=1 OPT_DOCKER=1 npm run safetest -- --watchAll=false --ci=1 --json --outputFile=results.json",
"safetest:regenerate-screenshots": "OPT_DOCKER=1 npm run safetest -- --watchAll=false --update-snapshot",
"start": "react-scripts start"
}
To integrate Safetest seamlessly into your React project, you'll need to configure your test runner appropriately. The following script demonstrates how to invoke the default test runner (react-scripts) while ensuring SafeTest compatibility. It involves setting specific flags and environment variables to enable Safetest with Jest and to include the testing of .safetest.tsx files. Depending on your project setup, you might need to make adjustments, such as using craco or react-app-rewired instead of react-scripts.
react-scripts test --env=safetest --testRegex="\\.safetest\\.tsx$"
This command ensures that Safetest is loaded and executed with Jest, allowing it to recognize and run tests from .safetest.tsx files. Adjustments may be necessary depending on your specific project configuration, especially if you're using alternative configurations like craco or react-app-rewired.
If you're using Vitest you'd use these scripts instead:
{
"scripts": {
"safetest": "OPT_URL=${OPT_URL:-http://localhost:3000/} vitest --config vite.safetest.config",
"safetest:ci": "rm -f artifacts.json && OPT_URL=${DEPLOYED_URL} OPT_CI=1 OPT_DOCKER=1 OPT_ARTIFACTS=artifacts.json npm run safetest -- --run --bail=5",
"safetest:regenerate-screenshots": "OPT_DOCKER=1 npm run safetest -- --run --update"
}
}
4- Add setup-safetest.tsx file:
Create a file called setup-safetest.tsx in the root of your project and add the following code:
import { setup } from 'safetest/setup';
setup({
bootstrappedAt: require.resolve('./src/main.tsx'),
});
This file contains the essential setup needed to integrate Safetest into your project seamlessly. It's also the central location where you can customize Safetest by providing options to the setup function.
If you're using vitest you'll need to add a vitest.safetest.config.ts file with the following config:
/// <reference types="vitest" />
import { defineConfig } from 'vite';
// https://vitejs.dev/config/
export default defineConfig({
test: {
globals: true,
testTimeout: 30000,
reporters: ['basic', 'json'],
outputFile: 'results.json',
setupFiles: ['setup-safetest'],
include: ['**/*.safetest.?(c|m)[jt]s?(x)'],
threads: process.env.CI ? true : false,
inspect: process.env.CI ? false : true,
},
});
5- Bootstrapping your application
To enable Safetest to work with your application, you need to bootstrap it to load properly. This is achieved by modifying your application's entry point, typically located at src/index.tsx, as follows:
import ReactDOM from "react-dom";
+import { bootstrap } from 'safetest/react';
import App from "./App";
-ReactDOM.render(
- <App />,
- document.getElementById("app")
-);
+const container = document.getElementById("app");
+const element = <App />;
+
+const isDev = process.env.NODE_ENV !== 'production';
+
+bootstrap({
+ element,
+ // render: (element) => ReactDOM.render(element, container),
+ // If using React 18:
+ render: (element) => ReactDOM.createRoot(container).render(element),
+
+ // Add one of the following depending on your bundler...
+
+ // Webpack:
+ webpackContext: isDev && import.meta.webpackContext('.', {
+ recursive: true,
+ regExp: /\.safetest$/,
+ mode: 'lazy'
+ })
+
+ // Vite:
+ // importGlob: isDev && import.meta.glob('./**/*.safetest.{j,t}s{,x}'),
+
+ // Using the `npx safetest generate-import-map src/Bootstrap.tsx src > src/imports.tsx` syntax:
+ // imports, // Where imports is defined as `import imports from './imports';`
+
+ // Other:
+ // import: isDev && async (s) => import(`${s.replace(/.*src/, '.').replace(/\.safetest$/, '')}.safetest`),
+
+});
The provided dynamic import mechanism utilizes Webpack Context or Vite Glob import (or any other suitable dynamic import flavor available) to bundle `.safetest.tsx` files separately in your project. This approach enables you to author tests for your application within the same project, eliminating the need for setting up a separate test project or worrying about tests being loaded in a non-test environment. The `isDev` check is primarily used to prevent tests from leaking into production, although it's not strictly required. In this project, it's disabled for simplicity in testing against the final deployed app.
6- Create your first test:
Now that Safetest is set up, you can begin writing your initial tests. Start by creating a file named src/App.safetest.tsx and include the following code:
7- Run your project:
Here are the instructions on how to run Safetest in your project:
A- Open Terminal:
Open your preferred terminal application.
B- Navigate to Your Project Path:
Use the `cd` command to navigate to the directory where your project is located. For example:
cd /path/to/your/project
C- Start Your Application:
Run the following command to start your application: npm start
D- Open Another Terminal and run Safetest:
In the new terminal, run the following command to execute Safetest:
npm run safetest
By following these steps, you'll be able to run Safetest and execute your tests alongside your application. This ensures that you can test your application's functionality seamlessly without interrupting its normal operation.
SafeTest Reports
SafeTest has released an HTML Test Reporter to help you easily check your test results. To use it, simply add details about the artefact to the results json. You can do this using the cli command provided by safetest.
Now, you have two options: You can either share the standalone HTML file at node_modules/safetest/report.html, or you can import the Report component from safetest/report and incorporate it into your application.
npx safetest add-artifact-info artifacts.json results.json
Here's an example of the vite-react example app report
import { Report } from 'safetest/report';
export const MyReport: React.FunctionComponent = () => {
return (
<Report
getTestUrl={(filename, test) => {
const relativeFile = `./${filename}`.replace(/\.[jt]sx?$/g, '');
const testName = test.trim().replace(/ /g, '+');
return `${process.env.DEPLOYED_URL}?test_path=${relativeFile}&test_name=${testName}`;
}}
renderArtifact={(type, path) => {
if (type === 'video')
return <video src={`${process.env.DEPLOYED_URL}${path}`} controls />;
// etc
}}
/>
);
};
Please keep in mind that the report includes a link to the test in the deployed environment. You achieve this by providing the getTestUrl prop to the Report component. This feature proves helpful when debugging tests that may fail in continuous integration but not on your local machine. Additionally, the renderArtifact prop is utilized to display the artifacts generated by the tests, such as videos or trace viewers.
Final thoughts
SafeTest is a popular testing framework at Netflix, and for good reason. It allows you to easily test components in isolation, while also being able to test applications as a whole. In comparison to other tools in the market, SafeTest's comprehensive approach to behaviour documentation and its user-friendly interface position it as a promising solution likely to be widely utilised across various industries.