TLDR
- Thunders exports your test cases as a Playwright TypeScript project that runs without Thunders.
- Each Thunders step becomes a named Playwright step, with your original plain-language instruction kept as a comment above its code.
- Secrets never leave Thunders. Passwords, API keys and saved logins are listed for you to set, never copied.
- On a real project, ~200 test cases, 95% of the steps exported as fully runnable code, and the project compiled with zero errors.
- The other 5% are the parts where Thunders uses proprietary AI or Tools, at run time: checks requiring visual understanding, data it generates with complex instructions, values it reads from the visual layer, and selectors that heal themselves, or Accessibility checks. Plain code cannot do those things on its own.
When you choose a testing platform, one question comes up early, often from procurement or legal: what happens to our tests if we leave?
With Thunders, the answer is simple. Your tests are yours. On request, Thunders exports your test cases as a standard Playwright project in TypeScript. You get a zip file that runs on your own machines and in your own CI, with no dependency on Thunders, using two commands:
npm ci
npx playwright test
We built this export for continuity, not for migration. Then we did what we always do: we tested it on real test cases, ran the exported code against a real application, and wrote down everything that came back. This article shares what we learned, including the parts that do not export, and what that tells you about where the value of your tests really lives.
Why does a test platform need an export? Avoiding vendor lock-in
Many companies must be able to keep running their tests if they stop using a vendor. Sometimes the reason is a contract. Sometimes it is a vendor-risk policy or an audit. Either way, "our tests live only inside a platform" is a hard answer to give.
The export gives a better answer. It is a one-time snapshot of your test cases, in an open format that any engineer can read and run. It is not a sync, and it is not a migration tool. It is a guarantee that you keep the work you put into your tests.
What does the Thunders Playwright export contain?
You receive one big folder for your Thunders project:
- One test file per test case. Each Thunders step becomes a
test.stepwith the same title, so the Playwright report reads like your Thunders test. - Your instructions, kept as comments. Above each block of code, the export keeps the plain-language step you wrote in Thunders. Anyone who opens the file sees what the code is meant to do.
- Shared modules for reused tests. A test case that other tests reuse, such as a login, becomes one shared function, not a copy in every file.
- One environment file per environment. You choose the environment at run time. Secret values are empty.
- Your test files. Files used by upload steps and the baseline images of visual checks come with the project.
- An export report. It starts with what you must set before the first run, then lists, for every test case, how many steps exported fully, partially, or as a TODO.
What happened when we ran it on real tests?
- 95% exported as full, runnable code: navigation, clicks, typing, dropdowns, API calls with their checks, JavaScript steps, and more.
- 3% steps were partial: the code is there, but it needs something from you, such as a saved login or a baseline image.
- 2% steps became TODOs: clear comments that say what to write, with your original instruction kept above them.
The exported project installed cleanly and compiled with zero errors. Then we ran it.
What does not export to Playwright, and why?
This is the most useful part of the exercise. Every step that did not export as full code had the same cause: in Thunders, that step relies on intelligence at run time, and a static script has none.
Checks written in plain language
In Thunders, a check is a sentence, and Thunders judges it on the real page, the way a careful tester would:
- "Verify the search results are sorted by price, lowest first."
- "Check that the error message tells the user how to fix the date."
- "Confirm the product photo matches the product name."
- "Verify the whole checkout page is in French, buttons included."
- "Check that the delivery date on the confirmation is a weekday after today."
Each of these takes one line in Thunders. In Playwright, each one is a small program: read every price and compare them, parse a date and check the calendar, or call an image or language model yourself. The export keeps your sentence as a comment, with a TODO where you write that program.
Selectors that heal
Thunders finds the right element even when your interface changes. The export can only freeze the selector that worked in the last successful run. Many of them look like this:
/html[1]/body[1]/div[3]/input[1]
They work today. They break the day a developer adds a banner above the form. In Thunders, the same test keeps passing.
Data that Thunders creates
"Generate a random customer name" produces fresh data on every Thunders run. When your step asks for a common kind of data, such as a name, an email, a phone number, a number in a range, a date, a password or an ID, the export writes code that generates it the same way Thunders does, so each run gets fresh data. When your step asks for something more specific, such as a vehicle number in a given format or a name that must start with a given letter, Thunders understands the request, but the export cannot. It replays the value from the last run, with a TODO to replace it with your own generator.
Values that Thunders reads from the page
Thunders can read a total, an order number or a code from the screen and use it in later steps. When Thunders knows which element it read, the export writes code that reads the same element at run time, so the value is always current. It never copies a value from an earlier run, because such a value can hold personal data or a token. Thunders goes one step further: it understands a request such as "keep only the number" or "take the second line", and it reshapes the value as you asked. In the export, that last part is a TODO.
Visual and file comparisons
Thunders compares screenshots and files with AI, so it ignores the differences that do not matter. Playwright compares pixels. The visual check exports, but you will likely tune its threshold. A file comparison becomes a comment that describes the original check.
Stability and retries
Thunders waits for the page to be stable and retries an action that fails for a moment. The exported code has no such logic. When an element is not ready, the test fails.
Mobile, Accessibility and Persona Testing.
These are platform features. They run with your steps inside Thunders and do not exist in a Playwright script.
Is the exported code faster than Thunders?
You might expect a plain script to beat a platform that thinks at every step.
On the tests that both can run, it does not. In our measurement, the same test cases ran [X]% faster in Thunders than in the exported Playwright project, on the same application and the same environment.
Three things explain the difference:
- Smarter waiting. A script waits with fixed rules: a timeout, or an element that becomes visible. Thunders watches the page itself and moves on as soon as the page is stable, so it does not wait longer than it must, and it does not act too early on a page that is still loading.
- Decisions at a lower level. Thunders talks to the browser directly. It decides how to click, type and wait from what the page really shows, instead of going through a generic action layer for every step.
- Infrastructure built for testing. Thunders runs your tests on browsers that are ready before the run starts, close to your application, and many tests at a time. A Playwright project runs where you put it: often a single CI machine that starts a browser for each run. Check out our article on Jev+Thunders
The exported project stays fast and lean, and you can tune it. But matching these three points is engineering work that your team would own.
So, should you run your tests in Thunders or in Playwright?
Both work. They do different jobs.
The Playwright export is a snapshot. It captures your test logic on the day of the export, and from that day on, your team maintains it. Every interface change is a selector to fix by hand. Every plain-language check is code to write. Every generated value is a generator to build.
Thunders is a living test suite. It heals selectors when your interface changes, checks plain-language assertions on the real page, creates fresh data on every run, decides conditions as a person would, and runs your tests on web and mobile. That is the 5% that no export can carry, and in our experience it is where most of the maintenance time of a test suite goes.
So the export answers "can we leave?" with a confident yes. It also shows, line by line, what you would take on if you did.
How do you get an export?
Cloud customers: ask your Thunders contact. Thunders prepares the export and sends you a secure download link that expires after 7 days.
Self-hosted customers: you run the export yourself on your own instance, with an API key of your organization.
In both cases, open the export report first. Its "Before you run" section lists the secrets to set and the logins to save. Then run:
npm ci
npx playwright install chromium
npx playwright test
Where Thunders fits in your test strategy
Thunders lets anyone describe an end-to-end test in plain language, runs it in a real browser or on a real device, and keeps it working when your product changes. The Playwright export is the guarantee behind that: your tests are always yours, in an open format, whenever you need them.
Create a free Thunders account or book a demo to see both on your own app.
.png)
