%20(1).jpg)
TLDR
We timed a full CI run of a 10-test suite in GitHub Actions. Thunders finished in 24s, ahead of Selenium (41s), Playwright (47s) and Cypress (51s). Here are the numbers and what drives them.
Every push triggers a wait. How long that wait lasts depends not on raw script speed, but on the full CI run: repository checkout, dependency install, test execution, and final result. That complete elapsed time is what this test automation speed benchmark measures.
In September 2026, we ran a 10-test suite through GitHub Actions with four frameworks and recorded one figure per run: the full CI run time.
Why full CI run time is the number that matters
Raw execution time (the seconds your scripts actually spend running) ignores the overhead that precedes every run. Before the first test can execute, the CI job must check out the repository and install the framework's dependencies. For browser-based frameworks, that includes downloading or locating browser binaries.
A developer waiting for a green light waits through all of it. So does the merge queue.
Full CI run time is the metric that reflects that reality. It runs from job start to the final pass or fail, and it is the number that decides how many times a team can push and iterate before the end of a session. To run tests in your CI pipeline efficiently, this is the figure worth tracking.
GitHub Actions bills in rounded-up minutes, which means a 24-second run and a 44-second run cost the same on the invoice. The developer feedback loop is not billed in minutes.
How the test automation speed benchmark was run
The benchmark covers a suite of 10 end-to-end tests against the same application, run once per framework in GitHub Actions on 14 September 2026. Each run is independent. The timer starts when the job starts and stops when the final test result is reported, including checkout and dependency installation.
The full results are available in the public benchmark repository.
A single dated run gives a precise snapshot, not a running average. The benchmark is repeated each sprint and the results compared as they accumulate.
The results
| Framework | Full CI run |
|---|---|
| Thunders | 24s |
| Selenium | 41s |
| Playwright | 47s |
| Cypress | 51s |
Full CI run of a 10-test suite in GitHub Actions, September 2026.
Thunders: 24s. The fastest of the four in this benchmark.
Selenium: 41s. See Thunders vs Selenium: 41% faster on the full CI run.
Playwright: 47s. Thunders finished 49% faster than Playwright end to end.
Cypress: 51s. Thunders finished 53% faster than Cypress end to end.
The gap is not in the test scripts themselves. It is in the overhead each framework carries into the CI job before a single test runs.
What drives the difference
Playwright, Selenium and Cypress run in a local or runner-provisioned environment. When a CI job starts, the runner must install the framework and its dependencies. For browser-based frameworks, that also means downloading browser binaries before any test can execute.
Thunders runs in the cloud. Tests execute in Thunders-managed browsers without a local install step on the critical path. The AI test generation layer resolves page elements at run time, removing a separate selector-maintenance pass before the run starts. Tests run in parallel, so the suite finishes without waiting for each test to complete before the next begins.
Cloud execution and parallelism compress the full CI run from both ends: the setup overhead before the first test, and the serial queuing that would otherwise extend the total.
What a faster full CI run changes for your pipeline
The effect is compounding. A developer who gets a result in 24 seconds instead of 47 seconds can push, see the result, fix, and push again in less time than a single Playwright run takes to return.
Across a team pushing on every pull request, shorter CI runs mean:
- Less waiting before a merge. The pipeline holds open for less time per push.
- More iterations per session. A sub-30-second feedback loop changes how teams work with tests, not just how fast the suite runs.
- Fewer queued merges. With multiple engineers pushing in parallel, a faster suite reduces the window during which the queue backs up.
Conclusion
On the metric that reflects what a developer actually waits for, Thunders completed a 10-test suite in 24 seconds in our September 2026 GitHub Actions benchmark: 41% faster than Selenium, 49% faster than Playwright, and 53% faster than Cypress on a full CI run.
The claim is narrow: one metric, one dated run, repeated each sprint. To see how Thunders performs in your own pipeline, book a demo.
