This makes several improvements to test output:
- Screenshots are no longer printed to console or saved to
an html file. This is unnecessary now that they are uploaded
to rdb.
- When a test is retried, the timeout hook was not correctly
hooked, which produced unexpected errors from multiple done
calls. This has been fixed.
- Screenshot error message now includes causes, which displays
the pending async stacks on timeout.
- Driveby fix to improper assertion in console-helpers.
Bug: None
Change-Id: Ia76f88ad9fbf1236a47a1f27eff41ee93d1a28a5
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/5349145
Commit-Queue: Eric Leese <leese@chromium.org>
Reviewed-by: Benedikt Meurer <bmeurer@chromium.org>
The part that takes a lot of time in running unit tests is actually
spinning up the browser and connecting to it.
`watch-unittests` command starts the file watcher for Karma
test runner which runs the tests on file change.
You can use this without `watch` script and by just rebuilding
the target.
When combined with `watch` command, this enables you to run unit tests
instantly after a file change.
Bug: none
Change-Id: Icd71230b5b455254dc25b4cef5f6ae72c5dbba06
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/5012787
Commit-Queue: Simon Zünd <szuend@chromium.org>
Auto-Submit: Ergün Erdoğmuş <ergunsh@chromium.org>
Reviewed-by: Simon Zünd <szuend@chromium.org>
This is the first in a long set of CLs that will begin to remove old
engine functionality we do not need.
I have started with the ThreadTracksSource enum and removed the
BOTH_ENGINES flag; updating all the screenshot tests to use the new
engine exclusively. In future CLs I will remove the enum and the flag
entirely, but first I will go through and remove code that only runs for
the OLD_ENGINE flag.
Bug: 1499182
Change-Id: I7ab164e500576d044f334705d5e99649b3523e94
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/5002389
Reviewed-by: Andres Olivares <andoli@chromium.org>
Auto-Submit: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Andres Olivares <andoli@chromium.org>
This CL changes the Puppeteer instance used by the e2e and interaction tests to work in the Tab target mode, that is the mode that supports MPArch and features like pre-rendering. In the next releases of Puppeteer it will be enabled by default and this configuration can be removed.
Bug: chromium:1484476
Change-Id: I771234935a1d95de143a6c4559aa297d7dd1b4ae
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/4853353
Auto-Submit: Alex Rudenko <alexrudenko@chromium.org>
Commit-Queue: Alex Rudenko <alexrudenko@chromium.org>
Reviewed-by: Simon Zünd <szuend@chromium.org>
This CL updates the run_test_suite.js script to support the --autoninja
flag alongside --target, in which case it will do a build before running
the tests.
Unfortunately I had to duplicate the package.json entry, else we get
into a world of confusion with passing command flags through npm run
commands, and I couldn't get it to work.
Bug: none
Change-Id: I0a339c72cf41a4ffcb0241e8b3efb1b35b2e5a9f
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/4684353
Reviewed-by: Andres Olivares <andoli@chromium.org>
Commit-Queue: Andres Olivares <andoli@chromium.org>
Auto-Submit: Jack Franklin <jacktfranklin@chromium.org>
This adds two scripts: one that divides the E2E tests command into
multiple ones that can be run separately, and another script that runs
these commands in parallel Python processes. This will allow us to run
the E2E tests locally but it is also necessary for running tests in
parallel on the builders. We also remove skipOnParallel since it is
not needed anymore.
Bug: 1419802
Change-Id: I7e4c2a6f1847c3a63cab1cc2365ddaa69b77f2fb
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/4543009
Reviewed-by: Philip Pfaffe <pfaffe@chromium.org>
Reviewed-by: Danil Somsikov <dsv@chromium.org>
Commit-Queue: Almothana Athamneh <almuthanna@chromium.org>
For convenience, since the bots will still use the JSON file.
Also, an option to define the folders on which coverage will be
calculated is added to the interaction tests. The flag works
identically to the COVERAGE_FOLDERS of unit tests, i.e. a pattern is
provided to define the included folders. For example: the flag can be
set like COVERAGE_FOLDERS=
front_end/{models/trace,models/timeline_model,panels/timeline}
These two changes should help to calculate the total OPP coverage in
go/opp-test-coverage
Bug: 1424729
Change-Id: Ie982f62dfac0014c05075e53173a8e7710d3b9e0
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/4345493
Commit-Queue: Andres Olivares <andoli@chromium.org>
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
Porting the internal script to devtools-frontend. Folders are created for each platform to prevent file not found errors.
The implementation is almost identical to the internal version, except:
- Support for windows presubmit bots, on which this didn't run before.
- Interactions root folder is passed as an argument to the script.
Bug: 1407638
Change-Id: Ia373ad258071cdcc0f7e5b417fcc2301d43084ce
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/4207837
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Andres Olivares <andoli@chromium.org>
This CL adds a `--log-level` flag to the unit test runner that will
allow the log level to be configured. This flag is passed right through
to Karma, but is also respected by the logs output by our test runner
before initialising Karma.
The default value for `--log-level` is `info`, so this CL doesn't change
the built in functionality; but you can now pass `debug`, `info`, `warn`
or `error` to configure it.
Bug: none
Change-Id: I82006a1a8283c3d93389f124317a37bac69b5038
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/3963112
Commit-Queue: Thiago Perrotta <tperrotta@chromium.org>
Reviewed-by: Thiago Perrotta <tperrotta@chromium.org>
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
This CL adds the ability to run:
```
npm run auto-unittest -- --mocha-fgrep=breadcrumb
```
To the Karma unit test suite to mirror the similar flag available in the
interactions and e2e test runner script. This is also why it's named
`fgrep`, as that's the same flag as used in the other script, and we
should be consistent.
Bug: none
Change-Id: Ic119f7186e0e97c83e00bf92a66d69181c931a8b
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/3452723
Auto-Submit: Jack Franklin <jacktfranklin@chromium.org>
Reviewed-by: Tim Van der Lippe <tvanderlippe@chromium.org>
Commit-Queue: Tim Van der Lippe <tvanderlippe@chromium.org>
ESLint 8 removed CLIEngine. Therefore, we need to migrate away from
CLIEngine to the main ESLint class (which already exists in ESLint 7).
However, that class is based on promises, which means we can't use them
in a CommonJS context.
Therefore, introduce a new `.mjs` version that uses ES modules (which
can do as we are on a recent version of Node) and use the promise
versions. There are no other functional changes compared to the old
`.js` script.
The reason that we first have to add this file here, is that our CQ
directly refers to the `.js` version. That means we first land this CL,
we update our CQ configuration and only then can we remove the `.js`
file from our repository.
R=jacktfranklin@chromium.org
Bug: none
Change-Id: Iad3656931ae09eb7f9f23b3e6ad7d39c8a761a31
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/3268299
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
Auto-Submit: Tim van der Lippe <tvanderlippe@chromium.org>
The CQ bots all run run_test_suite.js, and all our local `npm run X`
scripts have run run_test_suite.js for many months now. I think we are
now safe to remove the Python run_test_suite.py script, as all use cases
are covered by the new JS runner.
Bug: 1186163
Change-Id: Ie8b5fd52e0934176a7da61d61fb9624b87ed764b
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/3075181
Reviewed-by: Tim van der Lippe <tvanderlippe@chromium.org>
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
Once we've got the `yargsObject`, we pull out all the keys we want,
which are all the configuration flags bar the special yargs keys `$0`
and `_`. The bug here was that once we pulled out that configuration, we
didn't update it, and instead kept referring to `yargsObject` in the
rest of the code. This meant that when we dynamically set the
`test-server-type` flag, we set it on `yargsObject`, but we needed to
set it on the configuration flags object so that the settings propagate
down into the test runner.
Bug: chromium:1186163
Change-Id: Ia88bb25588651fac4c11783f551651d27ccd0334
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/3075379
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
Auto-Submit: Jack Franklin <jacktfranklin@chromium.org>
Reviewed-by: Tim van der Lippe <tvanderlippe@chromium.org>
Philip Pfaffe