This CL adds the ability to run:
```
npm run auto-unittest -- --mocha-fgrep=breadcrumb
```
To the Karma unit test suite to mirror the similar flag available in the
interactions and e2e test runner script. This is also why it's named
`fgrep`, as that's the same flag as used in the other script, and we
should be consistent.
Bug: none
Change-Id: Ic119f7186e0e97c83e00bf92a66d69181c931a8b
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/3452723
Auto-Submit: Jack Franklin <jacktfranklin@chromium.org>
Reviewed-by: Tim Van der Lippe <tvanderlippe@chromium.org>
Commit-Queue: Tim Van der Lippe <tvanderlippe@chromium.org>
ESLint 8 removed CLIEngine. Therefore, we need to migrate away from
CLIEngine to the main ESLint class (which already exists in ESLint 7).
However, that class is based on promises, which means we can't use them
in a CommonJS context.
Therefore, introduce a new `.mjs` version that uses ES modules (which
can do as we are on a recent version of Node) and use the promise
versions. There are no other functional changes compared to the old
`.js` script.
The reason that we first have to add this file here, is that our CQ
directly refers to the `.js` version. That means we first land this CL,
we update our CQ configuration and only then can we remove the `.js`
file from our repository.
R=jacktfranklin@chromium.org
Bug: none
Change-Id: Iad3656931ae09eb7f9f23b3e6ad7d39c8a761a31
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/3268299
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
Auto-Submit: Tim van der Lippe <tvanderlippe@chromium.org>
The CQ bots all run run_test_suite.js, and all our local `npm run X`
scripts have run run_test_suite.js for many months now. I think we are
now safe to remove the Python run_test_suite.py script, as all use cases
are covered by the new JS runner.
Bug: 1186163
Change-Id: Ie8b5fd52e0934176a7da61d61fb9624b87ed764b
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/3075181
Reviewed-by: Tim van der Lippe <tvanderlippe@chromium.org>
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
Once we've got the `yargsObject`, we pull out all the keys we want,
which are all the configuration flags bar the special yargs keys `$0`
and `_`. The bug here was that once we pulled out that configuration, we
didn't update it, and instead kept referring to `yargsObject` in the
rest of the code. This meant that when we dynamically set the
`test-server-type` flag, we set it on `yargsObject`, but we needed to
set it on the configuration flags object so that the settings propagate
down into the test runner.
Bug: chromium:1186163
Change-Id: Ia88bb25588651fac4c11783f551651d27ccd0334
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/3075379
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
Auto-Submit: Jack Franklin <jacktfranklin@chromium.org>
Reviewed-by: Tim van der Lippe <tvanderlippe@chromium.org>
This change replaces the 'spec' report with a 'mocha' report in karma
when the '--expanded-reporting' flag is set. If it is not set and the
tests fail, a hint is added to the console to enable the flag to get a
more verbose output of why the tests are failing.
Bug:None
Change-Id: Ia41615362332459a4d89aef0440cea0ec266157b
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2912098
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Jan Scheffler <janscheffler@chromium.org>
This script can be invoked to obtain the coverage from the unit
and interaction tests and writes them to a coverage-summary.json
file that lives in `test/`. This file can then be used in favor of
karma-coverage/coverage-summary.json to upload to the Chromium
infrastructure.
As a follow-up, we should turn on coverage collection on CQ and
hook up this script after both unit and interaction tests have
completed.
R=jacktfranklin@chromium.orgCC=liviurau@chromium.org
Bug: 1206705
Change-Id: I52199940d6747ec13f208233e04dcf81a618a8fb
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2884240
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
This CL fixes a bug where the test runner took the `--chrome-features`
flag but Conductor never read that value when booting up a browser
instance.
This CL updates it so it will read the value and pass it into the
relevant argument when executing `puppeteer.launch`.
I also changed the Python test suite runner to not prepend
`--enable-features`; that feels like something that Conductor should
do, not the test runner.
Bug: chromium:1186163
Change-Id: I05916d39d60bb98a606570cd82e688581dd78a40
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2790874
Reviewed-by: Tim van der Lippe <tvanderlippe@chromium.org>
Reviewed-by: Philip Pfaffe <pfaffe@chromium.org>
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
This reverts commit 7742a201a1.
Reason for revert: We do have a use case for this flag that we missed, so we need it. Will follow-up with a CL to fix the fact that the test frontend infra doesn't use it currently.
Original change's description:
> [TestRunner] remove --chrome-features flag
>
> The scripts (both the old Python one and the new JS one) took
> `--chrome-features` as a flag, which was then set as
> `process.env.CHROME_FEATURES`, however searching the codebase for
> `CHROME_FEATURES` revealed that whilst we set this value, we never
> read it. Therefore this CL removes it entirely, my logic being that if
> someone needed this flag to work, we would have found this bug a long
> time ago.
>
> We can reintroduce should we find a usecase for it in the future.
>
> Bug: chromium:1186163
> Change-Id: I11d900df4aaad232fd64b9edca2aac12c94fc988
> Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2782555
> Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
> Reviewed-by: Tim van der Lippe <tvanderlippe@chromium.org>
> Reviewed-by: Philip Pfaffe <pfaffe@chromium.org>
Bug: chromium:1186163
Change-Id: I336a4a14a3e720f3b0626d78148038ceab9a8774
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2786942
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Reviewed-by: Philip Pfaffe <pfaffe@chromium.org>
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
The scripts (both the old Python one and the new JS one) took
`--chrome-features` as a flag, which was then set as
`process.env.CHROME_FEATURES`, however searching the codebase for
`CHROME_FEATURES` revealed that whilst we set this value, we never
read it. Therefore this CL removes it entirely, my logic being that if
someone needed this flag to work, we would have found this bug a long
time ago.
We can reintroduce should we find a usecase for it in the future.
Bug: chromium:1186163
Change-Id: I11d900df4aaad232fd64b9edca2aac12c94fc988
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2782555
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
Reviewed-by: Tim van der Lippe <tvanderlippe@chromium.org>
Reviewed-by: Philip Pfaffe <pfaffe@chromium.org>
The current tests try to figure out the root directory and then glob
for _test.ts files within there, but this breaks easily if the folder
structure isn't exactly as expected. Instead we can set this via a
flag in the test runner.
This CL only updates the interaction tests, which use the new runner.
e2e tests will be supported in time.
Bug: chromium:1186163
Change-Id: I198b67bf0081ce7334c40d7d661fb8642c2d56d0
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2772045
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
Auto-Submit: Jack Franklin <jacktfranklin@chromium.org>
Reviewed-by: Paul Lewis <aerotwist@chromium.org>
This CL adds a flag to the new test runner to provide the test-server-type.
We intelligently set it if the test suite ends with `e2e` or
`interactions`, but otherwise it's required. Unfortunately this can't
be checked as part of Yargs' API, so we check for it ourselves at run
time.
Bug: chromium:1186163
Change-Id: I6487f1a95c6c86759e1489309d4c18aec1292073
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2767266
Auto-Submit: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Paul Lewis <aerotwist@chromium.org>
Reviewed-by: Paul Lewis <aerotwist@chromium.org>
That. Note that this currently isn't used or set because everyone +
all our scripts still run the Python version of the test runner
script, not the new script. But this needs to be in place before we
move people over to the new script.
We provide a fallback value such that this still works and is able to
run the tests despite the test runner not providing the configuration
via an environment variable.
Bug: chromium:1186163
Change-Id: I8255e5c7f9902be845e3c33add7b1703f6e4db9d
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2766904
Reviewed-by: Paul Lewis <aerotwist@chromium.org>
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
This CL adds two flags to the new test runner for configuring the
components server and then updates the server to detect those values.
Note that whilst this CL adds yet-another-way to configure the
components server, this is only being done for backwards compat whilst
we roll out this change to all the various places. We will remove old
options towards the end of this work.
Bug: chromium:1186163
Change-Id: I4c9b85a5745b9fe36894e3c492952938a5b8f648
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2763875
Auto-Submit: Jack Franklin <jacktfranklin@chromium.org>
Reviewed-by: Paul Lewis <aerotwist@chromium.org>
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
This CL lands the initial new run_test_suite.js that can take its
configuration from a config JSON file, or from flags.
This new script is only used to run `npm run auto-interactionstest`
and nothing more; I want to roll it out slowly and ensure that
everyone is aware of it before removing the old Python script. I will
create a doc with all the various steps, and required documentation,
as I've taken the chance to rename some options to make them clearer.
Bug: chromium:1186163
Change-Id: I5a199f82ac7ab0f323988acacaa7aae2d64e6349
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2763866
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
Reviewed-by: Paul Lewis <aerotwist@chromium.org>
The Localization presubmit steps include checking Loc V1 and V2:
V1: Done with migration, safe to remove
V2:
- check unused string in UIStrings (Moved to ESLint)
- check migrated directory (Done with migration, safe to remove)
- check the shape of API calls (Done by typescript or ESLint)
Since all checks are obsolete or moved, this CL removes the presubmit step to unhook the scripts inside scripts/localization. I will follow up a CL to remove the actual scripts.
The CL that introduced this step https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/1931279
Bug: 1136655
Change-Id: If203607588896693a6331de606c873458b74cd6e
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2746919
Reviewed-by: Simon Zünd <szuend@chromium.org>
Commit-Queue: Christy Chen <chrche@microsoft.com>
The current test suite runner assumes the paths; you pass
--test-suite=interactions and it assumes that they live in
out/TARGET/gen/test/interactions. This CL removes that assumption by
asking people to pass --test-suite-path, which would be
out/gen/interactions.
We will remove the --test-suite flag, but for now we're keeping it
around so the bots can still run their tests. I will file a bug to ask
that the bots have their recipes updated.
Bug:1182255
Change-Id: I6d9e5d419231845047394c5f7cbd0c74ebad3a58
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2722448
Reviewed-by: Paul Lewis <aerotwist@chromium.org>
Commit-Queue: Jack Franklin <jacktfranklin@chromium.org>
Jack Franklin