This CL adds an optional --chrome-binary argument to the unittest.py
script to use a specific Chrome binary to run unit tests against.
When no argument is specified, the script defaults to the downloaded
Chrome binary from `gclient sync` (same behavior as before).
We'd like to have the option of running unit tests on a Microsoft Edge
executable instead of the default downloaded Chrome executable.
Change-Id: Ice72f129086bcc1da56efa118280532cdbb25fd3
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2202628
Reviewed-by: Brandon Goddard <brgoddar@microsoft.com>
Reviewed-by: Tim van der Lippe <tvanderlippe@chromium.org>
Reviewed-by: Robert Paveza <Rob.Paveza@microsoft.com>
Commit-Queue: Brian Cui <brcui@microsoft.com>
This resurrects the perf test suite, as the older test-runner setup
was removed, but the perf test suite was not ported.
Example output:
Spawning hosted mode server
Boot performance
✓ run 1/37 (1185ms)
✓ run 2/37 (1234ms)
✓ run 3/37 (1213ms)
✓ run 4/37 (1376ms)
✓ run 5/37 (1327ms)
✓ run 6/37 (1235ms)
✓ run 7/37 (1352ms)
✓ run 8/37 (1439ms)
✓ run 9/37 (1332ms)
✓ run 10/37 (1290ms)
✓ run 11/37 (1229ms)
✓ run 12/37 (1241ms)
✓ run 13/37 (1233ms)
✓ run 14/37 (1203ms)
✓ run 15/37 (1177ms)
✓ run 16/37 (1336ms)
✓ run 17/37 (1526ms)
✓ run 18/37 (1465ms)
✓ run 19/37 (1537ms)
✓ run 20/37 (1709ms)
✓ run 21/37 (1760ms)
✓ run 22/37 (1704ms)
✓ run 23/37 (2549ms)
✓ run 24/37 (1387ms)
✓ run 25/37 (1934ms)
✓ run 26/37 (1860ms)
✓ run 27/37 (1309ms)
✓ run 28/37 (1295ms)
✓ run 29/37 (1391ms)
✓ run 30/37 (1569ms)
✓ run 31/37 (1562ms)
✓ run 32/37 (1442ms)
✓ run 33/37 (1431ms)
✓ run 34/37 (1646ms)
✓ run 35/37 (1927ms)
✓ run 36/37 (1600ms)
✓ run 37/37 (1587ms)
Mean boot time: 1475.28ms
50th percentile boot time: 1390.83ms
90th percentile boot time: 1860.02ms
99th percentile boot time: 2549.29ms
R=aerotwist@chromium.org
Change-Id: I1b6a79fdfc1b7c98f6156d3309b5b98ff8530476
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2209108
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
Reviewed-by: Paul Lewis <aerotwist@chromium.org>
This significantly speeds up Karma execution times to about 12 seconds on an unchanged build folder.
It uses the Ninja build output to find the unittest files.
You can run the new script with:
npm run unittest
npm run unittest -- --target=Release
To make sure you perform a minimal build and run tests right after it, run:
npm run auto-unittest
npm run auto-unittest -- --target=Release
If no ninja-build-name is set, it assumes that `out/Default` exists.
The `auto-unittest` command will run autoninja for you on the output folder.
R=jacktfranklin@chromium.org,aerotwist@chromium.org
No-Presubmit: true
Bug: 1061125
Change-Id: I45edd11e422c5cdc8a4fc0bbb6bc43e386519aa9
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2102717
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
Reviewed-by: Brandon Goddard <brgoddar@microsoft.com>
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
Reviewed-by: Paul Lewis <aerotwist@chromium.org>
This has two benefits:
1. We can now "pre-process" the arguments. It therefore allows us
to specifically set the tests we want to run. Before this change,
we would incorrectly include JavaScript files that were outputs
from (since-removed) TypeScript tests. Since adding these as
arguments to the Mocha invocation causes issues on Windows bots
with "arguments too long" errors, we should be using the config
file.
2. We can add additional arguments here without the need
of passing all the arguments from `run_test_suite.py` in.
R=petermarshall@chromium.orgTBR=aerotwist@chromium.org
Bug: 1071369
Change-Id: Icab02b1117f4095081987b65c8151ddf04239e11
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2157044
Reviewed-by: Tim van der Lippe <tvanderlippe@chromium.org>
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
Auto-Submit: Tim van der Lippe <tvanderlippe@chromium.org>
The shared test runner logic was reimplementing parts of Mocha,
in particular the test logging and filtering. Moreover, it was
booting the hosted mode server and puppeteer outside of Mocha.
Mocha supports root level hooks [1]. These hooks allow us to
perform work before a test starts. Moreover, by using the
before and after hook, we can run logic before and after
all tests.
By using these hooks, we can extract the "boot the hosted mode
server and hookup puppeteer" part to these root hooks.
Additionally, we can reset the pages in the `beforeEach`, which
means that tests themselves don't have to reset the pages.
We also put the implementation code into third_party/conductor,
as we would like to reuse this logic for the Puppeteer tests.
The Puppeteer test suite now also uses Mocha and has very
similar requirements as to our DevTools tests. By extracting
from DevTools, we can look into expanding the test runner to
other usecases, but that is out of scope for now.
[1]: https://mochajs.org/#root-level-hooks
Bug: 1071369
Change-Id: Ie9f954359d9de84da564b74b6f5517dd535db008
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2150458
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
Reviewed-by: Paul Lewis <aerotwist@chromium.org>
I ran into this while trying to reproduce Closure failures locally for
a different CL. Python's built-in remove method on lists returns None
instead of the modified list, which was causing the resulting value in
exec_command to be incorrect when a Java install without a server JVM
was used.
Change-Id: Ib2629b513c2f6401c51654e68792509796366e61
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2150731
Reviewed-by: Leo Lee <leolee@microsoft.com>
Reviewed-by: Yang Guo <yangguo@chromium.org>
Commit-Queue: Tony Ross <tross@microsoft.com>
We ignore some files for ESLint, as they are either generated or
third_party. However, when someone tries to change these files,
ESLint would emit a warning stating that the file in question is
ignored.
This issue was reported to ESLint in https://github.com/eslint/eslint/issues/9977
The suggested workaround is to use the CLIEngine to filter out
the problematic paths. However, since our script was written in
Python, that API is not accessible to us.
Therefore rewrite the script to Node and filter out the problematic
files. The calls to CLIEngine were mostly taken from eslint/lib/cli.js,
which was the previous file used by `node_modules/bin/eslint`.
R=jacktfranklin@chromium.orgCC=sigurds@chromium.org
Change-Id: Iee600f0e0d99fcb6eeeb203a952a50fe35f9aaf3
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2149316
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
Auto-Submit: Tim van der Lippe <tvanderlippe@chromium.org>
front_end/Runtime.js was doing multiple things: it was instantiating
side-effecty descriptors and initializing applications, as well as
storing all state for the modules/extensions, etc...
To support proper ES Modules, we need to separate these. root/Runtime.js
contains simple classes that store the data and can be retrieved, as well
as some helper functions that will load the respective data.
The RuntimeInstantiator.js code includes the start functions used in the
entrypoints themselves. They will use an instance of the Runtime to
properly load the application.
Along the way, I also fixed various TypeScript errors, mostly around
incorrect or missing type definitions.
R=aerotwist@chromium.org,jacktfranklin@chromium.org
Bug: 1011811
Change-Id: If3ddea9267813691b3faeb6ea4b0cf64edbde1f4
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2107523
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
This CL changes references to self.Common.settings (the global
instance of SDK.Common.Settings) over to
Common.Settings.Settings.instance(). To keep both TypeScript and
Closure happy we must make a method on the Settings class itself,
since it only allows private constructors to be accessed by static
methods on the class.
Bug: 1058320
Change-Id: I04afc8caf64acf29cdda13ef03ad05cfff4786a1
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2091450
Commit-Queue: Paul Lewis <aerotwist@chromium.org>
Reviewed-by: Tim van der Lippe <tvanderlippe@chromium.org>
For most of the PRESUBMIT invocations, the CL in question does not
change anything with regards to ESLint. Therefore, we can skip doing the
full check and only perform the check on relevant JavaScript and
TypeScript files.
However, if we do change the ESLint configuration (via one of the
involved build scripts/configuration files), we should run the full
check to make sure we are compliant.
`npm run check-lint` will still run the full lint check.
This saves about 15 seconds on a regular PRESUBMIT invocation, scaling
with the number of files changed.
R=jacktfranklin@chromium.org,aerotwist@chromium.org
Change-Id: I9c1bb055274bfa1d8cc47c9039a1196c92eba399
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2097993
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
This improves the PRESUBMIT performance as we can rely on the AST
parsing of ESLint, meaning we don't have to parse AST twice.
It also allows us to use the nice `--fix` solution to insert the proper
license header in the files.
This shaves an expected 5 seconds of the presubmit time
(non-scientifically computed based on 2 uploads).
Change-Id: I5c53e9b232585f9ec46f86b36557c50bf80f7add
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2097992
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
This checks in the InspectorBackendCommands in the source tree file
and ensures that it is kept in sync whenever the third_party
location has been updated.
In the process, I discovered multiple misconfigurations in the formatting
presubmit check. First of all, it was never running, because the
.eslintignore had an empty line. Second of all, it was running twice,
which is unnecessary since we now check for changed files at the end
of the presubmit. Lastly, it was only formatting JS files, while it
should check all files.
I have also updated the _CheckGeneratedFiles check to only run if it
is actually necessary. If there are no changes made to any affected of
the files, it will skip the step. This should thus reduce the presubmit
time and we will only pay the cost if we actually update any of the
files.
Lastly, it will now properly format and lint the generated files. This
makes reading the code a lot easier and makes it easier to digest the
diff when a protocol update goes through. I have verified that, in a
full build, the files are still minified. Thus, this has no impact on
the loading performance.
DISABLE_THIRD_PARTY_CHECK=Updating protocol generation
Fixed: 1056614
Change-Id: If49b0e749978ea1a7838992ec13507ee761ad76c
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2087765
Reviewed-by: Paul Lewis <aerotwist@chromium.org>
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
This instructs Ninja to generate the appropriate files for the test
files. It adds a separate `ts_library` definition for tests.
Some caveats:
- It currently hardcodes the Ninja build output name. We should fix out
a solution on how to make this irrespective of build name
- The test name has to be renamed to Color_test.ts. While I think this
is clearer for code navigation, it was necessary because `ts_library`
currently copies to `resources/inspector`, which is not necessary for
test files. We could therefore filter these out, based on the files that
end with `_test.ts`. We could also add a separate Ninja template,
something like `ts_test_library`. Not sure yet what is the best
approach.
DISABLE_THIRD_PARTY_CHECK=Updating TypeScript configuration
Bug: 1011811
Change-Id: If7448474e84a013d8d4b00a16743575a12112f58
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2064669
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
This CL migrates shared code from e2e, screenshots, and unit to a helper
file. It also introduces a unified runner for e2e and screenshots (and
perf once that's in place). Once this has landed I will remove
run_e2e.py and use run_test_suite.py instead.
Change-Id: I590e5eb6f3e8a4692767a2aa653b55685891614b
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2054547
Reviewed-by: Tim van der Lippe <tvanderlippe@chromium.org>
Commit-Queue: Paul Lewis <aerotwist@chromium.org>
There were multiple issues:
1. The e2e project was accidentally recompiling the shared files again,
thus we needed to use project references for that
2. There was a difference in configuration of the root project and the
e2etests. As such, make sure to extend the root config.
3. The code was being transpiled down to es5, which is not necessary as
we run a recent version of Node. Thus, we can use `esnext` instead.
Debugging scripts should now be a lot easier, especially as
`async-await` are no longer transpiled.
For now, strict is turned off, as there are various errors that pop up.
I will clean that up in a follow-up CL.
Bug: 1044632
Change-Id: Ie0cf29a1ddd1e027bcc107d2567283eaaaa7fead
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2047183
Reviewed-by: Paul Lewis <aerotwist@chromium.org>
Reviewed-by: Jack Franklin <jacktfranklin@chromium.org>
Commit-Queue: Tim van der Lippe <tvanderlippe@chromium.org>
We plan to have multiple suites using the same machinery that currently
runs the e2e. This CL moves that machinery out to a shared folder and
remaps the e2e code to use everything in the new locations.
Future CLs will introduce additional suites for capturing traces,
screenshots, and other artifact.
Change-Id: I452424d8f65fb03897f5a59606c592be996e91ab
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/2035949
Commit-Queue: Paul Lewis <aerotwist@chromium.org>
Reviewed-by: Mathias Bynens <mathias@chromium.org>
Tim van der Lippe