mirror of https://github.com/react/react-native-devtools-frontend.git synced 2026-10-30 09:54:02 +08:00

Files

T

Alex Rudenko bf9bcbf328 A helper script to convert to tsv

Bug: 379059135
Change-Id: I555f763d2123f1cb1b39fd3ae9468bb0f69613f1
Reviewed-on: https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/6038154
Reviewed-by: Ergün Erdoğmuş <ergunsh@chromium.org>
Commit-Queue: Ergün Erdoğmuş <ergunsh@chromium.org>
Auto-Submit: Alex Rudenko <alexrudenko@chromium.org>

2024-11-21 12:27:51 +00:00

eval

[AutoFreestyler] Basic side-by-side view for eval

2024-10-09 14:36:30 +00:00

auto_freestyler.js

[FreestylerEval] test using UI

2024-11-20 19:19:03 +00:00

OWNERS

[Freestyler] Fix ui translation issues and add a tab clean-up step

2024-08-29 12:15:55 +00:00

README.md

[Freestyler] Add label to auto freestyler runs for better inspection

2024-08-30 11:41:52 +00:00

to_tsv.mjs

A helper script to convert to tsv

2024-11-21 12:27:51 +00:00

README.md

Freestyler Evaluation

This directory contains scripts for the prompt iteration & evaluation process for Freestyler.

Mainly, auto_freestyler.js script takes example URLs, runs the examples and outputs the results to the data/ directory. Then, the HTML page in eval/ folder takes these results and presents them in a UI for evaluation.

Running auto freestyler

Prerequisites

You need to have at least Node v20 to run Auto Freestyler.
You need to have a version of Chrome that contains the Freestyler feature. (i.e. you also need to be logged in & have sync enabled)

Steps

Run a Freestyler enabled Chrome executable with --remote-debugging-port 9222 and --auto-open-devtools-for-tabs targeting about:blank. For example:

/Applications/Google\ Chrome\ Canary.app/Contents/MacOS/Google\ Chrome\ Canary --remote-debugging-port=9222 --auto-open-devtools-for-tabs about:blank

Make sure you use en-US locale in DevTools.
Close the DevTools window for the initial about:blank page but keep the tab open.
Run the following command

node scripts/freestyler/auto_freestyler.js --example-urls <example-url-1> <example-url-2>

At the end of these steps, the examples in the urls <example-url-1> and <example-url-2> should be run and the results must be saved to the data/ folder.

Tip: You can add a --label <label> argument to the run to label the dataset. For example:

node scripts/freestyler/auto_freestyler.js --label title-change --example-urls <example-url-1> <example-url-2>

Evaluating the results

Steps

Serve the scripts/freestyler folder by using a simple file server. For example:

python3 -m http.server

Visit http://localhost:8000/eval URL to see the UI.
Select the dataset you want to evaluate in the UI. (Dataset: selector)
Evaluate the examples one by one.

Tip: You can navigate the questions with Tab key and move between examples with ArrowLeft and ArrowRight.