Skip to content

Commit a7d8ebd

Browse files
authored
Add React benchmarking infrastructure (#9465)
* Initial commit for WIP benchmarking infrastructure * fixed lint issues and ran prettier * added <rootDir>/scripts/bench/ to ignore paths for Jest * tidied up code and fixed a few bugs in the runner.js * fixed eslint * improved the benchmark output from the runner * fixed typo * tided up print output in runner.js * throw error if chrome canary is not installed on mac * added better bench stats output (tables) * added benchmark diff to table results * adds bundle size comparisons to results * tidied up the results * fixed prettier output * attempt to trigger bech for circleci build * fixes flow exlclusion for lighthouse module * added class components benchmark * cleaned up stats.js * stability changes * circleci node version to 7 * added another benchmark * added colours to the different benchmarks to check if being cached * force no-cache headers * added more info messages * refactor chrome launching. * fixed an issue where launcher.kill might fail * Move server to runner. Launch it only once. * tidy up * changes the logic in how the remote repo is checked out * removes bench from circleci build * removed colors from benchmarks (no longer needed) * added CI integration comment * added hacker news benchmark * added skipBuild functionality * relabelled remote * Add confidence intervals * added first meaningful paint * removed some unused code * reverted code.json * updated benchmark runs back to 10 * no longer breaks when results contain missing bundles * adds CPU throttling * renamed build to remote-repo * small fix to build * fixed bad merge * upped runs to 10 from 2 again * properly pulls master * removes old-bench * runs benchmarks in headless mode * adds a --headless option * improved the git build process * added README * updated based feedback from review * adds merge base commit sha * addressing more PR feedback * remove built JS react files * updated .gitignore * added combined bundle load times to the metrics
1 parent 542dac4 commit a7d8ebd

38 files changed

+7394
-10624
lines changed

.eslintignore

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,6 @@ fixtures/
1313
# Ignore built files.
1414
build/
1515
coverage/
16-
scripts/bench/bench-*.js
16+
scripts/bench/benchmarks/**/*.js
1717
vendor/*
1818
**/node_modules

.flowconfig

+2
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,8 @@
44
<PROJECT_ROOT>/build/.*
55
<PROJECT_ROOT>/scripts/.*
66
<PROJECT_ROOT>/.*/node_modules/y18n/.*
7+
<PROJECT_ROOT>/node_modules/chrome-devtools-frontend/.*
8+
<PROJECT_ROOT>/node_modules/devtools-timeline-model/.*
79
<PROJECT_ROOT>/.*/__mocks__/.*
810
<PROJECT_ROOT>/.*/__tests__/.*
911

.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ static
77
_SpecRunner.html
88
__benchmarks__
99
build/
10+
remote-repo/
1011
coverage/
1112
.module-cache
1213
*.gem
@@ -29,3 +30,4 @@ chrome-user-data
2930
.vscode
3031
*.swp
3132
*.swo
33+
*react*min*.js

package.json

+9-1
Original file line numberDiff line numberDiff line change
@@ -62,14 +62,19 @@
6262
"glob-stream": "^6.1.0",
6363
"gzip-js": "~0.3.2",
6464
"gzip-size": "^3.0.0",
65+
"http-server": "^0.9.0",
66+
"http2": "^3.3.6",
6567
"jest": "^19.0.1",
6668
"jest-config": "^19.0.1",
6769
"jest-jasmine2": "^19.0.1",
6870
"jest-runtime": "^19.0.1",
71+
"lighthouse": "^1.6.3",
6972
"loose-envify": "^1.1.0",
7073
"merge-stream": "^1.0.0",
74+
"mime": "^1.3.4",
7175
"minimist": "^1.2.0",
7276
"ncp": "^2.0.0",
77+
"nodegit": "^0.18.0",
7378
"object-assign": "^4.1.1",
7479
"platform": "^1.1.0",
7580
"prettier": "^1.2.2",
@@ -84,6 +89,7 @@
8489
"rollup-plugin-replace": "^1.1.1",
8590
"rollup-plugin-uglify": "^1.0.1",
8691
"run-sequence": "^1.1.4",
92+
"stats-analysis": "^2.0.0",
8793
"through2": "^2.0.0",
8894
"tmp": "~0.0.28",
8995
"typescript": "~1.8.10",
@@ -99,6 +105,7 @@
99105
"version": 7
100106
},
101107
"scripts": {
108+
"bench": "npm run version-check && node scripts/bench/runner.js",
102109
"build": "npm run version-check && node scripts/rollup/build.js",
103110
"linc": "git diff --name-only --diff-filter=ACMRTUB `git merge-base HEAD master` | grep '\\.js$' | xargs eslint --",
104111
"lint": "node ./scripts/tasks/eslint.js",
@@ -112,7 +119,8 @@
112119
"modulePathIgnorePatterns": [
113120
"/.module-cache/",
114121
"<rootDir>/build/",
115-
"<rootDir>/scripts/rollup/shims/"
122+
"<rootDir>/scripts/rollup/shims/",
123+
"<rootDir>/scripts/bench/"
116124
],
117125
"rootDir": "",
118126
"transform": {

scripts/bench/README.md

+27-26
Original file line numberDiff line numberDiff line change
@@ -1,40 +1,41 @@
1-
Work-in-progress benchmarks.
1+
# React Benchmarking
22

3-
## Running the suite
3+
## Commands
44

5-
You'll need two folders to compare, each of them containing `react.min.js` and `react-dom-server.min.js`. You can run `npm run build` at the repo root to get a `build` folder with these files.
5+
In most cases, the only two commands you might want to use are:
66

7-
For example, if you want to compare a stable verion against master, you can create folders called `build-stable` and `build-master` and use the benchmark scripts like this:
7+
- `yarn bench`
8+
- `yarn build -- --type=UMD_PROD && yarn bench -- --skip-build`
89

9-
```
10-
$ ./measure.py build-stable stable.txt build-master master.txt
11-
$ ./analyze.py stable.txt master.txt
12-
```
10+
The first command will run benchmarks with all the default settings. A local and remote build will occcur on all bundles, both local and remote repos will be run against all benchmarks.
1311

14-
The test measurements (second argument to `analyze`, `master.txt` in this example) will be compared to the control measurements (first argument to `analyze`, `stable.txt` in this example).
12+
The second command will run all benchmarks but skip the build process. This is useful for when doing local performance tweaking and the remote repo has already had its bundles built. Bboth local and remote repos will be run against all benchmarks with this command too.
1513

16-
Changes with the `-` sign in the output mean `master` is faster than `stable`.
14+
The other commands are as follows:
1715

18-
You can name folders any way you like, this was just an example.
16+
```bash
17+
# will compare local repo vs remote merge base repo
18+
yarn bench
1919

20-
## Running one
21-
One thing you can do with them is benchmark initial render time for a realistic hierarchy:
20+
# will compare local repo vs remote merge base repo
21+
# this can significantly improve bench times due to no build
22+
yarn bench -- --skip-build
2223

23-
```
24-
$ which jsc
25-
/System/Library/Frameworks/JavaScriptCore.framework/Versions/Current/Resources/jsc
26-
$ jsc react-0.14.0.min.js bench-pe-es5.js -e 'var START=Date.now(); React.renderToString(React.createElement(Benchmark)); var END=Date.now(); print(END-START);'
27-
45
28-
```
24+
# will only build and run local repo against benchmarks (no remote values will be shown)
25+
yarn bench -- --local
2926

30-
Substitute `js` or `v8` for `jsc` to use SpiderMonkey or V8, respectively, if you've installed them.
27+
# will only build and run remote merge base repo against benchmarks (no local values will be shown)
28+
yarn bench -- --remote
3129

32-
## Creating one
30+
# will only build and run remote master repo against benchmarks
31+
yarn bench -- --remote=master
3332

34-
To create one, copy `extract-component.js` to your clipboard and paste it into the Chrome console on facebook.com, perhaps after changing the root ID if you don't want the tree with ID `.0`.
33+
# same as "yarn bench"
34+
yarn bench -- --remote --local
3535

36-
Then to convert it to ES5:
36+
# runs benchmarks with Chrome in headless mode
37+
yarn bench -- --headless
3738

38-
```
39-
babel --whitelist react,react.displayName --compact false bench-pe.js >bench-pe-es5.js
40-
```
39+
# runs only specific string matching benchmarks
40+
yarn bench -- --benchmark=hacker
41+
```

scripts/bench/analyze.py

-111
This file was deleted.

0 commit comments

Comments
 (0)