Skip to content

Remove msgpack from framework - Part II #2449

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Apr 13, 2021

Conversation

jumaffre
Copy link
Contributor

@jumaffre jumaffre commented Apr 12, 2021

Follow-up from #2343

kv::Map is no longer serialised with msgpack by default.

TODO:

  • Update docs.
  • Move msgpack dependency folder to test/ as we still use msgpack for TPCC.

@ghost
Copy link

ghost commented Apr 12, 2021

remove_msgpack@23340 aka 20210413.2 vs main ewma over 20 builds from 22975 to 23336

Click to see table
build_id build_number sb_sgx_cft^ sb_sgx_cft_mem sb_sgx_bft^ sb_sgx_bft_mem sb_ws_sgx_cft^ sb_ws_sgx_cft_mem sb_sig_sgx_cft^ sb_sig_sgx_cft_mem tpcc_sgx_cft^ tpcc_sgx_cft_mem tpcc_sgx_bft^ tpcc_sgx_bft_mem ls_sgx_cft^ ls_sgx_cft_mem ls_ws_sgx_cft^ ls_ws_sgx_cft_mem ls_jwt_sgx_cft^ ls_jwt_sgx_cft_mem ls_js_sgx_cft^ ls_js_sgx_cft_mem ls_js_jwt_sgx_cft^ ls_js_jwt_sgx_cft_mem CHAMP put (/s)^ CHAMP get (/s)^
22975 20210407.16 24927.9 1.19317e+07 14691.8 3.81461e+07 32626.6 1.27181e+07 4846.57 1.00967e+07 6506.04 8.63806e+07 3299.17 1.91762e+08 24302.8 1.71746e+07 31355.5 1.74367e+07 4404.51 1.45532e+07 2070.24 1.29803e+07 1447.19 8.2617e+06 1.3375e+06 3.5993e+07
23005 20210408.1 27229.9 1.21939e+07 14256.3 3.70975e+07 32560.8 1.27181e+07 4865.39 1.00967e+07 6830.24 8.48077e+07 3420.04 1.58994e+08 24781.2 1.71746e+07 31687.4 1.74367e+07 4019.78 1.4291e+07 1971.12 1.27181e+07 1380.23 7.99955e+06 1.33099e+06 3.66369e+07
23006 20210408.2 24902.5 1.21939e+07 14528.6 3.84083e+07 31561.7 1.2456e+07 4949.59 1.03588e+07 6324.8 8.42835e+07 3403.96 1.55324e+08 25640.5 1.74367e+07 30056.8 1.76989e+07 4389.86 1.45532e+07 2001.44 8.78598e+06 1440.09 9.31027e+06 1.30153e+06 3.59298e+07
23025 20210408.8 27003.6 1.21939e+07 14342.6 3.84083e+07 33791.5 1.29803e+07 4904.22 1.00967e+07 6502.3 8.45456e+07 3353.36 1.65548e+08 23086 1.69124e+07 28869.3 1.74367e+07 4195.23 1.45532e+07 1870.96 7.99955e+06 1381.17 8.2617e+06 1.34311e+06 3.56168e+07
23050 20210408.16 28117.6 1.2456e+07 14800.9 3.63111e+07 31778.9 1.4291e+07 4978.96 1.03588e+07 6267.46 8.61185e+07 3205.02 1.67645e+08 23256.9 1.69124e+07 31172.8 1.7961e+07 3996.57 1.4291e+07 1895.04 8.78598e+06 1388.83 9.31027e+06 1.33741e+06 3.5993e+07
23069 20210408.22 27412.1 1.2456e+07 13953.1 3.81461e+07 31122.8 1.27181e+07 4847.12 1.00967e+07 6566.49 8.5332e+07 3325.82 1.7682e+08 22045.9 1.74367e+07 30587.1 1.76989e+07 4155.28 1.4291e+07 2017.73 9.04813e+06 1445.6 7.99955e+06 1.34364e+06 3.55549e+07
23079 20210408.25 27847.3 1.2456e+07 14288 3.68354e+07 31087.8 1.27181e+07 4866.98 1.03588e+07 6578.06 8.50699e+07 3228.6 1.9019e+08 25983.1 1.71746e+07 28739.5 1.71746e+07 4003.31 1.4291e+07 1974.56 9.04813e+06 1433.32 7.73741e+06 1.3211e+06 3.61193e+07
23094 20210409.1 26993 1.21939e+07 14504.1 3.99811e+07 33739.2 1.29803e+07 4963.63 1.00967e+07 6700.18 8.50699e+07 3385.08 1.68694e+08 23941.4 1.69124e+07 30881.9 1.74367e+07 3825.47 1.40289e+07 1968.98 8.78598e+06 1427.8 8.2617e+06 1.3627e+06 3.5993e+07
23107 20210409.5 26984 1.2456e+07 15451.5 3.68354e+07 32370.8 1.27181e+07 5417.27 1.03588e+07 6572.9 8.50699e+07 3163.35 2.27152e+08 23913.3 1.69124e+07 32587.3 1.74367e+07 4223.19 1.4291e+07 1999.3 9.04813e+06 1504.13 7.47526e+06 1.32779e+06 3.61193e+07
23117 20210409.8 25904.1 1.21939e+07 14215.2 3.89325e+07 30392.5 1.29803e+07 4922.96 1.0621e+07 6551.68 8.48077e+07 2788.25 2.53366e+08 25015.3 1.71746e+07 30914 1.74367e+07 3853.23 1.40289e+07 2003.12 8.78598e+06 1463.83 7.73741e+06 1.33376e+06 3.62478e+07
23154 20210409.20 25455.3 1.2456e+07 14108.8 3.7884e+07 33597.4 1.29803e+07 4926.18 1.0621e+07 6639.66 8.5332e+07 3302.32 1.58208e+08 23577.4 1.7961e+07 31825 1.76989e+07 4019.73 1.4291e+07 2096.98 9.04813e+06 1441.42 7.99955e+06 1.3561e+06 3.63121e+07
23184 20210412.1 26532.9 1.27181e+07 14780.4 3.76218e+07 31976.2 1.27181e+07 4974.16 1.0621e+07 6596.69 8.58563e+07 3167.78 1.93335e+08 24313.6 1.71746e+07 31434.5 1.9796e+07 4196.96 1.45532e+07 2095.93 9.04813e+06 1395.1 7.99955e+06 1.32935e+06 3.65062e+07
23191 20210412.4 28168.2 1.21939e+07 13578.1 3.81461e+07 31048.7 1.27181e+07 5390.42 1.03588e+07 6693.37 8.50699e+07 3244.67 1.88879e+08 22873 1.76989e+07 31199.8 1.74367e+07 3656.45 1.40289e+07 1989.73 1.11453e+07 1408.86 9.31027e+06 1.3404e+06 3.62478e+07
23228 20210412.16 27094.5 1.2456e+07 14158 3.86704e+07 32506.2 1.29803e+07 4962.35 1.0621e+07 6724.78 8.5332e+07 3371.49 1.80752e+08 23486.6 1.69124e+07 31683.6 1.74367e+07 4059.8 1.48153e+07 2087.88 9.04813e+06 1465.14 7.99955e+06 1.34365e+06 3.58042e+07
23274 20210412.31 25939.2 1.2456e+07 15381.3 3.76218e+07 34876.6 1.27181e+07 4882.44 1.16696e+07 6475.3 8.40213e+07 3489.43 1.53227e+08 24076.5 1.69124e+07 30935.1 1.74367e+07 4201.1 1.53396e+07 2063.54 9.04813e+06 1409.9 8.2617e+06 1.32883e+06 3.58663e+07
23287 20210412.35 26181 1.19317e+07 15279.3 3.76218e+07 30442.5 1.27181e+07 4775.29 1.08831e+07 6243.42 8.37592e+07 3311.97 1.6581e+08 22556.2 1.71746e+07 30038.6 1.7961e+07 4179.6 1.4291e+07 2004.41 1.27181e+07 1398.76 1.00967e+07 1.34392e+06 3.53103e+07
23299 20210412.38 22458.5 1.21939e+07 14469 3.63111e+07 30947.7 1.32424e+07 4930.26 1.03588e+07 6645.11 8.50699e+07 2388.55 2.38424e+08 25354.5 1.71746e+07 30372.8 1.74367e+07 3931.02 1.45532e+07 2011.02 1.08831e+07 1452.91 9.83456e+06 1.3218e+06 3.54939e+07
23307 20210412.41 26681.9 1.21939e+07 14783.9 3.86704e+07 30591.8 1.2456e+07 4886.43 1.0621e+07 6590.81 8.45456e+07 3409.34 1.47722e+08 24744.1 1.76989e+07 31152 1.76989e+07 3734.74 1.37667e+07 2017.69 1.0621e+07 1392.95 8.2617e+06 1.32685e+06 3.59298e+07
23320 20210412.45 25555.7 1.21939e+07 13714.9 3.81461e+07 30542 1.27181e+07 4903.2 1.00967e+07 6234.66 8.40213e+07 3308.84 1.51654e+08 25564.5 1.71746e+07 27664.5 1.69124e+07 4277.3 1.48153e+07 1982.36 8.78598e+06 1412.9 7.73741e+06 1.31959e+06 3.60563e+07
23336 20210413.1 22699.4 1.19317e+07 15507.9 3.9719e+07 29730.3 1.2456e+07 4421.8 1.0621e+07 6756.09 8.5332e+07 2975.59 2.01462e+08 25559.6 1.90096e+07 30329.6 1.74367e+07 4497.1 1.48153e+07 2016.36 8.52384e+06 1385 7.99955e+06 1.33619e+06 3.58663e+07

images

@jumaffre jumaffre marked this pull request as ready for review April 12, 2021 10:58
@jumaffre jumaffre requested a review from a team as a code owner April 12, 2021 10:58
CHANGELOG.md Outdated

### Changed

- `kv::Map` is now an alias to `kv::JsonSerialisedMap`, which means all existing applications using `kv::Map` will now require `DECLARE_JSON...` macros (instead of `MSGPACK_DEFINE`) for custom key and value types. Note that this change may affect throughput of existing applications (#2449).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should also mention that msgpack is no longer installed or visible in their include paths. So any MSGPACK_DEFINEs in app code, that were previously necessary, actually need to be removed (and replaced with DECLARE_JSON_... if they didn't exist.

Also perhaps a suggestion to restore the lost throughput - "use Blit if you have simple types, or a custom serialiser"?

@@ -131,7 +131,7 @@ namespace tpcc
{
bool is_original = selected_rows.find(i) != selected_rows.end();
Stock s = generate_stock(i, wh_id, is_original);
auto stocks = args.tx.rw(tpcc::TpccTables::stocks);
auto stocks = args.tx.rw(TpccTables::stocks);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think all of the changes in this file are unnecessary? Or am I missing something?

Copy link
Member

@eddyashton eddyashton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code looks good, just think this needs a slightly more explicit warning in the CHANGELOG.

@achamayou achamayou merged commit 5707d03 into microsoft:main Apr 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants