document both MACE checkpoints tested and update the checkpoint on figshare to the one used for MBD v1 submission

Author: janosh

data/figshare/1.0.0.json

@@ -4,8 +4,8 @@
     "2023-06-02-pbenner-best-alignn-model.pth.zip"
   ],
   "mace_checkpoint": [
-    "https://figshare.com/ndownloader/files/41565618",
-    "2023-07-14-mace-universal-2-big-128-6.model"
+    "https://figshare.com/ndownloader/files/42374049",
+    "2023-08-14-mace-yuan-trained-mptrj-04.model"
   ],
   "mp_computed_structure_entries": [
     "https://figshare.com/ndownloader/files/40344436",

matbench_discovery/data.py

@@ -248,7 +248,7 @@ def _on_not_found(self, key: str, msg: str) -> None:  # type: ignore[override]
     )
     wbm_summary = "wbm/2022-10-19-wbm-summary.csv.gz"
     alignn_checkpoint = "2023-06-02-pbenner-best-alignn-model.pth.zip"
-    mace_checkpoint = "2023-07-14-mace-universal-2-big-128-6.model"
+    mace_checkpoint = "2023-08-14-mace-yuan-trained-mptrj-04.model"
 
 
 # data files can be downloaded and cached with matbench_discovery.data.load()

models/mace/readme.md

@@ -1,5 +1,6 @@
 ## MACE formation energy predictions on WBM test set
 
-This submission uses the [`2023-07-14-mace-universal-2-big-128-6.model`](https://figshare.com/ndownloader/files/41565618) checkpoint pre-trained on the [original M3GNet training set](https://figshare.com/articles/dataset/MPF_2021_2_8/19470599).
+This submission uses the [`2023-08-14-mace-yuan-trained-mptrj-04.model`](https://figshare.com/ndownloader/files/42374049) checkpoint trained by Yuan Chiang on the [MPtrj dataset](https://figshare.com/articles/dataset/23713842).
+We initially tested the `2023-07-14-mace-universal-2-big-128-6.model` checkpoint trained on the much smaller [original M3GNet training set](https://figshare.com/articles/dataset/MPF_2021_2_8/19470599) which we received directly from Ilyes Batatia. MPtrj-trained MACE performed better and was used for the Matbench Discovery v1 submission.
 
 MACE relaxed each test set structure until the maximum force in the training set dropped below 0.05 eV/Å or 500 optimization steps were reached, whichever occurred first.

site/src/routes/preprint/+page.md

@@ -285,7 +285,7 @@ All force-free models exhibit a much worse case of early-on precision drop, fall
 
 > @label:fig:rolling-mae-vs-hull-dist-models Universal potentials are more reliable classifiers because they exit the red triangle earliest.
 > These lines show the rolling MAE on the WBM test set as the energy to the convex hull of the MP training set is varied, lower is better.
-> The red-highlighted 'triangle of peril' shows where the models are most likely to misclassify structures.
+> The large red 'triangle of peril' shows where the models are most likely to misclassify structures.
 > As long as a model's rolling MAE remains inside the triangle, its mean error is larger than the distance to the convex hull.
 > If the model's error for a given prediction happens to point towards the stability threshold at 0 eV from the hull (the plot's center), its average error will change the stability classification of a material from true positive/negative to false negative/positive.
 > The width of the 'rolling window' box indicates the width over which errors hull distance prediction errors were averaged.