Skip to content

docs: add links to keys #981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 4, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions doc/run/example-of-machine.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

## DPDispatcher Update Note

DPDispatcher has updated and the api of machine.json is changed. DP-GEN will use the new DPDispatcher if the value of key "api_version" in machine.json is equal to or large than 1.0. And for now, DPDispatcher is maintained on a separate repo (https://github.com/deepmodeling/dpdispatcher). Please check the documents (https://deepmd.readthedocs.io/projects/dpdispatcher/en/latest/) for more information about the new DPDispatcher.
DPDispatcher has updated and the api of machine.json is changed. DP-GEN will use the new DPDispatcher if the value of key {dargs:argument}`api_version <run_mdata/api_version>` in machine.json is equal to or large than 1.0. And for now, DPDispatcher is maintained on a separate repo (https://github.com/deepmodeling/dpdispatcher). Please check the documents (https://deepmd.readthedocs.io/projects/dpdispatcher/en/latest/) for more information about the new DPDispatcher.

DP-GEN will use the old DPDispatcher if the key "api_version" is not specified in machine.json or the "api_version" is smaller than 1.0. This gurantees that the old machine.json still works.
DP-GEN will use the old DPDispatcher if the key {dargs:argument}`api_version <run_mdata/api_version>` is not specified in machine.json or the {dargs:argument}`api_version <run_mdata/api_version>` is smaller than 1.0. This gurantees that the old machine.json still works.

## New DPDispatcher

Expand Down Expand Up @@ -39,11 +39,11 @@ In this example, we perform the `train` task on a local workstation.
},
```

The "command" for the train task in the DeePMD-kit is "dp".
The {dargs:argument}`command <run_mdata/train/command>` for the train task in the DeePMD-kit is "dp".

In machine parameters, "batch_type" specifies the type of job scheduling system. If there is no job scheduling system, we can use the "Shell" to perform the task. "context_type" specifies the method of data transfer, and "local" means copying and moving data via local file storage systems (e.g. cp, mv, etc.). In DP-GEN, the paths of all tasks are automatically located and set by the software, and therefore "local_root" is always set to "./". The input file for each task will be sent to the "remote_root" and the task will be performed there, so we need to make sure that the path exists.
In machine parameters, {dargs:argument}`batch_type <run_mdata/train/machine/batch_type>` specifies the type of job scheduling system. If there is no job scheduling system, we can use the "Shell" to perform the task. {dargs:argument}`context_type <run_mdata/train/machine/context_type>` specifies the method of data transfer, and "local" means copying and moving data via local file storage systems (e.g. cp, mv, etc.). In DP-GEN, the paths of all tasks are automatically located and set by the software, and therefore {dargs:argument}`local_root <run_mdata/train/machine/local_root>` is always set to "./". The input file for each task will be sent to the {dargs:argument}`remote_root <run_mdata/train/machine/remote_root>` and the task will be performed there, so we need to make sure that the path exists.

In the resources parameter, "number_node", "cpu_per_node", and "gpu_per_node" specify the number of nodes, the number of CPUs, and the number of GPUs required for a task respectively. "group_size", which needs to be highlighted, specifies how many tasks will be packed into a group. In the training tasks, we need to train 4 models. If we only have one GPU, we can set the "group_size" to 4. If "group_size" is set to 1, 4 models will be trained on one GPU at the same time, as there is no job scheduling system. Finally, the environment variables can be activated by "source_list". In this example, "source /home/user1234/deepmd.env" is executed before "dp" to load the environment variables necessary to perform the training task.
In the resources parameter, {dargs:argument}`number_node <run_mdata/train/resources/number_node>`, {dargs:argument}`cpu_per_node <run_mdata/train/resources/cpu_per_node>`, and {dargs:argument}`gpu_per_node <run_mdata/train/resources/gpu_per_node>` specify the number of nodes, the number of CPUs, and the number of GPUs required for a task respectively. {dargs:argument}`group_size <run_mdata/train/resources/group_size>`, which needs to be highlighted, specifies how many tasks will be packed into a group. In the training tasks, we need to train 4 models. If we only have one GPU, we can set the {dargs:argument}`group_size <run_mdata/train/resources/group_size>` to 4. If {dargs:argument}`group_size <run_mdata/train/resources/group_size>` is set to 1, 4 models will be trained on one GPU at the same time, as there is no job scheduling system. Finally, the environment variables can be activated by {dargs:argument}`source_list <run_mdata/train/resources/source_list>`. In this example, "source /home/user1234/deepmd.env" is executed before "dp" to load the environment variables necessary to perform the training task.

### Perform model_devi task at a local Slurm cluster

Expand Down Expand Up @@ -71,11 +71,11 @@ In this example, we perform the model_devi task at a local Slurm workstation.
}
```

The "command" for the model_devi task in the LAMMPS is "lmp".
The {dargs:argument}`command <run_mdata/model_devi/command>` for the model_devi task in the LAMMPS is "lmp".

In the machine parameter, we specify the type of job scheduling system by changing the "batch_type" to "Slurm".
In the machine parameter, we specify the type of job scheduling system by changing the {dargs:argument}`batch_type <run_mdata/model_devi/machine/batch_type>` to "Slurm".

In the resources parameter, we specify the name of the queue to which the task is submitted by adding "queue_name". We can add additional lines to the calculation script via the "custom_flags". In the model_devi steps, there are frequently many short tasks, so we usually pack multiple tasks (e.g. 10) into a group for submission. Other parameters are similar to that of the local workstation.
In the resources parameter, we specify the name of the queue to which the task is submitted by adding {dargs:argument}`queue_name <run_mdata/model_devi/resources/queue_name>`. We can add additional lines to the calculation script via the {dargs:argument}`custom_flags <run_mdata/train/resources/custom_flags>`. In the model_devi steps, there are frequently many short tasks, so we usually pack multiple tasks (e.g. 10) into a group for submission. Other parameters are similar to that of the local workstation.

### Perform fp task in a remote PBS cluster

Expand Down Expand Up @@ -108,8 +108,8 @@ In this example, we perform the fp task at a remote PBS cluster that can be acce

VASP code is used for fp task and mpi is used for parallel computing, so "mpirun -n 32" is added to specify the number of parallel threads.

In the machine parameter, "context_type" is modified to "SSHContext" and "batch_type" is modified to "PBS". It is worth noting that "remote_root" should be set to an accessible path on the remote PBS cluster. "remote_profile" is added to specify the information used to connect the remote cluster, including hostname, username, port, etc.
In the machine parameter, {dargs:argument}`context_type <run_mdata/fp/machine/context_type>` is modified to "SSHContext" and {dargs:argument}`batch_type <run_mdata/fp/resources/batch_type>` is modified to "PBS". It is worth noting that {dargs:argument}`remote_root <run_mdata/fp/machine/remote_root>` should be set to an accessible path on the remote PBS cluster. {dargs:argument}`remote_profile <run_mdata/fp/machine[SSHContext]/remote_profile>` is added to specify the information used to connect the remote cluster, including hostname, username, port, etc.

In the resources parameter, we set "gpu_per_node" to 0 since it is cost-effective to use the CPU for VASP calculations.
In the resources parameter, we set {dargs:argument}`gpu_per_node <run_mdata/fp/resources/gpu_per_node>` to 0 since it is cost-effective to use the CPU for VASP calculations.

Explicit descriptions of keys in machine.json will be given in the following section.
14 changes: 7 additions & 7 deletions doc/run/example-of-param.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The basics related keys in param.json are given as follows
],
```

The basics related keys specify the basic information about the system. "type_map" gives the atom types, i.e. "H" and "C". "mass_map" gives the standard atom weights, i.e. "1" and "12".
The basics related keys specify the basic information about the system. {dargs:argument}`type_map <run_jdata/type_map>` gives the atom types, i.e. "H" and "C". {dargs:argument}`mass_map <run_jdata/mass_map>` gives the standard atom weights, i.e. "1" and "12".

## data

Expand All @@ -40,7 +40,7 @@ The data related keys in param.json are given as follows
],
```

The data related keys specify the init data for training initial DP models and structures used for model_devi calculations. "init_data_prefix" and "init_data_sys" specify the location of the init data. "sys_configs_prefix" and "sys_configs" specify the location of the structures.
The data related keys specify the init data for training initial DP models and structures used for model_devi calculations. {dargs:argument}`init_data_prefix <run_jdata/init_data_prefix>` and {dargs:argument}`init_data_sys <run_jdata/init_data_sys>` specify the location of the init data. {dargs:argument}`sys_configs_prefix <run_jdata/sys_configs_prefix>` and {dargs:argument}`sys_configs <run_jdata/sys_configs>` specify the location of the structures.

Here, the init data is provided at "...... /init/CH4.POSCAR.01x01x01/02.md/sys-0004-0001/deepmd". These structures are divided into two groups and provided at "....../init/CH4.POSCAR.01x01x01/01.scale_pert/sys-0004-0001/scale*/00000*/POSCAR" and "....../init/CH4.POSCAR.01x01x01/01.scale_pert/sys-0004-0001/scale*/00001*/POSCAR".

Expand All @@ -53,7 +53,7 @@ The training related keys in param.json are given as follows
"default_training_param": {
},
```
The training related keys specify the details of training tasks. "numb_models" specifies the number of models to be trained. "default_training_param" specifies the training parameters for `deepmd-kit`.
The training related keys specify the details of training tasks. {dargs:argument}`numb_models <run_jdata/numb_models>` specifies the number of models to be trained. "default_training_param" specifies the training parameters for `deepmd-kit`.

Here, 4 DP models will be trained in `00.train`. A detailed explanation of training parameters can be found in DeePMD-kit’s documentation (https://docs.deepmodeling.com/projects/deepmd/en/master/).

Expand Down Expand Up @@ -100,9 +100,9 @@ The exploration related keys in param.json are given as follows
}
],
```
The exploration related keys specify the details of exploration tasks. "model_devi_dt" specifies timestep for MD simulation. "model_devi_skip" specifies the number of structures skipped for saving in each MD. "model_devi_f_trust_lo" and "model_devi_f_trust_hi" specify the lower and upper bound of model_devi of forces for the selection. "model_devi_clean_traj" specifies whether to clean traj folders in MD. If type of model_devi_clean_traj is boolean type then it denote whether to clean traj folders in MD since they are too large.In "model_devi_jobs", "sys_idx" specifies the group of structures used for model_devi calculations, "temps" specifies the temperature (K) in MD, "press" specifies the pressure (Bar) in MD, "trj_freq" specifies the frequency of trajectory saved in MD, "nsteps" specifies the running steps of MD, "ensemble" specifies the ensemble used in MD, and "_idx" specifies the index of iteration.
The exploration related keys specify the details of exploration tasks. {dargs:argument}`model_devi_dt <run_jdata[model_devi_engine=lammps]/model_devi_dt>` specifies timestep for MD simulation. {dargs:argument}`model_devi_skip <run_jdata[model_devi_engine=lammps]/model_devi_skip>` specifies the number of structures skipped for saving in each MD. {dargs:argument}`model_devi_f_trust_lo <run_jdata[model_devi_engine=lammps]/model_devi_f_trust_lo>` and {dargs:argument}`model_devi_f_trust_hi <run_jdata[model_devi_engine=lammps]/model_devi_f_trust_hi>` specify the lower and upper bound of model_devi of forces for the selection. {dargs:argument}`model_devi_clean_traj <run_jdata[model_devi_engine=lammps]/model_devi_clean_traj>` specifies whether to clean traj folders in MD. If type of model_devi_clean_traj is boolean type then it denote whether to clean traj folders in MD since they are too large. In {dargs:argument}`model_devi_jobs <run_jdata[model_devi_engine=lammps]/model_devi_jobs>`, {dargs:argument}`sys_idx <run_jdata[model_devi_engine=lammps]/model_devi_jobs/sys_idx>` specifies the group of structures used for model_devi calculations, {dargs:argument}`temps <run_jdata[model_devi_engine=lammps]/model_devi_jobs/temps>` specifies the temperature (K) in MD, {dargs:argument}`press <run_jdata[model_devi_engine=lammps]/model_devi_jobs/press>` specifies the pressure (Bar) in MD, {dargs:argument}`trj_freq <run_jdata[model_devi_engine=lammps]/model_devi_jobs/trj_freq>` specifies the frequency of trajectory saved in MD, {dargs:argument}`nsteps <run_jdata[model_devi_engine=lammps]/model_devi_jobs/trj_freq>` specifies the running steps of MD, {dargs:argument}`ensemble <run_jdata[model_devi_engine=lammps]/model_devi_jobs/ensemble>` specifies the ensemble used in MD, and "_idx" specifies the index of iteration.

Here, MD simulations are performed at the temperature of 100 K and the pressure of 1.0 Bar with an integrator time of 2 fs under the nvt ensemble. Two iterations are set in "model_devi_jobs". MD simulations are run for 300 and 3000 time steps with the first and second groups of structures in "sys_configs" in 00 and 01 iterations. We choose to save all structures generated in MD simulations and have set `"trj_freq"` as 10, so 30 and 300 structures are saved in 00 and 01 iterations. If the "max_devi_f" of saved structure falls between 0.05 and 0.15, DP-GEN will treat the structure as a candidate. We choose to clean traj folders in MD since they are too large. If you want to save the most recent n iterations of traj folders, you can set "model_devi_clean_traj" to be an integer.
Here, MD simulations are performed at the temperature of 100 K and the pressure of 1.0 Bar with an integrator time of 2 fs under the nvt ensemble. Two iterations are set in {dargs:argument}`model_devi_jobs <run_jdata[model_devi_engine=lammps]/model_devi_jobs>`. MD simulations are run for 300 and 3000 time steps with the first and second groups of structures in {dargs:argument}`sys_configs <run_jdata/sys_configs>` in 00 and 01 iterations. We choose to save all structures generated in MD simulations and have set {dargs:argument}`trj_freq <run_jdata[model_devi_engine=lammps]/model_devi_jobs/trj_freq>` as 10, so 30 and 300 structures are saved in 00 and 01 iterations. If the "max_devi_f" of saved structure falls between 0.05 and 0.15, DP-GEN will treat the structure as a candidate. We choose to clean traj folders in MD since they are too large. If you want to save the most recent n iterations of traj folders, you can set {dargs:argument}`model_devi_clean_traj <run_jdata[model_devi_engine=lammps]/model_devi_clean_traj>` to be an integer.

## labeling

Expand All @@ -120,8 +120,8 @@ The labeling related keys in param.json are given as follows
"fp_incar": "....../INCAR_methane"
```

The labeling related keys specify the details of labeling tasks. "fp_style" specifies software for First Principles. "fp_task_max" and "fp_task_min" specify the minimum and maximum of structures to be calculated in `02.fp` of each iteration. "fp_pp_path" and "fp_pp_files" specify the location of the psuedo-potential file to be used for 02.fp. "fp_incar" specifies input file for VASP. INCAR must specify KSPACING and KGAMMA.
The labeling related keys specify the details of labeling tasks. {dargs:argument}`fp_style <run_jdata/fp_style>` specifies software for First Principles. {dargs:argument}`fp_task_max <run_jdata/fp_task_max>` and {dargs:argument}`fp_task_min <run_jdata/fp_task_min>` specify the minimum and maximum of structures to be calculated in `02.fp` of each iteration. {dargs:argument}`fp_pp_path <run_jdata[fp_style=vasp]/fp_pp_path>` and {dargs:argument}`fp_pp_files <run_jdata[fp_style=vasp]/fp_pp_files>` specify the location of the psuedo-potential file to be used for 02.fp. {dargs:argument}`run_jdata[fp_style=vasp]/fp_incar` specifies input file for VASP. INCAR must specify KSPACING and KGAMMA.

Here, a minimum of 1 and a maximum of 20 structures will be labeled using the VASP code with the INCAR provided at "....../INCAR_methane" and POTCAR provided at "....../methane/POTCAR" in each iteration. Note that the order of elements in POTCAR should correspond to the order in `type_map`.
Here, a minimum of 1 and a maximum of 20 structures will be labeled using the VASP code with the INCAR provided at "....../INCAR_methane" and POTCAR provided at "....../methane/POTCAR" in each iteration. Note that the order of elements in POTCAR should correspond to the order in {dargs:argument}`type_map <run_jdata/type_map>`.

All the keys of the DP-GEN are explained in detail in the section Parameters.
8 changes: 4 additions & 4 deletions doc/run/overview-of-the-run-process.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,14 +36,14 @@ In folder iter.000000/ 00.train:

In folder iter.000000/ 01.model_devi:

- Folder confs contains the initial configurations for LAMMPS MD converted from POSCAR you set in "sys_configs" of param.json.
- Folder confs contains the initial configurations for LAMMPS MD converted from POSCAR you set in {dargs:argument}`sys_configs <run_jdata/sys_configs>` of param.json.
- Folder task.000.00000x contains the input and output files of the LAMMPS. In folder task.000.00000x, file model_devi.out records the model deviation of concerned labels, energy and force in MD. It serves as the criterion for selecting which structures and doing first-principle calculations.

In folder iter.000000/ 02.fp:

- candidate.shuffle.000.out records which structures will be selected from last step 01.model_devi. There are always far more candidates than the maximum you expect to calculate at one time. In this condition, DP-GEN will randomly choose up to `"fp_task_max"` structures and form the folder task.*.
- rest_accurate.shuffle.000.out records the other structures where our model is accurate ("max_devi_f" is less than `"model_devi_f_trust_lo"`, no need to calculate any more),
- rest_failed.shuffled.000.out records the other structures where our model is too inaccurate (larger than `"model_devi_f_trust_hi"`, there may be some error).
- candidate.shuffle.000.out records which structures will be selected from last step 01.model_devi. There are always far more candidates than the maximum you expect to calculate at one time. In this condition, DP-GEN will randomly choose up to {dargs:argument}`fp_task_max <run_jdata/fp_task_max>` structures and form the folder task.*.
- rest_accurate.shuffle.000.out records the other structures where our model is accurate (`max_devi_f` is less than {dargs:argument}`model_devi_f_trust_lo <run_jdata[model_devi_engine=lammps]/model_devi_jobs/model_devi_f_trust_lo>`, no need to calculate any more),
- rest_failed.shuffled.000.out records the other structures where our model is too inaccurate (larger than {dargs:argument}`model_devi_f_trust_hi <run_jdata[model_devi_engine=lammps]/model_devi_jobs/model_devi_f_trust_hi>`, there may be some error).
- data.000: After first-principle calculations, DP-GEN will collect these data and change them into the format DeePMD-kit needs. In the next iteration's 00.train, these data will be trained together as well as the initial data.

DP-GEN identifies the stage of the run process by a record file, record.dpgen, which will be created and upgraded by codes. Each line contains two numbers: the first is the index of iteration, and the second, ranging from 0 to 9, records which stage in each iteration is currently running.
Expand Down