Skip to content

Commit 90dc2db

Browse files
douglasjacobsendouglasjacobsen
authored
Merge pull request #510 from GoogleCloudPlatform/release-v0.5.0
Merge release v0.5.0 into main
2 parents 2146246 + 7de4373 commit 90dc2db

File tree

358 files changed

+6393
-2906
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

358 files changed

+6393
-2906
lines changed

‎.github/workflows/spelling.yaml

+2-2
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,9 @@ jobs:
1111
formatting:
1212
runs-on: ubuntu-latest
1313
steps:
14-
- uses: actions/checkout@v3
14+
- uses: actions/checkout@v4
1515
- name: Check Spelling
16-
uses: crate-ci/typos@548ac37a5de9ce84871bf4db3c9b8c462896d480 # v1.16.24
16+
uses: crate-ci/typos@v1.20.4
1717
with:
1818
files: ./lib/ramble/ramble ./lib/ramble/docs ./examples ./share ./bin ./etc ./var ./README.md
1919
config: ./.typos.toml

‎.typos.toml

+2
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ extend-ignore-re = [
1717
"fom" = "fom"
1818
"namd" = "namd"
1919
"reord" = "reord"
20+
"PN" = "PN" # fixing tPN in IOR
21+
"repositor" = "repositor" # Fixing partial name without singular / plural suffix
2022

2123
[default.extend-identifiers]
2224
"ATPase" = "ATPase"

‎AUTHORS

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
# This is the list of Ramble's significant contributors.
2+
#
3+
# This does not necessarily list everyone who has contributed code,
4+
# especially since many employees of one corporation may be contributing.
5+
# To see the full list of contributors, see the revision history in
6+
# source control.
7+
Google LLC

‎LICENSE-APACHE

+1-1
Original file line numberDiff line numberDiff line change
@@ -187,7 +187,7 @@
187187
same "printed page" as the copyright notice for easier
188188
identification within third-party archives.
189189

190-
Copyright 2022-2024 Google LLC.
190+
Copyright 2022-2024 The Ramble Authors.
191191

192192
Licensed under the Apache License, Version 2.0 (the "License");
193193
you may not use this file except in compliance with the License.

‎README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ It works on Linux, macOS, and many supercomputers.
66
Ramble can be used to configure a variety of experiments for applications.
77
These can include anything from:
88
- Scientific parameter sweeps
9-
- Performance focused scalaing studies
9+
- Performance focused scaling studies
1010
- Compiler flag sweeps
1111

1212
To install ramble and configure your experiment workspace, make sure you have

‎bin/ramble

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
#!/bin/sh
2-
# Copyright 2022-2024 Google LLC
2+
# Copyright 2022-2024 The Ramble Authors
33
#
44
# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
55
# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license

‎bin/ramble-python

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright 2022-2024 Google LLC
1+
# Copyright 2022-2024 The Ramble Authors
22
#
33
# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
44
# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license

‎etc/ramble/defaults/spack.yaml

-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
11
spack:
2-
concretized: false
32
packages: {}
43
environments: {}

‎examples/basic_expansion_config.yaml

+3-4
Original file line numberDiff line numberDiff line change
@@ -29,16 +29,15 @@ ramble:
2929
n_ranks: '1'
3030
n_nodes: '1'
3131
spack:
32-
concretized: true
3332
packages:
3433
gcc9:
3534
spack_spec: [email protected] target=x86_64
3635
compiler_spec: [email protected]
3736
ompi412:
3837
spack_spec: [email protected] +legacylaunchers +pmi +thread_multiple +cxx target=x86_64
3938
compiler: gcc9
40-
impi2018:
41-
spack_spec: intel-mpi@2018.4.274
39+
impi2021:
40+
spack_spec: intel-oneapi-mpi@2021.11.0
4241
compiler: gcc9
4342
openfoam:
4443
spack_spec: openfoam-org@7
@@ -57,5 +56,5 @@ ramble:
5756
- openfoam
5857
wrfv4:
5958
packages:
60-
- impi2018
59+
- impi2021
6160
- wrfv4

‎examples/basic_gromacs_config.yaml

+3-4
Original file line numberDiff line numberDiff line change
@@ -38,13 +38,12 @@ ramble:
3838
size: '0003'
3939
type: 'rf'
4040
spack:
41-
concretized: true
4241
packages:
4342
gcc9:
4443
spack_spec: [email protected] target=x86_64
4544
compiler_spec: [email protected]
46-
impi2018:
47-
spack_spec: intel-mpi@2018.4.274 target=x86_64
45+
impi2021:
46+
spack_spec: intel-oneapi-mpi@2021.11.0 target=x86_64
4847
compiler: gcc9
4948
gromacs:
5049
spack_spec: [email protected]
@@ -53,4 +52,4 @@ ramble:
5352
gromacs:
5453
packages:
5554
- gromacs
56-
- impi2018
55+
- impi2021

‎examples/basic_hostname_config.yaml

-1
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,5 @@ ramble:
4444
n_nodes: '1'
4545
processes_per_node: '16'
4646
spack:
47-
concretized: true
4847
packages: {}
4948
environments: {}

‎examples/full_expansion_config.yaml

-1
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,6 @@ ramble:
2929
#(part1, 16, openfoam-skx, 2), (part1, 16, openfoam-skx, 4)
3030
#(part2, 32, openfoam-zen2, 2), (part2, 32, openfoam-zen2, 4)
3131
spack:
32-
concretized: true
3332
packages:
3433
gcc9:
3534
spack_spec: [email protected] target=x86_64

‎examples/slurm_execute_experiment.tpl

+3-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
#!/bin/bash
22
#SBATCH -N {n_nodes}
33
#SBATCH --ntasks-per-node {processes_per_node}
4+
#SBATCH -p {partition_name}
45
#SBATCH -J {application_name}_{workload_name}_{experiment_name}
56

67
# This is a template execution script for
@@ -20,5 +21,6 @@
2021

2122
cd {experiment_run_dir}
2223

23-
{command}
24+
scontrol show hostnames > {experiment_run_dir}/hostfile
2425

26+
{command}

‎examples/vector_gromacs_software_config.yaml

+3-4
Original file line numberDiff line numberDiff line change
@@ -26,13 +26,12 @@ ramble:
2626
- n_ranks
2727
- gromacs_version
2828
spack:
29-
concretized: true
3029
packages:
3130
gcc9:
3231
spack_spec: [email protected] target=x86_64
3332
compiler_spec: [email protected]
34-
impi2018:
35-
spack_spec: intel-mpi@2018.4.274 target=x86_64
33+
impi2021:
34+
spack_spec: intel-oneapi-mpi@2021.11.0 target=x86_64
3635
compiler: gcc9
3736
gromacs-{gromacs_version}:
3837
spack_spec: gromacs@{gromacs_version}
@@ -41,4 +40,4 @@ ramble:
4140
gromacs-{gromacs_version}:
4241
packages:
4342
- gromacs-{gromacs_version}
44-
- impi2018
43+
- impi2021

‎examples/vector_matrix_gromacs_config.yaml

+3-4
Original file line numberDiff line numberDiff line change
@@ -23,13 +23,12 @@ ramble:
2323
- type
2424
- n_ranks
2525
spack:
26-
concretized: true
2726
packages:
2827
gcc9:
2928
spack_spec: [email protected] target=x86_64
3029
compiler_spec: [email protected]
31-
impi2018:
32-
spack_spec: intel-mpi@2018.4.274 target=x86_64
30+
impi2021:
31+
spack_spec: intel-oneapi-mpi@2021.11.0 target=x86_64
3332
compiler: gcc9
3433
gromacs:
3534
spack_spec: [email protected]
@@ -38,4 +37,4 @@ ramble:
3837
gromacs:
3938
packages:
4039
- gromacs
41-
- impi2018
40+
- impi2021

‎lib/ramble/docs/conf.py

+8-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright 2022-2024 Google LLC
1+
# Copyright 2022-2024 The Ramble Authors
22
#
33
# Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
44
# https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
@@ -130,6 +130,7 @@ def setup(sphinx):
130130
"sphinx.ext.viewcode",
131131
"sphinxcontrib.programoutput",
132132
"sphinxcontrib.jquery",
133+
"sphinx_copybutton",
133134
]
134135

135136
# Set default graphviz options
@@ -396,3 +397,9 @@ class RambleStyle(DefaultStyle):
396397
intersphinx_mapping = {
397398
"python": ("https://docs.python.org/3", None),
398399
}
400+
401+
# sphinx_copybutton
402+
# Do not copy the prompt, or any console outputs.
403+
copybutton_exclude = '.gp, .go'
404+
# Escape hatch for turning off the copy button.
405+
copybutton_selector = "div:not(.hide-copy) > div.highlight > pre"

‎lib/ramble/docs/configuration_files.rst

+40-85
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.. Copyright 2022-2024 Google LLC
1+
.. Copyright 2022-2024 The Ramble Authors
22
33
Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
44
https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
@@ -54,31 +54,31 @@ Ramble provides several configuration scopes, which are used to denote
5454
precedence of configuration options. In precedence order (from lowest to
5555
highest) Ramble contains the following scopes:
5656

57-
(1) **default**: Stored in ``$(prefix)/etc/ramble/defaults/``. These are the
58-
default settings provided with Ramble. Users should generally not modify these
59-
settings, and instead use a higher precedence configuration scope. These
60-
defaults will change from version to version of Ramble.
61-
(2) **system**: Store in ``/etc/ramble/``. These are Ramble settings for an
62-
entire machine. These settings are typically managed by a systems
63-
administrator, or something with root access on the machine. Settings defined
64-
in this scope override settings in the **default** scope.
65-
(3) **site**: Stored in ``$(prefix)/etc/ramble/``. Settings here only affect
66-
*this instance* of Ramble, and they override both the **default** and
67-
**system** scopes.
68-
(4) **user**: Stored in ``~/.ramble/``. Settings here only affect a specific
69-
user, and override **default**, **system**, and **site** scopes.
70-
(5) **custom**: Stored in a custom directory, specified by ``--config-scope``.
71-
If multiple scopes are listed on the command line, they are ordered from lowest
72-
to highest precedence. Settings here override all previously defined scoped.
73-
(6) **workspace configs dir**: Stored in ``$(workspace_root)/configs``
74-
generally as a ``<config_section>.yaml`` file (i.e. ``variables.yaml``). These
75-
settings apply to a specific workspace, and override all previous configuration
76-
scopes.
77-
(7) **workspace configuration file**: Stored in
78-
``$(workspace_root)/configs/ramble.yaml``. Configuration scopes defined within
79-
this config file override all previously defined configuration scopes.
80-
(8) **command line**: Configuration options defined on the command line take
81-
precedence over all other scopes.
57+
1. **default**: Stored in ``$(prefix)/etc/ramble/defaults/``. These are the
58+
default settings provided with Ramble. Users should generally not modify these
59+
settings, and instead use a higher precedence configuration scope. These
60+
defaults will change from version to version of Ramble.
61+
2. **system**: Store in ``/etc/ramble/``. These are Ramble settings for an
62+
entire machine. These settings are typically managed by a systems
63+
administrator, or something with root access on the machine. Settings defined
64+
in this scope override settings in the **default** scope.
65+
3. **site**: Stored in ``$(prefix)/etc/ramble/``. Settings here only affect
66+
*this instance* of Ramble, and they override both the **default** and
67+
**system** scopes.
68+
4. **user**: Stored in ``~/.ramble/``. Settings here only affect a specific
69+
user, and override **default**, **system**, and **site** scopes.
70+
5. **custom**: Stored in a custom directory, specified by ``--config-scope``.
71+
If multiple scopes are listed on the command line, they are ordered from lowest
72+
to highest precedence. Settings here override all previously defined scoped.
73+
6. **workspace configs dir**: Stored in ``$(workspace_root)/configs``
74+
generally as a ``<config_section>.yaml`` file (i.e. ``variables.yaml``). These
75+
settings apply to a specific workspace, and override all previous configuration
76+
scopes.
77+
7. **workspace configuration file**: Stored in
78+
``$(workspace_root)/configs/ramble.yaml``. Configuration scopes defined within
79+
this config file override all previously defined configuration scopes.
80+
8. **command line**: Configuration options defined on the command line take
81+
precedence over all other scopes.
8282

8383
Each configuration directory may contain several configuration files, such as
8484
``config.yaml``, ``variables.yaml``, or ``modifiers.yaml``. When configurations
@@ -262,9 +262,14 @@ The format of this config section is as follows:
262262
The above example is general, and intended to show the available functionality
263263
of configuring environment variables. Below the ``env_vars`` level, one of four
264264
actions is available. These actions are:
265-
* ``set`` - Define a variable equal to a given value. Overwrites previously configured values
266-
* ``append`` - Append the given value to the end of a previous variable definition. Delimited for vars is defined by ``var_separator``, ``paths`` uses ``:``
267-
* ``prepend`` - Prepent the given value to the beginning of a previous variable definition. Only supports paths, delimiter is ``:``
265+
266+
* ``set`` - Define a variable equal to a given value. Overwrites previously
267+
configured values
268+
* ``append`` - Append the given value to the end of a previous variable
269+
definition. Delimited for vars is defined by ``var_separator``, ``paths``
270+
uses ``:``
271+
* ``prepend`` - Prepent the given value to the beginning of a previous variable
272+
definition. Only supports paths, delimiter is ``:``
268273
* ``unset`` - Remove a variable definition, if it is set.
269274

270275
.. _formatted-execs-config:
@@ -471,7 +476,6 @@ environments created from those packages. Its format is as follows:
471476
.. code-block:: yaml
472477
473478
spack:
474-
concretized: [True/False] # Should be false unless defined in a concretized workspace
475479
[variables: {}]
476480
packages:
477481
<package_name>:
@@ -517,70 +521,21 @@ Below is an annotated example of the spack dictionary.
517521
gcc9: # Abstract name to refer to this package
518522
spack_spec: [email protected] target=x86_64 # Spack spec for this package
519523
compiler_spec: [email protected] # Spack compiler spec for this package
520-
impi2018:
521-
spack_spec: intel-mpi@2018.4.274 target=x86_64
524+
impi2021:
525+
spack_spec: intel-oneapi-mpi@2021.11.0 target=x86_64
522526
compiler: gcc9 # Other package name to use as compiler for this package
523527
gromacs:
524528
spack_spec: [email protected]
525529
compiler: gcc9
526530
environments:
527531
gromacs:
528532
packages: # List of packages to include in this environment
529-
- impi2018
533+
- impi2021
530534
- gromacs
531535
532-
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
533-
Vector and Matrix Packages and Environments:
534-
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
535-
536-
Package and environment definitions can generate many packages and environments
537-
following Ramble's
538-
:ref:`vector<ramble-vector-logic>` / :ref:`matrix<ramble-matrix-logic>` logic.
539-
540-
Below is an example of using this logic within the spack dictionary:
541-
542-
.. code-block:: yaml
543-
544-
spack:
545-
packages:
546-
gcc-{ver}:
547-
variables:
548-
ver: ['9.3.0', '10.3.0', '12.2.0']
549-
spack_spec: gcc@{ver} target=x86_64
550-
compiler_spec: gcc@{ver}
551-
intel-mpi-{comp}:
552-
variables:
553-
comp: gcc-{ver}
554-
ver: ['9.3.0', '10.3.0', '12.2.0']
555-
spack_spec: [email protected]
556-
compiler: {comp}
557-
openmpi-{comp}:
558-
variables:
559-
comp: gcc-{ver}
560-
ver: ['9.3.0', '10.3.0', '12.2.0']
561-
spack_spec: [email protected]
562-
compiler: {comp}
563-
wrf-{comp}:
564-
variables:
565-
comp: gcc-{ver}
566-
ver: ['9.3.0', '10.3.0', '12.2.0']
567-
spack_spec: [email protected]
568-
compiler: {comp}
569-
environments:
570-
wrf-{comp}-{mpi}:
571-
variables:
572-
comp: gcc-{ver}
573-
ver: ['9.3.0', '10.3.0', '12.2.0']
574-
mpi: [intel-mpi-{comp}, openmpi-{comp}']
575-
matrix:
576-
- mpi
577-
packages:
578-
- {mpi}
579-
- wrf-{comp}
580-
581-
The above file will generate 3 versions of ``gcc``, 3 versions each of ``wrf``,
582-
``intel-mpi`` and ``openmpi`` built with each ``gcc`` version, and 6 spack
583-
environments, with each combination of the 2 ``mpi`` libraries and 3 compilers.
536+
Packages and environments defined inside the ``spack`` config section are
537+
merely templates. They will be rendered into explicit environments and packages
538+
by each individual experiment.
584539

585540
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
586541
External Spack Environment Support:

‎lib/ramble/docs/dev_guides.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.. Copyright 2022-2024 Google LLC
1+
.. Copyright 2022-2024 The Ramble Authors
22
33
Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
44
https://www.apache.org/licenses/LICENSE-2.0> or the MIT license

‎lib/ramble/docs/dev_guides/application_dev_guide.rst

+2-2
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
.. Copyright 2022-2024 Google LLC
1+
.. Copyright 2022-2024 The Ramble Authors
22
33
Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
44
https://www.apache.org/licenses/LICENSE-2.0> or the MIT license
@@ -265,7 +265,7 @@ directives that are intended to be package manager specific. As an example,
265265
there are directives for Spack defined by:
266266

267267
* :meth:`ramble.language.shared_language.software_spec`
268-
* :meth:`ramble.language.shared_language.default_compiler`
268+
* :meth:`ramble.language.shared_language.define_compiler`
269269
* :meth:`ramble.language.shared_language.required_package`
270270

271271
These provide Ramble with information about how Spack could install and require

0 commit comments

Comments
 (0)