Skip to content
This repository was archived by the owner on Dec 16, 2022. It is now read-only.

Commit e641543

Browse files
bryant1410DeNeutoy
bryant1410
authored and
DeNeutoy
committed
Add a test for the subcommands docstring help outputs (#3172)
* Add a test for the subcommands docstring help outputs * Fix subcommands help outputs in their module docstrings * Fix a PR comment
1 parent b6af6eb commit e641543

12 files changed

+274
-173
lines changed

‎allennlp/commands/configure.py

+4-5
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,15 @@
55
.. code-block:: bash
66
77
$ allennlp configure --help
8-
98
usage: allennlp configure [-h] [--port PORT]
10-
[--include-package INCLUDE_PACKAGE]
9+
[--include-package INCLUDE_PACKAGE]
1110
1211
Run the configuration wizard
1312
1413
optional arguments:
15-
-h, --help show this help message and exit
16-
--port PORT port to serve the wizard on
17-
--include-package INCLUDE_PACKAGE
14+
-h, --help show this help message and exit
15+
--port PORT port to serve the wizard on (default = 8123)
16+
--include-package INCLUDE_PACKAGE
1817
additional packages to include
1918
"""
2019

‎allennlp/commands/dry_run.py

+7-7
Original file line numberDiff line numberDiff line change
@@ -7,23 +7,23 @@
77
88
$ allennlp dry-run --help
99
usage: allennlp dry-run [-h] -s SERIALIZATION_DIR [-o OVERRIDES]
10-
[--include-package INCLUDE_PACKAGE]
11-
param_path
10+
[--include-package INCLUDE_PACKAGE]
11+
param_path
1212
1313
Create a vocabulary, compute dataset statistics and other training utilities.
1414
1515
positional arguments:
16-
param_path path to parameter file describing the model and its
16+
param_path path to parameter file describing the model and its
1717
inputs
1818
1919
optional arguments:
20-
-h, --help show this help message and exit
21-
-s SERIALIZATION_DIR, --serialization-dir SERIALIZATION_DIR
20+
-h, --help show this help message and exit
21+
-s SERIALIZATION_DIR, --serialization-dir SERIALIZATION_DIR
2222
directory in which to save the output of the dry run.
23-
-o OVERRIDES, --overrides OVERRIDES
23+
-o OVERRIDES, --overrides OVERRIDES
2424
a JSON structure used to override the experiment
2525
configuration
26-
--include-package INCLUDE_PACKAGE
26+
--include-package INCLUDE_PACKAGE
2727
additional packages to include
2828
"""
2929
import argparse

‎allennlp/commands/elmo.py

+48-38
Original file line numberDiff line numberDiff line change
@@ -14,44 +14,54 @@
1414
.. code-block:: console
1515
1616
$ allennlp elmo --help
17-
usage: allennlp elmo [-h] (--all | --top | --average)
18-
[--vocab-path VOCAB_PATH] [--options-file OPTIONS_FILE]
19-
[--weight-file WEIGHT_FILE] [--batch-size BATCH_SIZE]
20-
[--cuda-device CUDA_DEVICE] [--forget-sentences]
21-
[--use-sentence-keys] [--include-package INCLUDE_PACKAGE]
22-
input_file output_file
23-
24-
Create word vectors using ELMo.
25-
26-
positional arguments:
27-
input_file The path to the input file.
28-
output_file The path to the output file.
29-
30-
optional arguments:
31-
-h, --help show this help message and exit
32-
--all Output all three ELMo vectors.
33-
--top Output the top ELMo vector.
34-
--average Output the average of the ELMo vectors.
35-
--vocab-path VOCAB_PATH
36-
A path to a vocabulary file to generate.
37-
--options-file OPTIONS_FILE
38-
The path to the ELMo options file.
39-
--weight-file WEIGHT_FILE
40-
The path to the ELMo weight file.
41-
--batch-size BATCH_SIZE
42-
The batch size to use.
43-
--cuda-device CUDA_DEVICE
44-
The cuda_device to run on.
45-
--forget-sentences If this flag is specified, and --use-sentence-keys is
46-
not, remove the string serialized JSON dictionary that
47-
associates sentences with their line number (its HDF5
48-
key) that is normally placed in the
49-
"sentence_to_index" HDF5 key.
50-
--use-sentence-keys Normally a sentence's line number is used as the HDF5
51-
key for its embedding. If this flag is specified, the
52-
sentence itself will be used as the key.
53-
--include-package INCLUDE_PACKAGE
54-
additional packages to include
17+
usage: allennlp elmo [-h] (--all | --top | --average)
18+
[--vocab-path VOCAB_PATH] [--options-file OPTIONS_FILE]
19+
[--weight-file WEIGHT_FILE] [--batch-size BATCH_SIZE]
20+
[--file-friendly-logging] [--cuda-device CUDA_DEVICE]
21+
[--forget-sentences] [--use-sentence-keys]
22+
[--include-package INCLUDE_PACKAGE]
23+
input_file output_file
24+
25+
Create word vectors using ELMo.
26+
27+
positional arguments:
28+
input_file The path to the input file.
29+
output_file The path to the output file.
30+
31+
optional arguments:
32+
-h, --help show this help message and exit
33+
--all Output all three ELMo vectors.
34+
--top Output the top ELMo vector.
35+
--average Output the average of the ELMo vectors.
36+
--vocab-path VOCAB_PATH
37+
A path to a vocabulary file to generate.
38+
--options-file OPTIONS_FILE
39+
The path to the ELMo options file. (default = https://
40+
allennlp.s3.amazonaws.com/models/elmo/2x4096_512_2048c
41+
nn_2xhighway/elmo_2x4096_512_2048cnn_2xhighway_options
42+
.json)
43+
--weight-file WEIGHT_FILE
44+
The path to the ELMo weight file. (default = https://a
45+
llennlp.s3.amazonaws.com/models/elmo/2x4096_512_2048cn
46+
n_2xhighway/elmo_2x4096_512_2048cnn_2xhighway_weights.
47+
hdf5)
48+
--batch-size BATCH_SIZE
49+
The batch size to use. (default = 64)
50+
--file-friendly-logging
51+
outputs tqdm status on separate lines and slows tqdm
52+
refresh rate.
53+
--cuda-device CUDA_DEVICE
54+
The cuda_device to run on. (default = -1)
55+
--forget-sentences If this flag is specified, and --use-sentence-keys is
56+
not, remove the string serialized JSON dictionary that
57+
associates sentences with their line number (its HDF5
58+
key) that is normally placed in the
59+
"sentence_to_index" HDF5 key.
60+
--use-sentence-keys Normally a sentence's line number is used as the HDF5
61+
key for its embedding. If this flag is specified, the
62+
sentence itself will be used as the key.
63+
--include-package INCLUDE_PACKAGE
64+
additional packages to include
5565
"""
5666

5767
import argparse

‎allennlp/commands/evaluate.py

+12-12
Original file line numberDiff line numberDiff line change
@@ -18,35 +18,35 @@
1818
Evaluate the specified model + dataset
1919
2020
positional arguments:
21-
archive_file path to an archived trained model
22-
input_file path to the file containing the evaluation data
21+
archive_file path to an archived trained model
22+
input_file path to the file containing the evaluation data
2323
2424
optional arguments:
25-
-h, --help show this help message and exit
26-
--output-file OUTPUT_FILE
27-
path to output file to save metrics
28-
--weights-file WEIGHTS_FILE
25+
-h, --help show this help message and exit
26+
--output-file OUTPUT_FILE
27+
path to output file
28+
--weights-file WEIGHTS_FILE
2929
a path that overrides which weights file to use
30-
--cuda-device CUDA_DEVICE
30+
--cuda-device CUDA_DEVICE
3131
id of GPU to use (if any)
32-
-o OVERRIDES, --overrides OVERRIDES
32+
-o OVERRIDES, --overrides OVERRIDES
3333
a JSON structure used to override the experiment
3434
configuration
35-
--batch-weight-key BATCH_WEIGHT_KEY
35+
--batch-weight-key BATCH_WEIGHT_KEY
3636
If non-empty, name of metric used to weight the loss
3737
on a per-batch basis.
38-
--extend-vocab if specified, we will use the instances in your new
38+
--extend-vocab if specified, we will use the instances in your new
3939
dataset to extend your vocabulary. If pretrained-file
4040
was used to initialize embedding layers, you may also
4141
need to pass --embedding-sources-mapping.
42-
--embedding-sources-mapping EMBEDDING_SOURCES_MAPPING
42+
--embedding-sources-mapping EMBEDDING_SOURCES_MAPPING
4343
a JSON dict defining mapping from embedding module
4444
path to embeddingpretrained-file used during training.
4545
If not passed, and embedding needs to be extended, we
4646
will try to use the original file paths used during
4747
training. If they are not available we will use random
4848
vectors for embedding extension.
49-
--include-package INCLUDE_PACKAGE
49+
--include-package INCLUDE_PACKAGE
5050
additional packages to include
5151
"""
5252
from typing import Dict, Any

‎allennlp/commands/find_learning_rate.py

+35-35
Original file line numberDiff line numberDiff line change
@@ -6,41 +6,41 @@
66
.. code-block:: bash
77
88
$ allennlp find-lr --help
9-
usage: allennlp find-lr [-h] -s SERIALIZATION_DIR [-o OVERRIDES]
10-
[--start-lr START_LR] [--end-lr END_LR]
11-
[--num-batches NUM_BATCHES] [--linear]
12-
[--stopping-factor STOPPING_FACTOR] [--linear]
13-
[--include-package INCLUDE_PACKAGE]
14-
param_path
15-
16-
Find a learning rate range where the loss decreases quickly for the specified
17-
model and dataset.
18-
19-
positional arguments:
20-
param_path path to parameter file describing the model to be
21-
trained
22-
23-
optional arguments:
24-
-h, --help show this help message and exit
25-
-s SERIALIZATION_DIR, --serialization-dir SERIALIZATION_DIR
26-
directory in which to save Learning rate vs loss
27-
-f, --force overwrite the output directory if it exists
28-
-o OVERRIDES, --overrides OVERRIDES
29-
a JSON structure used to override the experiment
30-
configuration.
31-
--start-lr START_LR
32-
learning rate to start the search.
33-
--end-lr END_LR
34-
learning rate up to which search is done.
35-
--num-batches NUM_BATCHES
36-
number of mini-batches to run Learning rate finder.
37-
--stopping-factor STOPPING_FACTOR
38-
stop the search when the current loss exceeds the best
39-
loss recorded by multiple of stopping factor
40-
--linear increase learning rate linearly instead of exponential
41-
increase
42-
--include-package INCLUDE_PACKAGE
43-
additional packages to include
9+
usage: allennlp find-lr [-h] -s SERIALIZATION_DIR [-o OVERRIDES]
10+
[--start-lr START_LR] [--end-lr END_LR]
11+
[--num-batches NUM_BATCHES]
12+
[--stopping-factor STOPPING_FACTOR] [--linear] [-f]
13+
[--include-package INCLUDE_PACKAGE]
14+
param_path
15+
16+
Find a learning rate range where loss decreases quickly for the specified
17+
model and dataset.
18+
19+
positional arguments:
20+
param_path path to parameter file describing the model to be
21+
trained
22+
23+
optional arguments:
24+
-h, --help show this help message and exit
25+
-s SERIALIZATION_DIR, --serialization-dir SERIALIZATION_DIR
26+
The directory in which to save results.
27+
-o OVERRIDES, --overrides OVERRIDES
28+
a JSON structure used to override the experiment
29+
configuration
30+
--start-lr START_LR learning rate to start the search (default = 1e-05)
31+
--end-lr END_LR learning rate up to which search is done (default =
32+
10)
33+
--num-batches NUM_BATCHES
34+
number of mini-batches to run learning rate finder
35+
(default = 100)
36+
--stopping-factor STOPPING_FACTOR
37+
stop the search when the current loss exceeds the best
38+
loss recorded by multiple of stopping factor
39+
--linear increase learning rate linearly instead of exponential
40+
increase
41+
-f, --force overwrite the output directory if it exists
42+
--include-package INCLUDE_PACKAGE
43+
additional packages to include
4444
"""
4545

4646
from typing import List, Tuple

‎allennlp/commands/fine_tune.py

+47-1
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,53 @@
33
dataset` than the one it was originally trained on. It requires a saved model archive file, a path
44
to the data you will continue training with, and a directory in which to write the results.
55
6-
Run ``allennlp fine-tune --help`` for detailed usage information.
6+
.. code-block:: bash
7+
8+
$ allennlp fine-tune --help
9+
usage: allennlp fine-tune [-h] -m MODEL_ARCHIVE -c CONFIG_FILE -s
10+
SERIALIZATION_DIR [-o OVERRIDES] [--extend-vocab]
11+
[--file-friendly-logging]
12+
[--batch-weight-key BATCH_WEIGHT_KEY]
13+
[--embedding-sources-mapping EMBEDDING_SOURCES_MAPPING]
14+
[--include-package INCLUDE_PACKAGE]
15+
16+
Continues training a saved model on a new dataset.
17+
18+
optional arguments:
19+
-h, --help show this help message and exit
20+
-m MODEL_ARCHIVE, --model-archive MODEL_ARCHIVE
21+
path to the saved model archive from training on the
22+
original data
23+
-c CONFIG_FILE, --config-file CONFIG_FILE
24+
configuration file to use for training. Format is the
25+
same as for the "train" command, but the "model"
26+
section is ignored.
27+
-s SERIALIZATION_DIR, --serialization-dir SERIALIZATION_DIR
28+
directory in which to save the fine-tuned model and
29+
its logs
30+
-o OVERRIDES, --overrides OVERRIDES
31+
a JSON structure used to override the training
32+
configuration (only affects the config_file, _not_ the
33+
model_archive)
34+
--extend-vocab if specified, we will use the instances in your new
35+
dataset to extend your vocabulary. If pretrained-file
36+
was used to initialize embedding layers, you may also
37+
need to pass --embedding-sources-mapping.
38+
--file-friendly-logging
39+
outputs tqdm status on separate lines and slows tqdm
40+
refresh rate
41+
--batch-weight-key BATCH_WEIGHT_KEY
42+
If non-empty, name of metric used to weight the loss
43+
on a per-batch basis.
44+
--embedding-sources-mapping EMBEDDING_SOURCES_MAPPING
45+
a JSON dict defining mapping from embedding module
46+
path to embeddingpretrained-file used during training.
47+
If not passed, and embedding needs to be extended, we
48+
will try to use the original file paths used during
49+
training. If they are not available we will use random
50+
vectors for embedding extension.
51+
--include-package INCLUDE_PACKAGE
52+
additional packages to include
753
"""
854
import argparse
955
import json

‎allennlp/commands/make_vocab.py

+12-11
Original file line numberDiff line numberDiff line change
@@ -6,23 +6,24 @@
66
.. code-block:: bash
77
88
$ allennlp make-vocab --help
9-
10-
usage: allennlp make-vocab [-h] [-o OVERRIDES] [--include-package INCLUDE_PACKAGE] param_path
9+
usage: allennlp make-vocab [-h] -s SERIALIZATION_DIR [-o OVERRIDES]
10+
[--include-package INCLUDE_PACKAGE]
11+
param_path
1112
1213
Create a vocabulary from the specified dataset.
1314
1415
positional arguments:
15-
param_path path to parameter file describing the model and its
16-
inputs
16+
param_path path to parameter file describing the model and its
17+
inputs
1718
1819
optional arguments:
19-
-h, --help show this help message and exit
20-
-s SERIALIZATION_DIR, --serialization-dir SERIALIZATION_DIR
21-
directory in which to save the vocabulary directory
22-
-o OVERRIDES, --overrides OVERRIDES
23-
a JSON structure used to override the experiment
24-
configuration
25-
--include-package INCLUDE_PACKAGE
20+
-h, --help show this help message and exit
21+
-s SERIALIZATION_DIR, --serialization-dir SERIALIZATION_DIR
22+
directory in which to save the vocabulary directory
23+
-o OVERRIDES, --overrides OVERRIDES
24+
a JSON structure used to override the experiment
25+
configuration
26+
--include-package INCLUDE_PACKAGE
2627
additional packages to include
2728
"""
2829
import argparse

0 commit comments

Comments
 (0)