Skip to content
This repository was archived by the owner on Dec 16, 2022. It is now read-only.

Commit 354a19b

Browse files
guoquanschmmd
authored andcommitted
Update documents to sentence_splitter.py (#3023)
* Update sentence_splitter.py Update some legacy documents that look like copied from `WordSplitter`. * Update sentence_splitter.py
1 parent 427996d commit 354a19b

File tree

1 file changed

+4
-2
lines changed

1 file changed

+4
-2
lines changed

allennlp/data/tokenizers/sentence_splitter.py

+4-2
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,12 @@ class SentenceSplitter(Registrable):
1515

1616
def split_sentences(self, text: str) -> List[str]:
1717
"""
18-
Splits ``texts`` into a list of :class:`Token` objects.
18+
Splits a ``text`` :class:`str` paragraph into a list of :class:`str`, where each is a sentence.
1919
"""
2020
raise NotImplementedError
2121

2222
def batch_split_sentences(self, texts: List[str]) -> List[List[str]]:
2323
"""
24-
This method lets you take advantage of spacy's batch processing.
2524
Default implementation is to just iterate over the texts and call ``split_sentences``.
2625
"""
2726
return [self.split_sentences(text) for text in texts]
@@ -59,4 +58,7 @@ def split_sentences(self, text: str) -> List[str]:
5958

6059
@overrides
6160
def batch_split_sentences(self, texts: List[str]) -> List[List[str]]:
61+
"""
62+
This method lets you take advantage of spacy's batch processing.
63+
"""
6264
return [[sentence.string.strip() for sentence in doc.sents] for doc in self.spacy.pipe(texts)]

0 commit comments

Comments
 (0)