Skip to content

Commit ec26352

Browse files
maziyarpanahimaziyarpanahi
authored
Merge pull request #14437 from JohnSnowLabs/release/551-release-candidate
Release/551 release candidate
2 parents b314f62 + 1951ace commit ec26352

File tree

1,689 files changed

+31743
-5381
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,689 files changed

+31743
-5381
lines changed

‎CHANGELOG

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,36 @@
1+
========
2+
5.5.1
3+
========
4+
----------------
5+
New Features & Enhancements
6+
----------------
7+
* `BertForMultipleChoice` Transformer Added. Enhanced BERT’s capabilities to handle multiple-choice tasks such as standardized test questions and survey or quiz automation.
8+
* Integrated New Tasks and Documentation:
9+
* Added support and documentation for the following tasks:
10+
* Automatic Speech Recognition
11+
* Dependency Parsing
12+
* Image Captioning
13+
* Image Classification
14+
* Landing Page
15+
* Question Answering
16+
* Summarization
17+
* Table Question Answering
18+
* Text Classification
19+
* Text Generation
20+
* Text Preprocessing
21+
* Token Classification
22+
* Translation
23+
* Zero-Shot Classification
24+
* Zero-Shot Image Classification
25+
* `PromptAssembler` Annotator Introduced. Introduced a new annotator that constructs prompts for LLMs using a chat template and a sequence of messages. Accepts an array of tuples with roles (“system”, “user”, “assistant”) and message texts. Utilizes llama.cpp as a backend for template parsing, supporting basic template applications.
26+
27+
----------------
28+
Bug Fixes
29+
----------------
30+
* Resolved Pretrained Model Loading Issue on DBFS Systems.
31+
* Fixed a bug where pretrained models were not found when running AutoGGUF model pipelines on Databricks due to incorrect path handling of gguf files.
32+
33+
134
========
235
5.5.0
336
========

‎README.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ $ java -version
6363
$ conda create -n sparknlp python=3.7 -y
6464
$ conda activate sparknlp
6565
# spark-nlp by default is based on pyspark 3.x
66-
$ pip install spark-nlp==5.5.0 pyspark==3.3.1
66+
$ pip install spark-nlp==5.5.1 pyspark==3.3.1
6767
```
6868

6969
In Python console or Jupyter `Python3` kernel:
@@ -129,7 +129,7 @@ For a quick example of using pipelines and models take a look at our official [d
129129

130130
### Apache Spark Support
131131

132-
Spark NLP *5.5.0* has been built on top of Apache Spark 3.4 while fully supports Apache Spark 3.0.x, 3.1.x, 3.2.x, 3.3.x, 3.4.x, and 3.5.x
132+
Spark NLP *5.5.1* has been built on top of Apache Spark 3.4 while fully supports Apache Spark 3.0.x, 3.1.x, 3.2.x, 3.3.x, 3.4.x, and 3.5.x
133133

134134
| Spark NLP | Apache Spark 3.5.x | Apache Spark 3.4.x | Apache Spark 3.3.x | Apache Spark 3.2.x | Apache Spark 3.1.x | Apache Spark 3.0.x | Apache Spark 2.4.x | Apache Spark 2.3.x |
135135
|-----------|--------------------|--------------------|--------------------|--------------------|--------------------|--------------------|--------------------|--------------------|
@@ -157,7 +157,7 @@ Find out more about 4.x `SparkNLP` versions in our official [documentation](http
157157

158158
### Databricks Support
159159

160-
Spark NLP 5.5.0 has been tested and is compatible with the following runtimes:
160+
Spark NLP 5.5.1 has been tested and is compatible with the following runtimes:
161161

162162
| **CPU** | **GPU** |
163163
|--------------------|--------------------|
@@ -174,7 +174,7 @@ We are compatible with older runtimes. For a full list check databricks support
174174

175175
### EMR Support
176176

177-
Spark NLP 5.5.0 has been tested and is compatible with the following EMR releases:
177+
Spark NLP 5.5.1 has been tested and is compatible with the following EMR releases:
178178

179179
| **EMR Release** |
180180
|--------------------|
@@ -205,7 +205,7 @@ deployed to Maven central. To add any of our packages as a dependency in your ap
205205
from our official documentation.
206206

207207
If you are interested, there is a simple SBT project for Spark NLP to guide you on how to use it in your
208-
projects [Spark NLP SBT S5.5.0r](https://github.com/maziyarpanahi/spark-nlp-starter)
208+
projects [Spark NLP SBT S5.5.1r](https://github.com/maziyarpanahi/spark-nlp-starter)
209209

210210
### Python
211211

@@ -250,7 +250,7 @@ In Spark NLP we can define S3 locations to:
250250

251251
Please check [these instructions](https://sparknlp.org/docs/en/install#s3-integration) from our official documentation.
252252

253-
## Document5.5.0
253+
## Document5.5.1
254254

255255
### Examples
256256

@@ -283,7 +283,7 @@ the Spark NLP library:
283283
keywords = {Spark, Natural language processing, Deep learning, Tensorflow, Cluster},
284284
abstract = {Spark NLP is a Natural Language Processing (NLP) library built on top of Apache Spark ML. It provides simple, performant & accurate NLP annotations for machine learning pipelines that can scale easily in a distributed environment. Spark NLP comes with 1100+ pretrained pipelines and models in more than 192+ languages. It supports nearly all the NLP tasks and modules that can be used seamlessly in a cluster. Downloaded more than 2.7 million times and experiencing 9x growth since January 2020, Spark NLP is used by 54% of healthcare organizations as the world’s most widely used NLP library in the enterprise.}
285285
}
286-
}5.5.0
286+
}5.5.1
287287
```
288288

289289
## Community support

‎build.sbt

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ name := getPackageName(is_silicon, is_gpu, is_aarch64)
66

77
organization := "com.johnsnowlabs.nlp"
88

9-
version := "5.5.0"
9+
version := "5.5.1"
1010

1111
(ThisBuild / scalaVersion) := scalaVer
1212

@@ -185,8 +185,8 @@ val llamaCppDependencies =
185185
Seq(llamaCppGPU)
186186
else if (is_silicon.equals("true"))
187187
Seq(llamaCppSilicon)
188-
// else if (is_aarch64.equals("true"))
189-
// Seq(openVinoCPU)
188+
else if (is_aarch64.equals("true"))
189+
Seq(llamaCppAarch64)
190190
else
191191
Seq(llamaCppCPU)
192192

‎conda/meta.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
{% set name = "spark-nlp" %}
2-
{% set version = "5.5.0" %}
2+
{% set version = "5.5.1" %}
33

44
package:
55
name: {{ name|lower }}
66
version: {{ version }}
77

88
source:
99
url: https://pypi.io/packages/source/{{ name[0] }}/{{ name }}/spark-nlp-{{ version }}.tar.gz
10-
sha256: edc71585f462f548770bd13899686f10d88fa4a4a6e201bc1bf9c7711e398dc0
10+
sha256: e8ddaf939a1b0acbe0d7b6d6a67f7fa0c5a73339d9e4563e3c1aba1cf0039409
1111

1212
build:
1313
noarch: python

‎docs/_data/navigation.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,8 @@ sparknlp:
4444
url: /docs/en/pipelines
4545
- title: General Concepts
4646
url: /docs/en/concepts
47+
- title: Tasks
48+
url: /docs/en/tasks/landing_page
4749
- title: Annotators
4850
url: /docs/en/annotators
4951
- title: Transformers

‎docs/_layouts/landing.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -201,7 +201,7 @@ <h3 class="grey h3_title">{{ _section.title }}</h3>
201201
<div class="highlight-box">
202202
{% highlight bash %}
203203
# Using PyPI
204-
$ pip install spark-nlp==5.5.0
204+
$ pip install spark-nlp==5.5.1
205205

206206
# Using Anaconda/Conda
207207
$ conda install -c johnsnowlabs spark-nlp

‎docs/api/com/index.html

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33
<head>
44
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
55
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
6-
<title>Spark NLP 5.5.0 ScalaDoc - com</title>
7-
<meta name="description" content="Spark NLP 5.5.0 ScalaDoc - com" />
8-
<meta name="keywords" content="Spark NLP 5.5.0 ScalaDoc com" />
6+
<title>Spark NLP 5.5.1 ScalaDoc - com</title>
7+
<meta name="description" content="Spark NLP 5.5.1 ScalaDoc - com" />
8+
<meta name="keywords" content="Spark NLP 5.5.1 ScalaDoc com" />
99
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
1010

1111

@@ -28,7 +28,7 @@
2828
</head>
2929
<body>
3030
<div id="search">
31-
<span id="doc-title">Spark NLP 5.5.0 ScalaDoc<span id="doc-version"></span></span>
31+
<span id="doc-title">Spark NLP 5.5.1 ScalaDoc<span id="doc-version"></span></span>
3232
<span class="close-results"><span class="left">&lt;</span> Back</span>
3333
<div id="textfilter">
3434
<span class="input">

‎docs/api/com/johnsnowlabs/client/CloudClient.html

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33
<head>
44
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
55
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
6-
<title>Spark NLP 5.5.0 ScalaDoc - com.johnsnowlabs.client.CloudClient</title>
7-
<meta name="description" content="Spark NLP 5.5.0 ScalaDoc - com.johnsnowlabs.client.CloudClient" />
8-
<meta name="keywords" content="Spark NLP 5.5.0 ScalaDoc com.johnsnowlabs.client.CloudClient" />
6+
<title>Spark NLP 5.5.1 ScalaDoc - com.johnsnowlabs.client.CloudClient</title>
7+
<meta name="description" content="Spark NLP 5.5.1 ScalaDoc - com.johnsnowlabs.client.CloudClient" />
8+
<meta name="keywords" content="Spark NLP 5.5.1 ScalaDoc com.johnsnowlabs.client.CloudClient" />
99
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
1010

1111

@@ -28,7 +28,7 @@
2828
</head>
2929
<body>
3030
<div id="search">
31-
<span id="doc-title">Spark NLP 5.5.0 ScalaDoc<span id="doc-version"></span></span>
31+
<span id="doc-title">Spark NLP 5.5.1 ScalaDoc<span id="doc-version"></span></span>
3232
<span class="close-results"><span class="left">&lt;</span> Back</span>
3333
<div id="textfilter">
3434
<span class="input">

‎docs/api/com/johnsnowlabs/client/CloudManager.html

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33
<head>
44
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
55
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
6-
<title>Spark NLP 5.5.0 ScalaDoc - com.johnsnowlabs.client.CloudManager</title>
7-
<meta name="description" content="Spark NLP 5.5.0 ScalaDoc - com.johnsnowlabs.client.CloudManager" />
8-
<meta name="keywords" content="Spark NLP 5.5.0 ScalaDoc com.johnsnowlabs.client.CloudManager" />
6+
<title>Spark NLP 5.5.1 ScalaDoc - com.johnsnowlabs.client.CloudManager</title>
7+
<meta name="description" content="Spark NLP 5.5.1 ScalaDoc - com.johnsnowlabs.client.CloudManager" />
8+
<meta name="keywords" content="Spark NLP 5.5.1 ScalaDoc com.johnsnowlabs.client.CloudManager" />
99
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
1010

1111

@@ -28,7 +28,7 @@
2828
</head>
2929
<body>
3030
<div id="search">
31-
<span id="doc-title">Spark NLP 5.5.0 ScalaDoc<span id="doc-version"></span></span>
31+
<span id="doc-title">Spark NLP 5.5.1 ScalaDoc<span id="doc-version"></span></span>
3232
<span class="close-results"><span class="left">&lt;</span> Back</span>
3333
<div id="textfilter">
3434
<span class="input">

‎docs/api/com/johnsnowlabs/client/CloudResources$.html

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -3,9 +3,9 @@
33
<head>
44
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
55
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
6-
<title>Spark NLP 5.5.0 ScalaDoc - com.johnsnowlabs.client.CloudResources</title>
7-
<meta name="description" content="Spark NLP 5.5.0 ScalaDoc - com.johnsnowlabs.client.CloudResources" />
8-
<meta name="keywords" content="Spark NLP 5.5.0 ScalaDoc com.johnsnowlabs.client.CloudResources" />
6+
<title>Spark NLP 5.5.1 ScalaDoc - com.johnsnowlabs.client.CloudResources</title>
7+
<meta name="description" content="Spark NLP 5.5.1 ScalaDoc - com.johnsnowlabs.client.CloudResources" />
8+
<meta name="keywords" content="Spark NLP 5.5.1 ScalaDoc com.johnsnowlabs.client.CloudResources" />
99
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
1010

1111

@@ -28,7 +28,7 @@
2828
</head>
2929
<body>
3030
<div id="search">
31-
<span id="doc-title">Spark NLP 5.5.0 ScalaDoc<span id="doc-version"></span></span>
31+
<span id="doc-title">Spark NLP 5.5.1 ScalaDoc<span id="doc-version"></span></span>
3232
<span class="close-results"><span class="left">&lt;</span> Back</span>
3333
<div id="textfilter">
3434
<span class="input">

0 commit comments

Comments
 (0)