Skip to content

Commit 3395de4

Browse files
authored
Merge pull request jupyter#1256 from tbluejeans/fix-spark-properties
Fix spark config properties syntax in Pyspark notebook Docker image
2 parents 9fe5186 + 5b0b25d commit 3395de4

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

pyspark-notebook/Dockerfile

+2-2
Original file line numberDiff line numberDiff line change
@@ -52,8 +52,8 @@ RUN ln -s "spark-${APACHE_SPARK_VERSION}-bin-hadoop${HADOOP_VERSION}" spark && \
5252
# Fix Spark installation for Java 11 and Apache Arrow library
5353
# see: https://github.com/apache/spark/pull/27356, https://spark.apache.org/docs/latest/#downloading
5454
RUN cp -p "$SPARK_HOME/conf/spark-defaults.conf.template" "$SPARK_HOME/conf/spark-defaults.conf" && \
55-
echo 'spark.driver.extraJavaOptions="-Dio.netty.tryReflectionSetAccessible=true"' >> $SPARK_HOME/conf/spark-defaults.conf && \
56-
echo 'spark.executor.extraJavaOptions="-Dio.netty.tryReflectionSetAccessible=true"' >> $SPARK_HOME/conf/spark-defaults.conf
55+
echo 'spark.driver.extraJavaOptions -Dio.netty.tryReflectionSetAccessible=true' >> $SPARK_HOME/conf/spark-defaults.conf && \
56+
echo 'spark.executor.extraJavaOptions -Dio.netty.tryReflectionSetAccessible=true' >> $SPARK_HOME/conf/spark-defaults.conf
5757

5858
USER $NB_UID
5959

0 commit comments

Comments
 (0)