Skip to content

Commit e97b376

Browse files
jbouffardjbouffard
authored
Merge pull request #565 from jbouffard/0.3.0-release
0.3.0 Release Updates
2 parents d188d68 + cf405c7 commit e97b376

File tree

9 files changed

+188
-64
lines changed

9 files changed

+188
-64
lines changed

‎.travis.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ addons:
2929
install:
3030
- pushd geopyspark-backend &&
3131
./sbt "project geotrellis-backend" assembly &&
32-
cp geotrellis/target/scala-2.11/geotrellis-backend-assembly-0.2.2.jar ../geopyspark/jars &&
32+
cp geotrellis/target/scala-2.11/geotrellis-backend-assembly-0.3.0.jar ../geopyspark/jars &&
3333
popd
3434
- if [ ! -f archives/spark-2.1.1-bin-hadoop2.7.tgz ]; then
3535
pushd archives;

‎Makefile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,10 @@ export ASSEMBLED="assembled"
55
rwildcard=$(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2) $(filter $(subst *,%,$2),$d))
66

77
JAR-PATH := geopyspark/jars
8-
ASSEMBLYNAME := geotrellis-backend-assembly-0.2.2.jar
8+
ASSEMBLYNAME := geotrellis-backend-assembly-0.3.0.jar
99
BUILD-ASSEMBLY := geopyspark-backend/geotrellis/target/scala-2.11/${ASSEMBLYNAME}
1010
DIST-ASSEMBLY := ${JAR-PATH}/${ASSEMBLYNAME}
11-
WHEELNAME := geopyspark-0.1.0-py3-none-any.whl
11+
WHEELNAME := geopyspark-0.3.0-py3-none-any.whl
1212
WHEEL := dist/${WHEELNAME}
1313

1414
SCALA_SRC := $(call rwildcard, geopyspark-backend/geotrellis/src/, *.scala)

‎docs/CHANGELOG.rst

Lines changed: 176 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -1,46 +1,161 @@
11
Changelog
22
==========
33

4-
0.1.0
4+
5+
0.3.0
56
------
67

7-
The first release of GeoPySpark! After being in development for the past 6
8-
months, it is now ready for its initial release! Since nothing has been changed
9-
or updated per se, we'll just go over the features that will be present in
10-
0.1.0.
8+
New Features
9+
^^^^^^^^^^^^^
1110

11+
Aggregating a Layer By Cell
12+
****************************
1213

13-
**geopyspark.geotrellis**
14+
It is now possible to aggregate the cells of all values that share a key
15+
in a layer via the ``aggregate_by_cell`` method. This method is useful when
16+
you have a layer where you want to reduce all of the values by their key.
1417

15-
- Create a ``RasterRDD`` from GeoTiffs that are stored locally, on S3, or on
16-
HDFS.
17-
- Serialize Python RDDs to Scala and back.
18-
- Perform various tiling operations such as ``tile_to_layout``, ``cut_tiles``,
19-
and ``pyramid``.
20-
- Stitch together a ``TiledRasterRDD`` to create one ``Raster``.
21-
- ``rasterize`` geometries and turn them into ``RasterRDD``.
22-
- ``reclassify`` values of Rasters in RDDs.
23-
- Calculate ``cost_distance`` on a ``TiledRasterRDD``.
24-
- Perform local and focal operations on ``TiledRasterRDD``.
25-
- Read, write, and query GeoTrellis tile layers.
26-
- Read tiles from a layer.
27-
- Added ``PngRDD`` to make rendering to PNGs more efficient.
28-
- Added ``RDDWrapper`` to provide more functionality to the RDD classes.
29-
- Polygonal summary methods are now available to ``TiledRasterRDD``.
30-
- Euclidean distance added to ``TiledRasterRDD``.
31-
- Neighborhoods submodule added to make focal operations easier.
18+
.. code:: python3
3219
33-
**geopyspark.command**
20+
# A tiled layer which contains duplicate keys with different values
21+
# that we'd like to reduce so that there is one value per key.
22+
tiled_layer = ...
3423
35-
- GeoPySpark can now use a script to download the jar.
36-
Used when installing GeoPySpark from pip.
24+
# This will compute the aggregate SUM of each cell of values that share
25+
# a key within the layer.
26+
tiled_layer.aggregate_by_cell(gps.Operation.SUM)
3727
38-
**Documentation**
28+
# Similar to the above command, only this one is finding the STANDARD_DEVIATION
29+
# for each cell.
30+
tiled_layer.aggregate_by_cell(gps.Operation.STANDARD_DEVIATION)
3931
40-
- Added docstrings to all python classes, methods, etc.
41-
- Core-Concepts, rdd, geopycontext, and catalog.
42-
- Ingesting and creating a tile server with a greyscale raster dataset.
43-
- Ingesting and creating a tile server with data from Sentinel.
32+
Unioning Layers Together
33+
************************
34+
35+
Through the ``union`` method, it is now possible to union together an arbitrary number
36+
of either ``RasterLayer``\s or ``TiledRasterLayers``.
37+
38+
.. code:: python3
39+
40+
# Layers to be unioned together
41+
layers = [raster_layer_1, raster_layer_2, raster_layer_3]
42+
43+
unioned_layers = gps.union(layers)
44+
45+
Getting Point Values From a Layer
46+
**********************************
47+
48+
By using the ``get_point_values`` method, one can retrieve data points that falls
49+
on or near a given point.
50+
51+
.. code:: python3
52+
53+
from shapely.geometry import Point
54+
55+
# The points we'd like to collect data at
56+
p1 = Point(0, 0)
57+
p2 = Point(1, 1)
58+
p3 = Point(10, 10)
59+
60+
# The tiled layer which will be queried
61+
tiled_layer = ...
62+
63+
tiled_layer.get_point_values([p1, p2, p3])
64+
65+
The above code will return a ``[(Point, [float])]`` where each
66+
point given will be paired with all of the values it covers (one for
67+
each band of the Tile).
68+
69+
It is also possible to pass in a ``dict`` to ``get_point_values``.
70+
71+
.. code:: python3
72+
73+
labeled_points = {'p1': p1, 'p2': p2, 'p3': p3}
74+
75+
tiled_layer.get_point_values(labeled_points)
76+
77+
This will return a ``{k: (Point, [float])}`` which is similar to
78+
the above code only now the ``(Point, [float])`` is the value
79+
of the key that point had in the input ``dict``.
80+
81+
Combining Bands of Multiple Layers
82+
***********************************
83+
84+
``combine_bands`` will concatenate the bands of values that
85+
share a key together to produce a new, single value. This new
86+
Tile will contain all of the bands from all of the values
87+
that shared a key from the given layers.
88+
89+
This method is most useful when you have multiple layers
90+
that contain a single band from a multiband image; and you'd
91+
like to combine them together so that all or some of the bands
92+
are available from a single layer.
93+
94+
95+
.. code:: python3
96+
97+
# Three different layers that contain a single band from the
98+
# same scene
99+
band_1_layer = ...
100+
band_2_layer = ...
101+
band_3_layer = ...
102+
103+
# combined_layer will have values that contain three bands: the first
104+
# from band_1_layer, the second from band_2_layer, and the last from
105+
# band_3_layer
106+
combined_layer = gps.combine_bands([band_1_layer, band_2_layer, band_3_layer])
107+
108+
Other New Features
109+
*******************
110+
111+
- `Merge method for RasterLayer and TiledRasterLayer <https://github.com/locationtech-labs/geopyspark/pull/503>`__
112+
- `Filter a RasterLayer or a TiledRasterLayer by time <https://github.com/locationtech-labs/geopyspark/pull/518>`__
113+
- `Polygonal Summary on all bands <https://github.com/locationtech-labs/geopyspark/pull/519>`__
114+
- `Better temporal resolution control when writing layers <https://github.com/locationtech-labs/geopyspark/pull/542>`__
115+
- `TiledRasterLayers can now perform the abs local operation <https://github.com/locationtech-labs/geopyspark/pull/550>`__
116+
- `TiledRasterLayers can now perform the ** local operation <https://github.com/locationtech-labs/geopyspark/pull/551>`__
117+
118+
Bug Fixes
119+
^^^^^^^^^^
120+
121+
- `LayerType creation issue <https://github.com/locationtech-labs/geopyspark/pull/494>`__
122+
- `tuple serializer creation fix <https://github.com/locationtech-labs/geopyspark/pull/497>`__
123+
- `The TMS can now read from MultibandTile catalogs <https://github.com/locationtech-labs/geopyspark/pull/508>`__
124+
- `tileToLayout bug <https://github.com/locationtech-labs/geopyspark/pull/525>`__
125+
- `additional_jar_dirs fix <https://github.com/locationtech-labs/geopyspark/pull/532>`__
126+
- `stitch and saveStitch now work with MultibandTiles <https://github.com/locationtech-labs/geopyspark/pull/537>`__
127+
128+
0.2.2
129+
------
130+
131+
0.2.2 fixes the naming issue brought about in 0.2.1 where the backend jar and
132+
the docs had the incorrect version number.
133+
134+
135+
**geopyspark**
136+
137+
- Fixed version numbers for docs and jar.
138+
139+
140+
0.2.1
141+
------
142+
143+
0.2.1 adds two major bug fixes for the ``catalog.query`` and ``geotiff.get``
144+
functions as well as a few other minor changes/additions.
145+
146+
147+
**geopyspark**
148+
149+
- Updated description in ``setup.py``.
150+
151+
**geopyspark.geotrellis**
152+
153+
- Fixed a bug in ``catalog.query`` where the query would fail if the geometry
154+
used for querying was in a different projection than the source layer.
155+
- ``partition_bytes`` can now be set in the ``geotiff.get`` function when
156+
reading from S3.
157+
- Setting ``max_tile_size`` and ``num_partitions`` in ``geotiff.get`` will now
158+
work when trying to read geotiffs from S3.
44159

45160

46161
0.2.0
@@ -88,34 +203,43 @@ changes will be discussed below.
88203
- Example jupyter notebooks have been added.
89204

90205

91-
0.2.1
206+
0.1.0
92207
------
93208

94-
0.2.1 adds two major bug fixes for the ``catalog.query`` and ``geotiff.get``
95-
functions as well as a few other minor changes/additions.
96-
97-
98-
**geopyspark**
209+
The first release of GeoPySpark! After being in development for the past 6
210+
months, it is now ready for its initial release! Since nothing has been changed
211+
or updated per se, we'll just go over the features that will be present in
212+
0.1.0.
99213

100-
- Updated description in ``setup.py``.
101214

102215
**geopyspark.geotrellis**
103216

104-
- Fixed a bug in ``catalog.query`` where the query would fail if the geometry
105-
used for querying was in a different projection than the source layer.
106-
- ``partition_bytes`` can now be set in the ``geotiff.get`` function when
107-
reading from S3.
108-
- Setting ``max_tile_size`` and ``num_partitions`` in ``geotiff.get`` will now
109-
work when trying to read geotiffs from S3.
110-
111-
112-
0.2.2
113-
------
217+
- Create a ``RasterRDD`` from GeoTiffs that are stored locally, on S3, or on
218+
HDFS.
219+
- Serialize Python RDDs to Scala and back.
220+
- Perform various tiling operations such as ``tile_to_layout``, ``cut_tiles``,
221+
and ``pyramid``.
222+
- Stitch together a ``TiledRasterRDD`` to create one ``Raster``.
223+
- ``rasterize`` geometries and turn them into ``RasterRDD``.
224+
- ``reclassify`` values of Rasters in RDDs.
225+
- Calculate ``cost_distance`` on a ``TiledRasterRDD``.
226+
- Perform local and focal operations on ``TiledRasterRDD``.
227+
- Read, write, and query GeoTrellis tile layers.
228+
- Read tiles from a layer.
229+
- Added ``PngRDD`` to make rendering to PNGs more efficient.
230+
- Added ``RDDWrapper`` to provide more functionality to the RDD classes.
231+
- Polygonal summary methods are now available to ``TiledRasterRDD``.
232+
- Euclidean distance added to ``TiledRasterRDD``.
233+
- Neighborhoods submodule added to make focal operations easier.
114234

115-
0.2.2 fixes the naming issue brought about in 0.2.1 where the backend jar and
116-
the docs had the incorrect version number.
235+
**geopyspark.command**
117236

237+
- GeoPySpark can now use a script to download the jar.
238+
Used when installing GeoPySpark from pip.
118239

119-
**geopyspark**
240+
**Documentation**
120241

121-
- Fixed version numbers for docs and jar.
242+
- Added docstrings to all python classes, methods, etc.
243+
- Core-Concepts, rdd, geopycontext, and catalog.
244+
- Ingesting and creating a tile server with a greyscale raster dataset.
245+
- Ingesting and creating a tile server with data from Sentinel.

‎docs/conf.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@
3232
jar = 'geotrellis-backend-assembly-0.2.2.jar'
3333

3434
if not path.isfile(path.join('geopyspark/jars', jar)):
35-
url = 'https://github.com/locationtech-labs/geopyspark/releases/download/v0.2.2/'
35+
url = 'https://github.com/locationtech-labs/geopyspark/releases/download/v0.3.0/'
3636
subprocess.call(['curl', '-L', url+jar, '-o', path.join('geopyspark/jars', jar)])
3737

3838
sys.path.insert(0, path.abspath(os.curdir))
@@ -73,9 +73,9 @@
7373
# built documents.
7474
#
7575
# The short X.Y version.
76-
version = '0.2.2'
76+
version = '0.3.0'
7777
# The full version, including alpha/beta/rc tags.
78-
release = '0.2.2'
78+
release = '0.3.0'
7979

8080
# The language for content autogenerated by Sphinx. Refer to documentation
8181
# for a list of supported languages.

‎docs/index.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ a need for it.
104104
:caption: Home
105105
:hidden:
106106

107-
changelog <CHANGELOG>
107+
Changelog <CHANGELOG>
108108
contributing
109109

110110
.. toctree::

‎geopyspark-backend/project/Version.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
object Version {
2-
val geopyspark = "0.2.2"
3-
val geotrellis = "1.2.0-SNAPSHOT"
2+
val geopyspark = "0.3.0"
3+
val geotrellis = "1.2.0-RC2"
44
val scala = "2.11.11"
55
val crossScala = Seq("2.11.11", "2.10.6")
66
val scalaTest = "2.2.0"

‎geopyspark/command/configuration.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
from geopyspark.geopyspark_constants import JAR, CWD
1111

1212

13-
JAR_URL = 'https://github.com/locationtech-labs/geopyspark/releases/download/v0.2.2/' + JAR
13+
JAR_URL = 'https://github.com/locationtech-labs/geopyspark/releases/download/v0.3.0/' + JAR
1414
DEFAULT_JAR_PATH = path.join(CWD, 'jars')
1515
CONF = path.join(CWD, 'command', 'geopyspark.conf')
1616

‎geopyspark/geopyspark_constants.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
from os import path
33

44
"""GeoPySpark version."""
5-
VERSION = '0.2.2'
5+
VERSION = '0.3.0'
66

77
"""Backend jar name."""
88
JAR = 'geotrellis-backend-assembly-' + VERSION + '.jar'

‎setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77

88
setup_args = dict(
99
name='geopyspark',
10-
version='0.2.2',
10+
version='0.3.0',
1111
author='Jacob Bouffard, James McClain',
1212
author_email='[email protected], [email protected]',
1313
download_url='http://github.com/locationtech-labs/geopyspark',

0 commit comments

Comments
 (0)