You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
# Development - Contributing
2
2
3
-
PGStac uses a dockerized development environment. However,
3
+
PgSTAC uses a dockerized development environment. However,
4
4
it still needs a local install of pypgstac to allow an editable
5
5
install inside the docker container. This is installed automatically
6
6
if you have set up a virtual environment for the project. Otherwise
@@ -58,7 +58,7 @@ This will create a base migration for the new version and will create incrementa
58
58
All changes to SQL should only be made in the `/src/pgstac/sql` directory. SQL Files will be run in alphabetical order.
59
59
60
60
### Adding Tests
61
-
PGStac tests can be written using PGTap or basic SQL output comparisons. Additional testing is available using PyTest in the PyPgSTAC module. Tests can be run using the `scripts/test` command.
61
+
PgSTAC tests can be written using PGTap or basic SQL output comparisons. Additional testing is available using PyTest in the PyPgSTAC module. Tests can be run using the `scripts/test` command.
62
62
63
63
PGTap tests can be written using [PGTap](https://pgtap.org/) syntax. Tests should be added to the `/src/pgstac/tests/pgtap` directory. Any new sql files added to this directory must be added to `/src/pgstac/tests/pgtap.sql`.
Copy file name to clipboardExpand all lines: README.md
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@
28
28
29
29
PgSTAC provides functionality for STAC Filters, CQL2 search, and utilities to help manage the indexing and partitioning of STAC Collections and Items.
30
30
31
-
PgSTAC is used in production to scale to hundreds of millions of STAC items. PgSTAC implements core data models and functions to provide a STAC API from a PostgreSQL database. PgSTAC is entirely within the database and does not provide an HTTP-facing API. The [Stac FastAPI](https://github.com/stac-utils/stac-fastapi) PgSTAC backend and [Franklin](https://github.com/azavea/franklin) can be used to expose a PgSTAC catalog. Integrating PgSTAC with any other language with PostgreSQL drivers is also possible.
31
+
PgSTAC is used in production to scale to hundreds of millions of STAC items. PgSTAC implements core data models and functions to provide a STAC API from a PostgreSQL database. PgSTAC is entirely within the database and does not provide an HTTP-facing API. The [STAC FastAPI](https://github.com/stac-utils/stac-fastapi) PgSTAC backend and [Franklin](https://github.com/azavea/franklin) can be used to expose a PgSTAC catalog. Integrating PgSTAC with any other language with PostgreSQL drivers is also possible.
Copy file name to clipboardExpand all lines: docs/src/pgstac.md
+13-13Lines changed: 13 additions & 13 deletions
Original file line number
Diff line number
Diff line change
@@ -1,15 +1,15 @@
1
1
2
2
PGDatabase Schema and Functions for Storing and Accessing STAC collections and items in PostgreSQL
3
3
4
-
STAC Client that uses PGStac available in [STAC-FastAPI](https://github.com/stac-utils/stac-fastapi)
4
+
STAC Client that uses PgSTAC available in [STAC-FastAPI](https://github.com/stac-utils/stac-fastapi)
5
5
6
-
PGStac requires **Postgresql>=13** and **PostGIS>=3**. Best performance will be had using PostGIS>=3.1.
6
+
PgSTAC requires **Postgresql>=13** and **PostGIS>=3**. Best performance will be had using PostGIS>=3.1.
7
7
8
-
### PGStac Settings
9
-
PGStac installs everything into the pgstac schema in the database. This schema must be in the search_path in the postgresql session while using pgstac.
8
+
### PgSTAC Settings
9
+
PgSTAC installs everything into the pgstac schema in the database. This schema must be in the search_path in the postgresql session while using pgstac.
10
10
11
11
12
-
#### PGStac Users
12
+
#### PgSTAC Users
13
13
The pgstac_admin role is the owner of all the objects within pgstac and should be used when running things such as migrations.
14
14
15
15
The pgstac_ingest role has read/write privileges on all tables and should be used for data ingest or if using the transactions extension with stac-fastapi-pgstac.
@@ -28,7 +28,7 @@ To grant pgstac permissions to a current postgresql user:
28
28
GRANT pgstac_read TO <user>;
29
29
```
30
30
31
-
#### PGStac Search Path
31
+
#### PgSTAC Search Path
32
32
The search_path can be set at the database level or role level or by setting within the current session. The search_path is already set if you are directly using one of the pgstac users. If you are not logging in directly as one of the pgstac users, you will need to set the search_path by adding it to the search_path of the user you are using:
33
33
```sql
34
34
ALTER ROLE <user>SET SEARCH_PATH TO pgstac, public;
@@ -45,13 +45,13 @@ kwargs={
45
45
}
46
46
```
47
47
48
-
#### PGStac Settings Variables
48
+
#### PgSTAC Settings Variables
49
49
There are additional variables that control the settings used for calculating and displaying context (total row count) for a search, as well as a variable to set the filter language (cql-json or cql-json2).
50
50
The context is "off" by default, and the default filter language is set to "cql2-json".
51
51
52
52
Variables can be set either by passing them in via the connection options using your connection library, setting them in the pgstac_settings table or by setting them on the Role that is used to log in to the database.
53
53
54
-
Turning "context" on can be **very** expensive on larger databases. Much of what PGStac does is to optimize the search of items sorted by time where only fewer than 10,000 records are returned at a time. It does this by searching for the data in chunks and is able to "short circuit" and return as soon as it has the number of records requested. Calculating the context (the total count for a query) requires a scan of all records that match the query parameters and can take a very long time. Setting "context" to auto will use database statistics to estimate the number of rows much more quickly, but for some queries, the estimate may be quite a bit off.
54
+
Turning "context" on can be **very** expensive on larger databases. Much of what PgSTAC does is to optimize the search of items sorted by time where only fewer than 10,000 records are returned at a time. It does this by searching for the data in chunks and is able to "short circuit" and return as soon as it has the number of records requested. Calculating the context (the total count for a query) requires a scan of all records that match the query parameters and can take a very long time. Setting "context" to auto will use database statistics to estimate the number of rows much more quickly, but for some queries, the estimate may be quite a bit off.
55
55
56
56
Example for updating the pgstac_settings table with a new value:
57
57
```sql
@@ -92,19 +92,19 @@ The nohydrate conf item returns an unhydrated item bypassing the CPU intensive s
92
92
SELECT search('{"conf":{"nohydrate"=true}}');
93
93
```
94
94
95
-
#### PGStac Partitioning
96
-
By default PGStac partitions data by collection (note: this is a change starting with version 0.5.0). Each collection can further be partitioned by either year or month. **Partitioning must be set up prior to loading any data!** Partitioning can be configured by setting the partition_trunc flag on a collection in the database.
95
+
#### PgSTAC Partitioning
96
+
By default PgSTAC partitions data by collection (note: this is a change starting with version 0.5.0). Each collection can further be partitioned by either year or month. **Partitioning must be set up prior to loading any data!** Partitioning can be configured by setting the partition_trunc flag on a collection in the database.
97
97
```sql
98
98
UPDATE collections set partition_trunc='month' WHERE id='<collection id>';
99
99
```
100
100
101
101
In general, you should aim to keep each partition less than a few hundred thousand rows. Further partitioning (ie setting everything to 'month' when not needed to keep the partitions below a few hundred thousand rows) can be detrimental.
102
102
103
-
#### PGStac Indexes / Queryables
103
+
#### PgSTAC Indexes / Queryables
104
104
105
-
By default, PGStac includes indexes on the id, datetime, collection, and geometry. Further indexing can be added for additional properties globally or only on particular collections by modifications to the queryables table.
105
+
By default, PgSTAC includes indexes on the id, datetime, collection, and geometry. Further indexing can be added for additional properties globally or only on particular collections by modifications to the queryables table.
106
106
107
-
The `queryables` table controls the indexes that PGStac will build as well as the metadata that is returned from a [STAC Queryables endpoint](https://github.com/stac-api-extensions/filter#queryables).
107
+
The `queryables` table controls the indexes that PgSTAC will build as well as the metadata that is returned from a [STAC Queryables endpoint](https://github.com/stac-api-extensions/filter#queryables).
Copy file name to clipboardExpand all lines: docs/src/pypgstac.md
+9-9Lines changed: 9 additions & 9 deletions
Original file line number
Diff line number
Diff line change
@@ -2,12 +2,12 @@
2
2
3
3
PgSTAC includes a Python utility for bulk data loading and managing migrations.
4
4
5
-
PyPGStac is available on PyPI
5
+
pyPgSTAC is available on PyPI
6
6
```
7
7
pip install pypgstac
8
8
```
9
9
10
-
By default, PyPGStac does not install the `psycopg` dependency. If you want the database driver installed, use:
10
+
By default, pyPgSTAC does not install the `psycopg` dependency. If you want the database driver installed, use:
11
11
12
12
```
13
13
pip install pypgstac[psycopg]
@@ -39,7 +39,7 @@ Commands:
39
39
version Get version from a pgstac database.
40
40
```
41
41
42
-
PyPGStac will get the database connection settings from the **standard PG environment variables**:
42
+
pyPgSTAC will get the database connection settings from the **standard PG environment variables**:
43
43
44
44
- PGHOST=0.0.0.0
45
45
- PGPORT=5432
@@ -50,18 +50,18 @@ PyPGStac will get the database connection settings from the **standard PG enviro
50
50
It can also take a DSN database url "postgresql://..." via the **--dsn** flag.
51
51
52
52
### Migrations
53
-
PyPGStac has a utility to help apply migrations to an existing PGStac instance to bring it up to date.
53
+
pyPgSTAC has a utility to help apply migrations to an existing PgSTAC instance to bring it up to date.
54
54
55
55
There are two types of migrations:
56
-
-**Base migrations** install PGStac into a database with no current PGStac installation. These migrations follow the file pattern `"pgstac.[version].sql"`
57
-
-**Incremental migrations** are used to move PGStac from one version to the next. These migrations follow the file pattern `"pgstac.[version].[fromversion].sql"`
56
+
-**Base migrations** install PgSTAC into a database with no current PgSTAC installation. These migrations follow the file pattern `"pgstac.[version].sql"`
57
+
-**Incremental migrations** are used to move PgSTAC from one version to the next. These migrations follow the file pattern `"pgstac.[version].[fromversion].sql"`
58
58
59
-
Migrations are stored in ```pypgstac/pypgstac/migration`s``` and are distributed with the PyPGStac package.
59
+
Migrations are stored in ```pypgstac/pypgstac/migration`s``` and are distributed with the pyPgSTAC package.
60
60
61
61
### Running Migrations
62
-
PyPGStac has a utility for checking the version of an existing PGStac database and applying the appropriate migrations in the correct order. It can also be used to setup a database from scratch.
62
+
pyPgSTAC has a utility for checking the version of an existing PgSTAC database and applying the appropriate migrations in the correct order. It can also be used to setup a database from scratch.
63
63
64
-
To create an initial PGStac database or bring an existing one up to date, check you have the pypgstac version installed you want to migrate to and run:
64
+
To create an initial PgSTAC database or bring an existing one up to date, check you have the pypgstac version installed you want to migrate to and run:
0 commit comments