Skip to content

Commit fa6ebdf

Browse files
authored
📝 Source postgres: add workaround to sync a subset of columns (#10024)
* Add workaround for partial table permission issue in doc * Update doc * Update doc * Update doc * Remove dedundant sentence * Update doc
1 parent 1b1b94e commit fa6ebdf

File tree

1 file changed

+18
-5
lines changed

1 file changed

+18
-5
lines changed

docs/integrations/sources/postgres.md

Lines changed: 18 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -37,18 +37,18 @@ This is dependent on your networking setup. The easiest way to verify if Airbyte
3737

3838
#### 2. Create a dedicated read-only user with access to the relevant tables \(Recommended but optional\)
3939

40-
This step is optional but highly recommended to allow for better permission control and auditing. Alternatively, you can use Airbyte with an existing user in your database.
40+
This step is optional but highly recommended for better permission control and auditing. Alternatively, you can use Airbyte with an existing user in your database.
4141

4242
To create a dedicated database user, run the following commands against your database:
4343

4444
```sql
45-
CREATE USER airbyte PASSWORD 'your_password_here';
45+
CREATE USER <username> PASSWORD 'your_password_here';
4646
```
4747

4848
Then give it access to the relevant schema:
4949

5050
```sql
51-
GRANT USAGE ON SCHEMA <schema_name> TO airbyte
51+
GRANT USAGE ON SCHEMA <schema_name> TO <username>
5252
```
5353

5454
Note that to replicate data from multiple Postgres schemas, you can re-run the command above to grant access to all the relevant schemas, but you'll need to set up multiple sources connecting to the same db on multiple schemas.
@@ -58,10 +58,23 @@ Next, grant the user read-only access to the relevant tables. The simplest way i
5858
```sql
5959
GRANT SELECT ON ALL TABLES IN SCHEMA <schema_name> TO airbyte;
6060

61-
# Allow airbyte user to see tables created in the future
62-
ALTER DEFAULT PRIVILEGES IN SCHEMA <schema_name> GRANT SELECT ON TABLES TO airbyte;
61+
-- Allow user to see tables created in the future
62+
ALTER DEFAULT PRIVILEGES IN SCHEMA <schema_name> GRANT SELECT ON TABLES TO <username>;
6363
```
6464

65+
Currently, there is no way to sync a subset of columns using the Postgres source connector.
66+
- When setting up a connection, you can only choose which tables to sync, but not columns (issue [#2227](https://github.com/airbytehq/airbyte/issues/2227)).
67+
- If the user account can only access a subset of columns (i.e. has no `SELECT` permission for the full table), the connection check will pass. However, the data sync will fail with permission denied exception (issue [#9771](https://github.com/airbytehq/airbyte/issues/9771)).
68+
69+
The short-term workaround for both issues is to create a view on the specific columns, and grant the user read permission of that view:
70+
71+
```sql
72+
CREATE VIEW <view_name> as SELECT <columns> FROM <table>;
73+
GRANT SELECT ON TABLE <view_name> IN SCHEMA <schema_name> to <user_name>;
74+
```
75+
76+
A bug related to partial table permission is track in [issue #9771](https://github.com/airbytehq/airbyte/issues/9771).
77+
6578
#### 3. Optionally, set up CDC. Follow the guide [below](postgres.md#setting-up-cdc-for-postgres) to do so.
6679

6780
#### 4. That's it!

0 commit comments

Comments
 (0)