Skip to content

Commit 295903d

Browse files
Nicoretticpcloud
Nicoretti
authored and
cpcloud
committed
feat(exasol): add exasol backend
1 parent bf6ed1a commit 295903d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+1175
-96
lines changed

‎.github/workflows/ibis-backends.yml

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -159,6 +159,13 @@ jobs:
159159
- oracle
160160
services:
161161
- oracle
162+
- name: exasol
163+
title: Exasol
164+
serial: true
165+
extras:
166+
- exasol
167+
services:
168+
- exasol
162169
- name: flink
163170
title: Flink
164171
serial: true
@@ -283,6 +290,15 @@ jobs:
283290
- flink
284291
services:
285292
- flink
293+
- os: windows-latest
294+
backend:
295+
name: exasol
296+
title: Exasol
297+
serial: true
298+
extras:
299+
- exasol
300+
services:
301+
- exasol
286302
steps:
287303
- name: update and install system dependencies
288304
if: matrix.os == 'ubuntu-latest' && matrix.backend.sys-deps != null
@@ -604,7 +620,7 @@ jobs:
604620
- run: python -m pip install --upgrade pip 'poetry==1.7.1'
605621

606622
- name: remove deps that are not compatible with sqlalchemy 2
607-
run: poetry remove snowflake-sqlalchemy
623+
run: poetry remove snowflake-sqlalchemy sqlalchemy-exasol
608624

609625
- name: add sqlalchemy 2
610626
run: poetry add --lock --optional 'sqlalchemy>=2,<3'

‎ci/schema/exasol.sql

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
DROP SCHEMA IF EXISTS EXASOL CASCADE;
2+
CREATE SCHEMA EXASOL;
3+
4+
CREATE OR REPLACE TABLE EXASOL.diamonds
5+
(
6+
"carat" DOUBLE,
7+
"cut" VARCHAR(256),
8+
"color" VARCHAR(256),
9+
"clarity" VARCHAR(256),
10+
"depth" DOUBLE,
11+
"table" DOUBLE,
12+
"price" BIGINT,
13+
"x" DOUBLE,
14+
"y" DOUBLE,
15+
"z" DOUBLE
16+
);
17+
18+
CREATE OR REPLACE TABLE EXASOL.batting
19+
(
20+
"playerID" VARCHAR(256),
21+
"yearID" BIGINT,
22+
"stint" BIGINT,
23+
"teamID" VARCHAR(256),
24+
"logID" VARCHAR(256),
25+
"G" BIGINT,
26+
"AB" BIGINT,
27+
"R" BIGINT,
28+
"H" BIGINT,
29+
"X2B" BIGINT,
30+
"X3B" BIGINT,
31+
"HR" BIGINT,
32+
"RBI" BIGINT,
33+
"SB" BIGINT,
34+
"CS" BIGINT,
35+
"BB" BIGINT,
36+
"SO" BIGINT,
37+
"IBB" BIGINT,
38+
"HBP" BIGINT,
39+
"SH" BIGINT,
40+
"SF" BIGINT,
41+
"GIDP" BIGINT
42+
);
43+
44+
CREATE OR REPLACE TABLE EXASOL.awards_players
45+
(
46+
"playerId" VARCHAR(256),
47+
"awardID" VARCHAR(256),
48+
"yearID" VARCHAR(256),
49+
"logID" VARCHAR(256),
50+
"tie" VARCHAR(256),
51+
"notest" VARCHAR(256)
52+
);
53+
54+
CREATE OR REPLACE TABLE EXASOL.functional_alltypes
55+
(
56+
"id" INTEGER,
57+
"bool_col" BOOLEAN,
58+
"tinyint_col" SHORTINT,
59+
"small_int" SMALLINT,
60+
"int_col" INTEGER,
61+
"bigint_col" BIGINT,
62+
"float_col" FLOAT,
63+
"double_col" DOUBLE PRECISION,
64+
"date_string_col" VARCHAR(256),
65+
"string_col" VARCHAR(256),
66+
"timestamp_col" TIMESTAMP,
67+
"year" INTEGER,
68+
"month" INTEGER
69+
);
70+
71+
72+
IMPORT INTO EXASOL.diamonds FROM LOCAL CSV FILE '/data/diamonds.csv' COLUMN SEPARATOR = ',' SKIP = 1;
73+
IMPORT INTO EXASOL.batting FROM LOCAL CSV FILE '/data/batting.csv' COLUMN SEPARATOR = ',' SKIP = 1;
74+
IMPORT INTO EXASOL.awards_players FROM LOCAL CSV FILE '/data/awards_players.csv' COLUMN SEPARATOR = ',' SKIP = 1;
75+
IMPORT INTO EXASOL.functional_alltypes FROM LOCAL CSV FILE '/data/functional_alltypes.csv' COLUMN SEPARATOR = ',' SKIP = 1;

‎compose.yaml

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -412,6 +412,23 @@ services:
412412
volumes:
413413
- oracle:/opt/oracle/data
414414

415+
exasol:
416+
image: exasol/docker-db:7.1.23
417+
privileged: true
418+
ports:
419+
- 8563:8563
420+
healthcheck:
421+
interval: 10s
422+
retries: 9
423+
timeout: 90s
424+
test:
425+
- CMD-SHELL
426+
- /usr/opt/EXASuite-7/EXASolution-7.1.23/bin/Console/exaplus -c 127.0.0.1:8563 -u sys -p exasol -encryption OFF <<< 'SELECT 1'
427+
networks:
428+
- exasol
429+
volumes:
430+
- exasol:/data
431+
415432
flink-jobmanager:
416433
build: ./docker/flink
417434
image: ibis-flink
@@ -452,6 +469,7 @@ networks:
452469
trino:
453470
druid:
454471
oracle:
472+
exasol:
455473
flink:
456474

457475
volumes:
@@ -469,3 +487,4 @@ volumes:
469487
oracle:
470488
postgres:
471489
minio:
490+
exasol:

‎docs/backends/exasol.qmd

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Exasol
2+
3+
[https://www.exasol.com](https://www.exasol.com)
4+
5+
## Install
6+
7+
Install Ibis and dependencies for the Exasol backend:
8+
9+
::: {.panel-tabset}
10+
11+
## `pip`
12+
13+
Install with the `exasol` extra:
14+
15+
```{.bash}
16+
pip install 'ibis-framework[exasol]'
17+
```
18+
19+
And connect:
20+
21+
```{.python}
22+
import ibis
23+
24+
con = ibis.exasol.connect(...) # <1>
25+
```
26+
27+
1. Adjust connection parameters as needed.
28+
29+
## `conda`
30+
31+
Install for Exasol:
32+
33+
```{.bash}
34+
conda install -c conda-forge ibis-exasol
35+
```
36+
37+
And connect:
38+
39+
```{.python}
40+
import ibis
41+
42+
con = ibis.exasol.connect(...) # <1>
43+
```
44+
45+
1. Adjust connection parameters as needed.
46+
47+
## `mamba`
48+
49+
Install for Exasol:
50+
51+
```{.bash}
52+
mamba install -c conda-forge ibis-exasol
53+
```
54+
55+
And connect:
56+
57+
```{.python}
58+
import ibis
59+
60+
con = ibis.exasol.connect(...) # <1>
61+
```
62+
63+
1. Adjust connection parameters as needed.
64+
65+
:::
66+
67+
## Connect
68+
69+
### `ibis.exasol.connect`
70+
71+
```python
72+
con = ibis.exasol.connect(
73+
user = "username",
74+
password = "password",
75+
host = "localhost",
76+
port = 8563,
77+
schema = None,
78+
encryption = True,
79+
certificate_validation = True,
80+
encoding = "en_US.UTF-8"
81+
)
82+
```
83+
84+
::: {.callout-note}
85+
`ibis.exasol.connect` is a thin wrapper around [`ibis.backends.exasol.Backend.do_connect`](#ibis.backends.exasol.Backend.do_connect).
86+
:::
87+
88+
### Connection Parameters
89+
90+
```{python}
91+
#| echo: false
92+
#| output: asis
93+
from _utils import render_do_connect
94+
95+
render_do_connect("exasol")
96+
```

‎ibis/backends/base/__init__.py

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,16 +33,16 @@
3333

3434
__all__ = ("BaseBackend", "Database", "connect")
3535

36-
3736
_IBIS_TO_SQLGLOT_DIALECT = {
3837
"mssql": "tsql",
3938
"impala": "hive",
4039
"pyspark": "spark",
4140
"polars": "postgres",
4241
"datafusion": "postgres",
42+
# closest match see https://github.com/ibis-project/ibis/pull/7303#discussion_r1350223901
43+
"exa.websocket": "oracle",
4344
}
4445

45-
4646
_SQLALCHEMY_TO_SQLGLOT_DIALECT = {
4747
# sqlalchemy dialects of backends not listed here match the sqlglot dialect
4848
# name
@@ -52,6 +52,8 @@
5252
# druid allows double quotes for identifiers, like postgres:
5353
# https://druid.apache.org/docs/latest/querying/sql#identifiers-and-literals
5454
"druid": "postgres",
55+
# closest match see https://github.com/ibis-project/ibis/pull/7303#discussion_r1350223901
56+
"exa.websocket": "oracle",
5557
}
5658

5759

‎ibis/backends/base/sql/alchemy/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -554,7 +554,7 @@ def _schema_from_sqla_table(
554554
dtype = schema[name]
555555
else:
556556
dtype = cls.compiler.translator_class.get_ibis_type(
557-
column.type, nullable=column.nullable
557+
column.type, nullable=column.nullable or column.nullable is None
558558
)
559559
pairs.append((name, dtype))
560560
return sch.schema(pairs)

‎ibis/backends/conftest.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -539,6 +539,7 @@ def ddl_con(ddl_backend):
539539
params=_get_backends_to_test(
540540
keep=(
541541
"duckdb",
542+
"exasol",
542543
"mssql",
543544
"mysql",
544545
"oracle",

0 commit comments

Comments
 (0)