[ELE-489] Support dbt-trino connector

[radium226]

Is your feature request related to a problem? Please describe.
At $WORK, I heavily leverage on Trino to use multiple data sources within my DBT projects (using the dbt-trino connector).

It's also quite useful for storing DBT's test results in a different database than the analytical one (by using the --store-failures flag combined with the +schema and +database config).

Describe the solution you'd like
Elementary is IMHO a better solution than the raw --store-failures flag and it could be great to be able to make it work with the dbt-trino connector.

Describe alternatives you've considered
For now, none :) I'm still using the --store-failures flag but without the proper visualization, I can't make it very useful for the rest of the teams.

Additional context
Nope.

Would you be willing to contribute this feature?
Totally! I started to dig a little bit (I seached postgres__ to see what kind of macro needs to be implemented) but I need guidance on how to start and how to test.

_ELE-489

[Maayan-s]

Hi @radium226!
We would be happy to support your effort in making Elementary compatible with Trino.
To be honest I'm not familiar with it so I don't know how hard it would be.

Generally speaking, we implemented every platform-specific functionality using the adapter.dispatch functionality, as dbt recommends. You can see an example in this macro.
However, where there was a dbt_utils macro that we could use, we did. I do see utils in the dbt-trino adapter, so it looks promising in that sense.
Anyway, You can see here a workaround we did for a missing util in Spark.

I think you should approach it gradually -
Step 1 - Add support for uploading dbt artifacts and run results (in the dbt package).
Step 2 - Add support in the CLI for Slack alerts and UI generation.
Step 3 - Add support for data anomaly detection test (the most complex and platform-specific part of the code right now).

You can see here a guide for testing:
https://docs.elementary-data.com/general/contributions#contributing-to-the-dbt-package

[leniartek]

Hello @radium226
Happy to hear you benefit from Trino! (dbt-trino PM here).

This is the first time I see a request for Elementary integration, I will take a closer look.

Thanks for reaching out.
Piotr

[ndrluis]

Hello @leniartek and @Maayan-s,

In the coming weeks, I will be dedicated to implementing this integration, as we need Elementary working with Trino.

@leniartek, I would be very grateful if you have already identified any requirements related to this integration and could share them with me, so I can have a better idea of what I need to focus on.

In any case, I am available if you require help with anything.

[haritamar]

Hi @ndrluis !

That is great to hear.
FYI that in the upcoming Elementary release we are going to add Athena support thanks to the contribution of @artem-garmash with the help of @nicor88.
I know that Athena v3 is based on Trino so I'm wondering if there will be common code to both.

Also a couple of notes:

• Please see our contribution guide here
• Please see especially the note about integration tests in the dbt package - in particular these should help you to verify various Elementary features work as expected.
• I'll be more than happy to jump on a call to help you get started / assist in any other way.

Cheers,
Itamar