Proposal to Decouple Routing Rule Definitions from Storage and Enabling Database Storage

[Peiyingy]

Currently, the routing rule definitions and their storage are tightly coupled within Trino Gateway. The system only supports either config file-based rules (using MVEL expressions) or entirely external service-based routing rules, each introducing its own set of limitations.

Config File Based Routing Rules

• Deployment Dependency: Updating rule configurations requires a new deployment.

External Service Based Routing Rules

• Infrastructure Overhead: The use of an additional service introduces extra maintenance issues.

Proposal

This proposal suggests decoupling the abstract definition of routing rules from the storage mechanism and introducing a database for routing rules storage. (This approach is also mentioned in GitHub PR #540: "Database storage for MVEL rules will be added as a follow-up"). This decoupling allows for independent evolution of both aspects:

Decoupled Abstraction for Rule Definitions

• Common Core Fields: Every rule can share a set of common fields such as rule name, description, and priority.
• Extension for Specific Rule Types:
  • MVEL Rules: Extend with fields like condition and actions.
  • Value-Based Rules: Extend with fields like username, destination_routing_group, etc.
  • Mixed Rule Types: This abstraction enables us to mix rule types - using value-based rules for simplicity and MVEL rules for more complex logic - within the same framework.

Flexible Storage Options

• Database Storage: Rules can be stored in a database to leverage benefits such as rapid updates and enhanced validity checks.
• Config File Storage Solutions: Rules can be stored in YAML files if that better fits the use case, supporting a diverse range of storage strategies.

LinkedIn Experience

At LinkedIn, we implemented Presto-Gateway with a routing rules table stored in a database. Based on that experience, this simplifies management and introduces flexibility and agility. This change will benefit our open-source community by providing a more adaptable, maintainable, and robust routing rule system.

[Chaho12]

Yes, please to submit PR on this. @oneonestar @vishalya wdyt?