chore: Refine the project structure and readme

.github/dependabot.yml

@@ -1,7 +1,7 @@
 version: 2
 updates:
   - package-ecosystem: "maven"
-    directory: "/"
+    directory: "/wren-core-legacy"
     schedule:
       interval: "weekly"
     groups:
@@ -51,3 +51,16 @@ updates:
       - "dependencies"
       - "core"
       - "rust"
+  - package-ecosystem: "cargo"
+    directory: "/wren-core-base"
+    schedule:
+      interval: "weekly"
+    groups:
+      all:
+        patterns: [ "*" ]
+    commit-message:
+      prefix: "deps(core)"
+    labels:
+      - "dependencies"
+      - "core"
+      - "rust"

.github/workflows/build-image.yml

@@ -49,6 +49,7 @@ jobs:
           java-version: '21'
           cache: 'maven'
       - name: Build
+        working-directory: wren-core-legacy
         run: |
           ./mvnw clean install -B -DskipTests -P exec-jar
       - name: Docker meta
@@ -69,6 +70,7 @@ jobs:
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v3
       - name: Prepare
+        working-directory: wren-core-legacy
         id: prepare
         run: |
           WREN_VERSION=$(./mvnw --quiet help:evaluate -Dexpression=project.version -DforceStdout)
@@ -77,7 +79,7 @@ jobs:
       - name: Build and push
         uses: docker/build-push-action@v5
         with:
-          context: ./docker
+          context: ./wren-core-legacy/docker
           platforms: linux/amd64,linux/arm64
           tags: ${{ steps.meta.outputs.tags }}
           build-args: |

.github/workflows/ibis-ci.yml

@@ -27,7 +27,7 @@ jobs:
           java-version: '21'
           cache: 'maven'
       - name: Start Wren JAVA engine
-        working-directory: .
+        working-directory: ./wren-core-legacy
         run: |
           mkdir etc
           echo "node.environment=production" >> etc/config.properties

.github/workflows/maven-tests.yml

@@ -9,6 +9,12 @@ on:
       - reopened
       - labeled
       - unlabeled
+    paths:
+      - 'wren-core-legacy/**'
+
+defaults:
+  run:
+    working-directory: wren-core-legacy
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.event.number }}

.github/workflows/stable-release.yml

@@ -93,6 +93,7 @@ jobs:
           java-version: '21'
           cache: 'maven'
       - name: Build
+        working-directory: wren-core-legacy
         run: |
           ./mvnw clean install -B -DskipTests -P exec-jar
       - name: Docker meta
@@ -114,6 +115,7 @@ jobs:
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@v3
       - name: Prepare
+        working-directory: wren-core-legacy
         id: prepare
         run: |
           WREN_VERSION=$(./mvnw --quiet help:evaluate -Dexpression=project.version -DforceStdout)
@@ -122,7 +124,7 @@ jobs:
       - name: Build and push
         uses: docker/build-push-action@v5
         with:
-          context: ./docker
+          context: ./wren-core-legacy/docker
           platforms: linux/amd64,linux/arm64
           tags: ${{ steps.meta.outputs.tags }}
           build-args: |

.gitignore

@@ -3,7 +3,7 @@
 *.ipr
 *.iws
 target/
-.idea
+**/.idea
 .run
 .DS_Store
 .classpath

README.md

@@ -81,6 +81,13 @@ https://github.com/user-attachments/assets/dab9b50f-70d7-4eb3-8fc8-2ab55dc7d2ec
 ## 🚧 Project Status
 Wren Engine is currently in the beta version. The project team is actively working on progress and aiming to release new versions at least biweekly.
 
+## 🛠️ Developer GuideS
+The project consists of 4 main modules:
+1. [ibis-server](./ibis-server/): the Web server of Wren Engine powered by FastAPI and Ibis
+2. [wren-core](./wren-cores): the semantic core written in Rust powered by [Apache DataFusion](https://github.com/apache/datafusion)
+3. [wren-core-py](./wren-core-py): the Python binding for wren-core
+4. [mcp-server](./mcp-server/): the MCP server of Wren Engine powered by [MCP Python SDK](https://github.com/modelcontextprotocol/python-sdk)
+
 ## ⭐️ Community
 
 - Welcome to our [Discord server](https://discord.gg/5DvshJqG8Z) to give us feedback!

ibis-server/README.md

@@ -1,18 +1,12 @@
-# Ibis server
-Ibis server is a server base on Python that provides the API for rewriting the queries with [Modeling Definition Language (MDL)](https://docs.getwren.ai/engine/concept/what_is_mdl) manifest and connect data source via [Ibis](https://github.com/ibis-project/ibis). It is built on top of the [FastAPI](https://github.com/tiangolo/fastapi) framework. \
-We still need to run the Java engine for rewriting the queries. In the future, we will redesign the modeling core of java engine to [Rust](https://github.com/rust-lang/rust) and integrate the modeling core into the Ibis server. \
-We continuously integrate the Rust core with the Ibis server now.
-
-## Application structure
-The application consists of three main parts:
-1. [ibis-server](./): a Python web server powered by FastAPI and Ibis
-2. [wren-core](../wren-cores): a modeling core written in Rust powered by [Apache DataFusion](https://github.com/apache/datafusion)
-3. [wren-core-py](../wren-core-py): a Python adapter for the modeling core
+# Ibis Server Module
+This module is the API server of Wren Engine. It's built on top of [FastAPI](https://fastapi.tiangolo.com/). It provides several APIs for SQL queries. A SQL query will be planned by [wren-core](../wren-core/), transpiled by [sqlglot](https://github.com/tobymao/sqlglot), and then executed by [ibis](https://github.com/ibis-project/ibis) to query the database.
 
 ## Quick Start
 
 ### Running on Docker
 You can follow the steps below to run the Java engine and ibis.
+> Wren Engine is migrating to [wren-core](../wren-core/). However, we still recommend starting [the Java engine](../wren-core-legacy/) to enable the query fallback mechanism.
+
 Create `compose.yaml` file and add the following content, edit environment variables if needed (see [Environment Variables](docs/development#environment-variables))
 ```yaml
 services:
@@ -38,7 +32,6 @@ vim config.properties
 Add the following content to the `config.properties` file
 ```bash
 node.environment=production
-wren.directory=/usr/src/app/etc/mdl
 ```
 Run the docker compose
 ```bash
@@ -79,7 +72,7 @@ just run
 ```
 
 ### Enable Tracing
-We uses OpenTelemetry as its tracing framework. Refer to OpenTelemetry zero-code instrumentation to install the required dependencies.
+We use OpenTelemetry as its tracing framework. Refer to OpenTelemetry zero-code instrumentation to install the required dependencies.
 Then, use the following just command to start the Ibis server, which exports tracing logs to the console:
 ```
 just run-trace-console

wren-core-legacy/README.md

@@ -0,0 +1,37 @@
+# Legacy Wren Core
+It's the legacy version of the Wren core implemented in Java. This module is utilized by the API v2 of the ibis-server for SQL planning and is responsible for building the `wren-engine` Docker image.
+Currently, Wren engine has been migrated to [a new Rust implementation](../wren-core/) 🚀.
+
+## Requirements
+- JDK 21+
+
+## Running Wren Core Server
+We recommend running Wren core server using Docker. It's the easiest way to start the wren core server:
+```
+docker run --name java-engine -p 8080:8080 -v $(pwd)/docker/etc:/usr/src/app/etc ghcr.io/canner/wren-engine:latest  
+```
+
+### Maven Build
+For developing, you can build Wren core by yourself. Wren core is a standard maven project. We can build an executable jar using the following command:
+```
+./mvnw clean install -DskipTests -P exec-jar
+```
+Then, start Wren core server
+```
+java -Dconfig=docker/etc/config.properties --add-opens=java.base/java.nio=ALL-UNNAMED -jar wren-server/target/wren-server-0.15.2-SNAPSHOT-executable.jar
+```
+
+### Running Wren Engine in IDE
+After building with Maven, you can run the project in your IDE. We recommend using [IntelliJ IDEA](http://www.jetbrains.com/idea/). Since Wren core is a standard Maven project, you can easily import it into your IDE. In IntelliJ, choose `Open Project from the Quick Start` box or select `Open` from the File menu and choose the root `pom.xml` file.
+
+After opening the project in IntelliJ, ensure that the Java SDK is properly configured for the project:
+
+1. Open the File menu and select **Project Structure**.
+2. In the **SDKs** section, ensure that JDK 21 is selected (create one if it does not exist).
+3. In the **Project** section, ensure the Project language level is set to 21.
+
+Set up the running profile with the following configuration:
+- **SDK**: The JDK you configured.
+- **Main class**: `io.wren.server.WrenServer`
+- **VM options**: `-Dconfig=docker/etc/config.properties`
+- **Working directory**: The path to `wren-core-legacy`.