Skip to content

Commit e9ad896

Browse files
authored
docs: migrate from MySQL in v0.9 (#1258)
1 parent 750ee06 commit e9ad896

File tree

3 files changed

+181
-1
lines changed

3 files changed

+181
-1
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# 从 MySQL 迁移
2+
3+
本文档将指引您完成从 MySQL 迁移到 GreptimeDB。
4+
5+
## 在开始迁移之前
6+
7+
请注意,尽管 GreptimeDB 支持 MySQL 的协议,并不意味着 GreptimeDB 实现了 MySQL
8+
的所有功能。你可以参考 "[ANSI 兼容性](/reference/sql/compatibility.md)" 查看在 GreptimeDB 中使用 SQL 的约束。
9+
10+
## 迁移步骤
11+
12+
### 在 GreptimeDB 中创建数据库和表
13+
14+
在从 MySQL 迁移数据之前,你首先需要在 GreptimeDB 中创建相应的数据库和表。
15+
由于 GreptimeDB 有自己的 SQL 语法用于创建表,因此你不能直接重用 MySQL 生成的建表 SQL。
16+
17+
当你为 GreptimeDB 编写创建表的 SQL 时,首先请了解其“[数据模型](/user-guide/concepts/data-model.md)”。然后,在创建表的
18+
SQL 中请考虑以下几点:
19+
20+
1. 由于 time index 列在表创建后无法更改,所以你需要仔细选择 time index
21+
列。时间索引最好设置为数据生成时的自然时间戳,因为它提供了查询数据的最直观方式,以及最佳的查询性能。例如,在 IOT
22+
场景中,你可以使用传感器采集数据时的时间作为 time index;或者在可观测场景中使用事件的发生时间。
23+
2. 不建议在此迁移过程中另造一个时间戳用作时间索引,例如使用 `DEFAULT current_timestamp()` 创建的新列。也不建议使用具有随机时间戳的列。
24+
3. 选择合适的 time index 精度也至关重要。和 time index 的选择一样,一旦表创建完毕,time index
25+
的精度就无法变更了。请根据你的数据集在[这里](/reference/sql/data-types#data-types-compatible-with-mysql-and-postgresql)
26+
找到最适合的时间戳类型。
27+
4. 根据您的查询模式选择最适合的 tag 列。tag 列存储经常被查询的元数据,其中的值是数据源的标签,通常用于描述数据的特征。tag 列具有索引,所以使用 tag 列的查询具备良好的性能。
28+
29+
请参考[CREATE](/reference/sql/create.md) SQL 文档,了解如何选择正确的数据类型以及“ttl”或“compaction”选项等。
30+
31+
### 双写 GreptimeDB 和 MySQL
32+
33+
双写 GreptimeDB 和 MySQL 是迁移过程中防止数据丢失的有效策略。通过使用 MySQL 的客户端库(JDBC + 某个 MySQL
34+
驱动),你可以建立两个客户端实例 —— 一个用于 GreptimeDB,另一个用于 MySQL。有关如何使用 SQL 将数据写入
35+
GreptimeDB,请参考[写入数据](/user-guide/ingest-data/for-iot/sql.md)部分。
36+
37+
若无需保留所有历史数据,你可以双写一段时间以积累业务所需的最新数据,然后停止向 MySQL 写入数据并仅使用
38+
GreptimeDB。如果需要完整迁移所有历史数据,请按照接下来的步骤操作。
39+
40+
### 从 MySQL 导出数据
41+
42+
[mysqldump](https://dev.mysql.com/doc/refman/8.4/en/mysqldump.html) 是一个常用的、从 MySQL 导出数据的工具。使用
43+
mysqldump,我们可以从 MySQL 中导出后续可直接导入到 GreptimeDB 的数据。例如,如果我们想要从 MySQL 导出两个数据库 `db1`
44+
`db2`,我们可以使用以下命令:
45+
46+
```bash
47+
mysqldump -h127.0.0.1 -P3306 -umysql_user -p --compact -cnt --skip-extended-insert --databases db1 db2 > /path/to/output.sql
48+
```
49+
50+
替换 `-h``-P``-u` 参数为 MySQL 服务的正确值。`--databases` 参数用于指定要导出的数据库。输出将写入
51+
`/path/to/output.sql` 文件。
52+
53+
`/path/to/output.sql` 文件应该具有如下内容:
54+
55+
```plaintext
56+
~ ❯ cat /path/to/output.sql
57+
58+
USE `db1`;
59+
INSERT INTO `foo` (`ts`, `a`, `b`) VALUES (1,'hello',1);
60+
INSERT INTO ...
61+
62+
USE `db2`;
63+
INSERT INTO `foo` (`ts`, `a`, `b`) VALUES (2,'greptime',2);
64+
INSERT INTO ...
65+
```
66+
67+
### 将数据导入 GreptimeDB
68+
69+
[MySQL Command-Line Client](https://dev.mysql.com/doc/refman/8.4/en/mysql.html) 可用于将数据导入
70+
GreptimeDB。继续上面的示例,假设数据导出到文件 `/path/to/output.sql`,那么我们可以使用以下命令将数据导入 GreptimeDB:
71+
72+
```bash
73+
mysql -h127.0.0.1 -P4002 -ugreptime_user -p -e "source /path/to/output.sql"
74+
```
75+
76+
替换 `-h``-P``-u` 参数为你的 GreptimeDB 服务的值。`source` 命令用于执行 `/path/to/output.sql` 文件中的 SQL
77+
命令。若需要进行 debug,添加 `-vvv` 以查看详细的执行结果。
78+
79+
总结一下,数据迁移步骤如下图所示:
80+
81+
![migrate mysql data steps](/migration-mysql.jpg)
82+
83+
数据迁移完成后,你可以停止向 MySQL 写入数据,并继续使用 GreptimeDB!
Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
# Migrate from MySQL
2+
3+
This document will guide you through the migration process from MySQL to GreptimeDB.
4+
5+
## Before you start the migration
6+
7+
Please be aware that though GreptimeDB supports the wire protocol of MySQL, it does not mean GreptimeDB implements all
8+
MySQL's features. You may refer to the "[ANSI Compatibility](/reference/sql/compatibility.md)" to see the
9+
constraints regarding using SQL in GreptimeDB.
10+
11+
## Migration steps
12+
13+
### Create the databases and tables in GreptimeDB
14+
15+
Before migrating the data from MySQL, you first need to create the corresponding databases and tables in GreptimeDB.
16+
GreptimeDB has its own SQL syntax for creating tables, so you cannot directly reuse the table creation SQLs that are produced
17+
by MySQL.
18+
19+
When you write the table creation SQL for GreptimeDB, it's important to understand
20+
its "[data model](/user-guide/concepts/data-model.md)" first. Then, please take the following considerations in
21+
your create table SQL:
22+
23+
1. Since the time index column cannot be changed after the table is created, you need to choose the time index column
24+
carefully. The time index is best set to the natural timestamp when the data is generated, as it provides the most
25+
intuitive way to query the data, and the best query performance. For example, in the IOT scenes, you can use the
26+
collection time of sensor data as the time index; or the occurrence time of an event in the observability scenes.
27+
2. In this migration process, it's not recommend to create another synthetic timestamp, such as a new column created
28+
with `DEFAULT current_timestamp()` as the time index column. It's not recommend to use the random timestamp as the
29+
time index either.
30+
3. It's vital to set the most fit timestamp precision for your time index column, too. Like the chosen of time index
31+
column, the precision of it cannot be changed as well. Find the most fit timestamp type for your
32+
data set [here](/reference/sql/data-types#data-types-compatible-with-mysql-and-postgresql).
33+
4. Choose the most fit tag columns based on your query patterns. The tag columns store the metadata that is
34+
commonly queried. The values in the tag columns are labels attached to the collected sources, generally used to
35+
describe a particular characteristic of these sources. The tag columns are indexed, making queries on them
36+
performant.
37+
38+
Finally please refer to "[CREATE](/reference/sql/create.md)" SQL document for more details for choosing the
39+
right data types and "ttl" or "compaction" options, etc.
40+
41+
### Write data to both GreptimeDB and MySQL simultaneously
42+
43+
Writing data to both GreptimeDB and MySQL simultaneously is a practical strategy to avoid data loss during migration. By
44+
utilizing MySQL's client libraries (JDBC + a MySQL driver), you can set up two client instances - one for GreptimeDB
45+
and another for MySQL. For guidance on writing data to GreptimeDB using SQL, please refer to the [Ingest Data](/user-guide/ingest-data/for-iot/sql.md) section.
46+
47+
If retaining all historical data isn't necessary, you can simultaneously write data to both GreptimeDB and MySQL for a
48+
specific period to accumulate the required recent data. Subsequently, cease writing to MySQL and continue exclusively
49+
with GreptimeDB. If a complete migration of all historical data is needed, please proceed with the following steps.
50+
51+
### Export data from MySQL
52+
53+
[mysqldump](https://dev.mysql.com/doc/refman/8.4/en/mysqldump.html) is a commonly used tool to export data from MySQL.
54+
Using it, we can export the data that can be later imported into GreptimeDB directly. For example, if we want to export
55+
two databases, `db1` and `db2` from MySQL, we can use the following command:
56+
57+
```bash
58+
mysqldump -h127.0.0.1 -P3306 -umysql_user -p --compact -cnt --skip-extended-insert --databases db1 db2 > /path/to/output.sql
59+
```
60+
61+
Replace the `-h`, `-P` and `-u` flags with the appropriate values for your MySQL server. The `--databases` flag is used
62+
to specify the databases to be exported. The output will be written to the `/path/to/output.sql` file.
63+
64+
The content in the `/path/to/output.sql` file should be like this:
65+
66+
```plaintext
67+
~ ❯ cat /path/to/output.sql
68+
69+
USE `db1`;
70+
INSERT INTO `foo` (`ts`, `a`, `b`) VALUES (1,'hello',1);
71+
INSERT INTO ...
72+
73+
USE `db2`;
74+
INSERT INTO `foo` (`ts`, `a`, `b`) VALUES (2,'greptime',2);
75+
INSERT INTO ...
76+
```
77+
78+
### Import data into GreptimeDB
79+
80+
The [MySQL Command-Line Client](https://dev.mysql.com/doc/refman/8.4/en/mysql.html) can be used to import data into
81+
GreptimeDB. Continuing the above example, say the data is exported to file `/path/to/output.sql`, then we can use the
82+
following command to import the data into GreptimeDB:
83+
84+
```bash
85+
mysql -h127.0.0.1 -P4002 -ugreptime_user -p -e "source /path/to/output.sql"
86+
```
87+
88+
Replace the `-h`, `-P` and `-u` flags with the appropriate values for your GreptimeDB server. The `source` command is
89+
used to execute the SQL commands in the `/path/to/output.sql` file. Add `-vvv` to see the detailed execution results for
90+
debugging purpose.
91+
92+
To summarize, data migration steps can be illustrate as follows:
93+
94+
![migrate mysql data steps](/migration-mysql.jpg)
95+
96+
After the data migration is completed, you can stop writing data to MySQL and continue using GreptimeDB exclusively!

versioned_sidebars/version-0.9-sidebars.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,8 @@
130130
"type": "category",
131131
"label": "Migrate to GreptimeDB",
132132
"items": [
133-
"user-guide/migrate-to-greptimedb/migrate-from-influxdb"
133+
"user-guide/migrate-to-greptimedb/migrate-from-influxdb",
134+
"user-guide/migrate-to-greptimedb/migrate-from-mysql"
134135
]
135136
},
136137
{

0 commit comments

Comments
 (0)