Skip to content

Commit 80ff3a9

Browse files
PavelShilin89Pavel_Shilin
PavelShilin89
and
Pavel_Shilin
authored
SELECT does not wait for FREEZE + UPDATE (#2805)
* Added a new case that SELECT does not wait for FREEZE + UPDATE * Changing manticore-load output to json --------- Co-authored-by: Pavel_Shilin <[email protected]>
1 parent f02cb67 commit 80ff3a9

File tree

3 files changed

+82
-53
lines changed

3 files changed

+82
-53
lines changed

‎test/clt-tests/base/searchd-with-flexible-ports.conf

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ searchd {
1414
data_dir = /var/log/manticore-${INSTANCE}
1515
query_log_format = sphinxql
1616
query_log_commands = 1
17+
server_id = ${INSTANCE}
1718
}
1819
1920
EOF

‎test/clt-tests/core/show-settings.rec

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ mysql -h0 -P1306 -e "SHOW SETTINGS;"
1919
| searchd.data_dir | /var/log/manticore-1 |
2020
| searchd.query_log_format | sphinxql |
2121
| searchd.query_log_commands | 1 |
22+
| searchd.server_id | 1 |
2223
| searchd.binlog_path | /var/log/manticore-1/binlog |
2324
| common.plugin_dir | /usr/local/lib/manticore |
2425
| common.lemmatizer_base | /usr/share/manticore/ |

‎test/clt-tests/core/test-freeze-unfreeze-update.rec

Lines changed: 80 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -1,71 +1,98 @@
1+
––– block: ../base/start-searchd-with-buddy –––
12
––– input –––
2-
echo -e "searchd {\n listen = 9315:mysql\n listen = 9316\n log = /tmp/searchd.log\n pid_file = /tmp/searchd.pid\n data_dir = /tmp/data\n watchdog = 0\n}" > /tmp/116.conf
3+
manticore-load --quiet --json --init="CREATE TABLE test (id bigint, usertest text stored indexed, category string, age int, salary float, discount float, is_active bool, last_login timestamp, product_codes multi, large_values multi64, additional_info json, location_vector float_vector)" --load="INSERT INTO test (id, usertest, category, age, salary, discount, is_active, last_login, product_codes, large_values, additional_info, location_vector) VALUES (<increment>, '<text/5/20>', '<text/5/10>', <int/1/100>, <float/1.0/100.0>, <float/0.0/10.0>, <int/0/1>, <int/1609459200/1704067200>, (<array/1/10/1/100>), (<array/1/10/-100000/100000>), '{\"key\":\"value\"}', (<array/4/4/1.0/10.0>))" --batch-size=10000 --threads=4 --total=1000000
34
––– output –––
5+
{
6+
"threads": %{NUMBER},
7+
"batch_size": %{NUMBER},
8+
"time": "%{NUMBER}:%{NUMBER}",
9+
"total_operations": %{NUMBER},
10+
"operations_per_second": %{NUMBER},
11+
"qps": {
12+
"avg": %{NUMBER},
13+
"p99": %{NUMBER},
14+
"p95": %{NUMBER},
15+
"p5": %{NUMBER},
16+
"p1": %{NUMBER}
17+
},
18+
"latency": {
19+
"avg": %{NUMBER}.%{NUMBER},
20+
"p50": %{NUMBER},
21+
"p95": %{NUMBER},
22+
"p99": %{NUMBER}
23+
}
24+
}
425
––– input –––
5-
mkdir -p /tmp/data; rm -f /tmp/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd -c /tmp/116.conf > /dev/null
26+
mysql -h0 -P9306 -e "SELECT * FROM test ORDER BY id ASC LIMIT 100000 OPTION max_matches=100000;" > /tmp/test_data.txt
627
––– output –––
728
––– input –––
8-
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /tmp/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /tmp/searchd.log;fi
29+
md5sum /tmp/test_data.txt
930
––– output –––
10-
Buddy started!
31+
99494585e9f5bad83f5f6ec41a62d292 /tmp/test_data.txt
1132
––– input –––
12-
mysql -P9315 -h0 -e "drop table if exists t; create table t(s string); insert into t values(1, 'a'); flush ramchunk t"; echo $?
33+
mysql -h0 -P9306 -e "SHOW TABLES"
1334
––– output –––
14-
0
35+
+-------+------+
36+
| Table | Type |
37+
+-------+------+
38+
| test | rt |
39+
+-------+------+
1540
––– input –––
16-
mysql -P9315 -h0 -e "select * from t;"
41+
mysql -P9306 -h0 -e "SELECT COUNT(*) FROM test;"
1742
––– output –––
18-
+------+------+
19-
| id | s |
20-
+------+------+
21-
| 1 | a |
22-
+------+------+
43+
+----------+
44+
| count(*) |
45+
+----------+
46+
| 1000000 |
47+
+----------+
2348
––– input –––
24-
(rm -f /tmp/update.sql; for n in `seq 1 10000`; do echo "update t set s='b' where id=1;" >> /tmp/update.sql; done; nohup sh -c "while true; do mysql -P9315 -h0 < /tmp/update.sql | break; done" &>/dev/null &) 2>/dev/null
49+
mysql -P9306 -h0 -e "DESCRIBE test;"
2550
––– output –––
51+
+-----------------+--------------+----------------+
52+
| Field | Type | Properties |
53+
+-----------------+--------------+----------------+
54+
| id | bigint | |
55+
| usertest | text | indexed stored |
56+
| category | string | |
57+
| age | uint | |
58+
| salary | float | |
59+
| discount | float | |
60+
| is_active | bool | |
61+
| last_login | timestamp | |
62+
| product_codes | mva | |
63+
| large_values | mva64 | |
64+
| additional_info | json | |
65+
| location_vector | float_vector | |
66+
+-----------------+--------------+----------------+
2667
––– input –––
27-
sleep 30; mysql -P9315 -h0 -e "select * from t;"
68+
mysql -h0 -P9306 -e "SELECT * FROM test ORDER BY id ASC LIMIT 10;"
2869
––– output –––
29-
+------+------+
30-
| id | s |
31-
+------+------+
32-
| 1 | b |
33-
+------+------+
70+
+------+-------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+----------------------------------------------------------------+-----------------+---------------------------------------+
71+
| id | usertest | category | age | salary | discount | is_active | last_login | product_codes | large_values | additional_info | location_vector |
72+
+------+-------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+----------------------------------------------------------------+-----------------+---------------------------------------+
73+
| 1 | Incorrectly heavy soft carefully late sit; put draw afraid; gave hardly. | Been vaguely 1000 this at understood gave always east fake. | 8 | 20.299999 | 1.500000 | 0 | 1676104873 | 10,16,40,71,85,87,100 | -88054,-30114,-15979,5487,72220,72277,91879,93074,97442 | {"key":"value"} | 2.000000,8.000000,10.000000,5.000000 |
74+
| 2 | Called, afraid foolish! happy? busy taken dead shall make seen privately understood short universally. | Was? seen feel economically beautiful legally. | 30 | 86.900002 | 4.900000 | 1 | 1697110603 | 1,40,47,48,52,66,80,93 | -87441,-74083,-38090,-21342,-13596,49456,57513,90976,92580 | {"key":"value"} | 6.000000,10.000000,6.000000,3.000000 |
75+
| 3 | Calm incorrectly modern obviously soft certainly, quietly need tell poor spring evening drove young? lost. | Probably came knew felt has. | 47 | 21.600000 | 6.700000 | 0 | 1678595569 | 3 | 26497,73063,73768,87840 | {"key":"value"} | 10.000000,5.000000,9.000000,10.000000 |
76+
| 4 | Cruel; emotionally beautiful modern the. Thirsty do poor dead; lazy let sick; kept angry politically closed understand. | Regularly became? fell. Brought with stand south. | 3 | 4.200000 | 7.600000 | 0 | 1650601281 | 24,28,62 | -99076 | {"key":"value"} | 4.000000,8.000000,8.000000,3.000000 |
77+
| 5 | Scientifically sad afraid, meet positively! were spent hard. | Get wrote excited deep suddenly foolish they unsuccessfully. | 34 | 4.600000 | 5.700000 | 1 | 1672897587 | 7,9,10,16,29,36,52,82,99 | 23225,29377 | {"key":"value"} | 10.000000,2.000000,7.000000,1.000000 |
78+
| 6 | Happy certainly, carelessly scientifically lie saw grew, hurt hot full monthly cut may calm be sharp. | Cut serious; day! harshly regionally bitter happy quickly stale the. | 65 | 92.599998 | 4.500000 | 0 | 1699959055 | 3,20,74,92 | -58459,-31703,-8868,-8398,1620,24456,76572,79957,86553,95523 | {"key":"value"} | 7.000000,10.000000,8.000000,3.000000 |
79+
| 7 | Easily often set hot new it become. | Be quiet cold nightly poor. | 59 | 19.400000 | 6.000000 | 1 | 1685126023 | 6,11,34,35,47,60,86,98 | -54811,-28523,-16029,-8383,266,6217,31166,67799,99092 | {"key":"value"} | 9.000000,2.000000,5.000000,5.000000 |
80+
| 8 | Politically grew rich loud tried seen. Nowhere am salty dawn quietly? gentle permanently gone? drew kept. | Big specifically weak held she; constantly send down spiritually. | 15 | 34.700001 | 1.000000 | 0 | 1647818588 | 77,86 | -55160,-5014,31069,31847,34334,36035,61084,62322,81203,95016 | {"key":"value"} | 4.000000,7.000000,4.000000,7.000000 |
81+
| 9 | Far, usually light west educationally. Fake wet salty wrong maybe! does went usually? humble send are. | Economically commonly wrote specifically healthy wrong culturally? 4, small rich. | 51 | 4.400000 | 0.500000 | 1 | 1611948425 | 2,5,16,18,35,38,44,79,95 | -77137,-67602,-19932,-10666,3139,22804,45013,65922,71495,87473 | {"key":"value"} | 1.000000,5.000000,4.000000,10.000000 |
82+
| 10 | We nowhere bad hold! happily. | Unusually hold clever that! sometimes illegally real late suddenly. | 6 | 70.699997 | 0.800000 | 0 | 1689780079 | 14,94 | -91152,-66001,-52476,-36462,6015,35599,50917,92421 | {"key":"value"} | 3.000000,6.000000,4.000000,9.000000 |
83+
+------+-------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+----------------------------------------------------------------+-----------------+---------------------------------------+
3484
––– input –––
35-
stdbuf -oL mysql -P9315 -h0 -e "freeze t"
85+
mysql -h0 -P9306 -e "flush ramchunk test; freeze test; `for n in \`seq 1 100\`; do echo UPDATE test SET category='updated' WHERE id=$n; done`" &
3686
––– output –––
37-
+------------------------+------------------------+
38-
| file | normalized |
39-
+------------------------+------------------------+
40-
| /tmp/data/t/t.0.spa | /tmp/data/t/t.0.spa |
41-
| /tmp/data/t/t.0.spb | /tmp/data/t/t.0.spb |
42-
| /tmp/data/t/t.0.spd | /tmp/data/t/t.0.spd |
43-
| /tmp/data/t/t.0.spe | /tmp/data/t/t.0.spe |
44-
| /tmp/data/t/t.0.sph | /tmp/data/t/t.0.sph |
45-
| /tmp/data/t/t.0.sphi | /tmp/data/t/t.0.sphi |
46-
| /tmp/data/t/t.0.spi | /tmp/data/t/t.0.spi |
47-
| /tmp/data/t/t.0.spidx | /tmp/data/t/t.0.spidx |
48-
| /tmp/data/t/t.0.spm | /tmp/data/t/t.0.spm |
49-
| /tmp/data/t/t.0.spp | /tmp/data/t/t.0.spp |
50-
| /tmp/data/t/t.0.spt | /tmp/data/t/t.0.spt |
51-
| /tmp/data/t/t.meta | /tmp/data/t/t.meta |
52-
| /tmp/data/t/t.ram | /tmp/data/t/t.ram |
53-
| /tmp/data/t/t.settings | /tmp/data/t/t.settings |
54-
+------------------------+------------------------+
5587
––– input –––
56-
mysql -P9315 -h0 -e "unfreeze t"; echo $?
88+
mysql -h0 -P9306 -e "SELECT * FROM test ORDER BY id ASC LIMIT 5;"
5789
––– output –––
58-
0
59-
––– input –––
60-
mysql -P9315 -h0 -e "DESCRIBE t"
61-
––– output –––
62-
+-------+--------+------------+
63-
| Field | Type | Properties |
64-
+-------+--------+------------+
65-
| id | bigint | |
66-
| s | string | |
67-
+-------+--------+------------+
68-
––– input –––
69-
mysql -P9315 -h0 -e "UPDATE t SET s='с' WHERE id=1; FLUSH RAMCHUNK t"; echo $?
70-
––– output –––
71-
0
90+
+------+-------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+------------------------------------------------------------+-----------------+---------------------------------------+
91+
| id | usertest | category | age | salary | discount | is_active | last_login | product_codes | large_values | additional_info | location_vector |
92+
+------+-------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+------------------------------------------------------------+-----------------+---------------------------------------+
93+
| 1 | Incorrectly heavy soft carefully late sit; put draw afraid; gave hardly. | Been vaguely 1000 this at understood gave always east fake. | 8 | 20.299999 | 1.500000 | 0 | 1676104873 | 10,16,40,71,85,87,100 | -88054,-30114,-15979,5487,72220,72277,91879,93074,97442 | {"key":"value"} | 2.000000,8.000000,10.000000,5.000000 |
94+
| 2 | Called, afraid foolish! happy? busy taken dead shall make seen privately understood short universally. | Was? seen feel economically beautiful legally. | 30 | 86.900002 | 4.900000 | 1 | 1697110603 | 1,40,47,48,52,66,80,93 | -87441,-74083,-38090,-21342,-13596,49456,57513,90976,92580 | {"key":"value"} | 6.000000,10.000000,6.000000,3.000000 |
95+
| 3 | Calm incorrectly modern obviously soft certainly, quietly need tell poor spring evening drove young? lost. | Probably came knew felt has. | 47 | 21.600000 | 6.700000 | 0 | 1678595569 | 3 | 26497,73063,73768,87840 | {"key":"value"} | 10.000000,5.000000,9.000000,10.000000 |
96+
| 4 | Cruel; emotionally beautiful modern the. Thirsty do poor dead; lazy let sick; kept angry politically closed understand. | Regularly became? fell. Brought with stand south. | 3 | 4.200000 | 7.600000 | 0 | 1650601281 | 24,28,62 | -99076 | {"key":"value"} | 4.000000,8.000000,8.000000,3.000000 |
97+
| 5 | Scientifically sad afraid, meet positively! were spent hard. | Get wrote excited deep suddenly foolish they unsuccessfully. | 34 | 4.600000 | 5.700000 | 1 | 1672897587 | 7,9,10,16,29,36,52,82,99 | 23225,29377 | {"key":"value"} | 10.000000,2.000000,7.000000,1.000000 |
98+
+------+-------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+------------------------------------------------------------+-----------------+---------------------------------------+

0 commit comments

Comments
 (0)