Skip to content

SELECT does not wait for FREEZE + UPDATE #2805

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Mar 19, 2025
Merged

SELECT does not wait for FREEZE + UPDATE #2805

Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
0c771b3
Added a new case that SELECT does not wait for FREEZE + UPDATE
Dec 2, 2024
9fdabe6
Update test-freeze-unfreeze-update.rec
Dec 2, 2024
daa96a1
Added set +b to silence notifications
Dec 2, 2024
df776b0
Fix test-freeze-unfreeze-update.rec
Dec 2, 2024
6f18ad3
Merge branch 'master' into update/test-freeze-unfreeze-update
Dec 8, 2024
ddb544e
Update test-freeze-unfreeze-update.rec
Dec 8, 2024
b891398
Added removing /tmp/data/t
Dec 9, 2024
68a2b91
Update test-freeze-unfreeze-update.rec
Dec 10, 2024
cc78a37
Return of unfreeze, query optimization
Dec 10, 2024
7d12c81
Deleted "wait"
Dec 10, 2024
9be416d
Update test-freeze-unfreeze-update.rec
PavelShilin89 Dec 11, 2024
ed5a13b
Update test-freeze-unfreeze-update
Dec 17, 2024
3ade90a
Added timeouts and result control
Dec 18, 2024
b57a6a5
modification of the load_names_attr.php script, adding new attributes
PavelShilin89 Dec 24, 2024
b8ed54b
Update searchd-with-flexible-ports.conf
PavelShilin89 Dec 24, 2024
65bec8e
Merge branch 'master' into update/test-freeze-unfreeze-update
PavelShilin89 Mar 13, 2025
cb5b57e
Update test/clt-tests/core/test-freeze-unfreeze-update.rec, update lo…
PavelShilin89 Mar 17, 2025
d2f3f6f
Update test-freeze-unfreeze-update.rec using manticore load, undo cha…
PavelShilin89 Mar 17, 2025
5529b2b
Update show-settings.rec
PavelShilin89 Mar 17, 2025
2065275
Changing manticore-load output to json
PavelShilin89 Mar 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions test/clt-tests/base/searchd-with-flexible-ports.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ searchd {
data_dir = /var/log/manticore-${INSTANCE}
query_log_format = sphinxql
query_log_commands = 1
server_id = ${INSTANCE}
}
EOF
1 change: 1 addition & 0 deletions test/clt-tests/core/show-settings.rec
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ mysql -h0 -P1306 -e "SHOW SETTINGS;"
| searchd.data_dir | /var/log/manticore-1 |
| searchd.query_log_format | sphinxql |
| searchd.query_log_commands | 1 |
| searchd.server_id | 1 |
| searchd.binlog_path | /var/log/manticore-1/binlog |
| common.plugin_dir | /usr/local/lib/manticore |
| common.lemmatizer_base | /usr/share/manticore/ |
Expand Down
115 changes: 62 additions & 53 deletions test/clt-tests/core/test-freeze-unfreeze-update.rec
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,71 +1,80 @@
––– block: ../base/start-searchd-with-buddy –––
––– input –––
echo -e "searchd {\n listen = 9315:mysql\n listen = 9316\n log = /tmp/searchd.log\n pid_file = /tmp/searchd.pid\n data_dir = /tmp/data\n watchdog = 0\n}" > /tmp/116.conf
manticore-load --quiet --init="CREATE TABLE test (id bigint, usertest text stored indexed, category string, age int, salary float, discount float, is_active bool, last_login timestamp, product_codes multi, large_values multi64, additional_info json, location_vector float_vector)" --load="INSERT INTO test (id, usertest, category, age, salary, discount, is_active, last_login, product_codes, large_values, additional_info, location_vector) VALUES (<increment>, '<text/5/20>', '<text/5/10>', <int/1/100>, <float/1.0/100.0>, <float/0.0/10.0>, <int/0/1>, <int/1609459200/1704067200>, (<array/1/10/1/100>), (<array/1/10/-100000/100000>), '{\"key\":\"value\"}', (<array/4/4/1.0/10.0>))" --batch-size=10000 --threads=4 --total=1000000
––– output –––
Threads ; Batch ; Time ; Total Docs ; Docs/Sec ; Avg QPS ; p99 QPS ; p95 QPS ; p5 QPS ; p1 QPS ; Lat Avg ; Lat p50 ; Lat p95 ; Lat p99 ;
4 ; 10000 ; %{NUMBER}:%{NUMBER} ; %{NUMBER} ; %{NUMBER} ; %{NUMBER} ; %{NUMBER} ; %{NUMBER} ; %{NUMBER} ; %{NUMBER} ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ; %{NUMBER}.%{NUMBER} ;
––– input –––
mkdir -p /tmp/data; rm -f /tmp/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd -c /tmp/116.conf > /dev/null
mysql -h0 -P9306 -e "SELECT * FROM test ORDER BY id ASC LIMIT 100000 OPTION max_matches=100000;" > /tmp/test_data.txt
––– output –––
––– input –––
if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /tmp/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /tmp/searchd.log;fi
md5sum /tmp/test_data.txt
––– output –––
Buddy started!
99494585e9f5bad83f5f6ec41a62d292 /tmp/test_data.txt
––– input –––
mysql -P9315 -h0 -e "drop table if exists t; create table t(s string); insert into t values(1, 'a'); flush ramchunk t"; echo $?
mysql -h0 -P9306 -e "SHOW TABLES"
––– output –––
0
+-------+------+
| Table | Type |
+-------+------+
| test | rt |
+-------+------+
––– input –––
mysql -P9315 -h0 -e "select * from t;"
mysql -P9306 -h0 -e "SELECT COUNT(*) FROM test;"
––– output –––
+------+------+
| id | s |
+------+------+
| 1 | a |
+------+------+
+----------+
| count(*) |
+----------+
| 1000000 |
+----------+
––– input –––
(rm -f /tmp/update.sql; for n in `seq 1 10000`; do echo "update t set s='b' where id=1;" >> /tmp/update.sql; done; nohup sh -c "while true; do mysql -P9315 -h0 < /tmp/update.sql | break; done" &>/dev/null &) 2>/dev/null
mysql -P9306 -h0 -e "DESCRIBE test;"
––– output –––
+-----------------+--------------+----------------+
| Field | Type | Properties |
+-----------------+--------------+----------------+
| id | bigint | |
| usertest | text | indexed stored |
| category | string | |
| age | uint | |
| salary | float | |
| discount | float | |
| is_active | bool | |
| last_login | timestamp | |
| product_codes | mva | |
| large_values | mva64 | |
| additional_info | json | |
| location_vector | float_vector | |
+-----------------+--------------+----------------+
––– input –––
sleep 30; mysql -P9315 -h0 -e "select * from t;"
mysql -h0 -P9306 -e "SELECT * FROM test ORDER BY id ASC LIMIT 10;"
––– output –––
+------+------+
| id | s |
+------+------+
| 1 | b |
+------+------+
+------+-------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+----------------------------------------------------------------+-----------------+---------------------------------------+
| id | usertest | category | age | salary | discount | is_active | last_login | product_codes | large_values | additional_info | location_vector |
+------+-------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+----------------------------------------------------------------+-----------------+---------------------------------------+
| 1 | Incorrectly heavy soft carefully late sit; put draw afraid; gave hardly. | Been vaguely 1000 this at understood gave always east fake. | 8 | 20.299999 | 1.500000 | 0 | 1676104873 | 10,16,40,71,85,87,100 | -88054,-30114,-15979,5487,72220,72277,91879,93074,97442 | {"key":"value"} | 2.000000,8.000000,10.000000,5.000000 |
| 2 | Called, afraid foolish! happy? busy taken dead shall make seen privately understood short universally. | Was? seen feel economically beautiful legally. | 30 | 86.900002 | 4.900000 | 1 | 1697110603 | 1,40,47,48,52,66,80,93 | -87441,-74083,-38090,-21342,-13596,49456,57513,90976,92580 | {"key":"value"} | 6.000000,10.000000,6.000000,3.000000 |
| 3 | Calm incorrectly modern obviously soft certainly, quietly need tell poor spring evening drove young? lost. | Probably came knew felt has. | 47 | 21.600000 | 6.700000 | 0 | 1678595569 | 3 | 26497,73063,73768,87840 | {"key":"value"} | 10.000000,5.000000,9.000000,10.000000 |
| 4 | Cruel; emotionally beautiful modern the. Thirsty do poor dead; lazy let sick; kept angry politically closed understand. | Regularly became? fell. Brought with stand south. | 3 | 4.200000 | 7.600000 | 0 | 1650601281 | 24,28,62 | -99076 | {"key":"value"} | 4.000000,8.000000,8.000000,3.000000 |
| 5 | Scientifically sad afraid, meet positively! were spent hard. | Get wrote excited deep suddenly foolish they unsuccessfully. | 34 | 4.600000 | 5.700000 | 1 | 1672897587 | 7,9,10,16,29,36,52,82,99 | 23225,29377 | {"key":"value"} | 10.000000,2.000000,7.000000,1.000000 |
| 6 | Happy certainly, carelessly scientifically lie saw grew, hurt hot full monthly cut may calm be sharp. | Cut serious; day! harshly regionally bitter happy quickly stale the. | 65 | 92.599998 | 4.500000 | 0 | 1699959055 | 3,20,74,92 | -58459,-31703,-8868,-8398,1620,24456,76572,79957,86553,95523 | {"key":"value"} | 7.000000,10.000000,8.000000,3.000000 |
| 7 | Easily often set hot new it become. | Be quiet cold nightly poor. | 59 | 19.400000 | 6.000000 | 1 | 1685126023 | 6,11,34,35,47,60,86,98 | -54811,-28523,-16029,-8383,266,6217,31166,67799,99092 | {"key":"value"} | 9.000000,2.000000,5.000000,5.000000 |
| 8 | Politically grew rich loud tried seen. Nowhere am salty dawn quietly? gentle permanently gone? drew kept. | Big specifically weak held she; constantly send down spiritually. | 15 | 34.700001 | 1.000000 | 0 | 1647818588 | 77,86 | -55160,-5014,31069,31847,34334,36035,61084,62322,81203,95016 | {"key":"value"} | 4.000000,7.000000,4.000000,7.000000 |
| 9 | Far, usually light west educationally. Fake wet salty wrong maybe! does went usually? humble send are. | Economically commonly wrote specifically healthy wrong culturally? 4, small rich. | 51 | 4.400000 | 0.500000 | 1 | 1611948425 | 2,5,16,18,35,38,44,79,95 | -77137,-67602,-19932,-10666,3139,22804,45013,65922,71495,87473 | {"key":"value"} | 1.000000,5.000000,4.000000,10.000000 |
| 10 | We nowhere bad hold! happily. | Unusually hold clever that! sometimes illegally real late suddenly. | 6 | 70.699997 | 0.800000 | 0 | 1689780079 | 14,94 | -91152,-66001,-52476,-36462,6015,35599,50917,92421 | {"key":"value"} | 3.000000,6.000000,4.000000,9.000000 |
+------+-------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+----------------------------------------------------------------+-----------------+---------------------------------------+
––– input –––
stdbuf -oL mysql -P9315 -h0 -e "freeze t"
mysql -h0 -P9306 -e "flush ramchunk test; freeze test; `for n in \`seq 1 100\`; do echo UPDATE test SET category='updated' WHERE id=$n; done`" &
––– output –––
+------------------------+------------------------+
| file | normalized |
+------------------------+------------------------+
| /tmp/data/t/t.0.spa | /tmp/data/t/t.0.spa |
| /tmp/data/t/t.0.spb | /tmp/data/t/t.0.spb |
| /tmp/data/t/t.0.spd | /tmp/data/t/t.0.spd |
| /tmp/data/t/t.0.spe | /tmp/data/t/t.0.spe |
| /tmp/data/t/t.0.sph | /tmp/data/t/t.0.sph |
| /tmp/data/t/t.0.sphi | /tmp/data/t/t.0.sphi |
| /tmp/data/t/t.0.spi | /tmp/data/t/t.0.spi |
| /tmp/data/t/t.0.spidx | /tmp/data/t/t.0.spidx |
| /tmp/data/t/t.0.spm | /tmp/data/t/t.0.spm |
| /tmp/data/t/t.0.spp | /tmp/data/t/t.0.spp |
| /tmp/data/t/t.0.spt | /tmp/data/t/t.0.spt |
| /tmp/data/t/t.meta | /tmp/data/t/t.meta |
| /tmp/data/t/t.ram | /tmp/data/t/t.ram |
| /tmp/data/t/t.settings | /tmp/data/t/t.settings |
+------------------------+------------------------+
––– input –––
mysql -P9315 -h0 -e "unfreeze t"; echo $?
mysql -h0 -P9306 -e "SELECT * FROM test ORDER BY id ASC LIMIT 5;"
––– output –––
0
––– input –––
mysql -P9315 -h0 -e "DESCRIBE t"
––– output –––
+-------+--------+------------+
| Field | Type | Properties |
+-------+--------+------------+
| id | bigint | |
| s | string | |
+-------+--------+------------+
––– input –––
mysql -P9315 -h0 -e "UPDATE t SET s='с' WHERE id=1; FLUSH RAMCHUNK t"; echo $?
––– output –––
0
+------+-------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+------------------------------------------------------------+-----------------+---------------------------------------+
| id | usertest | category | age | salary | discount | is_active | last_login | product_codes | large_values | additional_info | location_vector |
+------+-------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+------------------------------------------------------------+-----------------+---------------------------------------+
| 1 | Incorrectly heavy soft carefully late sit; put draw afraid; gave hardly. | Been vaguely 1000 this at understood gave always east fake. | 8 | 20.299999 | 1.500000 | 0 | 1676104873 | 10,16,40,71,85,87,100 | -88054,-30114,-15979,5487,72220,72277,91879,93074,97442 | {"key":"value"} | 2.000000,8.000000,10.000000,5.000000 |
| 2 | Called, afraid foolish! happy? busy taken dead shall make seen privately understood short universally. | Was? seen feel economically beautiful legally. | 30 | 86.900002 | 4.900000 | 1 | 1697110603 | 1,40,47,48,52,66,80,93 | -87441,-74083,-38090,-21342,-13596,49456,57513,90976,92580 | {"key":"value"} | 6.000000,10.000000,6.000000,3.000000 |
| 3 | Calm incorrectly modern obviously soft certainly, quietly need tell poor spring evening drove young? lost. | Probably came knew felt has. | 47 | 21.600000 | 6.700000 | 0 | 1678595569 | 3 | 26497,73063,73768,87840 | {"key":"value"} | 10.000000,5.000000,9.000000,10.000000 |
| 4 | Cruel; emotionally beautiful modern the. Thirsty do poor dead; lazy let sick; kept angry politically closed understand. | Regularly became? fell. Brought with stand south. | 3 | 4.200000 | 7.600000 | 0 | 1650601281 | 24,28,62 | -99076 | {"key":"value"} | 4.000000,8.000000,8.000000,3.000000 |
| 5 | Scientifically sad afraid, meet positively! were spent hard. | Get wrote excited deep suddenly foolish they unsuccessfully. | 34 | 4.600000 | 5.700000 | 1 | 1672897587 | 7,9,10,16,29,36,52,82,99 | 23225,29377 | {"key":"value"} | 10.000000,2.000000,7.000000,1.000000 |
+------+-------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------+------+-----------+----------+-----------+------------+--------------------------+------------------------------------------------------------+-----------------+---------------------------------------+
Loading