directory is not empty when thread is busy with flushing diskchunk

[donhardman]

Bug Description:

In some cases when we drop a table and create a table within a short time interval, we see the following error.

I tried using a different approach and found that this case happens when we have another table, work with it, and it flushes data periodically due to multiple selects. When at some point during this process we perform drop and create operations sequentially, we encounter the issue with "folder exists already."

Here is the script. Simply save it as mre, run chmod +x mre, and execute it with ./mre to reproduce the issue.

#!/usr/bin/env bash

# Trap function to handle cleanup
cleanup() {
	echo -e "\nStopping all processes..."
	kill $(jobs -p) 2>/dev/null
	wait
	exit 0
}

# Set up trap for Ctrl+C
trap cleanup SIGINT SIGTERM

# First thread - Insert into table a
insert_data() {
	while true; do
		mysql -h0 -P9306 << EOF
		INSERT INTO a (id, value) VALUES (0, 'hello world');
EOF
		sleep 2
	done > /dev/null
}

# Second thread - Select from table a
select_data() {
	while true; do
		mysql -h0 -P9306 << EOF
		select * from a where match('world');
EOF
		sleep 0.2
		done > /dev/null
	}

# Second thread - Drop and Create table b
drop_and_create() {
	while true; do
		mysql -h0 -P9306 << EOF
		DROP TABLE IF EXISTS b;
		CREATE TABLE b (id bigint, value string);
EOF
	done
}

# Start all threads in background
insert_data &
select_data &
drop_and_create &

echo -e "All threads started.\nWaiting for error.\nPress Ctrl+C to stop."

# Wait indefinitely until Ctrl+C is pressed
while true; do
	sleep 1
done

Manticore Search Version:

Latest Dev Version

Operating System Version:

Ubuntu Jammy

Have you tried the latest development version?

None

Internal Checklist:

^{To be completed by the assignee. Check off tasks that have been completed or are not applicable.}

• Implementation completed
• Tests developed
• Documentation updated
• Documentation reviewed
• Changelog updated

[donhardman]

Setting diskchunk_flush_write_timeout = -1 helps to eliminate the issue

[klirichek]

@donhardman done in commit above

[donhardman]

Test implemented: #3194
P.S. It requires last CLT version