CI: memcheck.sh improvements; github workflow for nightly memleak tests and other minor changes

Enhanced memory checking script with automatic leak detection and reporting. The script now:
- Scans memory checker log files
- Identifies memory leaks using predefined patterns
- Provides detailed output for detected leaks
- Exits with an error code if leaks are found

Configured a scheduled GitHub Actions workflow to run memory leak detection tests using Valgrind. The workflow:
- Runs nightly
- Uses a specialized Valgrind-enabled Docker container
- Executes memcheck.sh for comprehensive memory leak testing
- Uploads test artifacts for further analysis

Author: sanikolaev

.github/workflows/clt_nightly.yml

@@ -1,5 +1,5 @@
 name: Nightly tests
-run-name: Nightly tests ${{ github.sha }}
+run-name: 🌙 Nightly tests ${{ github.sha }}
 
 on:
   schedule:

.github/workflows/nightly_dumps.yml

@@ -1,5 +1,5 @@
 name: CLT testing of supported versions of mysqldump
-run-name: CLT testing of supported versions of mysqldump ${{ github.sha }}
+run-name: 🌙 CLT testing of supported versions of mysqldump ${{ github.sha }}
 
 on:
   schedule:

.github/workflows/nightly_memleaks.yml

@@ -0,0 +1,54 @@
+name: Nightly memleaks tests
+run-name: 🌙 Nightly memleaks tests ${{ github.sha }}
+
+on:
+  schedule:
+    - cron: '00 20 * * *'
+  push:
+    paths:
+      - '.github/workflows/nightly_memleaks.yml'    
+  
+# cancels the previous workflow run when a new one appears in the same branch (e.g. master or a PR's branch)
+concurrency:
+  group: nightly_memleaks_${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  memleaks:
+    name: Memleak tests
+    runs-on: ubuntu-22.04
+    defaults:
+      run:
+        shell: bash
+    timeout-minutes: 300
+    container:
+      image: manticoresearch/manticore_valgrind:jammy
+      env:
+        DIAGNOSTIC: 1
+        VERBOSE: 1
+        CTEST_CONFIGURATION_TYPE: "RelWithDebInfo"
+        UNITY_BUILD: 1
+        CACHEB: "../cache"
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          set-safe-directory: true
+      - name: Check out cache before building
+        uses: actions/cache/[email protected]
+        with:
+          path: cache
+          key: build_linux_debug_x86_64
+      - name: memcheck.sh
+        run: |
+          mysqld &
+          bash memcheck.sh
+      - name: Upload build artifacts
+        if: always()
+        continue-on-error: true
+        uses: manticoresoftware/upload_artifact_with_retries@v4
+        with:
+          name: memleaks_${{ github.sha }}
+          path: build/Testing
+  

memcheck.sh

@@ -3,11 +3,14 @@
 # This script can be run on the host, or in the docker
 
 chmod +x valgrind
+mkdir -p build
 cd build
 
-[[ ! -z "${CACHEB}" ]] && export CACHEB=../cache
-[[ ! -z "${uid}" ]] && export uid=`id -u`
-[[ ! -z "${gid}" ]] && export gid=`id -g`
+[[ -z "${CACHEB}" ]] && export CACHEB=../cache
+[[ ! -e "${CACHEB}" ]] && mkdir -p ${CACHEB}
+[[ -z "${uid}" ]] && export uid=`id -u`
+[[ -z "${gid}" ]] && export gid=`id -g`
+trap "chown -R $uid:$gid ." EXIT
 
 export CTEST_CONFIGURATION_TYPE=RelWithDebInfo
 export CTEST_CMAKE_GENERATOR=Ninja
@@ -24,4 +27,42 @@ export NO_TESTS=0
 rm -f Testing/Temporary/MemoryChecker.*.log
 time ctest -V -S ../misc/ctest/memcheck.cmake
 find Testing/Temporary/MemoryChecker.*.log -size 0 -delete
-chown -R $uid:$gid .
+
+# Analysis of memory leak reports
+echo "Analyzing memory leak reports..."
+LEAK_FOUND=0
+LEAK_PATTERNS=("definitely lost")
+
+# Function to check for memory leaks in a file
+check_leaks() {
+    local file=$1
+    local test_num=$(basename "$file" | sed -E 's/MemoryChecker\.([0-9]+)\.log/\1/')
+    
+    for pattern in "${LEAK_PATTERNS[@]}"; do
+        if grep -i "$pattern" "$file" > /dev/null; then
+            echo "===================================================================="
+            echo "MEMORY LEAK DETECTED in test_$test_num"
+            echo "Pattern found: $pattern"
+            echo "===================================================================="
+            echo "Full log content:"
+            cat "$file"
+            echo "===================================================================="
+            LEAK_FOUND=1
+        fi
+    done
+}
+
+# Process all memory checker log files
+for log_file in Testing/Temporary/MemoryChecker.*.log; do
+    if [ -f "$log_file" ]; then
+        check_leaks "$log_file"
+    fi
+done
+
+if [ $LEAK_FOUND -eq 1 ]; then
+    echo "Memory leaks were detected. Check the logs above for details."
+    exit 1
+else
+    echo "No memory leaks detected."
+    exit 0
+fi

memdocker.sh

@@ -1,7 +1,8 @@
 #!/bin/bash
 #Run shell in valgrind docker
 
-BUILD_DOCKER=manticore_valgrind:jammy
+# Use environment variable BUILD_DOCKER if set, otherwise use default value
+BUILD_DOCKER=${BUILD_DOCKER:-manticoresearch/manticore_valgrind:jammy}
 CONFIG=RelWithDebInfo
 WORKDIR=/work
 

misc/ctest/memcheck.md

@@ -20,7 +20,7 @@ It is recommended to clean the `/manticore/source/dir/build` folder. This may no
 
 After running, you may notice the text near the end of the output:
 ```
-MemCheck log files can be found here: (<#> corresponds to the test number)
+MemCheck log files can be found here: (<#> corresponds to test number)
 /manticore/source/dir/build/Testing/Temporary/MemoryChecker.<#>.log
 ```