Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Pr validator fixes and testing #6449

Open
wants to merge 18 commits into
base: main
Choose a base branch
from
Open

fix: Pr validator fixes and testing #6449

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
2b11397
test
SATYAsasini Mar 17, 2025
8b35f54
add centralise workflow
SATYAsasini Mar 17, 2025
31efc17
fix: permisssion
SATYAsasini Mar 18, 2025
48672db
chore: changes in flow
SATYAsasini Mar 18, 2025
bef7807
chore: changes in trigger
SATYAsasini Mar 18, 2025
ce9fa6e
chore: trigger event changes
SATYAsasini Mar 18, 2025
036d45f
misc: chore
SATYAsasini Mar 18, 2025
f6f2a99
misc: chore
SATYAsasini Mar 18, 2025
55af3d3
misc: chore
SATYAsasini Mar 18, 2025
7f26a5b
misc: chore
SATYAsasini Mar 18, 2025
7146660
misc: chore
SATYAsasini Mar 18, 2025
4b805a0
misc: chore
SATYAsasini Mar 18, 2025
74b071a
test
SATYAsasini Mar 18, 2025
36c6572
test
SATYAsasini Mar 18, 2025
f4368b3
test
SATYAsasini Mar 18, 2025
8926dea
test
SATYAsasini Mar 18, 2025
80445a2
chore: misc
SATYAsasini Mar 18, 2025
98e37ea
chore: test
SATYAsasini Mar 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/workflows/pr-issue-test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Validate Pull Request-TEST(pull_request_target)

on:
pull_request_target:
types:
- opened
- synchronize
- edited
# branches:
# - 'main'
# - 'release-**'
# - 'develop'
# - 'hotfix-**'
# - 'fix/pr-validator'
jobs:
pr-validation:
runs-on: ubuntu-latest
steps:
- name: test
run: echo "workflow triggered"
# uses: devtron-labs/utilities/.github/workflows/pr-validator.yaml@feat/central-pr-validator
# secrets: inherit
# with:
# validate_sql: true
274 changes: 8 additions & 266 deletions .github/workflows/pr-issue-validator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,284 +2,26 @@ name: Validate Pull Request

on:
pull_request:
types:
types:
- opened
- synchronize
- edited
- reopened
branches:
- 'main'
- 'release-**'
- 'develop'
- 'hotfix-v0**'
- 'hotfix-**'
- 'fix/pr-validator'
# paths-ignore:
# - 'docs/**'
# - '.github/'
# - 'CHANGELOG/'
# - 'charts/'
# - 'manifests/'
# - 'sample-docker-templates/'

jobs:
validate-PR-issue:
runs-on: ubuntu-latest
permissions:
issues: write
contents: read
pull-requests: write
repository-projects: read
steps:
- name: Checkout repository
uses: actions/checkout@v2
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0

- name: Validate Issue Reference
env:
GH_TOKEN: ${{ github.token }}
PR_BODY: ${{ github.event.pull_request.body }}
PRNUM: ${{ github.event.pull_request.number }}
TITLE: ${{ github.event.pull_request.title }}
run: |

echo "base or target repo : ${{ github.event.pull_request.base.repo.full_name }}"
echo "head or source repo : ${{ github.event.pull_request.head.repo.full_name }}"

if [[ ${{ github.event.pull_request.head.repo.full_name }} == ${{ github.event.pull_request.base.repo.full_name }} ]]; then
export forked=false
else
export forked=true
fi

set -x
# Skip validation for documentation or chore PRs
if [[ "$TITLE" =~ ^(doc:|docs:|chore:|misc:|Release:|release:|Sync:|sync:) ]]; then
echo "Skipping validation for docs/chore PR."
echo "PR NUMBER-: $PRNUM "
gh pr edit $PRNUM --remove-label "PR:Issue-verification-failed"
gh pr edit $PRNUM --add-label "PR:Ready-to-Review"
exit 0
fi

# Define all issue matching patterns
patterns=(
"((Fixes|Resolves) #[0-9]+)"
"((Fixes|Resolves) https://github.com/devtron-labs/devtron/issues/[0-9]+)"
"((Fixes|Resolves) devtron-labs/devtron#[0-9]+)"
"(Fixes|Resolves):?\\s+\\[#([0-9]+)\\]"
"((Fixes|Resolves):? #devtron-labs/devops-sprint/issues/[0-9]+)"
"((Fixes|Resolves):? #devtron-labs/sprint-tasks/issues/[0-9]+)"
"((Fixes|Resolves) https://github.com/devtron-labs/devops-sprint/issues/[0-9]+)"
"((Fixes|Resolves) https://github.com/devtron-labs/sprint-tasks/issues/[0-9]+)"
"((Fixes|Resolves):? #devtron-labs/sprint-tasks#[0-9]+)"
)

# Extract issue number and repo from PR body
extract_issue_number() {
local pattern="$1" # Get the pattern as the first argument to the function

# Check if PR_BODY matches the provided pattern using Bash's =~ regex operator
if [[ "$PR_BODY" =~ $pattern ]]; then
echo "matched for this pattern $pattern"

issue_num=$(echo "$PR_BODY" | grep -oE "$pattern" | grep -oE "[0-9]+")

# Extract the repository name (e.g., devtron-labs/devtron) from PR_BODY using grep
repo=$(echo "$PR_BODY" | grep -oE "devtron-labs/[a-zA-Z0-9_-]+")
echo "Extracted issue number: $issue_num from repo: $repo"

return 0 # Return success
else
echo "No match for the pattern $pattern"
fi
return 1 # Return failure if no match
}

issue_num=""
repo="devtron-labs/devtron" # Default repo
for pattern in "${patterns[@]}"; do
echo "Now checking for $pattern"
extract_issue_number "$pattern" && break
done

if [[ -z "$issue_num" ]]; then
echo "No valid issue number found."
gh pr edit $PRNUM --add-label "PR:Issue-verification-failed"
gh pr edit $PRNUM --remove-label "PR:Ready-to-Review"
exit 1
fi

# Form the issue API URL dynamically
issue_api_url="https://api.github.com/repos/$repo/issues/$issue_num"
echo "API URL: $issue_api_url"

if [[ $repo == "devtron-labs/devtron" || $repo == "devtron-labs/devtron-services" || $repo == "devtron-labs/dashboard" ]]; then
echo "No extra arguments needed: public repository detected."
response_code=$(curl -s -o /dev/null -w "%{http_code}" \
"$issue_api_url")

else
echo "Adding extra arguments for authentication: private repository detected."
response_code=$(curl -s -o /dev/null -w "%{http_code}" \
--header "authorization: Bearer ${{ secrets.GH_PR_VALIDATOR_TOKEN }}" \
"$issue_api_url")
fi

echo "Response Code: $response_code"
if [[ "$response_code" -eq 200 ]]; then
echo "Issue #$issue_num is valid and exists in $repo."

# Fetch the current state of the issue (open/closed) from the private repository.
if [[ $repo == "devtron-labs/devtron" || $repo == "devtron-labs/devtron-services" || $repo == "devtron-labs/dashboard" ]]; then
echo "No extra arguments needed: public repository detected."
issue_status=$(curl -s \
"$issue_api_url"| jq '.state'|tr -d \")
else
echo "Adding extra arguments for authentication: private repository detected."
issue_status=$(curl -s \
--header "authorization: Bearer ${{ secrets.GH_PR_VALIDATOR_TOKEN }}" \
"$issue_api_url"| jq '.state'|tr -d \")
fi
echo "Issue Number : $issue_num Status: $issue_status"
# Check if the issue is still open.
# if [[ "$issue_status" == open ]]; then
# echo "Issue #$issue_num is opened."
if [[ $forked == true ]]; then
echo "PR:Ready-to-Review, exiting gracefully"
exit 0
fi
# Remove the 'Issue-verification-failed' label (if present) and add 'Ready-to-Review'.
gh pr edit $PRNUM --remove-label "PR:Issue-verification-failed"
gh pr edit $PRNUM --add-label "PR:Ready-to-Review"
echo "PR:Ready-to-Review, exiting gracefully"
exit 0
# else
# echo "Issue #$issue_num is closed. Please link an open issue to proceed."
# if [[ $forked == true ]]; then
# echo "PR:Ready-to-Review, exiting gracefully"
# exit 0
# fi
# Add a comment to the PR indicating the issue is not linked correctly.
# gh pr comment $PRNUM --body "PR is linked to a closed issue. Please link an open issue to proceed."

# Add the 'Issue-verification-failed' label and remove 'Ready-to-Review'.
# gh pr edit $PRNUM --add-label "PR:Issue-verification-failed"
# gh pr edit $PRNUM --remove-label "PR:Ready-to-Review"
# exit 1
#fi
else
echo "Issue not found. Invalid URL or issue number."
# Add a comment to the PR indicating the issue is not linked correctly.
gh pr comment $PRNUM --body "PR is not linked to a valid issue. Please update the issue link."

# Apply 'Issue-verification-failed' label and remove 'Ready-to-Review' label.
gh pr edit $PRNUM --add-label "PR:Issue-verification-failed"
gh pr edit $PRNUM --remove-label "PR:Ready-to-Review"
exit 1
fi
- name: Check SQL file format and duplicates
shell: bash
env:
pr_no: ${{ github.event.pull_request.number }}
GH_TOKEN: ${{ github.token }}
run: |
# Fetch the latest changes from the main branch
git fetch origin main

# Get the list of changed files
git diff origin/main...HEAD --name-only > diff

# Specify the directory containing migration files
MIGRATION_DIR="scripts/sql"
ls
pwd

# Print changed files
echo "Changed files:"
cat diff

changed_files=""
while IFS= read -r file; do
if [[ $file == $MIGRATION_DIR/* && $file == *.up.sql ]]; then
changed_files+="$file\n"
fi
done < diff

# Print the filtered .up.sql files
echo "Filtered .up.sql files:"
echo -e "$changed_files"

# Check if there are any .up.sql migration files in the changed files list
if [ -z "$changed_files" ]; then
echo "No .up.sql migration files found in the changes."
else
# Extract unique migration numbers from the directory (considering only .up.sql files)
existing_migrations=$(ls $MIGRATION_DIR | grep -E "\.up\.sql$" | grep -oE "[0-9]{3}[0-9]{3}[0-9]{2}" | sort | uniq)

# Exclude migration numbers from changed files in existing_migrations
while read -r file; do
migration_number=$(basename "$file" | grep -oE "[0-9]{3}[0-9]{3}[0-9]{2}")
existing_migrations=$(echo "$existing_migrations" | grep -v "$migration_number")
done <<< "$changed_files"

# Validate each changed .up.sql migration file
is_valid=true
processed_migrations=()
while read -r file; do
# Extract migration number from the filename
migration_number=$(basename "$file" | grep -oE "[0-9]{3}[0-9]{3}[0-9]{2}")

# Check if the filename has the full XXXPPPNN format
if [[ ! $(basename "$file") =~ ^[0-9]{3}[0-9]{3}[0-9]{2}_ ]]; then
echo "Error: Migration file $file does not have the complete XXXPPPNN format."
is_valid=false
continue
fi

if [ -z "$migration_number" ]; then
echo "Warning: Could not extract migration number from $file."
continue
fi

# Check if this migration number has already been processed
if [[ " ${processed_migrations[@]} " =~ " $migration_number " ]]; then
continue
fi
processed_migrations+=("$migration_number")

# Check if the migration number is unique
if echo "$existing_migrations" | grep -q "$migration_number"; then
echo "Error: Migration number $migration_number already exists."
is_valid=false
fi

# Check if the migration number is greater than previous ones
last_migration=$(echo "$existing_migrations" | tail -n 1)
if [ "$migration_number" -le "$last_migration" ]; then
echo "Error: Migration number $migration_number is not greater than the latest ($last_migration)."
is_valid=false
fi

# Check for sequential hotfix requirement (if NN > 01, check for NN-1)
hotfix_number=$(echo "$migration_number" | grep -oE "[0-9]{2}$")
if [ "$hotfix_number" -gt "01" ]; then
previous_hotfix=$(printf "%02d" $((10#$hotfix_number - 1)))
expected_previous_number="${migration_number:0:6}$previous_hotfix"
if ! echo "$existing_migrations" | grep -q "$expected_previous_number"; then
echo "Error: Previous hotfix migration $expected_previous_number not found for $migration_number."
is_valid=false
fi
fi

done <<< "$changed_files"

if [ "$is_valid" = false ]; then
echo "Validation failed. Please fix the errors before merging."
gh pr comment $pr_no --body "The Migration files providede inside of the PR does not pass the criteria!!"
exit 1
fi

echo "All .up.sql migration file validations passed."
gh pr comment $pr_no --body "The migration files have successfully passed the criteria!!"
fi
pr-validation:
uses: devtron-labs/utilities/.github/workflows/pr-validator.yaml@feat/central-pr-validator
secrets: inherit
with:
validate_sql: true # Enable/disable SQL validationsq
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ Get updates on Devtron's development and chat with project maintainers, contribu
- Articles, Howtos, Tutorials - [Devtron Blogs](https://devtron.ai/blog/)
- Devtron is trusted by communities all across the globe. The list of organizations using Devtron can be found [here](./USERS.md).

### Join us at Discord channel
### Join us at Discord Channel
<p>
<a href="https://discord.gg/jsRG5qx2gp">
<img
Expand Down
Loading