Skip to content

Sample Data Loading and Test in GitHub Action #3223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 98 commits into from
Feb 22, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
44d3e1a
DB testing Workflow
JaiPannu-IITI Feb 15, 2025
723f9a5
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 16, 2025
ee982a5
workflow
JaiPannu-IITI Feb 16, 2025
1909d58
Merge branch 'PR-Workflow-(DB)' of https://github.com/JaiPannu-IITI/t…
JaiPannu-IITI Feb 16, 2025
bbf88b9
fix data readibility
JaiPannu-IITI Feb 16, 2025
29ea81b
dynamic insertion
JaiPannu-IITI Feb 16, 2025
4ec6777
dynamic testing
JaiPannu-IITI Feb 16, 2025
0143804
prettier
JaiPannu-IITI Feb 16, 2025
432148a
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 16, 2025
9aeb522
Clean Code
JaiPannu-IITI Feb 16, 2025
be189c7
Merge branch 'PR-Workflow-(DB)' of https://github.com/JaiPannu-IITI/t…
JaiPannu-IITI Feb 16, 2025
54e0c50
finalizing
JaiPannu-IITI Feb 16, 2025
fd473ca
comment
JaiPannu-IITI Feb 16, 2025
edebdd5
graceful exit
JaiPannu-IITI Feb 16, 2025
8a44750
pretty
JaiPannu-IITI Feb 16, 2025
e94acf2
pretty
JaiPannu-IITI Feb 16, 2025
8f48c4a
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 17, 2025
d79e082
Test case
JaiPannu-IITI Feb 17, 2025
597bd66
Merge branch 'PR-Workflow-(DB)' of https://github.com/JaiPannu-IITI/t…
JaiPannu-IITI Feb 17, 2025
ce6efd6
pretty
JaiPannu-IITI Feb 17, 2025
060df44
basic testing
JaiPannu-IITI Feb 17, 2025
ca64626
pretty
JaiPannu-IITI Feb 17, 2025
cb788ea
Empty commit
JaiPannu-IITI Feb 17, 2025
2db0ddb
type check
JaiPannu-IITI Feb 17, 2025
b01653f
pretty
JaiPannu-IITI Feb 17, 2025
19e96b4
coderabbit
JaiPannu-IITI Feb 17, 2025
51c04ac
Empty commit
JaiPannu-IITI Feb 17, 2025
1872ac0
changes
JaiPannu-IITI Feb 17, 2025
11943bb
change
JaiPannu-IITI Feb 17, 2025
a183cd6
pretty
JaiPannu-IITI Feb 17, 2025
b74aecb
Final test
JaiPannu-IITI Feb 17, 2025
6e41bc5
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 18, 2025
4b0be6e
major changes
JaiPannu-IITI Feb 18, 2025
ef8f3e3
Merge branch 'PR-Workflow-(DB)' of https://github.com/JaiPannu-IITI/t…
JaiPannu-IITI Feb 18, 2025
1ca9efe
correction
JaiPannu-IITI Feb 18, 2025
3eebbcd
docs
JaiPannu-IITI Feb 18, 2025
4061fd7
prettier
JaiPannu-IITI Feb 18, 2025
9b1d59f
finalizing
JaiPannu-IITI Feb 18, 2025
023a6ae
test modifications
JaiPannu-IITI Feb 18, 2025
3ed16e2
test modify
JaiPannu-IITI Feb 18, 2025
fbb0103
pretty
JaiPannu-IITI Feb 18, 2025
2704f49
remove test
JaiPannu-IITI Feb 18, 2025
cb28075
environment setup
JaiPannu-IITI Feb 18, 2025
b899925
changes
JaiPannu-IITI Feb 18, 2025
db5c885
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 18, 2025
6788693
documentation updated
JaiPannu-IITI Feb 18, 2025
5b309de
Merge branch 'PR-Workflow-(DB)' of https://github.com/JaiPannu-IITI/t…
JaiPannu-IITI Feb 18, 2025
ad53429
documentation
JaiPannu-IITI Feb 18, 2025
67c6e91
documentation
JaiPannu-IITI Feb 18, 2025
4b25561
documentation
JaiPannu-IITI Feb 18, 2025
bdd8891
test
JaiPannu-IITI Feb 19, 2025
9e902f6
module
JaiPannu-IITI Feb 19, 2025
021e964
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 19, 2025
914cc79
major testing improvement
JaiPannu-IITI Feb 20, 2025
2250ee3
Merge branch 'PR-Workflow-(DB)' of https://github.com/JaiPannu-IITI/t…
JaiPannu-IITI Feb 20, 2025
e6fd5b0
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 20, 2025
e0c6b4a
test
JaiPannu-IITI Feb 20, 2025
8453f8e
pretty
JaiPannu-IITI Feb 20, 2025
a097800
xoldd, coderabbit ai implementation
JaiPannu-IITI Feb 20, 2025
ed37435
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 20, 2025
c1bfc4d
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 21, 2025
451724e
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 21, 2025
40dad6b
changes
JaiPannu-IITI Feb 21, 2025
209231c
Merge branch 'PR-Workflow-(DB)' of https://github.com/JaiPannu-IITI/t…
JaiPannu-IITI Feb 21, 2025
7f8387e
fix
JaiPannu-IITI Feb 21, 2025
cbc445e
improvements
JaiPannu-IITI Feb 21, 2025
6e6a847
improvements
JaiPannu-IITI Feb 21, 2025
df35d8d
code quality
JaiPannu-IITI Feb 21, 2025
f30d560
removed unnecessary packages
JaiPannu-IITI Feb 21, 2025
1e67c4b
fix
JaiPannu-IITI Feb 21, 2025
7b8a492
remove unwanted package (xoldd)
JaiPannu-IITI Feb 21, 2025
5d843e2
fix
JaiPannu-IITI Feb 21, 2025
038942d
test database
JaiPannu-IITI Feb 21, 2025
c51af3c
code-quality
JaiPannu-IITI Feb 21, 2025
5587783
manipulate db
JaiPannu-IITI Feb 21, 2025
4dd5400
fix
JaiPannu-IITI Feb 21, 2025
fddd190
backward compatibility
JaiPannu-IITI Feb 21, 2025
6cb21d7
fix
JaiPannu-IITI Feb 21, 2025
d528e97
final
JaiPannu-IITI Feb 21, 2025
29924bd
cleaning
JaiPannu-IITI Feb 21, 2025
d430bc0
corrected environment
JaiPannu-IITI Feb 21, 2025
3016e41
host managment
JaiPannu-IITI Feb 21, 2025
3b88beb
fixed code quality
JaiPannu-IITI Feb 21, 2025
039640b
sample
JaiPannu-IITI Feb 21, 2025
c116227
removed unncessary dependencies
JaiPannu-IITI Feb 21, 2025
c6adb1e
roll back to postgres env
JaiPannu-IITI Feb 21, 2025
9657f10
pretty
JaiPannu-IITI Feb 21, 2025
cd84244
pretty
JaiPannu-IITI Feb 21, 2025
8b0921d
fix
JaiPannu-IITI Feb 21, 2025
55b7146
added tests
JaiPannu-IITI Feb 21, 2025
0a63e0d
fix
JaiPannu-IITI Feb 21, 2025
9287f8d
vitest is sufficient
JaiPannu-IITI Feb 21, 2025
78c4696
coderabbit
JaiPannu-IITI Feb 21, 2025
0f20d5b
docker environment configuration
JaiPannu-IITI Feb 22, 2025
33dffac
docs
JaiPannu-IITI Feb 22, 2025
4e83255
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 22, 2025
9058603
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 22, 2025
f257c72
Merge branch 'develop-postgres' into PR-Workflow-(DB)
JaiPannu-IITI Feb 22, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,28 @@ jobs:
path: "./coverage/lcov.info"
min_coverage: 45.0

Database-Access-Check:
name: Checking database connection and access
runs-on: ubuntu-latest
needs: [Run-Tests]
steps:
- name: Checkout this repository
uses: actions/[email protected]
- name: Create .env file for talawa api testing environment
run: cp ./envFiles/.env.ci ./.env
- name: Building Talawa API
run: docker compose build
- name: Run Container
run: docker compose up -d
- name: Adding tables to testing environment
run: docker compose exec api pnpm apply_drizzle_test_migrations
- name: Import Sample Data into Database
run: docker compose exec api sh -c "yes yes | pnpm import:sample-data"
- name: Validate Database Records
run: docker compose exec api pnpm test:db-connection
- name: Stop Services
run: docker compose down

Test-Docusaurus-Deployment:
name: Test Deployment to https://docs-api.talawa.io
runs-on: ubuntu-latest
Expand Down
2 changes: 1 addition & 1 deletion envFiles/.env.ci
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ API_POSTGRES_USER=talawa
# https://vitest.dev/config/#watch
CI=true
# https://blog.platformatic.dev/handling-environment-variables-in-nodejs#heading-set-nodeenvproduction-for-all-environments
NODE_ENV=production
NODE_ENV=test

########## docker compose `api` container service ##########

Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,7 @@
"generate_drizzle_migrations": "drizzle-kit generate",
"generate_graphql_sdl_file": "tsx ./scripts/generateGraphQLSDLFile.ts",
"generate_gql_tada": "gql.tada generate-output && gql.tada turbo --fail-on-warn",
"test:db-connection": "tsx ./src/utilities/testDbConnection.ts",
"import:sample-data": "tsx ./src/utilities/loadSampleData.ts",
"push_drizzle_schema": "drizzle-kit push",
"push_drizzle_test_schema": "drizzle-kit push --config=./test/drizzle.config.ts",
Expand Down
62 changes: 61 additions & 1 deletion src/utilities/loadSampleData.ts
Original file line number Diff line number Diff line change
@@ -1,19 +1,25 @@
import fs from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
import { hash } from "@node-rs/argon2";

Check warning on line 4 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L4

Added line #L4 was not covered by tests
import dotenv from "dotenv";
import { sql } from "drizzle-orm";
import { drizzle } from "drizzle-orm/postgres-js";
import inquirer from "inquirer";
import postgres from "postgres";
import { uuidv7 } from "uuidv7";

Check warning on line 10 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L10

Added line #L10 was not covered by tests
import * as schema from "../drizzle/schema";

dotenv.config();

const dirname: string = path.dirname(fileURLToPath(import.meta.url));

const isTestEnvironment = process.env.NODE_ENV === "test";

Check warning on line 17 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L17

Added line #L17 was not covered by tests

const queryClient = postgres({
host: process.env.API_POSTGRES_HOST,
host: isTestEnvironment
? process.env.API_POSTGRES_TEST_HOST
: process.env.API_POSTGRES_HOST,

Check warning on line 22 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L20-L22

Added lines #L20 - L22 were not covered by tests
port: Number(process.env.API_POSTGRES_PORT),
database: process.env.API_POSTGRES_DATABASE,
username: process.env.API_POSTGRES_USER,
Expand Down Expand Up @@ -91,11 +97,65 @@
* @param collections - Array of collection/table names to insert data into
* @param options - Options for loading data
*/

async function ensureAdministratorExists(): Promise<void> {
console.log("Checking if the administrator user exists...");

Check warning on line 102 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L101-L102

Added lines #L101 - L102 were not covered by tests

const email = process.env.API_ADMINISTRATOR_USER_EMAIL_ADDRESS;
if (!email) {
console.error(
"ERROR: API_ADMINISTRATOR_USER_EMAIL_ADDRESS is not defined.",
);
return;
}

Check warning on line 110 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L104-L110

Added lines #L104 - L110 were not covered by tests

const existingUser = await db.query.usersTable.findFirst({
columns: { id: true, role: true },
where: (fields, operators) => operators.eq(fields.emailAddress, email),
});

Check warning on line 115 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L112-L115

Added lines #L112 - L115 were not covered by tests

if (existingUser) {
if (existingUser.role !== "administrator") {
console.log("Updating user role to administrator...");
await db
.update(schema.usersTable)
.set({ role: "administrator" })
.where(sql`email_address = ${email}`);
console.log("Administrator role updated.");
} else {
console.log("Administrator user already exists.");
}
return;
}

Check warning on line 129 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L117-L129

Added lines #L117 - L129 were not covered by tests

console.log("Creating administrator user...");
const userId = uuidv7();
const password = process.env.API_ADMINISTRATOR_USER_PASSWORD;
if (!password) {
throw new Error("API_ADMINISTRATOR_USER_PASSWORD is not defined.");
}
const passwordHash = await hash(password);

Check warning on line 137 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L131-L137

Added lines #L131 - L137 were not covered by tests

await db.insert(schema.usersTable).values({
id: userId,
emailAddress: email,
name: process.env.API_ADMINISTRATOR_USER_NAME || "",
passwordHash,
role: "administrator",
isEmailAddressVerified: true,
creatorId: userId,
});

Check warning on line 147 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L139-L147

Added lines #L139 - L147 were not covered by tests

console.log("Administrator user created successfully.");
}

Check warning on line 150 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L149-L150

Added lines #L149 - L150 were not covered by tests

async function insertCollections(
collections: string[],
options: LoadOptions = {},
): Promise<void> {
try {
await ensureAdministratorExists();

Check warning on line 157 in src/utilities/loadSampleData.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/loadSampleData.ts#L157

Added line #L157 was not covered by tests

if (options.format) {
await formatDatabase();
}
Expand Down
133 changes: 133 additions & 0 deletions src/utilities/testDbConnection.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
import dotenv from "dotenv";
import { sql } from "drizzle-orm";
import { drizzle } from "drizzle-orm/postgres-js";
import postgres from "postgres";
import * as schema from "../drizzle/schema";

Check warning on line 5 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L1-L5

Added lines #L1 - L5 were not covered by tests

dotenv.config();

Check warning on line 7 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L7

Added line #L7 was not covered by tests

const isTestEnvironment = process.env.NODE_ENV === "test";

Check warning on line 9 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L9

Added line #L9 was not covered by tests

// Setup PostgreSQL connection
const queryClient = postgres({
host: isTestEnvironment
? process.env.API_POSTGRES_TEST_HOST
: process.env.API_POSTGRES_HOST,
port: Number(process.env.API_POSTGRES_PORT),
database: process.env.API_POSTGRES_DATABASE,
username: process.env.API_POSTGRES_USER,
password: process.env.API_POSTGRES_PASSWORD,
ssl: process.env.API_POSTGRES_SSL_MODE === "true",
});

Check warning on line 21 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L12-L21

Added lines #L12 - L21 were not covered by tests

const db = drizzle(queryClient, { schema });

Check warning on line 23 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L23

Added line #L23 was not covered by tests

const expectedCounts: Record<string, number> = {
users: 16,
organizations: 4,
organization_memberships: 18,
};

Check warning on line 29 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L25-L29

Added lines #L25 - L29 were not covered by tests

/**
* Checks record counts in specified tables after data insertion.
* @returns {Promise<boolean>} - Returns true if data matches expected values.
*/
async function checkCountAfterImport(): Promise<boolean> {
try {
const tables = [
{ name: "users", table: schema.usersTable },
{ name: "organizations", table: schema.organizationsTable },
{
name: "organization_memberships",
table: schema.organizationMembershipsTable,
},
];
let allValid = true;
for (const { name, table } of tables) {
const result = await db
.select({ count: sql<number>`count(*)` })
.from(table);
const actualCount = Number(result[0]?.count ?? 0); // Convert actual count to number
const expectedCount = expectedCounts[name]; // Expected count is already a number

Check warning on line 51 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L35-L51

Added lines #L35 - L51 were not covered by tests

if (actualCount !== expectedCount) {
console.error(
`ERROR: Record count mismatch in ${name} (Expected ${expectedCount}, Found ${actualCount})`,
);
allValid = false;
}
}

Check warning on line 59 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L53-L59

Added lines #L53 - L59 were not covered by tests

return allValid;
} catch (error) {
console.error(`ERROR: ${error}`);
return false;
}
}

Check warning on line 66 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L61-L66

Added lines #L61 - L66 were not covered by tests

/**
* Makes an update in the database (Modify the first user's name).
* @returns {Promise<boolean>} - Returns true if the update was successful.
*/
async function updateDatabase(): Promise<boolean> {
const updatedName = "Test User";

Check warning on line 73 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L72-L73

Added lines #L72 - L73 were not covered by tests

try {
const user = await db.select().from(schema.usersTable).limit(1);
if (user.length === 0) {
console.error("ERROR: No user found to update!");
return false;
}

Check warning on line 80 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L75-L80

Added lines #L75 - L80 were not covered by tests

const userId = user[0]?.id;

Check warning on line 82 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L82

Added line #L82 was not covered by tests

await db
.update(schema.usersTable)
.set({ name: updatedName })
.where(sql`id = ${userId}`);

Check warning on line 87 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L84-L87

Added lines #L84 - L87 were not covered by tests

const updatedUser = await db
.select()
.from(schema.usersTable)
.where(sql`id = ${userId}`);

Check warning on line 92 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L89-L92

Added lines #L89 - L92 were not covered by tests

if (updatedUser[0]?.name !== updatedName) {
console.error("ERROR: Database update failed!");
return false;
}
return true;
} catch (error) {
console.error(`ERROR: ${error}`);
return false;
}
}

Check warning on line 103 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L94-L103

Added lines #L94 - L103 were not covered by tests

/**
* Runs the validation and update process.
*/
async function runValidation(): Promise<void> {
try {
const validRecords = await checkCountAfterImport();
if (!validRecords) {
console.error("\nERROR: Database validation failed!");
process.exit(1);
}
console.log("\nDatabase Validation : Success");
const updateSuccess = await updateDatabase();
if (!updateSuccess) {
console.error("\nERROR: Database update validation failed!");
process.exit(1);
}
console.log("Database Updation : Success");
process.exit(0);
} catch (error) {
if (error instanceof Error) {
console.error(`\nERROR: ${error.message}`);
} else {
console.error(`\nERROR: ${String(error)}`);
}
process.exit(1);
}
}

Check warning on line 131 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L108-L131

Added lines #L108 - L131 were not covered by tests

runValidation();

Check warning on line 133 in src/utilities/testDbConnection.ts

View check run for this annotation

Codecov / codecov/patch

src/utilities/testDbConnection.ts#L133

Added line #L133 was not covered by tests
Loading