Skip to content

Commit e22cf98

Browse files
0xkenj10xkenj1
committed
Merge remote-tracking branch 'origin/dev' into fix/copycache-script
2 parents 29d1348 + e88db0b commit e22cf98

File tree

2 files changed

+40
-140
lines changed

2 files changed

+40
-140
lines changed

‎.github/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,8 @@ Before deploying the application, you need to set up the basic infrastructure:
4545
- Sets up Docker image registry
4646
- Run once before the first deployment
4747

48+
Note: Once you deploy ECR registry, you will need to set `ECR_REGISTRY` in the repository settings.
49+
4850
### Base Deployment
4951

5052
Two workflows handle the base deployment operations:
@@ -100,7 +102,6 @@ To properly configure your GitHub repository, set up the following environment v
100102
- `AWS_ACCESS_KEY_ID`
101103
- `AWS_SECRET_ACCESS_KEY`
102104
- `ECR_REGISTRY`
103-
- `ECR_REPOSITORY`
104105

105106
2. Add Repository Environment Variables
106107
Under GitHub Repository Settings → Environment Variables, add:
@@ -248,6 +249,5 @@ The `push-to-ecr.yaml` workflow automatically pushes images to Amazon ECR when c
248249
Required secrets for ECR:
249250

250251
- `ECR_REGISTRY`
251-
- `ECR_REPOSITORY`
252252
- `AWS_ACCESS_KEY_ID`
253253
- `AWS_SECRET_ACCESS_KEY`

‎.github/production_operations.md

Lines changed: 38 additions & 138 deletions
Original file line numberDiff line numberDiff line change
@@ -6,139 +6,20 @@ This guide is for production operations on the Gitcoin Data Layer.
66

77
![Architecture diagram](./architecture_diagram.png)
88

9+
## Prerequisites
10+
11+
- Make sure you have already created the S3 bucket and ECR repository. Refer to the [README.md](./README.md) for more information.
12+
- Make sure you have already created environment variables and secrets in the repository settings. Refer to the [README.md](./README.md) for more information.
13+
914
## First Deployment
1015

11-
Fill the secrets and variables in github actions secrets and variables. those are:
12-
13-
Repository secrets:
14-
15-
- `AWS_ACCESS_KEY_ID`
16-
- `AWS_SECRET_ACCESS_KEY`
17-
- `COINGECKO_API_KEY`
18-
- `DATALAYER_HASURA_ADMIN_SECRET`
19-
- `DATALAYER_PG_PASSWORD`
20-
- `DATALAYER_PG_USER`
21-
- `ECR_REGISTRY`
22-
23-
Repository Variables:
24-
25-
- `APP_NAME`
26-
- `AWS_REGION`
27-
- `TERRAFORM_VARS` (Mostly all the changes will be just the image tags)
28-
29-
example TERRAFORM_VARS:
30-
31-
```json
32-
{
33-
"GREEN_PROCESSING_IMAGE_TAG": "d8cece196697abbdafa5a7027e0b12f0ffe8bd77",
34-
"BLUE_PROCESSING_IMAGE_TAG": "d8cece196697abbdafa5a7027e0b12f0ffe8bd77",
35-
"GREEN_API_REPOSITORY_URL": "registry.hub.docker.com/hasura/graphql-engine",
36-
"GREEN_API_IMAGE_TAG": "v2.23.0",
37-
"GREEN_NODE_ENV": "production",
38-
"GREEN_RETRY_MAX_ATTEMPTS": 10,
39-
"GREEN_RETRY_BASE_DELAY_MS": 200,
40-
"GREEN_RETRY_MAX_DELAY_MS": 1000,
41-
"GREEN_RETRY_FACTOR": 1.5,
42-
"GREEN_CHAINS": [
43-
{
44-
"id": 10,
45-
"name": "optimism",
46-
"rpcUrls": [
47-
"https://optimism.llamarpc.com",
48-
"https://rpc.ankr.com/optimism",
49-
"https://optimism.gateway.tenderly.co",
50-
"https://optimism.blockpi.network/v1/rpc/public",
51-
"https://mainnet.optimism.io",
52-
"https://opt-mainnet.g.alchemy.com/v2/demo"
53-
],
54-
"fetchLimit": 1000,
55-
"fetchDelayMs": 2000
56-
},
57-
{
58-
"id": 1,
59-
"name": "mainnet",
60-
"rpcUrls": ["https://eth.llamarpc.com", "https://rpc.flashbots.net/fast"],
61-
"fetchLimit": 1000,
62-
"fetchDelayMs": 2000
63-
}
64-
],
65-
"GREEN_INDEXER_GRAPHQL_URL": "https://indexer.dev.hyperindex.xyz/e6a0458/v1/graphql",
66-
"GREEN_METADATA_SOURCE": "public-gateway",
67-
"GREEN_PUBLIC_GATEWAY_URLS": [
68-
"https://ipfs.io",
69-
"https://dweb.link",
70-
"https://cloudflare-ipfs.com",
71-
"https://gateway.pinata.cloud",
72-
"https://ipfs.infura.io",
73-
"https://ipfs.fleek.co",
74-
"https://ipfs.eth.aragon.network",
75-
"https://ipfs.jes.xxx",
76-
"https://ipfs.lol",
77-
"https://ipfs.mle.party"
78-
],
79-
"GREEN_PRICING_SOURCE": "coingecko",
80-
"GREEN_COINGECKO_API_TYPE": "pro",
81-
"GREEN_LOG_LEVEL": "info",
82-
"GREEN_DATALAYER_PG_DB_NAME": "GitcoinDatalayerGreen",
83-
"BLUE_API_REPOSITORY_URL": "registry.hub.docker.com/hasura/graphql-engine",
84-
"BLUE_API_IMAGE_TAG": "v2.23.0",
85-
"BLUE_NODE_ENV": "production",
86-
"BLUE_RETRY_MAX_ATTEMPTS": 10,
87-
"BLUE_RETRY_BASE_DELAY_MS": 200,
88-
"BLUE_RETRY_MAX_DELAY_MS": 1000,
89-
"BLUE_RETRY_FACTOR": 1.5,
90-
"BLUE_CHAINS": [
91-
{
92-
"id": 10,
93-
"name": "optimism",
94-
"rpcUrls": [
95-
"https://optimism.llamarpc.com",
96-
"https://rpc.ankr.com/optimism",
97-
"https://optimism.gateway.tenderly.co",
98-
"https://optimism.blockpi.network/v1/rpc/public",
99-
"https://mainnet.optimism.io",
100-
"https://opt-mainnet.g.alchemy.com/v2/demo"
101-
],
102-
"fetchLimit": 1000,
103-
"fetchDelayMs": 2000
104-
},
105-
{
106-
"id": 1,
107-
"name": "mainnet",
108-
"rpcUrls": ["https://eth.llamarpc.com", "https://rpc.flashbots.net/fast"],
109-
"fetchLimit": 1000,
110-
"fetchDelayMs": 2000
111-
}
112-
],
113-
"BLUE_INDEXER_GRAPHQL_URL": "https://indexer.dev.hyperindex.xyz/e6a0458/v1/graphql",
114-
"BLUE_METADATA_SOURCE": "public-gateway",
115-
"BLUE_PUBLIC_GATEWAY_URLS": [
116-
"https://ipfs.io",
117-
"https://dweb.link",
118-
"https://cloudflare-ipfs.com",
119-
"https://gateway.pinata.cloud",
120-
"https://ipfs.infura.io",
121-
"https://ipfs.fleek.co",
122-
"https://ipfs.eth.aragon.network",
123-
"https://ipfs.jes.xxx",
124-
"https://ipfs.lol",
125-
"https://ipfs.mle.party"
126-
],
127-
"BLUE_PRICING_SOURCE": "coingecko",
128-
"BLUE_COINGECKO_API_TYPE": "pro",
129-
"BLUE_LOG_LEVEL": "info",
130-
"BLUE_DATALAYER_PG_DB_NAME": "GitcoinDatalayerBlue"
131-
}
132-
```
133-
134-
Once variables are set, you can deploy the first time by running the `Deploy to AWS (First deployment)` workflow.
135-
136-
Follow this instructions to make the first deployment work:
137-
138-
1. Log in to your AWS account
139-
2. Copy your database endpoint from RDS > Databases > gitcoin-data-layer-staging-rds > Connectivity and security > Endpoint
140-
3. Go to EC2 > Instances > gitcoin-data-layer-production-bastion > Connect > Session Manager > Connect ( IF YOU CAN’T USE `SessionManager` try rebooting the instance)
141-
4. Once in the terminal run:
16+
### Steps
17+
18+
1. Run `Deploy to AWS` workflow, and wait until it finishes.
19+
2. Log in to your AWS account
20+
3. Copy your database endpoint from RDS > Databases > gitcoin-data-layer-staging-rds > Connectivity and security > Endpoint
21+
4. Go to EC2 > Instances > gitcoin-data-layer-production-bastion > Connect > Session Manager > Connect ( IF YOU CAN’T USE `SessionManager` try rebooting the instance)
22+
5. Once in the terminal run:
14223

14324
1. sudo su
14425
2. cd ~
@@ -238,9 +119,10 @@ Follow this instructions to make the first deployment work:
238119

239120
## Upgrade using blue deployment
240121

241-
1. Update the TERRAFORM_VARS with the new image tag, or changes. (You can run `Current Deployment State` workflow to see the current state of the deployment, and the active environment. If the deployment state is `single`, it means that there is just one deployment and is the active one. If the deployment state is `deployment`, it means that there are two deployments, the blue and the green, and the active environment is `active_deployment`.)
242-
2. Run `Deploy Blue Green (Start upgrade - Step 1)` workflow and deploy the target deployment
243-
3. Follow the instructions to set up the target deployment:
122+
1. If you made any changes to Envio indexer, you need to wait until the indexer is ready and stable.
123+
2. Update the TERRAFORM_VARS with the new image tag, or changes. (You can run `Current Deployment State` workflow to see the current state of the deployment, and the active environment. If the deployment state is `single`, it means that there is just one deployment and is the active one. If the deployment state is `deployment`, it means that there are two deployments, the blue and the green, and the active environment is `active_deployment`.)
124+
3. Run `Deploy Blue Green (Start upgrade - Step 1)` workflow and deploy the target deployment
125+
4. Follow the instructions to set up the target deployment:
244126

245127
- Set migrations with target environment nano ./scripts/migrations/.env
246128
```tsx
@@ -252,7 +134,25 @@ Follow this instructions to make the first deployment work:
252134
- pnpm db:copy-cache -f {{ green | blue (should be the source environment) }}
253135
- pnpm db:migrate
254136

255-
4. Wait until the new deployment is stable.
256-
5. Run `Promote Blue Green (Start upgrade - Step 2)` workflow
257-
6. Validate that is stable and working (You can rollback running again `Promote Blue Green (Start upgrade - Step 2)`)
258-
7. Once you are sure that the new deployment is stable, you can destroy the old deployment by running `Destroy Blue Green (Start upgrade - Step 3)` workflow
137+
5. Wait until the new deployment is stable.
138+
6. Run `Promote Blue Green (Start upgrade - Step 2)` workflow
139+
7. Validate that is stable and working, you can go to the hasura api task on ECS and get the IP address of the task and check if the api is working. (You can rollback running again `Promote Blue Green (Start upgrade - Step 2)`)
140+
8. Once you are sure that the new deployment is stable, you can destroy the old deployment by running `Destroy Blue Green (Start upgrade - Step 3)` workflow
141+
142+
## For New chain
143+
144+
1. Wait for the new chain to be fully indexed on Envio indexer.
145+
2. Update the TERRAFORM_VARS with the new chain in `CHAINS` environment variable.
146+
3. Run `Upgrade current deployment` workflow and deploy it to the current working deployment.
147+
148+
## For new event
149+
150+
1. Wait until envio indexer is ready and stable.
151+
2. Update TERRAFORM_VARS with the new event image tag.
152+
3. Do blue green deployment
153+
154+
## For new strategy
155+
156+
1. Wait until envio indexer is ready and stable.
157+
2. Update TERRAFORM_VARS with the new image tag.
158+
3. Do blue green deployment

0 commit comments

Comments
 (0)