Conversation
validation change
field validation change
HG-304 AEF: Excel and CSV Report generation
fixes for HG-196, HG-199, HG-200, HG-203
Reports file download
cyclic issue fix
field change
modified AEF reports and fixed issues in AEF report
Hg 356,HG-408,HG-378
…hen loggin project stage
Signed-off-by: palindaa <palinda@xeptagon.com>
| name: Deploy Pre Check | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| backend-changes: ${{ steps.changes.outputs.backend-changes }} | ||
| all-changes: ${{ steps.changes.outputs.all-changes }} | ||
| workflows-changes: ${{ steps.changes.outputs.workflows-changes }} | ||
| frontend-changes: ${{ steps.changes.outputs.frontend-changes }} | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| with: | ||
| fetch-depth: 0 | ||
| ref: ${{ github.event.inputs.branch_name || github.ref }} | ||
| - name: Use Node.js ${{ matrix.node-version }} | ||
| uses: actions/setup-node@v3 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
| - name: Determine changed services | ||
| id: changes | ||
| run: | | ||
| CHANGED_FILES=$(git diff --name-only HEAD^ HEAD) | ||
|
|
||
| if echo "$CHANGED_FILES" | grep -q "docker-"; then | ||
| echo "All changes detected." | ||
| echo "all-changes=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "No All changes detected." | ||
| echo "all-changes=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| if echo "$CHANGED_FILES" | grep -q ".github/workflows/"; then | ||
| echo "Workflow changes detected." | ||
| echo "workflows-changes=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "No Workflow changes detected." | ||
| echo "workflows-changes=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| if echo "$CHANGED_FILES" | grep -q "web/"; then | ||
| echo "Frontend changes detected." | ||
| echo "frontend-changes=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "No Frontend changes detected." | ||
| echo "frontend-changes=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| if echo "$CHANGED_FILES" | grep -q "backend/"; then | ||
| echo "Backend changes detected." | ||
| echo "backend-changes=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "No Backend changes detected." | ||
| echo "backend-changes=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| deploy_frontend: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 months ago
To fix the issue, we need to add a permissions block to the workflow. This block should specify the minimal permissions required for the workflow to function correctly. Based on the operations performed in the workflow, the contents: read permission is sufficient for most jobs. For jobs that require additional permissions, such as deploying to AWS, we can add specific permissions as needed.
The permissions block can be added at the root level of the workflow to apply to all jobs or at the job level to customize permissions for individual jobs. In this case, we will add the permissions block at the root level to ensure all jobs inherit the least privilege by default.
| @@ -1,2 +1,4 @@ | ||
| name: Carbon Registry Demo Deployment | ||
| permissions: | ||
| contents: read | ||
| on: |
| name: Carbon Registry Frontend Deploy | ||
| needs: changes | ||
| if: needs.changes.outputs.frontend-changes == 'true' || needs.changes.outputs.workflows-changes == 'true' || needs.changes.outputs.all-changes == 'true' | ||
| runs-on: ubuntu-latest | ||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }}-'frontend' | ||
| cancel-in-progress: true | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| with: | ||
| fetch-depth: 5 | ||
| - name: Check Docker variable changes | ||
| run: | | ||
| echo "docker-compose file Change...." | ||
| git diff HEAD~ -- HEAD -z ./docker-compose* | ||
| - name: Cache modules | ||
| uses: actions/cache@v3 | ||
| id: yarn-cache | ||
| with: | ||
| path: node_modules | ||
| key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }} | ||
| restore-keys: ${{ runner.os }}-yarn- | ||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v1 | ||
| with: | ||
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| aws-region: us-east-1 | ||
| - name: Use Node.js ${{ matrix.node-version }} | ||
| uses: actions/setup-node@v3 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
| - name: Login to Amazon ECR | ||
| id: login-ecr | ||
| uses: aws-actions/amazon-ecr-login@v1 | ||
| - name: Build, tag, and push the image to Amazon ECR | ||
| id: build-image | ||
| env: | ||
| ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }} | ||
| ECR_REPOSITORY: carbon-web | ||
| IMAGE_TAG: ${{ github.head_ref || github.ref_name }} | ||
| run: | | ||
| # Build a docker container and push it to ECR | ||
| docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG -f web/Dockerfile . --build-arg PORT=3030 --build-arg VITE_APP_BACKEND=http://localhost:3000 --build-arg VITE_APP_COUNTRY_NAME="CountryX" --build-arg VITE_APP_REGISTRY_NAME="CountryXRegistry" --build-arg VITE_APP_MAP_TYPE="Mapbox" --build-arg VITE_APP_MAPBOXGL_ACCESS_TOKEN=${{ secrets.MAPBOXGL_ACCESS_TOKEN }} --build-arg VITE_APP_MAXIMUM_FILE_SIZE=5242880 --build-arg COUNTRY_FLAG_URL="https://carbon-common-dev.s3.amazonaws.com/flag.png" | ||
| echo "Pushing image to ECR..." | ||
| docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG | ||
| echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" | ||
| - name: Deploy docker image to Amazon EC2 | ||
| if: github.ref == 'refs/heads/main' | ||
| env: | ||
| ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }} | ||
| ECR_REPOSITORY: carbon-web | ||
| IMAGE_TAG: ${{ github.head_ref || github.ref_name }} | ||
| PRIVATE_KEY: ${{ secrets.AWS_SSH_KEY_PRIVATE_DEMO }} | ||
| HOSTNAME: ${{secrets.HOST_IP_DEMO }} | ||
| USER_NAME: ec2-user | ||
| run: | | ||
| echo "$PRIVATE_KEY" > private_key && chmod 600 private_key | ||
| ssh -o StrictHostKeyChecking=no -i private_key ${USER_NAME}@${HOSTNAME} ' | ||
| carbon/prod_frontend_deploy.sh ' | ||
|
|
||
| deploy_backend: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 months ago
To fix the issue, we need to add a permissions block to the workflow. This block should specify the minimal permissions required for the workflow to function correctly. Since the workflow primarily involves reading repository contents and deploying Docker images, the contents: read permission is sufficient. If specific jobs require additional permissions, they can be defined within the respective job blocks.
The permissions block should be added at the root level of the workflow to apply to all jobs. Alternatively, permissions can be set for individual jobs if different levels of access are required.
| @@ -1,2 +1,4 @@ | ||
| name: Carbon Registry Demo Deployment | ||
| permissions: | ||
| contents: read | ||
| on: |
| name: Carbon Registry Backend Deploy | ||
| needs: changes | ||
| if: needs.changes.outputs.backend-changes == 'true' || needs.changes.outputs.workflows-changes == 'true' || needs.changes.outputs.all-changes == 'true' | ||
| runs-on: ubuntu-latest | ||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }}-'backend' | ||
| cancel-in-progress: true | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| with: | ||
| fetch-depth: 5 | ||
| - name: Check Docker variable changes | ||
| run: | | ||
| echo "docker-compose file Change...." | ||
| git diff HEAD~ -- HEAD -z ./docker-compose* | ||
| - name: Use Node.js ${{ matrix.node-version }} | ||
| uses: actions/setup-node@v3 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v1 | ||
| with: | ||
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| aws-region: us-east-1 | ||
| - name: Login to Amazon ECR | ||
| id: login-ecr | ||
| uses: aws-actions/amazon-ecr-login@v1 | ||
| - name: Build, tag, and push the image to Amazon ECR | ||
| id: build-image | ||
| env: | ||
| ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }} | ||
| ECR_REPOSITORY: carbon-services | ||
| IMAGE_TAG: ${{ github.head_ref || github.ref_name }} | ||
| run: | | ||
| # Build a docker container and push it to ECR | ||
| docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG -f backend/services/Dockerfile . | ||
| echo "Pushing image to ECR..." | ||
| docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG | ||
| echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" | ||
| - name: Deploy docker image to Amazon EC2 | ||
| if: github.ref == 'refs/heads/main' | ||
| env: | ||
| PRIVATE_KEY: ${{ secrets.AWS_SSH_KEY_PRIVATE_DEMO }} | ||
| HOSTNAME: ${{secrets.HOST_IP_DEMO }} | ||
| USER_NAME: ec2-user | ||
| run: | | ||
| echo "$PRIVATE_KEY" > private_key && chmod 600 private_key | ||
| ssh -o StrictHostKeyChecking=no -i private_key ${USER_NAME}@${HOSTNAME} ' | ||
| carbon/prod_backend_deploy.sh ' |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 months ago
To fix the issue, we will add a permissions block to the root of the workflow file. This block will define the minimal permissions required for the workflow to function correctly. Based on the tasks performed in the workflow (e.g., checking out code, configuring AWS credentials, and deploying to EC2), the contents: read permission is sufficient for most steps. If any specific steps require additional permissions (e.g., pull-requests: write), they can be added explicitly.
| @@ -1,2 +1,4 @@ | ||
| name: Carbon Registry Demo Deployment | ||
| permissions: | ||
| contents: read | ||
| on: |
| name: Deploy Pre Check | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| backend-changes: ${{ steps.changes.outputs.backend-changes }} | ||
| all-changes: ${{ steps.changes.outputs.all-changes }} | ||
| workflows-changes: ${{ steps.changes.outputs.workflows-changes }} | ||
| frontend-changes: ${{ steps.changes.outputs.frontend-changes }} | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| with: | ||
| fetch-depth: 0 | ||
| ref: ${{ github.event.inputs.branch_name || github.ref }} | ||
| - name: Use Node.js ${{ matrix.node-version }} | ||
| uses: actions/setup-node@v3 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
| - name: Determine changed services | ||
| id: changes | ||
| run: | | ||
| CHANGED_FILES=$(git diff --name-only HEAD^ HEAD) | ||
|
|
||
| if echo "$CHANGED_FILES" | grep -q "docker-"; then | ||
| echo "All changes detected." | ||
| echo "all-changes=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "No All changes detected." | ||
| echo "all-changes=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| if echo "$CHANGED_FILES" | grep -q ".github/workflows/"; then | ||
| echo "Workflow changes detected." | ||
| echo "workflows-changes=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "No Workflow changes detected." | ||
| echo "workflows-changes=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| if echo "$CHANGED_FILES" | grep -q "web/"; then | ||
| echo "Frontend changes detected." | ||
| echo "frontend-changes=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "No Frontend changes detected." | ||
| echo "frontend-changes=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| if echo "$CHANGED_FILES" | grep -q "backend/"; then | ||
| echo "Backend changes detected." | ||
| echo "backend-changes=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "No Backend changes detected." | ||
| echo "backend-changes=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| deploy_frontend: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 months ago
To fix the issue, we need to add a permissions block to the workflow. This block should specify the minimal permissions required for the workflow to function correctly. Based on the operations performed in the workflow, the following permissions are recommended:
contents: readfor accessing repository contents.packages: writefor pushing Docker images to Amazon ECR.id-token: writeif the workflow uses OpenID Connect for authentication with AWS.
The permissions block can be added at the root level of the workflow to apply to all jobs or at the job level for more granular control.
| @@ -1,2 +1,6 @@ | ||
| name: Carbon Registry Test Deployment | ||
| permissions: | ||
| contents: read | ||
| packages: write | ||
| id-token: write | ||
| on: |
| name: Carbon Registry Frontend Deploy | ||
| needs: changes | ||
| if: needs.changes.outputs.frontend-changes == 'true' || needs.changes.outputs.workflows-changes == 'true' || needs.changes.outputs.all-changes == 'true' | ||
| runs-on: ubuntu-latest | ||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }}-'frontend' | ||
| cancel-in-progress: true | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| with: | ||
| fetch-depth: 5 | ||
| - name: Check Docker variable changes | ||
| run: | | ||
| echo "docker-compose file Change...." | ||
| git diff HEAD~ -- HEAD -z ./docker-compose* | ||
| - name: Cache modules | ||
| uses: actions/cache@v3 | ||
| id: yarn-cache | ||
| with: | ||
| path: node_modules | ||
| key: ${{ runner.os }}-yarn-${{ hashFiles('**/yarn.lock') }} | ||
| restore-keys: ${{ runner.os }}-yarn- | ||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v1 | ||
| with: | ||
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| aws-region: us-east-1 | ||
| - name: Use Node.js ${{ matrix.node-version }} | ||
| uses: actions/setup-node@v3 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
| - name: Login to Amazon ECR | ||
| id: login-ecr | ||
| uses: aws-actions/amazon-ecr-login@v1 | ||
| - name: Build, tag, and push the image to Amazon ECR | ||
| id: build-image | ||
| env: | ||
| ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }} | ||
| ECR_REPOSITORY: carbon-web | ||
| IMAGE_TAG: ${{ github.head_ref || github.ref_name }} | ||
| run: | | ||
| # Build a docker container and push it to ECR | ||
| docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG -f web/Dockerfile . --build-arg PORT=3030 --build-arg VITE_APP_BACKEND=http://localhost:3000 --build-arg VITE_APP_COUNTRY_NAME="CountryX" --build-arg VITE_APP_REGISTRY_NAME="CountryXRegistry" --build-arg VITE_APP_MAP_TYPE="Mapbox" --build-arg VITE_APP_MAPBOXGL_ACCESS_TOKEN=${{ secrets.MAPBOXGL_ACCESS_TOKEN }} --build-arg VITE_APP_MAXIMUM_FILE_SIZE=5242880 --build-arg COUNTRY_FLAG_URL="https://carbon-common-dev.s3.amazonaws.com/flag.png" | ||
| echo "Pushing image to ECR..." | ||
| docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG | ||
| echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" | ||
| - name: Deploy docker image to Amazon EC2 | ||
| if: github.ref == 'refs/heads/develop' | ||
| env: | ||
| ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }} | ||
| ECR_REPOSITORY: carbon-web | ||
| IMAGE_TAG: ${{ github.head_ref || github.ref_name }} | ||
| PRIVATE_KEY: ${{ secrets.AWS_SSH_KEY_PRIVATE_TEST }} | ||
| HOSTNAME: ${{secrets.HOST_IP_TEST }} | ||
| USER_NAME: ec2-user | ||
| run: | | ||
| echo "$PRIVATE_KEY" > private_key && chmod 600 private_key | ||
| ssh -o StrictHostKeyChecking=no -i private_key ${USER_NAME}@${HOSTNAME} ' | ||
| carbon/dev_frontend_deploy.sh ' | ||
|
|
||
| deploy_backend: |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 months ago
To fix the issue, we will add a permissions block to the root of the workflow file. This block will define the minimal permissions required for the workflow to function correctly. Based on the actions used in the workflow, such as actions/checkout, aws-actions/configure-aws-credentials, and aws-actions/amazon-ecr-login, the workflow primarily requires contents: read for accessing the repository contents. Additional permissions like pull-requests: write or others can be added if specific steps require them.
| @@ -1,2 +1,4 @@ | ||
| name: Carbon Registry Test Deployment | ||
| permissions: | ||
| contents: read | ||
| on: |
| name: Carbon Registry Backend Deploy | ||
| needs: changes | ||
| if: needs.changes.outputs.backend-changes == 'true' || needs.changes.outputs.workflows-changes == 'true' || needs.changes.outputs.all-changes == 'true' | ||
| runs-on: ubuntu-latest | ||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }}-'backend' | ||
| cancel-in-progress: true | ||
| steps: | ||
| - uses: actions/checkout@v3 | ||
| with: | ||
| fetch-depth: 5 | ||
| - name: Check Docker variable changes | ||
| run: | | ||
| echo "docker-compose file Change...." | ||
| git diff HEAD~ -- HEAD -z ./docker-compose* | ||
| - name: Use Node.js ${{ matrix.node-version }} | ||
| uses: actions/setup-node@v3 | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
| - name: Configure AWS credentials | ||
| uses: aws-actions/configure-aws-credentials@v1 | ||
| with: | ||
| aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} | ||
| aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} | ||
| aws-region: us-east-1 | ||
| - name: Login to Amazon ECR | ||
| id: login-ecr | ||
| uses: aws-actions/amazon-ecr-login@v1 | ||
| - name: Build, tag, and push the image to Amazon ECR | ||
| id: build-image | ||
| env: | ||
| ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }} | ||
| ECR_REPOSITORY: carbon-services | ||
| IMAGE_TAG: ${{ github.head_ref || github.ref_name }} | ||
| run: | | ||
| # Build a docker container and push it to ECR | ||
| docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG -f backend/services/Dockerfile . | ||
| echo "Pushing image to ECR..." | ||
| docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG | ||
| echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" | ||
| - name: Deploy docker image to Amazon EC2 | ||
| if: github.ref == 'refs/heads/develop' | ||
| env: | ||
| PRIVATE_KEY: ${{ secrets.AWS_SSH_KEY_PRIVATE_TEST }} | ||
| HOSTNAME: ${{secrets.HOST_IP_TEST }} | ||
| USER_NAME: ec2-user | ||
| run: | | ||
| echo "$PRIVATE_KEY" > private_key && chmod 600 private_key | ||
| ssh -o StrictHostKeyChecking=no -i private_key ${USER_NAME}@${HOSTNAME} ' | ||
| carbon/dev_backend_deploy.sh ' |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 months ago
To fix the issue, we need to add a permissions block to the workflow. This block should specify the minimal permissions required for the workflow to function correctly. Since the workflow involves checking out code, interacting with AWS, and deploying Docker images, the permissions should be limited to contents: read for accessing repository files and id-token: write for AWS authentication if necessary.
The permissions block can be added at the root level of the workflow to apply to all jobs or within each job to customize permissions for specific tasks. In this case, adding it at the root level is sufficient and ensures consistency across all jobs.
| @@ -1,2 +1,5 @@ | ||
| name: Carbon Registry Test Deployment | ||
| permissions: | ||
| contents: read | ||
| id-token: write | ||
| on: |
|
Hi @palindaa! I'm new to this team and still being onboarded. Given this is such a large PR, do you have an overview of the additions, features, bugfixes, etc that are included in this patch? |
|
Hi @Nolski , These are the changes and bug fixes requested and approved by @zungundp . Please find the full task list in this document. UNDP Carbon Registry Task List.csv |
In some configurations of docker, this causes the builds to fail
Fixed some issues when building and running Phase 2
No description provided.