AWS - CodeDeploy / GitHub Actions _ Flask

목표
 back-flask server GitHub Action을 사용해 workflow 자동화
 AWS CodeDeploy 사용하여 EC2에 자동 배포
 
 참고 
 https://github.com/Kwak-Minju/FIRSTSTEP_BE_AWS.git
 
 * CodeDeploy 먼저 설명되어있지만 GithubAction부터 해보는 것이 더 편리할 것 같음.

CodeDeploy

#1 flask app 과 CodeDeploy 에서 사용할 코드 작성

### appspec.yml ###

version: 0.0
os: linux
files:
  - souce: /
    destination: /home/ubuntu/FIRSTSTEP_BE_AWS
hooks:
  BeforeInstall:
    - location: scripts/before_install.sh
      runas: root
  AfterInstall:
    - location: scripts/after_install.sh
      runas: root

### scripts/before_install.sh ###

#!/bin/bash
rm -rf /home/ubuntu/FIRSTSTEP_BE_AWS
mkdir /home/ubuntu/FIRSTSTEP_BE_AWS

### scripts/after_install.sh ###

#!/bin/bash
cd /home/ubuntu/FIRSTSTEP_BE_AWS

echo ">>> pip install ================="
pip install -r requirements.txt

echo ">>>remove template files =============="
rm -rf appspec.yml requirements.txt

echo ">>> change owner to ubuntu ================"
chown -R ubuntu /home/ubuntu/FIRSTSTEP_BE_AWS

echo ">>> run app ======================"
python3 app.py

#2 git repository 소스 등록

User@DESKTOP-UDC737O MINGW64 /c/FirstStep_project/FIRSTSTEP_BE_AWS (main)
$ git add .

User@DESKTOP-UDC737O MINGW64 /c/FirstStep_project/FIRSTSTEP_BE_AWS (main)
$ git commit -m "codedeploy edit"

User@DESKTOP-UDC737O MINGW64 /c/FirstStep_project/FIRSTSTEP_BE_AWS (main)
$ git push origin

#3 어플리케이션 생성

FlaskApplication

#4 CodeDeploy에서 사용할 역할 생성 - firststepCodeDeployRole

#5 EC2 인스턴스에 적용할 정책과 역할을 생성

firststepCodeDeployEC2Policy / firststepCodeDeployEC2Role

#6 flaskserver 인스턴스 접속 - codedeploy agent 설치 후 실행

# PHP 설치
ubuntu@ip-10-0-2-206:~$ sudo apt install -y php
ubuntu@ip-10-0-2-206:~$ sudo systemctl restart apache2
ubuntu@ip-10-0-2-206:~$ cd /var/www/html

sudo apt update
sudo apt install ruby-full
sudo apt install wget

cd /home/ubuntu
# wget https://bucket-name.s3.region-identifier.amazonaws.com/latest/install
# 서울 region의 경우
wget https://aws-codedeploy-ap-northeast-2.s3.ap-northeast-2.amazonaws.com/latest/install

chmod +x ./install

ubuntu@ip-10-0-10-203:~$ sudo service codedeploy-agent status
● codedeploy-agent.service - LSB: AWS CodeDeploy Host Agent
     Loaded: loaded (/etc/init.d/codedeploy-agent; generated)
     Active: active (running) since Thu 2023-11-23 05:23:24 UTC; 34s ago
       Docs: man:systemd-sysv-generator(8)
    Process: 30031 ExecStart=/etc/init.d/codedeploy-agent start (code=exited, status=0/SUCCESS)
      Tasks: 2 (limit: 1121)
     Memory: 57.3M
        CPU: 1.069s
     CGroup: /system.slice/codedeploy-agent.service
             ├─30037 "codedeploy-agent: master 30037" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" >
             └─30039 "codedeploy-agent: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller of >

Nov 23 05:23:24 ip-10-0-10-203 systemd[1]: Starting LSB: AWS CodeDeploy Host Agent...
Nov 23 05:23:24 ip-10-0-10-203 codedeploy-agent[30031]: Starting codedeploy-agent:
Nov 23 05:23:24 ip-10-0-10-203 systemd[1]: Started LSB: AWS CodeDeploy Host Agent.
lines 1-15/15 (END)

<region별 버킷 참조>

https://docs.aws.amazon.com/ko_kr/codedeploy/latest/userguide/resource-kit.html#resource-kit-bucket-names

#7 배포 그룹 생성

FlaskApplicationDeployGroup

#8 배포 생성

- commit id 확인

#9 flaskserver 인스턴스로 접속해서 로그확인

ubuntu@ip-10-0-10-203:~$ sudo systemctl status codedeploy-agent
● codedeploy-agent.service - LSB: AWS CodeDeploy Host Agent
     Loaded: loaded (/etc/init.d/codedeploy-agent; generated)
     Active: active (running) since Thu 2023-11-23 05:23:24 UTC; 19min ago
       Docs: man:systemd-sysv-generator(8)
    Process: 30031 ExecStart=/etc/init.d/codedeploy-agent start (code=exited, status=0/SUCCESS)
      Tasks: 2 (limit: 1121)
     Memory: 57.8M
        CPU: 1.163s
     CGroup: /system.slice/codedeploy-agent.service
             ├─30037 "codedeploy-agent: master 30037" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" >
             └─30039 "codedeploy-agent: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller of >

Nov 23 05:23:24 ip-10-0-10-203 systemd[1]: Starting LSB: AWS CodeDeploy Host Agent...
Nov 23 05:23:24 ip-10-0-10-203 codedeploy-agent[30031]: Starting codedeploy-agent:
Nov 23 05:23:24 ip-10-0-10-203 systemd[1]: Started LSB: AWS CodeDeploy Host Agent.

배포 작업 디렉터리 확인

**ERROR

ERROR [codedeploy-agent(3039) ]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller:Missing credentials - please check if this instance was started with am IAM instance profile

해당 에러가 발생하게 된 원인

1. IAM 역할을 지정하지 않고 인스턴스를 실행

2. 이후 실행시킨 인스턴스에 CodeDeploy Agent를 설치

3. 이때 설치된 CodeDeploy에는 해당 역할을 실행 할 수 있는 자격증명이 없기에 위와 같은 에러가 발생 하는 것

해결

아무 권한이 없이 역할을 하나 생성해서 EC2에 부여

flaskDummyRole

- 아무것도 지정하지 않고 다음

- codedeploy agent 재시작

sudo service codedeploy-agent restart
ubuntu@ip-10-0-10-203:/opt/codedeploy-agent$ sudo service codedeploy-agent status
● codedeploy-agent.service - LSB: AWS CodeDeploy Host Agent
     Loaded: loaded (/etc/init.d/codedeploy-agent; generated)
     Active: active (running) since Thu 2023-11-23 06:01:21 UTC; 5s ago
       Docs: man:systemd-sysv-generator(8)
    Process: 30234 ExecStart=/etc/init.d/codedeploy-agent start (code=exited, status=0/SUCCESS)
      Tasks: 2 (limit: 1121)
     Memory: 56.4M
        CPU: 1.046s
     CGroup: /system.slice/codedeploy-agent.service
             ├─30241 "codedeploy-agent: master 30241" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "" "">
             └─30243 "codedeploy-agent: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller of master 30241"

Nov 23 06:01:20 ip-10-0-10-203 systemd[1]: Stopped LSB: AWS CodeDeploy Host Agent.
Nov 23 06:01:20 ip-10-0-10-203 systemd[1]: codedeploy-agent.service: Consumed 1.686s CPU time.
Nov 23 06:01:20 ip-10-0-10-203 systemd[1]: Starting LSB: AWS CodeDeploy Host Agent...
Nov 23 06:01:21 ip-10-0-10-203 codedeploy-agent[30234]: Starting codedeploy-agent:
Nov 23 06:01:21 ip-10-0-10-203 systemd[1]: Started LSB: AWS CodeDeploy Host Agent.

**ERROR

CodeDeploy agent was not able to receive the lifecycle event. Check the CodeDeploy agent logs on your host and make sure the agent is running and can commect to the CodeDeploy server.

CodeDeploy에서 이벤트 주기가 가능하지 않으니, 배포하는 서버의 log를 보라는것

경로 : var/log/aws/codedeploy-agent/codedeploy-agent.log

ubuntu@ip-10-0-10-203:/opt/codedeploy-agent$ cat /var/log/aws/codedeploy-agent/codedeploy-agent.log
2023-11-23T06:07:56 WARN  [codedeploy-agent(30243)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Calling PutHostCommandComplete: "Code Error"

WARN [codedeploy-gent(30243)]: InstanceAgent::Plugins::CodeDeployPlugin::CommandPoller: Calling PutHostCommandComplete: "Code Error"

=> code에서 발생하는 오류

해결 : 코드 확인

=> source 가 아닌 souce였다.....ㅎ

### appspec.yml ###

version: 0.0
os: linux
files:
  - source: /
    destination: /home/ubuntu/FIRSTSTEP_BE_AWS
hooks:
  BeforeInstall:
    - location: scripts/before_install.sh
      runas: root
  AfterInstall:
    - location: scripts/after_install.sh
      runas: root

** ERROR

### run_flask.sh ###

echo ">>> run app ==================="
python3 -u app.py > /dev/null 2> /home/ubuntu/flask.log </dev/null &

### after_install.sh ###

#!/bin/bash
cd /home/ubuntu/FIRSTSTEP_BE_AWS

echo ">>> pip install requirements.txt ================="
pip install -r requirements.txt

echo ">>>remove template files =============="
rm -rf appspec.yml requirements.txt

echo ">>> change owner to ubuntu ================"
chown -R ubuntu /home/ubuntu/FIRSTSTEP_BE_AWS

포그라운드에서 실행시 사용자가 다른 작업을 못함

* python3 -u app.py > /dev/null 2> /home/ubuntu/flask.log </dev/null &

- -u: Python의 출력 버퍼링을 사용하지 않도록 하는 옵션

> /dev/null : 표준 출력(stout)을 /dev/null로 리다이렉션하여 모든 출력을 무시

2> /dev/null: 표준 에러 출력을 /dev/null로 리다이렉션하여 모든 에러 메시지를 무시

< /dev/null: 표준 입력을 /dev/null에서 읽도록 지정

/dev/null은 아무것도 없음을 나타내는 파일로, 이 경우에는 표준 입력이 비어있는 것

& 백그라운드 작업으로 실행

=> 출력 및 에러를 무시하면서 백그라운드에서 실행

Github Actions

#1 버킷 생성

firststep-back

#2 Github 레포지터리에 Secrets를 추가

#3 Github Actions 워크플로우를 작성

name: Deploy to Amazon EC2

on:
  push:
    branches: [ "main" ]

env:
  AWS_REGION: ap-northeast-2
  S3_BUCKET_NAME: firststepbucket
  CODE_DEPLOY_APPLICATION_NAME: FlaskApplication
  CODE_DEPLOY_DEPLOY_GROUP_NAME: FlaskApplicationDeployGroup

permissions:
  contents: read

jobs:
  deploy:
    name: Deploy
    runs-on: ubuntu-latest
    environment: production
    steps:
    - name: Checkout
      uses: actions/checkout@v3

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: ${{ env.AWS_REGION }}

    - name: Upload to AWS S3
      run: |
        aws deploy push \
          --application-name ${{ env.CODE_DEPLOY_APPLICATION_NAME }} \
          --s3-location s3://$S3_BUCKET_NAME/$GITHUB_SHA.zip \
          --ignore-hidden-files \
          --source . 

    - name: Deploy to AWS EC2 from S3
      run: |
        aws deploy create-deployment \
          --application-name ${{ env.CODE_DEPLOY_APPLICATION_NAME }} \
          --deployment-config-name CodeDeployDefault.AllAtOnce \
          --deployment-group-name ${{ env.CODE_DEPLOY_DEPLOY_GROUP_NAME }} \
          --s3-location bucket=$S3_BUCKET_NAME,key=$GITHUB_SHA.zip,bundleType=zip

#4 Action 실행 결과를 확인

#5 배포 로그 확인

**ERROR

The overall deployment failed because too many individual instances failed deployment, too few healthy instances are available for deployment, or some instances in your deployment group are experiencing problems.

Access Denied