Merge pull request #982 from CityOfBoston/develop

Catch-up with main branch Deploys

.github/config/releaseNotesConfig.yml

@@ -0,0 +1,53 @@
+username: 'Digital Team Notices'
+icon_url: https://boston.gov/digitalteamicon.png
+
+pretext: A production deployment has been completed and release notes have been published.
+title: <{{payload.release.html_url}}|Code Release>
+title_link: {{payload.release.html_url}}
+
+text: |
+ A new release has been made for <{{payload.repository.homepage}}|{{payload.repository.name}}>.
+  _{{payload.repository.description}}_
+  <{{payload.release.html_url}}|Click here> to see the Release Notes.
+
+fallback: |-
+  [GitHub] {{payload.release.name}} is released for {{repositoryName}}
+
+blocks:
+  - type: "divider"
+  - type: "header"
+    text: 
+      type: "plain_text"
+      text: "Release Information"
+  - type: section
+    fields:
+    - type: mrkdwn
+      text: "Release Title\n<{{payload.release.html_url}}|{{payload.release.name}}>"
+    - type: mrkdwn
+      text: "Repository Tag\n{{payload.release.tag_name}}"
+    - type: mrkdwn
+      text: "Release Date\n{{truncate payload.release.created_at 10}}"      
+  - type: context
+    elements:
+      - type: image
+        image_url: "{{payload.release.author.avatar_url}}"
+        alt_text: "{{#if (payload.release.author.name)}}{{payload.release.author.name}}{{else}}{{payload.release.author.login}}{{/if}}"
+      - type: mrkdwn
+        text: "Released By <{{payload.release.author.html_url}}|{{#if (payload.release.author.name)}}{{payload.release.author.name}}{{else}}{{payload.release.author.login}}{{/if}}>"
+  - type: "divider"
+
+footer: >-
+  {{payload.enterprise.name}}, <{{payload.repository.homepage}}|{{payload.repository.name}}> <{{payload.release.html_url}}|code release>
+
+colors:
+  success: '#5DADE2'
+  failure: '#884EA0'
+  cancelled: '#A569BD'
+  default: '#7D3C98'
+
+icons:
+  success: ':white_check_mark:'
+  failure: ':grimacing:'
+  cancelled: ':x:'
+  skipped: ':heavy_minus_sign:'
+  default: ':interrobang:'

.github/workflows/releaseNotes.yml

@@ -0,0 +1,19 @@
+name: Broadcast Release Notes
+on:
+  release:
+    types: [released]
+  workflow_dispatch:
+jobs:
+  release:
+    runs-on: ubuntu-latest
+    env:
+      SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
+    steps:
+      - name: Checkout
+        uses: actions/[email protected]    
+      - name: Release Notes
+        uses: act10ns/[email protected]
+        with:
+          status: ${{ job.status }}
+          channel: ${{ vars.SLACK_RELEASE_CHANNEL }}
+          config: .github/config/releaseNotesConfig.yml

RELEASE.md

@@ -0,0 +1,48 @@
+# Release Methodology
+
+## Deploy Pipeline Overview
+For this repository, the deploy pipeline follows these steps:
+- a new working branch is made off `develop`,
+- when ready, the developer then creates a PR for a merge of their working branch back into the `develop` branch, this triggers a build/verification for all apps/services in Travis,
+- Travis finally reports its build status to Slack, 
+- the developer then selects the apps/services to deploy using Slack, triggering an AWS CodeBuild process to build containers and deploy to AWS-ECS. CodeBuild internally updates branches `develop/service-name` and `production/service-name` as needed, (Note to Phill - not sure how deploys to staging cluster and production cluster are differentiated ...)
+- the developer commits the merge for the PR (for the branch to `develop`).
+
+## Lead Developer: Tag and release `Production` branch
+After an app/service is deployed to production, verified and the PR committed, the lead developer must tag and release the felevant `production/service-name` branch/es so that the Project Manager/s can complete their Release Notes.
+1. goto the [release section](https://github.com/CityOfBoston/digital/releases) of the repository,
+2. note the last release number for the app/service, (format _service-name:vYYYY.n_ where YYYY is the year and n is an incrementing integer)
+3. click the "Draft a New Release" button
+4. click on "Choose a Tag" and create a new tag (which follows the release numbering pattern _service-name:vYYYY.n_)
+5. ensure the Target is the correct `production/service-name` branch
+6. give the release a title.  This will be the same as the tag in step 4 above.
+7. in the Description, copy and paste in the template below, then click the `Generate release notes` button to append the commits to be bottom of the textbox. Update the "Jira Tickets` section with all tickets that have been addressed in this release.
+8. click "Set as the latest release",
+9. click the `Save draft` button.
+
+## Project Manager: Release `Production/service-name` branch
+The Project Manager will edit the draft release notes, finalize and publish them.
+1. goto the [release section](https://github.com/CityOfBoston/digital/releases) of the repository,
+2. edit the draft release,
+3. update the *[PM to complete]* block with narrative related to the release,
+4. click "Set as the latest release",
+5. click the `Publish release` button.
+
+A Github action <img src="https://s3-us-west-2.amazonaws.com/slack-files2/bot_icons/2023-02-09/4779927044435_48.png" alt="" style="width: 20px; height: 20px"/> will now fire which will post a message to the slack [#jira-releases channel](https://cityofboston-doit.slack.com/archives/C03UZ01E5N2).
+
+# Release Description Template 
+```
+## [Copy title of production PR]
+
+### Release Notes
+[PM to complete]
+
+### Related Jira tickets
+[Add a list of Jira Tickets addressed in this Release, with links to the Jira website]
+example: Dig-1839 - [Update residential exemption application in Assessing Online](https://bostondoit.atlassian.net/browse/DIG-1839)
+
+```
+## Project Manager: Release Jira Tickets 
+1. In Jira create a release with the following convention RepositoryName/release version (e.g. digital/service-name:v2023.2) 
+2. The release description should include what was updated and a link to the release notes (e.g. service-name code updates [Release Notes](https://github.com/CityOfBoston/boston.gov-d8/releases/tag/service-name:v2023.2))
+3. Attached release fix version to tickets before releasing the tickets. 

deploy/buildspec.yml

@@ -7,7 +7,7 @@ phases:
       - echo $DOCKERHUB_PASSWORD | docker login --username $DOCKERHUB_USERNAME --password-stdin
   install:
     runtime-versions:
-      nodejs: 10
+      nodejs: 14
     commands:
       - node -v
       - npm config set scripts-prepend-node-path true

package.json

@@ -55,5 +55,8 @@
     "lint-staged": "^8.1.0",
     "prettier": "^1.17.0",
     "typescript": "^4.4.2"
+  },
+  "dependencies": {
+    "make": "^0.8.1"
   }
 }

scripts/service-entrypoint.sh

@@ -10,18 +10,16 @@ echo "entrypoint.sh: start"
 if [ -z "$AWS_S3_CONFIG_URL" ]; then
   echo >&2 'error: missing AWS_S3_CONFIG_URL environment variable'
 else
-  aws s3 sync $AWS_S3_CONFIG_URL .
-  if [ -z "$DEPLOY_VARIANT" ]; then
-    echo >&2 'DEPLOY_VARIANT not set'
-  else
-    echo >&2 "syncing $AWS_S3_CONFIG_URL/$DEPLOY_VARIANT"
-    aws s3 sync "$AWS_S3_CONFIG_URL/$DEPLOY_VARIANT" .
-  fi
+  echo "TESTING > service-entrypoint.sh"
+   aws s3 sync $AWS_S3_CONFIG_URL .
+   if [ -z "$DEPLOY_VARIANT" ]; then
+     echo >&2 'DEPLOY_VARIANT not set'
+   else
+     echo >&2 "syncing $AWS_S3_CONFIG_URL/$DEPLOY_VARIANT"
+     aws s3 sync "$AWS_S3_CONFIG_URL/$DEPLOY_VARIANT" .
+   fi
 fi
 
-# Useful for debugging the AWS syncs
-ls
-
 echo "entrypoint.sh: command"
 
 exec "$@"

services-js/access-boston/Dockerfile

@@ -0,0 +1,123 @@
+# syntax=docker/dockerfile:1
+FROM node:14.19.1-alpine as build_phase
+
+# ARG S3_ENV_PATH='s3://cob-digital-apps-staging-config/access-boston/test'
+
+ENV WORKSPACE=access-boston
+ENV NODE_ENV development
+
+WORKDIR /app
+
+#ENV PYTHONUNBUFFERED=1
+RUN apk add --no-cache git openssl \
+   && apk add --update --no-cache python3 curl unzip \
+   && ln -sf python3 /usr/bin/python \
+   && python3 -m ensurepip \
+   && pip3 install --no-cache --upgrade pip setuptools
+
+# To prevent “Error: could not get uid/gid”
+RUN npm config set unsafe-perm true
+
+# Need to upgrade yarn to at least 1.6
+RUN yarn global add yarn@^1.6.0
+
+ADD . /app/
+
+# This is the tar'd up collection of package.json files created by
+# build-service-container.sh. Working with it and the lockfiles means we can
+# cache the yarn install across builds when there are no dependency changes.
+#ADD package-json.tar /app/
+#ADD yarn.lock lerna.json .yarnrc /app/
+
+RUN /app/scripts/generate-ssl-key.sh /app/services-js/$WORKSPACE
+
+RUN yarn install
+
+# This is the time consuming step +/-300secs
+WORKDIR /app/services-js/$WORKSPACE
+RUN yarn install --ignore-scripts
+
+# syntax=docker/dockerfile:1
+FROM node:14.19.1-alpine as deploy_phase
+
+ENV WORKSPACE access-boston
+ENV NODE_ENV development
+ENV USE_SSL true
+
+COPY --from=build_phase /app /app
+
+ENV PYTHONUNBUFFERED=1
+RUN apk add --no-cache git openssl \
+   && apk add --update --no-cache python3 curl unzip \
+   && ln -sf python3 /usr/bin/python \
+   && python3 -m ensurepip \
+   && pip3 install --no-cache --upgrade pip setuptools \
+   && cd /tmp \
+   && curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip" \
+   && unzip awscli-bundle.zip \
+   && ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws \
+   && rm awscli-bundle.zip \
+   && rm -rf awscli-bundle
+
+WORKDIR /app/services-js/${WORKSPACE}
+RUN yarn run build
+
+#ADD ./.aws /root/.aws
+#RUN --mount=type=secret,id=aws,target=/root/.aws/credentials
+
+RUN --mount=type=secret,id=aws,target=/root/.aws/credentials \
+  BUILD_ID=$(cat /app/services-js/access-boston/build/.next/BUILD_ID) && \
+  aws s3 cp --recursive --acl public-read /app/services-js/access-boston/build/.next/static/${BUILD_ID} s3://cob-digital-apps-staging-static/access-boston/_next/static/${BUILD_ID}/ && \
+  aws s3 cp --recursive --acl public-read /app/services-js/access-boston/build/.next/static/chunks s3://cob-digital-apps-staging-static/access-boston/_next/static/chunks/ && \
+  aws s3 cp --recursive --acl public-read /app/services-js/access-boston/build/.next/static/runtime s3://cob-digital-apps-staging-static/access-boston/_next/static/runtime/ && \
+  aws s3 cp --recursive --acl public-read /app/services-js/access-boston/build/.next/static/${BUILD_ID} s3://cob-digital-apps-prod-static/access-boston/_next/static/${BUILD_ID}/ && \
+  aws s3 cp --recursive --acl public-read /app/services-js/access-boston/build/.next/static/chunks s3://cob-digital-apps-prod-static/access-boston/_next/static/chunks/ && \
+  aws s3 cp --recursive --acl public-read /app/services-js/access-boston/build/.next/static/runtime s3://cob-digital-apps-prod-static/access-boston/_next/static/runtime/ || \
+    echo "OOPS"
+
+EXPOSE 3000
+
+ENV NODE_ENV production
+
+ENTRYPOINT ["/app/scripts/service-entrypoint.sh"]
+CMD ["yarn", "start"]
+
+###################################
+# NOTES:
+#   To use this file:
+#       [a] clone the digital repo
+#       [b] from a terminal, in the access-boston folder, run
+#           DOCKER_BUILDKIT=1 docker build --pull --cache-from local-test/access-boston:latest -f ./Dockerfile -t local-test/access-boston:latest --secret id=aws,src=$HOME/.aws/credentials ../..
+#           -> this will create an image tagged local-test/access-boston:latest on the local machine
+#       [c] from a terminal, in the access-boston folder, run
+#           docker compose up --no-build -d access-boston
+#           -> this will create a container which should start on your local machine, with your cloned repo mounted into
+#               so that your changes are immediately effective in the browser.
+#           -> you should be able to see the webapp at https://127.0.0.1:300/group-mgmt ..etc
+#       [d] In a terminal app, open a session in the container by running:
+#           docker exec -it access-boston /bin/sh
+#           -> This has effectively ssh'd you into the container as the defaul (root) user.
+#       [e] In the container session, you can start the yarn watcher
+#           yarn dev
+#           -> this will watch the /app folder (which is a mounted copy from your local computer) and recompile as changes
+#               are made.  Thus, you can code and test simultaneously (slight delay as the rebuilds occur)
+#   WHEN YOU ARE READY TO DEPLOY TO STAGE:
+#       [1] Check your code into github
+#       [2] Close down any shells into the container and stop the container:
+#           docker stop access-boston
+#       [2.5] Re-login into ECR with this command --> aws ecr get-login-password --region us-east-1 --profile=cityofboston | docker login --username AWS --password-stdin 251803681989.dkr.ecr.us-east-1.amazonaws.com 
+#       [3] Rebuild the container and apps by running:
+#           DOCKER_BUILDKIT=1 docker build --pull --cache-from local-test/access-boston:latest -f ./Dockerfile -t 251803681989.dkr.ecr.us-east-1.amazonaws.com/cob-digital-apps-staging/access-boston:deploy-new-stage --secret id=aws,src=$HOME/.aws/credentials ../..
+#           -> this is essentially the same command as in [b] above, just uses a different tag and ensures the /app folder
+#              is physically there and not mounted (using the docker-compose command in [c] above mounts the repo over
+#              whatever was added into the the image's /app folder during the docker build command)
+#       [4] Push the image to AWS by running
+#           docker push 251803681989.dkr.ecr.us-east-1.amazonaws.com/cob-digital-apps-staging/access-boston:deploy-new-stage
+#           -> the deploy should start once the image is transferred/uploaded
+#
+#   WHEN YOU ARE READY TO DEPLOY TO PROD:
+#       [1] Tag the image you pushed to stage with a production tag:
+#           docker tag 251803681989.dkr.ecr.us-east-1.amazonaws.com/cob-digital-apps-staging/access-boston:deploy-new-stage 251803681989.dkr.ecr.us-east-1.amazonaws.com/cob-digital-apps-prod/access-boston:deploy-new-prod
+#       [2] Push the image to AWS by running
+#           docker push 251803681989.dkr.ecr.us-east-1.amazonaws.com/cob-digital-apps-prod/access-boston:deploy-new-prod
+#           -> the deploy should start once the image is transferred/uploaded

services-js/access-boston/README.md

@@ -93,8 +93,10 @@ We also need to generate keys.
 http://localhost:3000/
 
 ### Staging
-DEV: https://access-boston.dev.digital-staging.boston.gov/
-TEST: http://access-test.boston.gov/
+DEV:
+ - https://access-boston-dev.digital-staging.boston.gov/
+TEST: 
+ - http://access-test.boston.gov/
 
 ### PROD
 https://access-boston.boston.gov/