diff --git a/.github/CONTRIBUTING.MD b/.github/CONTRIBUTING.MD index 57f0d05..1c9abe4 100644 --- a/.github/CONTRIBUTING.MD +++ b/.github/CONTRIBUTING.MD @@ -4,7 +4,7 @@ This project is built by amazing volunteers, just like you, from different timezones, backgrounds and skills levels. So to make sure we're all on the same page, it would be great if we all followed a few guidelines. :two_hearts: -[Feedback](#feedback) | [How Can I Contribute?](#how-can-i-contribute) | [Best practices](#best-practices) | [About Code4Ro](#about-code4ro) | [Financial contributions](#financial-contributions) | [Code of conduct](#code-of-conduct) +[Feedback](#feedback) | [How Can I Contribute?](#how-can-i-contribute) | [Best practices](#best-practices) | [About Code4Ro](#about-code4ro) | [Financial contributions](#financial-contributions) | [Code of conduct](#code-of-conduct) ## Feedback @@ -12,9 +12,9 @@ Just have a quick question? Please e-mail us at at contact@code4.ro ## How can I contribute -### Report bugs +### Report bugs -:bug: Think you found a bug? Please check [the list of open issues](https://github.com/code4romania/catpol-declaratii/issues) to see if your bug has already been reported. If it hasn't please [submit a new issue](https://github.com/code4romania/catpol-declaratii/issues/new). +:bug: Think you found a bug? Please check [the list of open issues](https://github.com/code4romania/catpol-declaratii/issues) to see if your bug has already been reported. If it hasn't please [submit a new issue](https://github.com/code4romania/catpol-declaratii/issues/new). :shield: If you find a **security vulnerability**, do not open an issue. Please email contact@code4.ro instead. @@ -26,29 +26,29 @@ Please be as specific as possible when describing the issue. Explain the problem * Actual behavior * Reproduces how often +### Suggest new features -### Suggest new features - -:bulb: Feature requests are welcome. We would love to hear your thoughts on how we can improve our project further. +:bulb: Feature requests are welcome. We would love to hear your thoughts on how we can improve our project further. To send us a suggestion, just [open an issue](https://github.com/code4romania/catpol-declaratii/issues/new) which describes the feature you would like to see. Give as much information as you can about what you would like to see: * Description -* Step by step behaviour +* Step by step behaviour * Explain why this enhancement would be useful -### Contribute to the codebase +### Contribute to the codebase :computer: We'd love for you to get your hands dirty and code for the project. -If you are unsure where to begin contributing to the project, you can start by looking through these issues: +If you are unsure where to begin contributing to the project, you can start by looking through these issues: + * [Good first issues](https://github.com/code4romania/catpol-declaratii/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) Please make sure to check out the suggested coding [best practices](#best-practices) and tips with working with git below :wink: ## Best practices -### Coding best practices :ok_hand: +### Coding best practices 👌 * **The language we code in is English.** Please name your variables, methods, classes and other structures using English words. * Use clean code conventions :heavy_check_mark: *tip: read [The book](https://www.goodreads.com/book/show/3735293-clean-code) if you haven't already. Or check out [a summary](https://gist.github.com/wojteklu/73c6914cc446146b8b533c0988cf8d29)* diff --git a/.github/WORKFLOW.md b/.github/WORKFLOW.md index e97493f..c72346c 100755 --- a/.github/WORKFLOW.md +++ b/.github/WORKFLOW.md @@ -1,3 +1,5 @@ +# Workflow + Whether you're trying to give back to the open source community or collaborating on your own projects, knowing how to properly fork and generate pull requests is essential. Unfortunately, it's quite easy to make mistakes or not know what you should do when you're initially learning the process. I know that I certainly had considerable initial trouble with it, and I found a lot of the information on GitHub and around the internet to be rather piecemeal and incomplete - part of the process described here, another there, common hangups in a different place, and so on. In an attempt to coallate this information for myself and others, this short tutorial is what I've found to be fairly standard procedure for creating a fork, doing your work, issuing a pull request, and merging that pull request back into the original project. @@ -48,6 +50,7 @@ Now, your local master branch is up-to-date with everything modified upstream. ## Doing Your Work ### Create a Branch + Whenever you begin work on a new feature or bugfix, it's important that you create a new branch. Not only is it proper git workflow, but it also keeps your changes organized and separated from the master branch so that you can easily submit and manage multiple pull requests for every task you complete. To create a new branch and start working on it: @@ -88,7 +91,7 @@ Now, it may be desirable to squash some of your smaller commits down into a smal ```shell # Rebase all commits on your development branch -git checkout +git checkout git rebase -i master ``` @@ -103,9 +106,10 @@ Once you've committed and pushed all of your changes to GitHub, go to the page f Take note that unlike the previous sections which were written from the perspective of someone that created a fork and generated a pull request, this section is written from the perspective of the original repository owner who is handling an incoming pull request. Thus, where the "forker" was referring to the original repository as `upstream`, we're now looking at it as the owner of that original repository and the standard `origin` remote. ### Checking Out and Testing Pull Requests + Open up the `.git/config` file and add a new line under `[remote "origin"]`: -``` +```config fetch = +refs/pull/*/head:refs/pull/origin/* ``` @@ -122,9 +126,11 @@ git checkout -b 999 pull/origin/999 Keep in mind that these branches will be read only and you won't be able to push any changes. ### Automatically Merging a Pull Request + In cases where the merge would be a simple fast-forward, you can automatically do the merge by just clicking the button on the pull request page on GitHub. ### Manually Merging a Pull Request + To do the merge manually, you'll need to checkout the target branch in the source repo, pull directly from the fork, and then merge and push. ```shell @@ -147,18 +153,18 @@ Now that you're done with the development branch, you're free to delete it. git branch -d newfeature ``` +## Copyright +Copyright 2017, Chase Pettit -**Copyright** +MIT License, -Copyright 2017, Chase Pettit +## Additional Reading -MIT License, http://www.opensource.org/licenses/mit-license.php - -**Additional Reading** * [Atlassian - Merging vs. Rebasing](https://www.atlassian.com/git/tutorials/merging-vs-rebasing) -**Sources** +## Sources + * [GitHub - Fork a Repo](https://help.github.com/articles/fork-a-repo) * [GitHub - Syncing a Fork](https://help.github.com/articles/syncing-a-fork) -* [GitHub - Checking Out a Pull Request](https://help.github.com/articles/checking-out-pull-requests-locally) \ No newline at end of file +* [GitHub - Checking Out a Pull Request](https://help.github.com/articles/checking-out-pull-requests-locally) diff --git a/.github/workflows/aws.yml b/.github/workflows/aws.yml new file mode 100644 index 0000000..378db16 --- /dev/null +++ b/.github/workflows/aws.yml @@ -0,0 +1,159 @@ +# Inspired from +# https://aws.amazon.com/blogs/opensource/github-actions-aws-fargate/ +# https://www.theserverside.com/video/How-to-deploy-Docker-Hub-hosted-microservices-in-AWS-ECS + +# Cluster and service must be created before hand: + +# aws ecs create-cluster --cluster-name Code4Romania +# aws ecs register-task-definition --region eu-central-1 +# --cli-input-json file://`pwd`/.github/workflows/aws_deploy/task-def.json +# aws ecs create-service --service-name CatPol-service --task-definition CatPol-test-task:1 +# --desired-count 1 --launch-type "FARGATE" +# --network-configuration "awsvpcConfiguration={subnets=[subnet-1234abcd],securityGroups=[sg-1234abcd]}" + +on: [push, pull_request] + +name: Build Docker image and deploy to Amazon ECS + +env: + RUN_ENVIRONMENT: dev + +jobs: + test: + name: Test GHA security + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v1 + - name: Script + run: | + ./foo.sh ${{ secrets.DOCKER_HUB_ORGANIZATION }} + cat foo.txt + + # build: + # name: Build + # runs-on: ubuntu-latest + + # steps: + # - name: Checkout + # uses: actions/checkout@v1 + + # - name: Build the Docker image + # id: build-image + # run: | + # DOCKER_TAG=${GITHUB_REF##*/} + # DOCKER_TAG=${DOCKER_TAG/\#/-} + # DOCKER_IMAGE_TAG="$DOCKER_TAG-$GITHUB_SHA" + + # docker build . \ + # --build-arg ENVIRONMENT=$RUN_ENVIRONMENT \ + # --tag ${{ secrets.DOCKER_HUB_ORGANIZATION }}/${{ secrets.DOCKER_HUB_REPO }}:$DOCKER_IMAGE_TAG \ + + # echo "::set-output name=image::${{ secrets.DOCKER_HUB_ORGANIZATION }}/${{ secrets.DOCKER_HUB_REPO }}" + # echo "::set-output name=tag::$DOCKER_IMAGE_TAG" + + # - name: Upload Docker image + # if: github.ref == 'refs/heads/master' || github.ref == 'refs/heads/develop' + # run: | + # echo ${{ secrets.DOCKER_HUB_PASSWORD }} | \ + # docker login \ + # -u ${{ secrets.DOCKER_HUB_USERNAME }} \ + # --password-stdin + # docker push ${{ steps.build-image.outputs.image }}:${{ steps.build-image.outputs.tag}} + + # - name: Tag Staging + # if: github.ref == 'refs/heads/develop' + # run: | + # docker tag \ + # ${{ steps.build-image.outputs.image }}:${{ steps.build-image.outputs.tag}} \ + # ${{ steps.build-image.outputs.image }}:staging + # docker push ${{ steps.build-image.outputs.image }}:staging + + # - name: Tag Latest + # if: github.ref == 'refs/heads/master' + # run: | + # docker tag \ + # ${{ steps.build-image.outputs.image }}:${{ steps.build-image.outputs.tag}} \ + # ${{ steps.build-image.outputs.image }}:latest + # docker push ${{ steps.build-image.outputs.image }}:latest + + infra: + name: Prepare Infrastructure + # if: github.ref == 'refs/heads/master' || github.ref == 'refs/heads/develop' + runs-on: ubuntu-latest + # needs: build + + steps: + - name: Checkout + uses: actions/checkout@v1 + + - name: 'Terraform Format' + uses: hashicorp/terraform-github-actions@master + with: + tf_actions_version: 0.12.21 + tf_actions_subcommand: 'fmt' + tf_actions_working_dir: '.github/workflows/aws_deploy/terraform' + tf_actions_comment: 'false' + - name: 'Terraform Init' + uses: hashicorp/terraform-github-actions@master + with: + tf_actions_version: 0.12.21 + tf_actions_subcommand: 'init' + tf_actions_working_dir: '.github/workflows/aws_deploy/terraform' + tf_actions_comment: 'false' + - name: 'Terraform Validate' + uses: hashicorp/terraform-github-actions@master + with: + tf_actions_version: 0.12.21 + tf_actions_subcommand: 'validate' + tf_actions_working_dir: '.github/workflows/aws_deploy/terraform' + tf_actions_comment: 'false' + - name: 'Terraform Plan' + uses: hashicorp/terraform-github-actions@master + with: + tf_actions_version: 0.12.21 + tf_actions_subcommand: 'plan' + tf_actions_working_dir: '.github/workflows/aws_deploy/terraform' + tf_actions_comment: 'false' + args: "--var secret=${{ secrets.DOCKER_HUB_ORGANIZATION }}" + # - name: 'Terraform Apply' + # uses: hashicorp/terraform-github-actions@master + # with: + # tf_actions_version: 0.12.21 + # tf_actions_subcommand: 'apply' + # tf_actions_working_dir: '.github/workflows/aws_deploy/terraform' + # tf_actions_comment: 'false' + # args: "--var secret=${{ secrets.DOCKER_HUB_ORGANIZATION }}" + + # deploy: + # name: Deploy + # if: github.ref == 'refs/heads/master' + # runs-on: ubuntu-latest + # needs: infra + + # steps: + # - name: Checkout + # uses: actions/checkout@v1 + + # - name: Configure AWS credentials + # uses: aws-actions/configure-aws-credentials@v1 + # with: + # aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + # aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + # aws-region: eu-central-1 + + # - name: Fill in the new image ID in the Amazon ECS task definition + # id: task-def + # uses: aws-actions/amazon-ecs-render-task-definition@v1 + # with: + # task-definition: .github/workflows/aws_deploy/task-def.json + # container-name: catpol + # image: ${{ secrets.DOCKER_HUB_ORGANIZATION }}/${{ secrets.DOCKER_HUB_REPO }} + + # - name: Deploy Amazon ECS task definition + # uses: aws-actions/amazon-ecs-deploy-task-definition@v1 + # with: + # task-definition: ${{ steps.task-def.outputs.task-definition }} + # service: CatPol-service + # cluster: CatPol + # wait-for-service-stability: true diff --git a/.github/workflows/aws_deploy/database.tf b/.github/workflows/aws_deploy/database.tf new file mode 100644 index 0000000..0bbf59f --- /dev/null +++ b/.github/workflows/aws_deploy/database.tf @@ -0,0 +1,29 @@ +resource "aws_db_instance" "main" { + name = local.prefix + engine = "postgres" + engine_version = "11.6" + instance_class = "db.t2.micro" + + allocated_storage = 10 + apply_immediately = true + backup_retention_period = 5 + db_subnet_group_name = aws_db_subnet_group.main.name + multi_az = true + skip_final_snapshot = true + vpc_security_group_ids = [aws_security_group.intra.id] + + username = aws_ssm_parameter.db_username.value + password = aws_ssm_parameter.db_password.value + + tags = { + Name = local.prefix + } +} + +resource "aws_db_subnet_group" "main" { + name = local.prefix + subnet_ids = aws_subnet.private-db.*.id + tags = { + Name = local.prefix + } +} diff --git a/.github/workflows/aws_deploy/load-balancing.tf b/.github/workflows/aws_deploy/load-balancing.tf new file mode 100644 index 0000000..eb73626 --- /dev/null +++ b/.github/workflows/aws_deploy/load-balancing.tf @@ -0,0 +1,31 @@ +resource "aws_alb" "main" { + load_balancer_type = "application" + name = local.prefix + subnets = aws_subnet.public.*.id + tags = { + Name = local.prefix + } +} + +output "Load-Balancer DNS" { + value = aws_alb.main.dns_name +} + +resource "aws_alb_listener" "main" { + load_balancer_arn = aws_alb.main.arn + port = 80 + + default_action { + type = "forward" + target_group_arn = aws_alb_target_group.main.id + } +} + +resource "aws_alb_target_group" "main" { + name = local.prefix + vpc_id = aws_vpc.app.id + + tags = { + Name = local.prefix + } +} diff --git a/.github/workflows/aws_deploy/main.tf b/.github/workflows/aws_deploy/main.tf new file mode 100644 index 0000000..8355de3 --- /dev/null +++ b/.github/workflows/aws_deploy/main.tf @@ -0,0 +1,7 @@ +provider "aws" { + region = "eu-central-1" +} + +locals { + prefix = "catpol" +} diff --git a/.github/workflows/aws_deploy/networking.tf b/.github/workflows/aws_deploy/networking.tf new file mode 100644 index 0000000..362bdaa --- /dev/null +++ b/.github/workflows/aws_deploy/networking.tf @@ -0,0 +1,163 @@ +locals { + subnet_count_app = 1 + subnet_count_public = 2 + subnet_count_db = 2 +} + +data "aws_availability_zones" "available" { + state = "available" +} + +resource "aws_vpc" "app" { + cidr_block = "10.0.0.0/16" + + tags = { + Name = local.prefix + } +} + +output "VPC CIDR" { + value = aws_vpc.app.cidr_block +} + +################################################# +# Subnets +################################################# + +resource "aws_subnet" "public" { + count = local.subnet_count_public + vpc_id = aws_vpc.app.id + cidr_block = cidrsubnet(aws_vpc.app.cidr_block, 8, count.index) + availability_zone = data.aws_availability_zones.available.names[count.index] + + tags = { + Name = "${local.prefix}-public" + } +} + +output "Subnet Public CIDR" { + value = aws_subnet.public.*.cidr_block +} + +resource "aws_subnet" "private-app" { + count = local.subnet_count_app + vpc_id = aws_vpc.app.id + cidr_block = cidrsubnet(aws_vpc.app.cidr_block, 8, local.subnet_count_public + count.index) + availability_zone = data.aws_availability_zones.available.names[count.index] + + tags = { + Name = "${local.prefix}-private-app" + } +} + +output "Subnet Private App CIDR" { + value = aws_subnet.private-app.*.cidr_block +} + +resource "aws_subnet" "private-db" { + count = local.subnet_count_db + vpc_id = aws_vpc.app.id + cidr_block = cidrsubnet(aws_vpc.app.cidr_block, 8, local.subnet_count_public + local.subnet_count_app + count.index) + availability_zone = data.aws_availability_zones.available.names[count.index] + + tags = { + Name = "${local.prefix}-private-db" + } +} + +output "Subnet Private DB CIDR" { + value = aws_subnet.private-db.*.cidr_block +} + +################################################# +# Gateways +################################################# + +resource "aws_internet_gateway" "public" { + vpc_id = aws_vpc.app.id + + tags = { + Name = "${local.prefix}-public" + } +} + +resource "aws_eip" "private" { + vpc = true + + tags = { + Name = "${local.prefix}-private-app" + } +} + +output "NAT Egress Elastic IP" { + value = aws_eip.private.private_ip +} + +resource "aws_nat_gateway" "private" { + allocation_id = aws_eip.private.id + subnet_id = element(aws_subnet.public.*.id, 0) + + tags = { + Name = "${local.prefix}-private-app" + } +} + +################################################# +# Route tables +################################################# + +resource "aws_route_table" "public" { + vpc_id = aws_vpc.app.id + + tags = { + Name = "${local.prefix}-public" + } +} + +resource "aws_route_table" "private-app" { + vpc_id = aws_vpc.app.id + + tags = { + Name = "${local.prefix}-private-app" + } +} + +resource "aws_route_table" "private-db" { + vpc_id = aws_vpc.app.id + + tags = { + Name = "${local.prefix}-private-db" + } +} + +################################################# +# Routes +################################################# + +resource "aws_route_table_association" "public-gw" { + route_table_id = aws_route_table.public.id + gateway_id = aws_internet_gateway.public.id +} + +resource "aws_route_table_association" "private-gw" { + route_table_id = aws_route_table.private-app.id + gateway_id = aws_nat_gateway.private.id +} + +resource "aws_route_table_association" "public" { + count = local.subnet_count_public + route_table_id = aws_route_table.private-app.id + subnet_id = element(aws_subnet.public.*.id, count.index) +} + +resource "aws_route_table_association" "private-app" { + count = local.subnet_count_app + route_table_id = aws_route_table.private-app.id + subnet_id = element(aws_subnet.private-app.*.id, count.index) +} + +resource "aws_route_table_association" "private-db" { + count = local.subnet_count_db + route_table_id = aws_route_table.private-db.id + subnet_id = element(aws_subnet.private-db.*.id, count.index) +} diff --git a/.github/workflows/aws_deploy/secrets.tf b/.github/workflows/aws_deploy/secrets.tf new file mode 100644 index 0000000..ba93c67 --- /dev/null +++ b/.github/workflows/aws_deploy/secrets.tf @@ -0,0 +1,28 @@ +resource "aws_ssm_parameter" "db_username" { + name = "/${local.prefix}/db_username" + value = var.db_username + type = "SecureString" + key_id = aws_kms_key.main.id + + tags = { + Name = local.prefix + } +} + +resource "aws_ssm_parameter" "db_password" { + name = "/${local.prefix}/db_password" + value = var.db_password + type = "SecureString" + key_id = aws_kms_key.main.id + + tags = { + Name = local.prefix + } +} + +resource "aws_kms_key" "main" { + description = "Key for ${local.prefix} for secret variables" + tags = { + Name = local.prefix + } +} diff --git a/.github/workflows/aws_deploy/security.tf b/.github/workflows/aws_deploy/security.tf new file mode 100644 index 0000000..97581d4 --- /dev/null +++ b/.github/workflows/aws_deploy/security.tf @@ -0,0 +1,39 @@ +resource "aws_security_group" "public" { + name = "${local.prefix}-public" + description = "Public access" + vpc_id = aws_vpc.app.id + tags = { + Name = "${local.prefix}-public" + } +} + +resource "aws_security_group" "intra" { + name = "${local.prefix}-intra" + description = "Intra-service access. Used for App-DB communication." + vpc_id = aws_vpc.app.id + tags = { + Name = "${local.prefix}-intra" + } +} + +resource "aws_security_group_rule" "public" { + description = "Give public access on HTTP" + security_group_id = aws_security_group.public + + cidr_blocks = "0.0.0.0/0" + from_port = 80 + to_port = 80 + protocol = "tcp" + type = "ingress" +} + +resource "aws_security_group_rule" "intra" { + description = "Allow intra-service communication" + security_group_id = aws_security_group.intra + + from_port = -1 + to_port = -1 + protocol = "all" + type = "ingress" + self = true +} diff --git a/.github/workflows/aws_deploy/service.tf b/.github/workflows/aws_deploy/service.tf new file mode 100644 index 0000000..dd6b1c8 --- /dev/null +++ b/.github/workflows/aws_deploy/service.tf @@ -0,0 +1,48 @@ +locals { + container_name = "catpol" + container_port = 8000 +} + +resource "aws_ecs_cluster" "app" { + name = local.prefix + + tags = { + Name = local.prefix + } +} + +resource "aws_ecs_service" "app" { + name = local.prefix + cluster = aws_ecs_cluster.app.id + task_definition = aws_ecs_task_definition.app.id + desired_count = 1 + launch_type = "FARGATE" + # iam_role = + # depends_on = [] + + load_balancer { + target_group_arn = aws_alb_target_group.main.id + container_name = local.container_name + container_port = local.container_port + } +} + +resource "aws_ecs_task_definition" "app" { + family = local.prefix + container_definitions = data.template_file.task-def.rendered + requires_compatibilities = ["FARGATE"] + cpu = 1 + memory = 1024 +} + +data "template_file" "task-def" { + template = file("task-def_template.json") + vars = { + container_name = local.container_name + container_port = local.container_port + container_image = var.docker_image + + db_username = aws_ssm_parameter.db_username.value + db_password = aws_ssm_parameter.db_password.value + } +} diff --git a/.github/workflows/aws_deploy/task-def_template.json b/.github/workflows/aws_deploy/task-def_template.json new file mode 100644 index 0000000..884886c --- /dev/null +++ b/.github/workflows/aws_deploy/task-def_template.json @@ -0,0 +1,23 @@ +[ + { + "name": "${container_name}", + "image": "${container_image}", + "portMappings": [ + { + "protocol": "tcp", + "containerPort": ${container_port} + } + ], + "essential": true, + "secrets": [ + { + "name": "db_username", + "valueFrom": "${db_username}" + }, + { + "name": "db_password", + "valueFrom": "${db_password}" + } + ] + } +] diff --git a/.github/workflows/aws_deploy/variables.tf b/.github/workflows/aws_deploy/variables.tf new file mode 100644 index 0000000..5c7d5f9 --- /dev/null +++ b/.github/workflows/aws_deploy/variables.tf @@ -0,0 +1,5 @@ +variable "docker_image" {} + +variable "db_username" {} + +variable "db_password" {} diff --git a/.gitignore b/.gitignore index 5898f22..1f27df4 100644 --- a/.gitignore +++ b/.gitignore @@ -34,4 +34,9 @@ wheels/ *.egg MANIFEST -.DS_STORE \ No newline at end of file +.DS_STORE + +# Terraform +.terraform +*terraform.tfstate* +*.tfvars diff --git a/foo.sh b/foo.sh new file mode 100755 index 0000000..2ea1127 --- /dev/null +++ b/foo.sh @@ -0,0 +1,11 @@ +#!/bin/bash + +SOME_VAR="foo-$1" +echo $SOME_VAR + +echo $SOME_VAR > foo.txt + +foo="$1" +for (( i=0; i<${#foo}; i++ )); do + echo "${foo:$i:1}" +done