Skip to content
This repository was archived by the owner on Mar 27, 2023. It is now read-only.

Commit 9e13ba8

Browse files
briancaffeybriancaffey
authored
Merge pull request #11 from briancaffey/10-fix-local-dev-issues
10 fix local dev issues
2 parents c940f27 + 4e3da4a commit 9e13ba8

File tree

103 files changed

+480
-3588
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

103 files changed

+480
-3588
lines changed

Diff for: ‎.env.template

-10
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,6 @@
11
SECRET_KEY=secret
22
DEBUG=True
33

4-
POSTGRES_NAME=postgres
5-
POSTGRES_USERNAME=postgres
6-
POSTGRES_PASSWORD=postgres
7-
POSTGRES_SERVICE_HOST=postgres
8-
POSTGRES_SERVICE_PORT=5432
9-
10-
114
# social auth
125
GITHUB_KEY=yourkey
136
GITHUB_SECRET=yoursecret
@@ -18,9 +11,6 @@ GOOGLE_OAUTH2_SECRET=abcde-fghij
1811
SOCIAL_AUTH_FACEBOOK_KEY = ''
1912
SOCIAL_AUTH_FACEBOOK_SECRET = ''
2013

21-
CELERY_BROKER_URL=redis://redis:6379/0
22-
CELERY_RESULT_BACKEND=redis://redis:6379/1
23-
2414
DJANGO_EMAIL_HOST=mailhog
2515
DJANGO_EMAIL_PORT=1025
2616

Diff for: ‎.gitlab-ci.yml

+2-4
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,14 @@ stages:
55
- build
66
- integration
77
- release
8-
- plan
98
- deploy
109

1110
# gitlab ci deployment options
1211
include:
1312
- local: /gitlab-ci/documentation.yml
1413
- local: /gitlab-ci/renovate.yml
15-
- local: /gitlab-ci/aws/cdk.yml
16-
# - local: /gitlab-ci/aws/cloudformation.yml
17-
#- local: /gitlab-ci/gcp/terraform.yml
14+
- local: /gitlab-ci/aws/dev.yml
15+
- local: /gitlab-ci/aws/app.yml
1816

1917
.Pytest:
2018
image: python:3.8

Diff for: ‎.python-version

-1
This file was deleted.

Diff for: ‎README.md

+80-9
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,92 @@ Documentation for this project can be found here:
44

55
[https://verbose-equals-true.gitlab.io/django-postgres-vue-gitlab-ecs/](https://verbose-equals-true.gitlab.io/django-postgres-vue-gitlab-ecs/)
66

7+
8+
## Project Architecture Overview
9+
10+
Here is an overview of the project architecture, including the CI/CD pipeline and the AWS infrastructure that will be automatically provisioned through the AWS Cloud Development Kit:
11+
12+
![png](/architecture.png)
13+
14+
(This diagram was created with [draw.io](https://draw.io). Here's the link to the a read-only version of the diagram on draw.io: [https://drive.google.com/file/d/1gU61zjoW80fCusUcswU1zhEE5VFB1Z5U/view?usp=sharing](https://drive.google.com/file/d/1gU61zjoW80fCusUcswU1zhEE5VFB1Z5U/view?usp=sharing)
15+
16+
### Legend
17+
18+
1 - GitLab is used to host the source code, test the source code and deploy the application to AWS.
19+
20+
2 - Unit testing (see `.gitlab-ci.yml`)
21+
22+
2a - Pytest
23+
24+
2b - Jest
25+
26+
2c - Cypress
27+
28+
3 - Deployment phase (see `/gitlab-ci/aws/cdk.yml`)
29+
30+
3a - Quasar PWA assets are built if there are changes in the `quasar` directory
31+
32+
3b - AWS Cloud Development Kit (CDK) defines all infrastructure in AWS (4a - 12)
33+
34+
3c - AWS CLI is used to run Fargate tasks through manual GitLab CI jobs
35+
36+
4 - CDK Assets (ECR and S3 buckets that CDK uses internally to manage build assets and artifacts)
37+
38+
4a - Elastic Container Repository is used to manage the Django docker image used in various parts of the application
39+
40+
4b - S3 bucket used to store files associated with CDK and CloudFormation
41+
42+
5 - Route53 is used to route traffic to the CloudFront distribution
43+
44+
6 - CloudFront distribution that serves as the "front desk" of the application. It routes requests to to the correct CloudFront Origin
45+
46+
7 - CloudFront Origin Configurations
47+
48+
7a - S3 bucket for Quasar PWA assets
49+
50+
7b - Application Load Balancer for Django application (`/api/`, `/admin/`, `/flower/`, `/ws/`, `/graphql/`)
51+
52+
7c - S3 bucket for Django assets (static files, public media and private media)
53+
54+
8 - Web server and websocket servers
55+
56+
8a - Fargate service running uvicorn process (REST, GraphQL, Django Channels)
57+
58+
8b - Autoscaling Group for Fargate Service that serves Django API
59+
60+
9 - Celery and celery worker autoscaling
61+
62+
9a - Fargate service that is autoscaled between 0 and `N` Fargate tasks for a given celery queue
63+
64+
9b - Scheduled Event that triggers a Lambda to make a request to Django backend which collects celery queue metrics and published metrics to CloudWatch using boto3
65+
66+
9c - Lambda event the makes a request to `/api/celery-metrics/`
67+
68+
9d - CloudWatch alarm that is used to scale the Fargate service for a celery queue
69+
70+
9e - Autoscaling group for celery Fargate service
71+
72+
10 - Fargate tasks that run Django management commands such as `migrate` and `collectstatic`. These are triggered from manual GitLab CI jobs using the AWS CLI (3c)
73+
74+
11 - ElastiCache for Redis, used for Caching, Celery Broker, Channels Layer, etc.
75+
76+
12 - Aurora Postgres Serverless
77+
778
## Local Development
879

980
First, copy `.env.template` to a new file in the project's root directory called `.env`. This file will be read by `docker-compose` in the next step. Adjust any of the values in this file if needed, or add new variables for any secret information you need to pass to docker-compose (or to docker containers).
1081

82+
```sh
83+
docker-compose up
84+
```
85+
86+
Open `http://localhost` in your browser.
87+
88+
You can specify environment variables for docker-compose by adding an `.env` file to the root of the project based on `.env.template`.
89+
1190
### Social Authentication Keys
1291

13-
To use social sign on in development, you will need to create an application with the given provider.
92+
To use social sign on in local development, you will need to create an application with the given provider such as GitHub, Google, Facebook, etc.
1493

1594
#### GitHub
1695

@@ -23,14 +102,6 @@ Go to [https://github.com/settings/applications/new](https://github.com/settings
23102

24103
In the `.env` file, add the `Client ID` of your GitHub OAuth App as the `GITHUB_KEY` variable, and add the `Client Secret` as the `GITHUB_SECRET` variable.
25104

26-
```sh
27-
docker-compose up
28-
```
29-
30-
Open `http://localhost` in your browser.
31-
32-
You can specify environment variables for docker-compose by adding an `.env` file to the root of the project based on `.env.template`.
33-
34105
## VuePress Documentation
35106

36107
This project uses VuePress for documentation. To view the documentation site locally, run the following command:

Diff for: ‎architecture.png

1.08 MB
Loading

Diff for: ‎awscdk/app.py

+2-2
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,8 @@
1313
# if the the production environent subdomain should nott be included in the URL
1414
# redefine `full_domain_name` to `base_domain_name` for that environment
1515
full_domain_name = f"{environment_name}.{base_domain_name}" # dev.mysite.com
16-
# if environment_name == "prod":
17-
# full_domain_name = base_domain_name
16+
if environment_name == "app":
17+
full_domain_name = base_domain_name
1818
base_app_name = os.environ.get("APP_NAME", "mysite-com")
1919
full_app_name = f"{environment_name}-{base_app_name}" # dev-mysite-com
2020
aws_region = os.environ.get("AWS_DEFAULT_REGION", "us-east-1")

Diff for: ‎backend/.dockerignore

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
media
2+
static

Diff for: ‎backend/apps/core/management/commands/watch_celery.py

-37
This file was deleted.

Diff for: ‎backend/apps/core/management/commands/watch_celery_beat.py

-33
This file was deleted.

Diff for: ‎backend/backend/settings/base.py

+4-6
Original file line numberDiff line numberDiff line change
@@ -11,15 +11,13 @@
1111
"""
1212

1313
import os
14+
from pathlib import Path
1415

1516
from kombu import Queue
1617
import redis
1718

18-
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
19-
BASE_DIR = os.path.dirname(
20-
os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
21-
)
22-
19+
# Build paths inside the project like this: BASE_DIR / 'subdir'.
20+
BASE_DIR = Path(__file__).resolve(strict=True).parent.parent.parent
2321

2422
# Quick-start development settings - unsuitable for production
2523
# See https://docs.djangoproject.com/en/2.1/howto/deployment/checklist/
@@ -269,7 +267,7 @@
269267
# Assets
270268

271269
STATIC_URL = "/static/"
272-
STATIC_ROOT = "/static/"
270+
STATIC_ROOT = os.path.join(BASE_DIR, "static")
273271

274272
MEDIA_URL = "/media/"
275273
MEDIA_ROOT = os.path.join(BASE_DIR, "media")

Diff for: ‎backend/requirements/base.txt

+1
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ djangorestframework==3.11.0
1717
djangorestframework_simplejwt==4.4.0
1818
django-debug-toolbar==2.2
1919

20+
gevent==20.6.2
2021

2122
# Python Social Auth
2223

Diff for: ‎backend/requirements/dev.txt

+5-1
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,8 @@ django-extensions==2.2.9 # NOTE: in installed_apps
33
Werkzeug==1.0.1 # used for runserver_plus exception console
44
ipython==7.14.0
55
jupyter==1.0.0
6-
gql==2.0.0
6+
gql==2.0.0
7+
8+
watchdog==0.10.3
9+
pyyaml==5.3.1
10+
argh==0.26.2

Diff for: ‎backend/scripts/ci/Dockerfile

-49
This file was deleted.

Diff for: ‎backend/scripts/prod/start_prod.sh

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
#!/bin/bash
22

3-
gunicorn -t 300 -b 0.0.0.0:8000 backend.wsgi
3+
gunicorn -t 300 -k gevent -w 4 -b 0.0.0.0:8000 backend.wsgi

Diff for: ‎cloudformation/infrastructure/certs.yaml

-29
This file was deleted.

0 commit comments

Comments
 (0)