You now have a repo to start writing your backend logic! The Go api comes with an endpoint returning the status of the app.
<%- if eq (index .Params backendApplicationHosting) "serverless" %>
Your application is deployed to AWS using SAM. The main building blocks includes
- API Gateway
- Lambda Authorizer
- Lambda container Application (Your application)
The application is configured by template.yaml and declares the API gateway, Authorizer, and Application. The config file is also responsible for per environment and environment variables and invoke Lambda roles and Domain setup with API gateway.
There are also a Parameters file with your repository .sam-params-stage and .sam-params-prod, they are used for CI to pick up these values, consist of Environment, HostedZoneId and GatewayCertificateArn. This is created during the init step and committed to your repo, The cert ARN and hostedZoneID are not sensitive but requires list all permissions in IAM, therefore are created here instead of granting the CI user those permission to fetch on the fly, and they should not change often.
The deployment is configured from config.toml, and triggered from Github Actions .github/workflows/sam.yml, this declares the staging and production SAM deployment artifacts destination and settings.
<%- else %>
Your application is deployed on your EKS cluster through circleCI, you can see the pod status on kubernetes in your application namespace:
kubectl -n <% .Name %> get pods
You can update the resource limits in the kubernetes/base/deployment.yml, and control fine-grain customizations based on environment and specific deployments such as Scaling out your production replicas from the overlays configurations
This project is set up with a local/cloud hybrid dev environment. This means you can do fast local development of a single service, even if that service depends on other resources in your cluster. Make a change to your service, run it, and you can immediately see the new service in action in a real environment. You can also use any tools like your local IDE, debugger, etc. to test/debug/edit/run your service.
Usually when developing you would run the service locally with a local database and any other dependencies running either locally or in containers using docker-compose, minikube, etc.
Now your service will have access to any dependencies within a namespace running in the EKS cluster, with access to resources there.
Telepresence is used to provide this functionality.
Development workflow:
- Run
start-dev-env.sh- You will be dropped into a shell that is the same as your local machine, but works as if it were running inside a pod in your k8s cluster - Change code and run the server - As you run your local server, using local code, it will have access to remote dependencies, and will be sent traffic by the load balancer
- Test on your cloud environment with real dependencies -
https://<your name>-<% index .ParamsstagingBackendSubdomain%><% index .ParamsstagingHostRoot%> - git commit & auto-deploy to Staging through the build pipeline
<%if eq (index .Params CIVendor) "circleci" %>## Circle CI
Your repository comes with a end-to-end CI/CD pipeline, which includes the following steps:
- Checkout
- Unit Tests
- Build and Push Image
- Deploy Staging
- Integration Tests
- Deploy Production
See details on CircleCi
<% else if eq (index .Params CIVendor) "github-actions" %>## Github actions
Your repository comes with a end-to-end CI/CD pipeline, which includes the following steps:
- Checkout
- Unit Tests
- Build Image
- Upload Image to ECR
- Deploy image to Staging cluster
- Integration tests
- Deploy image to Production cluster
Note: you can add a approval step using Github Environments(Available for Public repos/Github pro), you can configure an environment in the Settings tab then simply add the follow to your ci manifest (./github/workflows/ci.yml)
deploy-production: # or any step you would like to require Explicit approval
enviroments:
name: <env-name>Your repository comes with a protected branch master and you can edit Branch protection in Branches tab of Github settings. This ensures code passes tests before getting merged into your default branch.
By default it requires [lint, unit-test] to be passing to allow Pull requests to merge.
<% end %>
Your application is assumed to rely on a database. An application-specific user has been created for you by the /scripts/create-db-user.sh script in the infrastructure project.
A Kubernetes secret will exist in your application namespace (<% .Name %>) that will provide these credentials to your application.
Along with the database credentials, any other secrets that need to be provided to the application can be managed in AWS Secrets Manager.
Secrets have been created for each environment called <% .Name %>/application/<environment>/<% .Name %> which contain a list of environment variables that will be synced with the kubernetes secret in your namespace via a tool called external-secrets
Any secrets managed by external-secrets will be synced to kubernetes every 15 seconds. Keep in mind that any changes must be made in Secrets Manager, as any that are made to the secret on the kubernetes side will be overwritten.
You can see the external-secrets configuration in kubernetes/overlays/staging/external-secret.yml (this is the one for staging)
To work with the secret in AWS you can use the web interface or the cli tool:
aws secretsmanager get-secret-value --secret=<% .Name %>/application/stage/<% .Name %>
The intent is that the last part of the secret name is the component of your application this secret is for. For example: if you were adding a new billing service, the secret might be called <% .Name %>/application/stage/billing
An example cron job is specified in kubernetes/base/cronjob.yml.
The default configuration specifies suspend: true to ensure this cronjob does not run unless you want to enable it.
When you are ready for your cron job to run, make sure to set suspend: false.
The default cron job specifies three parameters that you will need to change depending on your application's needs:
See a detailed specification of the cron schedule format. This will need to be modified to fit the constraints of your application.
The default image specified is a barebones busybox base image. You likely want to run processes dependent on your backend codebase; so the image will likely be the same as for your backend application.
As per the image attribute noted above, you will likely be running custom arguments in the context of that image. You should specify those arguments as per the documentation.
Database migrations are handled with Flyway. Migrations run in a docker container started in the Kubernetes cluster by CircleCI or the local dev environment startup process.
The migration job is defined in kubernetes/migration/job.yml and your SQL scripts should be in database/migration/.
Migrations will be automatically run against your dev environment when running ./start-dev-env.sh. After merging the migration it will be run against other environments automatically as part of the pipeline.
The SQL scripts need to follow Flyway naming convention here, which allow you to create different types of migrations:
- Versioned - These have a numerically incrementing version id and will be kept track of by Flyway. Only versions that have not yet been applied will be run during the migration process.
- Undo - These have a matching version to a versioned migration and can be used to undo the effects of a migration if you need to roll back.
- Repeatable - These will be run whenever their content changes. This can be useful for seeding data or updating views or functions.
Here are some example migrations:
V1__create_tables.sql
CREATE TABLE address (
id INT(6) UNSIGNED AUTO_INCREMENT PRIMARY KEY,
person_id INT(6),
street_number INT(10),
street_name VARCHAR(50),
reg_date TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
);V2__add_columns.sql
ALTER TABLE address
ADD COLUMN city VARCHAR(30) AFTER street_name,
ADD COLUMN province VARCHAR(30) AFTER city<% end %>
<% if eq (index .Params billingEnabled) "yes" %>
A subscription and checkout example using Stripe, coupled with the frontend repository to provide an end-to-end checkout example for you to customize. We also setup a webhook and an endpoint in the backend to receive webhook when events occur.
The following example content has been set up in Stripe:
- 1 product
- 3 prices(subscriptions) [annual, monthly, daily]
- 1 webhook [
charge.failed,charge.succeeded,customer.created,subscription_schedule.created] See link for available webhooks: https://stripe.com/docs/api/webhook_endpoints/create?lang=curl#create_webhook_endpoint-enabled_events
this is setup using the script scripts/stripe-example-setup.sh
The deployment only requires the environment variables:
- STRIPE_API_SECRET_KEY (created in AWS secret then deployed via Kubernetes Secret)
- FRONTEND_URL (used for sending user back to frontend upon checkouts)
- BACKEND_URL (used for redirects after checkout and webhooks)
<% end %>