EP01: DevOps Capstone Project — Initial Setup

In today’s tech-driven world, DevOps has become the backbone of efficient software development and deployment. But learning DevOps isn’t just about understanding the theory; the real insights come from rolling up your sleeves and building something from scratch. That’s the essence of this series.

If you've ever wanted to unify containerization, continuous integration, continuous delivery, infrastructure as code, container orchestration, and monitoring into a single, effective workflow, you're in the right place. Across this series, we'll build a complete DevOps project from the ground up, immersing you in the practical tools and techniques that DevOps professionals rely on every day.

Project Overview

In this project, we’ll work with a sample application that converts URLs into QR codes, using AWS as our cloud provider. The app is divided into three main components: Front-end, API, and Storage. We’ll containerize and deploy the Front-end and API on AWS, while using AWS S3 for storing the generated QR code images.

Here’s how the application is structured:

• Front-End Container: This container will host the user interface where users can input URLs to generate QR codes.

• API Container: This component will handle the logic for generating QR codes based on the input URLs.

• AWS S3: Used for storing the generated QR codes, accessed via API calls from the Front-End and API components.

To give you a clear picture of what we’ll be building, here’s an overview of our project’s architecture.

While the application code is provided, our task is to apply core DevOps practices. We’ll focus on containerizing the app, setting up CI/CD pipelines, and ensuring observability and monitoring are in place to maintain performance and scalability on AWS.

Here are the topics we will tackle in this series and as soon as an article is ready, I will add it here:

1. Containerization: Containerize both the front-end and API of the QR Code Application by creating Dockerfiles.

2. CI/CD: Write CI/CD pipeline for automating the process of pushing front-end and API images to Dockerhub.

3. Infrastructure as Code (IaC): Use OpenTofu to define and deploy the cloud Infrastructure(EKS).

4. Kubernetes YAML Files: Create Deployment, Service, Service account, Ingress, and ConfigMap YAML files for both the Next.js front-end and the FastAPI backend to deploy it on Kubernetes.

5. Install Nginx Ingress Controller: We will install the Nginx Ingress Controller, which in turn will create an AWS Application Load Balancer (ALB) for the EKS cluster.

6. DNS Mapping of Load Balancer: We will get the URL of the Load Balancer and map it to our domain name in Cloudflare.

7. Install Cert-Manager: To provide an automated way of ensuring TLS certificates for our site, we will install Cert-Manager.

8. Helm Chart for application: Create a Helm chart for the application to facilitate easy deployment through the Helm command.

9. Monitoring & Logging Setup: Implement monitoring and logging for the EKS cluster using Helm, Prometheus, Loki, and Grafana.

10. ArgoCD Installation: Install and set up ArgoCD for continuous delivery and GitOps.

Prerequisites:

Before starting the project, ensure you have the following prerequisites:

• An AWS account with the necessary permissions to create resources.

• OpenTofu/Terraform, Docker and AWS CLI installed on your local machine.

• A domain managed through Cloudflare.

• An S3 bucket set up in AWS.

• Basic familiarity with Kubernetes, Docker, Github Actions, DNS mapping, and DevOps principles.

Initial Setup

Having outlined the project's scope and objectives, it's time to roll up our sleeves and dive into the initial setup. Here, we'll configure the QR Code Generator application locally, ensuring each component front end, back end, and storage is operational before moving forward. This foundational setup is key, as it prepares us to apply DevOps practices like containerization, CI/CD, and observability as we build out the project step by step.

Our application is built with a Next.js front end for a smooth user experience and a FastAPI back end in Python to handle QR code generation efficiently.

Application Overview

• Front-End: A web interface where users can submit URLs for QR code generation.

• API: A backend service that processes submitted URLs, generates QR codes, and securely stores them in cloud storage using an AWS S3 bucket.

Repository Setup & AWS S3 Configuration

Start by cloning the QR code application repository where we’ll implement DevOps practices.

1. Visit this repository, or use the command below to clone it into your local machine:

   $ git clone https://github.com/Ayaan49/qr-code-app.git

2. Navigate into the qr-code-app directory and then into the api directory:

   $ cd qr-code-app/api

3. Edit the main.py file and make the following changes:

   • Allowing CORS for Local Testing: Add the URL of your own domain where you’ll be hosting the site. For example:

      origins = [
          "http://localhost:3000",
          "https://dev.qr-app.devfun.me",  # Example URL
          "https://your-own-site.com"      # Add your own site URL here
      ]

     This configuration ensures CORS permissions are set up for your hosting domain. The Example URL is my own hosting domain, so remember to replace it with your own.

   • AWS S3 Configuration: Change the bucket_name placeholder to your actual S3 bucket name.

     # AWS S3 Configuration
     s3 = boto3.client('s3')
     bucket_name = '<YOUR_BUCKET>'  # Add your bucket name here

4. Set the necessary permissions for your S3 bucket:

   • IAM User Access: Assign the AmazonS3FullAccess policy to your IAM user in the AWS console.

   • Bucket Permissions:

     • Turn Block all public access to off.

       

     • Set Object Ownership to Bucket owner preferred.

       

     • Add the following bucket policy to grant public read access to the qr_codes folder:

       

       This policy allows public read access to the files in the qr_codes folder, enabling users to view or download QR code images directly from your S3 bucket without authentication.

5. Edit the .env file in the api folder and fill your own AWS credentials

   AWS_ACCESS_KEY=Your-AWS-Access-Key
   AWS_SECRET_KEY=Your-AWS-Secret-Access-Key

Running locally

Setting Up the API Server

The API code is located in the api directory. Follow these steps to run the API server locally:

1. Navigate to the api directory:

   $ cd api

2. Create a virtual environment:

   $ python -m venv .venv

3. Install dependencies:

   $ pip install -r requirements.txt

4. Start the API server:

   $ uvicorn main:app --reload

Your API server will be running at localhost:8000

Front-End Setup

The front-end code is located in the front-end-nextjs directory. To run the front-end server locally:

1. Navigate to the front-end-nextjs directory:

   $ cd front-end-nextjs

2. Install dependencies:

   $ npm install

3. Start the Next.js server:

   $ npm run dev

The front-end server will be accessible at localhost:3000

Now, let’s generate a QR code from a URL.

It works! Give it a try yourself.

Wrapping Up!

In this section, we covered the foundational setup and configuration of our QR Code Generator application, preparing it for deployment by integrating AWS S3 for QR code storage and testing the app locally. With the application now running on your machine, we have a solid base to build upon as we start implementing DevOps practices.

In the next section, we'll dive into containerizing the application with Docker. This will allow us to encapsulate the front end and API into efficient, portable containers, setting the stage for streamlined deployment and scalability.

See you in the next section as we begin Dockerizing our app!

---