What is GitOps?

GitOps is an operational model where:

1. The desired state of your system lives in git
2. An agent continuously compares desired state to actual state
3. Any drift is automatically corrected (or flagged)

With ArgoCD, you push a change to git and the cluster catches up — you never kubectl apply to production directly.

Installing ArgoCD

$ kubectl create namespace argocd

$ kubectl apply -n argocd -f \
  https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# wait for pods to be ready
$ kubectl wait --for=condition=available deployment \
  -l app.kubernetes.io/name=argocd-server \
  -n argocd --timeout=120s

# port-forward the UI
$ kubectl port-forward svc/argocd-server -n argocd 8080:443

Get the initial admin password:

$ kubectl get secret argocd-initial-admin-secret -n argocd \
  -o jsonpath='{.data.password}' | base64 -d
r8Kf9pXqZmN2wQ4T

Visit https://localhost:8080, log in as admin, and you’re in.

Install the CLI:

$ brew install argocd

$ argocd login localhost:8080 \
  --username admin \
  --password r8Kf9pXqZmN2wQ4T \
  --insecure

Creating Your First Application

An ArgoCD Application links a git repository path to a cluster namespace. Here’s the YAML approach:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
  namespace: argocd
spec:
  project: default

  source:
    repoURL: https://github.com/myorg/k8s-manifests
    targetRevision: main
    path: apps/myapp

  destination:
    server: https://kubernetes.default.svc
    namespace: production

  syncPolicy:
    automated:
      prune: true       # delete resources removed from git
      selfHeal: true    # revert manual changes to the cluster
    syncOptions:
      - CreateNamespace=true

$ kubectl apply -f myapp-argocd.yaml
application.argoproj.io/myapp created

$ argocd app get myapp
Name:               myapp
Project:            default
Server:             https://kubernetes.default.svc
Namespace:          production
URL:                https://localhost:8080/applications/myapp
Repo:               https://github.com/myorg/k8s-manifests
Target:             main
Path:               apps/myapp
Sync Policy:        Automated (Prune, Self Heal)
Sync Status:        Synced to main (a3f2c9d)
Health Status:      Healthy

Sync Policies

ArgoCD can sync manually or automatically.

Manual sync — ArgoCD shows drift but waits for you to approve:

$ argocd app sync myapp

Automated sync — ArgoCD applies changes within seconds of a git push. Use prune: true to also delete resources that were removed from git, and selfHeal: true to revert any manual kubectl edits.

For production, many teams prefer automated sync with manual promotion: automated in staging, manual approval gate for production. You can set up ArgoCD sync windows to restrict when automated syncs are allowed.

Repository Structure Patterns

ArgoCD works with raw manifests, Helm charts, or Kustomize. A common layout for multiple apps and environments:

k8s-manifests/
├── apps/
│   ├── myapp/
│   │   ├── base/          # shared manifests
│   │   └── overlays/
│   │       ├── staging/   # kustomize patches for staging
│   │       └── production/
│   └── api-gateway/
└── argocd/
    └── applications/      # ArgoCD Application YAMLs

Kustomize source:

source:
  repoURL: https://github.com/myorg/k8s-manifests
  path: apps/myapp/overlays/production
  targetRevision: main

Helm chart source:

source:
  repoURL: https://github.com/myorg/k8s-manifests
  path: helm/myapp
  targetRevision: main
  helm:
    valueFiles:
      - values/production.yaml

App of Apps Pattern

Managing dozens of ArgoCD Application resources manually doesn’t scale. The App of Apps pattern uses one root Application to manage all others:

# root-app.yaml
spec:
  source:
    path: argocd/applications   # directory of Application YAMLs

Push a new Application YAML to argocd/applications/ and ArgoCD picks it up automatically — no manual kubectl apply needed.

Checking Sync Status

# list all apps
$ argocd app list
NAME    CLUSTER        NAMESPACE   PROJECT  STATUS  HEALTH   SYNCPOLICY
myapp   in-cluster     production  default  Synced  Healthy  Auto-Prune

# see what's out of sync (before syncing)
$ argocd app diff myapp

# force a sync
$ argocd app sync myapp --prune

# roll back to a previous git commit
$ argocd app rollback myapp <revision-id>

Conclusion

ArgoCD enforces a discipline that’s hard to achieve with pipelines alone: every change to the cluster goes through git, drift is visible and correctable, and the history of every deployment is a git commit. The combination of automated sync, self-healing, and pull-request-based change management gives you Kubernetes deployments that are auditable, reproducible, and recoverable. Pair it with Helm or Kustomize for per-environment configuration and you have a complete GitOps workflow.

Basic Structure

Every awk program follows the same pattern:

awk 'pattern { action }' file

• pattern — a condition that must be true for the action to run (optional)
• action — what to do with the matching line (optional; default is print)

If you omit the pattern, the action runs on every line. If you omit the action, awk prints the matching lines.

Fields and the Field Separator

awk splits each line on whitespace by default. Fields are numbered $1, $2, …, and $0 is the entire line.

$ echo "Alice 30 Engineer" | awk '{ print $1, $3 }'
Alice Engineer

Change the field separator with -F:

$ echo "root:x:0:0:root:/root:/bin/bash" | awk -F: '{ print $1, $7 }'
root /bin/bash

Set a multi-character or regex separator:

$ echo "key=value=extra" | awk -F'=' '{ print $2 }'
value

Built-in Variables

$ awk '{ print NR, NF, $0 }' /etc/hosts
1 2 127.0.0.1 localhost
2 1 ::1
3 4 255.255.255.255 broadcasthost

Pattern Matching

Match lines containing a regex:

$ awk '/ERROR/' app.log
2024-01-15 ERROR connection refused on port 5432
2024-01-15 ERROR disk quota exceeded

Negate with !~:

$ awk '!/DEBUG/' app.log

Match a specific field against a pattern:

$ awk '$3 ~ /ERROR/' app.log

Numeric comparisons work directly:

$ awk '$5 > 1000' access.log

BEGIN and END Blocks

BEGIN runs before any input is read. END runs after all input is processed. Both are optional.

$ awk 'BEGIN { print "=== Report ===" } { print $1 } END { print "=== Done ===" }' names.txt
=== Report ===
Alice
Bob
Carol
=== Done ===

A classic use: summing a column.

$ awk '{ sum += $3 } END { print "Total:", sum }' sales.txt
Total: 48320

Printing and Formatting

print separates items with OFS (default space). printf works just like C:

$ awk '{ printf "%-20s %5d\n", $1, $2 }' data.txt
Alice                   30
Bob                     25

Set OFS to change the output delimiter:

$ echo "Alice 30 Engineer" | awk 'BEGIN { OFS="," } { print $1, $2, $3 }'
Alice,30,Engineer

Print the last field regardless of how many there are:

$ echo "a b c d e" | awk '{ print $NF }'
e

Print all but the first field:

$ echo "timestamp INFO message goes here" | awk '{ $1=""; print $0 }'
 INFO message goes here

Conditionals and Loops

awk has full if/else, for, and while support inside action blocks:

$ awk '{ if ($2 >= 90) print $1, "PASS"; else print $1, "FAIL" }' scores.txt
Alice PASS
Bob FAIL
Carol PASS

$ awk 'BEGIN { for (i=1; i<=5; i++) print i }'
1
2
3
4
5

Arrays

awk arrays are associative (like a hash map). You don’t declare them — just use them:

$ awk '{ count[$1]++ } END { for (user in count) print user, count[user] }' access.log
alice 42
bob 17
carol 8

Check if a key exists with in:

$ awk '$1 in seen { next } { seen[$1]=1; print }' file.txt

This prints only the first occurrence of each value in column 1 — a simple dedup.

Common One-Liners

Print lines between two patterns (inclusive):

$ awk '/START/,/END/' file.txt

Count lines matching a pattern:

$ awk '/ERROR/ { count++ } END { print count }' app.log
14

Remove duplicate lines (preserving order):

$ awk '!seen[$0]++' file.txt

Print every other line:

$ awk 'NR % 2 == 0' file.txt

Sum the second column of a CSV:

$ awk -F, '{ sum += $2 } END { print sum }' data.csv

Convert whitespace-delimited output to CSV:

$ ps aux | awk 'NR>1 { print $1","$2","$11 }'
root,1,/sbin/launchd
_windowserver,188,/System/Library/PrivateFrameworks/SkyLight.framework/...

Find lines longer than 80 characters:

$ awk 'length($0) > 80' source.c

Print unique values of a field:

$ awk -F: '!seen[$1]++ { print $1 }' /etc/passwd
root
nobody
daemon

Multiple Files and FILENAME

When processing multiple files, FILENAME tells you which file the current line came from:

$ awk '{ print FILENAME, NR, $0 }' *.log
access.log 1 GET /index.html 200
error.log 1 PHP Warning: ...

Reset a counter per file using FNR (file-relative record number) vs NR (global):

$ awk 'FNR==1 { print "--- File:", FILENAME }' *.log
--- File: access.log
--- File: error.log

Conclusion

awk covers an enormous range of tasks — anywhere from a quick column extraction to a multi-pass report with totals and grouping. The key mental model is: pattern gates the action, fields are $1…$NF, and BEGIN/END handle setup and teardown. Once those three ideas are solid, most awk one-liners read naturally. For anything more complex, sed handles in-place substitution and jq handles JSON — but for structured plaintext, awk is still the sharpest tool in the box.

How cron works

crond is a daemon that runs in the background and wakes up every minute to check if any scheduled job is due. Jobs are defined in a crontab (cron table) — a plain text file with one job per line. Each line specifies a schedule and a command.

$ crontab -l
# no crontab for mukul

To edit your crontab:

$ crontab -e

This opens the file in $EDITOR (usually vi or nano). Changes take effect immediately after saving.

Crontab syntax

Every cron entry follows this structure:

* * * * * /path/to/command
│ │ │ │ │
│ │ │ │ └─ Day of week  (0–7, 0 and 7 are Sunday)
│ │ │ └─── Month        (1–12)
│ │ └───── Day of month (1–31)
│ └─────── Hour         (0–23)
└───────── Minute       (0–59)

A * means “every valid value”. Here are some examples:

Step values

*/n means “every n units”. */15 in the minute field means every 15 minutes.

Range and list

• A range: 1-5 (Monday through Friday)
• A list: 1,3,5 (Monday, Wednesday, Friday)
• Combined: 0 9 * * 1,3,5 — 9 AM on Mon, Wed, Fri

Practical examples

Run a backup script nightly at 2 AM

0 2 * * * /home/mukul/scripts/backup.sh >> /var/log/backup.log 2>&1

The >> ... 2>&1 redirects both stdout and stderr to a log file. Without redirection, cron mails the output to the system user — which often disappears silently.

Clear a temp directory every Sunday at 3 AM

0 3 * * 0 rm -rf /tmp/myapp/cache/*

Restart a service if it crashes (crude watchdog)

*/5 * * * * pgrep myapp || systemctl restart myapp

Pull latest code and restart an app every hour

0 * * * * cd /srv/myapp && git pull && systemctl restart myapp

Environment in cron

Cron jobs run with a minimal environment — no ~/.bashrc, no PATH beyond /usr/bin:/bin. This is the most common source of “it works manually but not in cron” bugs.

Fix it by setting PATH at the top of your crontab:

PATH=/usr/local/bin:/usr/bin:/bin:/home/mukul/.local/bin

0 2 * * * backup.sh

Or use absolute paths everywhere inside the script.

System-wide cron directories

On Linux, you don’t always need crontab -e. Drop a script into one of these directories and cron will run it automatically:

/etc/cron.hourly/
/etc/cron.daily/
/etc/cron.weekly/
/etc/cron.monthly/

Files in these directories must be executable and must not have an extension (no .sh).

$ sudo cp myscript /etc/cron.daily/myscript
$ sudo chmod +x /etc/cron.daily/myscript

macOS specifics

macOS ships with cron but Apple recommends launchd for new automations. That said, crontab still works fine on macOS for most use cases. One gotcha: full disk access permissions can block cron from accessing certain directories under modern macOS security restrictions. If a cron job silently fails, check System Settings → Privacy & Security → Full Disk Access and add cron or your terminal emulator.

You can also use launchd plists in ~/Library/LaunchAgents/ for finer-grained scheduling (e.g., run at login, retry on failure), but for simple recurring jobs crontab is less boilerplate.

Debugging cron jobs

Check the system mail:

$ mail
# or
$ cat /var/spool/mail/$USER

Check syslog for cron activity:

$ grep CRON /var/log/syslog | tail -20
May 23 02:00:01 hostname CRON[12345]: (mukul) CMD (/home/mukul/scripts/backup.sh)

Test the command manually with a clean environment:

$ env -i HOME=$HOME PATH=/usr/bin:/bin bash -c 'your_command_here'

This strips most environment variables and closely mimics what cron sees.

Common mistakes

• Forgetting 2>&1 — error output silently disappears
• Relative paths — use absolute paths or set PATH in the crontab
• No newline at end of file — some cron implementations ignore the last line without a trailing newline
• Editing /etc/crontab directly instead of crontab -e — system crontab has an extra user field; mixing formats causes parse errors

Conclusion

cron is one of those tools you can learn in 20 minutes and rely on for years. The syntax is compact but expressive enough for nearly every scheduling pattern you’ll encounter. Once you get the hang of redirecting output, setting PATH, and testing jobs in a clean environment, you’ll stop wrestling with silent failures and start trusting your scheduled jobs to just run.

How DNS Resolution Works

When you type example.com into a browser, here’s what actually happens:

Browser
  ↓
Local DNS cache (checked first)
  ↓
Recursive resolver (usually your ISP or 8.8.8.8)
  ↓
Root nameserver (.) — knows where .com nameservers are
  ↓
TLD nameserver (.com) — knows where example.com nameservers are
  ↓
Authoritative nameserver (ns1.example.com) — returns the actual record
  ↑
Answer propagates back up and is cached at each layer

The whole process typically takes 20–120 ms for a cold lookup. Subsequent lookups are answered from cache in under 1 ms.

DNS Record Types

A Record

Maps a hostname to an IPv4 address. The most fundamental record.

example.com.    300    IN    A    93.184.216.34
www.example.com. 300   IN    A    93.184.216.34

You can have multiple A records for the same hostname — DNS returns all of them, and the client picks one (usually the first, sometimes randomly). This is the simplest form of load balancing, though it has no health checking.

AAAA Record

Maps a hostname to an IPv6 address. Same concept as A, but for IPv6.

example.com.    300    IN    AAAA    2606:2800:220:1:248:1893:25c8:1946

CNAME Record

An alias — maps one hostname to another. The target must be an A (or AAAA) record, not an IP address.

www.example.com.    3600    IN    CNAME    example.com.
blog.example.com.   3600    IN    CNAME    mysite.netlify.app.

The CNAME constraint: you cannot put a CNAME on the root domain (example.com) because the root needs SOA and NS records, which can’t coexist with a CNAME. Use an A record for the root and CNAME for subdomains. Some DNS providers offer “CNAME flattening” or “ALIAS” records to work around this.

CNAME chain length: avoid chaining CNAMEs (a → b → c). Each hop costs a DNS lookup and adds latency.

MX Record

Specifies mail servers for a domain, with a priority value (lower = higher priority):

example.com.    3600    IN    MX    10    mail1.example.com.
example.com.    3600    IN    MX    20    mail2.example.com.

When email is sent to user@example.com, the sender’s mail server looks up the MX records and tries them in priority order.

TXT Record

Arbitrary text. Used for domain verification, SPF, DKIM, and DMARC:

; SPF — which servers are allowed to send email for this domain
example.com.    TXT    "v=spf1 include:_spf.google.com ~all"

; Domain ownership verification (Google Search Console, etc.)
example.com.    TXT    "google-site-verification=abc123..."

; DKIM public key (for email signing)
mail._domainkey.example.com.    TXT    "v=DKIM1; k=rsa; p=MIGfMA0..."

NS Record

Nameserver — delegates authority for a zone to specific nameservers:

example.com.    NS    ns1.cloudflare.com.
example.com.    NS    ns2.cloudflare.com.

NS records are set at your domain registrar and point to whoever hosts your DNS zone.

SOA Record

Start of Authority — metadata about the DNS zone itself (primary nameserver, admin email, serial number, refresh intervals). You rarely set this manually; your DNS provider manages it.

PTR Record

Reverse DNS — maps an IP address back to a hostname. Used by mail servers to verify that sending servers are legitimate.

34.216.184.93.in-addr.arpa.    PTR    example.com.

TTL: Time to Live

TTL is how long (in seconds) resolvers and clients should cache a DNS record before asking again.

example.com.    300    IN    A    93.184.216.34
              ↑
           TTL = 300 seconds = 5 minutes

Implications:

• Low TTL (60–300s): DNS changes propagate quickly. Good before a migration, but generates more DNS traffic.
• High TTL (3600–86400s): Changes are slow to propagate (up to 24 hours), but faster for end users.

Best practice: lower the TTL to 300 before a planned IP change, wait for the old TTL to expire, make the change, then raise the TTL back after confirming everything works.

Debugging DNS with dig

dig is your primary DNS debugging tool:

# Basic A record lookup
$ dig example.com

;; ANSWER SECTION:
example.com.    300    IN    A    93.184.216.34

# Lookup a specific record type
$ dig example.com MX
$ dig example.com TXT
$ dig example.com NS

# Use a specific resolver (bypass your local cache)
$ dig @8.8.8.8 example.com

# Check the full resolution chain (+trace)
$ dig +trace example.com

# Short output
$ dig +short example.com
93.184.216.34

# Reverse DNS lookup
$ dig -x 93.184.216.34

Debugging with nslookup

nslookup is available on Windows and macOS by default:

$ nslookup example.com
Server:    192.168.1.1
Address:   192.168.1.1#53

Non-authoritative answer:
Name:    example.com
Address: 93.184.216.34

# Query a specific record type
$ nslookup -type=MX example.com

# Query a specific server
$ nslookup example.com 8.8.8.8

Common DNS Pitfalls

“My DNS change isn’t propagating” — DNS propagation isn’t magic. What’s actually happening is that resolvers around the world are holding cached copies of the old record until their TTL expires. Check the old TTL before making changes.

CNAME on the root domain — This breaks things. Use an A record for example.com and a CNAME for www.example.com.

Missing the trailing dot — In zone files, hostnames are written with a trailing dot (example.com.) to indicate they’re absolute. Without it, many DNS tools append the zone name, turning mail into mail.example.com.. dig adds trailing dots in output; most DNS UIs handle it for you.

Forgetting SPF/DKIM/DMARC — Email without these records gets flagged as spam or rejected outright. Set them up when you configure MX records.

Conclusion

DNS is deceptively simple on the surface — point a domain at an IP, done — but the details matter when things go wrong or when you’re setting up email, CDNs, or multi-region infrastructure. The record types you’ll use most are A, CNAME, MX, and TXT. Keep TTLs low before planned changes, use dig to inspect what resolvers actually see (not what your DNS panel shows), and remember that “DNS propagation” is just cache TTL expiry — not something that happens on its own schedule.

Images vs Containers

An image is a read-only blueprint. A container is a running instance of that blueprint. The relationship is the same as a class and an object in code — one image can spawn many containers.

Pull the official Nginx image and you’ve downloaded a layered filesystem snapshot:

$ docker pull nginx:alpine
alpine: Pulling from library/nginx
Digest: sha256:2d194184...
Status: Downloaded newer image for nginx:alpine

Spin up a container from it:

$ docker run -d -p 8080:80 --name web nginx:alpine
c3a1f9e7b2d4...

$ docker ps
CONTAINER ID   IMAGE          COMMAND                  CREATED         STATUS         PORTS                  NAMES
c3a1f9e7b2d4   nginx:alpine   "/docker-entrypoint.…"   5 seconds ago   Up 4 seconds   0.0.0.0:8080->80/tcp   web

-d runs it in the background. -p 8080:80 maps host port 8080 to container port 80. Visit http://localhost:8080 and you’ll see Nginx’s welcome page.

Building Your Own Image

A Dockerfile is a recipe for your image. Each instruction adds a layer.

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["python", "app.py"]

Build it:

$ docker build -t myapp:latest .
[+] Building 12.3s
 => [1/5] FROM python:3.12-slim
 => [2/5] WORKDIR /app
 => [3/5] COPY requirements.txt .
 => [4/5] RUN pip install --no-cache-dir -r requirements.txt
 => [5/5] COPY . .
 => exporting to image

The COPY requirements.txt step is deliberately before COPY . . — Docker caches each layer. If you change application code but not requirements.txt, the pip install layer is reused and rebuilds are fast.

Essential Container Commands

# list running containers
$ docker ps

# list all containers including stopped ones
$ docker ps -a

# stream logs
$ docker logs -f web

# open a shell inside a running container
$ docker exec -it web sh

# stop and remove
$ docker stop web && docker rm web

# remove the image
$ docker rmi nginx:alpine

Volumes: Persisting Data

Containers are ephemeral — when you remove one, all data written inside it disappears. Volumes solve this by mounting a storage location from the host (or a Docker-managed directory) into the container.

# named volume — Docker manages the location
$ docker run -d \
  -v pgdata:/var/lib/postgresql/data \
  -e POSTGRES_PASSWORD=secret \
  --name db \
  postgres:16

# bind mount — explicit host path
$ docker run -d \
  -v /home/mukul/site:/usr/share/nginx/html:ro \
  -p 8080:80 \
  nginx:alpine

Named volumes survive container removal. Bind mounts are great for local development because you edit files on the host and the container picks up changes immediately.

List and inspect volumes:

$ docker volume ls
DRIVER    VOLUME NAME
local     pgdata

$ docker volume inspect pgdata
[
  {
    "Name": "pgdata",
    "Mountpoint": "/var/lib/docker/volumes/pgdata/_data",
    ...
  }
]

Cleaning Up

Docker accumulates unused images, stopped containers, and dangling volumes quickly.

# remove all stopped containers, unused networks, dangling images
$ docker system prune
WARNING! This will remove:
  - all stopped containers
  - all networks not used by at least one container
  - all dangling images
  - all dangling build cache

Total reclaimed space: 1.23GB

# nuclear option — also removes unused images
$ docker system prune -a

Conclusion

Images are immutable blueprints, containers are running instances, and volumes are where persistent data lives. With just docker build, docker run, and a well-written Dockerfile, you can package any application into a portable, reproducible artifact. Once you’re comfortable running single containers, the natural next step is coordinating multiple services with Docker Compose.

The docker-compose.yml File

Compose describes your application as a set of services, each of which maps to a container. Here’s a practical example: a FastAPI backend backed by PostgreSQL and Redis.

services:
  api:
    build: .
    ports:
      - "8000:8000"
    environment:
      DATABASE_URL: postgresql://user:secret@db:5432/appdb
      REDIS_URL: redis://cache:6379
    depends_on:
      db:
        condition: service_healthy
      cache:
        condition: service_started
    volumes:
      - .:/app

  db:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: appdb
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d appdb"]
      interval: 5s
      timeout: 5s
      retries: 5

  cache:
    image: redis:7-alpine
    volumes:
      - redisdata:/data

volumes:
  pgdata:
  redisdata:

Services can reference each other by service name as the hostname — db and cache are resolvable from the api container because Compose creates a shared network automatically.

Starting and Stopping

# start everything in the background
$ docker compose up -d
[+] Running 3/3
 ✔ Container myapp-db-1     Healthy
 ✔ Container myapp-cache-1  Started
 ✔ Container myapp-api-1    Started

# tail logs from all services
$ docker compose logs -f

# tail just the api service
$ docker compose logs -f api

# stop and remove containers (volumes are preserved)
$ docker compose down

# stop AND remove volumes (wipes the database)
$ docker compose down -v

Rebuilding After Code Changes

When you change your Dockerfile or source code, Compose won’t automatically pick it up unless you rebuild.

$ docker compose up -d --build

For local development with a bind mount (- .:/app), code changes reflect immediately without rebuilding — only dependency changes in your Dockerfile need a rebuild.

Running One-Off Commands

# open a postgres shell
$ docker compose exec db psql -U user -d appdb

# run database migrations
$ docker compose run --rm api python manage.py migrate

# run tests inside the api container
$ docker compose run --rm api pytest

exec runs a command in an already-running container. run spins up a fresh container for the command and exits when done. --rm cleans it up afterwards.

Scaling a Service

$ docker compose up -d --scale api=3

This starts three instances of the api service. You’d typically put a load balancer (Nginx or Traefik) in front of them, but it’s a fast way to test horizontal scaling locally.

Environment Files

Hardcoding secrets in docker-compose.yml is fine for local dev, but for shared repos or staging environments, use a .env file:

# .env
POSTGRES_PASSWORD=secret
REDIS_URL=redis://cache:6379

Compose reads .env automatically. Reference variables with ${VAR_NAME}:

environment:
  DATABASE_URL: postgresql://user:${POSTGRES_PASSWORD}@db:5432/appdb

Add .env to .gitignore and commit a .env.example with placeholder values instead.

Useful Inspection Commands

# show running containers and their status
$ docker compose ps

# show which host ports are mapped
$ docker compose port api 8000
0.0.0.0:8000

# show resource usage
$ docker compose stats

Conclusion

Docker Compose turns a sprawling set of docker run commands into a single, readable file that anyone on the team can spin up identically. The depends_on with healthchecks ensures services start in the right order, named volumes keep data safe across restarts, and the built-in network means services find each other by name without any manual configuration. Once you’re running containers locally with Compose, moving to production with Kubernetes or a managed container service becomes a much smaller leap.

Basic Syntax

find [path] [expression]

The path is where to start searching (recursively). The expression is a combination of tests and actions.

$ find . -name "*.log"
./app/logs/access.log
./app/logs/error.log

Start with . to search the current directory tree, or give an absolute path like /var/log.

Searching by Name

-name is case-sensitive. Use -iname for a case-insensitive match.

$ find /etc -name "nginx.conf"
/etc/nginx/nginx.conf

$ find . -iname "readme*"
./README.md
./docs/readme.txt

Use shell wildcards (*, ?) inside quotes to prevent the shell from expanding them before find sees them.

Searching by Type

-type filters results by file type:

$ find . -type d -name "__pycache__"
./src/__pycache__
./tests/__pycache__

Searching by Size

-size accepts a number with a unit suffix:

Prefix with + for “greater than” or - for “less than”:

$ find /var/log -type f -size +100M
/var/log/syslog.1

$ find . -type f -size -1k
./config/.gitkeep
./config/empty.conf

Searching by Modification Time

-mtime filters by last modification time in days. -mmin works the same way in minutes.

# Files modified in the last 24 hours
$ find . -type f -mtime -1

# Files not touched in over 30 days
$ find /tmp -type f -mtime +30

# Files modified in the last 60 minutes
$ find . -type f -mmin -60

There are three time flags:

• -mtime — last modification time (content changed)
• -atime — last access time (file was read)
• -ctime — last status change time (permissions or ownership changed)

Searching by Permissions

-perm matches files by their permission bits:

# Files with exactly 644 permissions
$ find . -type f -perm 644

# Files that are world-writable (dangerous to have in web roots)
$ find /var/www -type f -perm -o+w

The - prefix means “at least these bits are set.” Without it, find requires an exact match.

Combining Tests with AND, OR, NOT

By default, multiple tests are ANDed together. Use -o for OR and ! for NOT:

# .log OR .tmp files
$ find . \( -name "*.log" -o -name "*.tmp" \) -type f

# Everything that is NOT a directory
$ find . ! -type d

Executing Commands with -exec

This is where find gets genuinely powerful. -exec runs a command for each matched file. Use {} as the placeholder for the filename and terminate the command with \;:

# Delete all .pyc files
$ find . -name "*.pyc" -exec rm {} \;

# Change ownership on all files in a web directory
$ find /var/www -type f -exec chown www-data:www-data {} \;

# Print the size of each matching file
$ find . -name "*.log" -exec du -sh {} \;
4.0K    ./app/logs/access.log
128M    ./app/logs/error.log

Replace \; with + to batch arguments into a single command invocation (faster for large result sets):

$ find . -name "*.pyc" -exec rm {} +

Using xargs for Better Performance

xargs is often more flexible than -exec ... +. Pipe find output into it:

$ find . -name "*.log" -print0 | xargs -0 grep "ERROR"

-print0 and -0 use the null byte as a delimiter, which handles filenames with spaces correctly.

Excluding Directories with -prune

-prune tells find to skip a directory entirely:

# Search everything except node_modules
$ find . -name "node_modules" -prune -o -name "*.js" -print

# Exclude multiple directories
$ find . \( -name ".git" -o -name "vendor" -o -name "node_modules" \) -prune -o -type f -print

The -o -print at the end is necessary — without it, -prune would suppress output for the non-pruned results.

Practical One-Liners

# Find and delete all empty directories
$ find . -type d -empty -delete

# Find duplicate filenames (same name, different location)
$ find . -type f -printf "%f\n" | sort | uniq -d

# List the 10 largest files under /var
$ find /var -type f -printf "%s %p\n" | sort -rn | head -10

# Find files owned by a specific user
$ find /home -user alice -type f

# Find SUID binaries (common security audit)
$ find / -perm -4000 -type f 2>/dev/null

Conclusion

find rewards learning its flags properly. The combination of type filtering, time-based searches, permission checks, and -exec makes it a complete filesystem query language. The key patterns to internalize are: use -print0 | xargs -0 for safe pipelining, use -prune to skip large directories, and always quote your wildcard patterns. Most tasks that feel like they need a shell loop can be handled more cleanly with a single find command.

How GitHub Actions Works

Actions are triggered by events (push, pull_request, schedule, etc.) and run workflows — YAML files in .github/workflows/. Each workflow has jobs, each job runs on a runner (a fresh VM), and each job has steps that run sequentially.

Event → Workflow → Jobs (parallel by default) → Steps (sequential)

A Basic CI Workflow

Create .github/workflows/ci.yml:

name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install dependencies
        run: |
          pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run tests
        run: pytest --tb=short -q

      - name: Lint
        run: ruff check .

Push this file and GitHub immediately starts running the workflow. Every subsequent pull request will show a green or red status check before you can merge.

Caching Dependencies

By default, every run reinstalls dependencies from scratch. Caching the pip download cache cuts minutes off each run:

      - name: Cache pip packages
        uses: actions/cache@v4
        with:
          path: ~/.cache/pip
          key: $-pip-$
          restore-keys: |
            $-pip-

The cache key includes a hash of requirements.txt, so the cache is invalidated whenever dependencies change.

Matrix Builds: Test Across Multiple Versions

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.10", "3.11", "3.12"]

    steps:
      - uses: actions/checkout@v4

      - name: Set up Python $
        uses: actions/setup-python@v5
        with:
          python-version: $

      - name: Install and test
        run: |
          pip install -r requirements.txt
          pytest

This runs three parallel jobs — one per Python version — and reports them all in the pull request.

A Deployment Workflow

Separation of concerns: the CI workflow runs on every PR, the deploy workflow runs only on pushes to main. Create .github/workflows/deploy.yml:

name: Deploy

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    environment: production

    steps:
      - uses: actions/checkout@v4

      - name: Build Docker image
        run: docker build -t myapp:$ .

      - name: Log in to Docker Hub
        uses: docker/login-action@v3
        with:
          username: $
          password: $

      - name: Push image
        run: |
          docker tag myapp:$ myuser/myapp:latest
          docker push myuser/myapp:latest

      - name: Deploy to server
        uses: appleboy/ssh-action@v1
        with:
          host: $
          username: $
          key: $
          script: |
            docker pull myuser/myapp:latest
            docker compose up -d --no-deps api

Managing Secrets

Never hardcode credentials in workflow files. Store them in Settings → Secrets and variables → Actions and reference them as $. Secret values are masked in logs.

For per-environment secrets (staging vs production), use Environments (environment: production in the job), which lets you require manual approval before a deployment job runs.

Useful Workflow Patterns

Run a job only on specific file changes:

on:
  push:
    paths:
      - "src/**"
      - "requirements.txt"

Share data between jobs:

jobs:
  build:
    runs-on: ubuntu-latest
    outputs:
      image_tag: $
    steps:
      - id: tag
        run: echo "tag=$" >> $GITHUB_OUTPUT

  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - run: echo "Deploying $"

Conditional steps:

      - name: Notify Slack on failure
        if: failure()
        uses: slackapi/slack-github-action@v1
        with:
          payload: '{"text":"Deploy failed on $"}'
        env:
          SLACK_WEBHOOK_URL: $

Conclusion

GitHub Actions gives you a full CI/CD pipeline without leaving your repository. The CI workflow on pull requests catches regressions before they merge, the deployment workflow automates what would otherwise be a manual, error-prone release process, and secrets management keeps credentials out of your codebase. Start with a simple test job and add deployment steps incrementally — a working pipeline you understand beats a sophisticated one that nobody maintains.

Basic Syntax

$ grep "pattern" file.txt
$ grep "pattern" file1.txt file2.txt
$ command | grep "pattern"

Essential Flags

grep vs egrep vs fgrep

• grep — basic regex (BRE): \(, \), \+ need backslashes
• egrep / grep -E — extended regex (ERE): (, ), +, | work without backslashes
• fgrep / grep -F — no regex, literal string matching; fastest

In practice, always use grep -E instead of egrep — it’s the same behavior, just more explicit.

Basic Examples

$ grep "ERROR" app.log
2024-01-15 10:23:11 ERROR Connection refused
2024-01-15 10:25:44 ERROR Timeout after 30s

Case-insensitive:

$ grep -i "error" app.log

Count matches:

$ grep -c "ERROR" app.log
14

Show surrounding context (5 lines before and after):

$ grep -C 5 "segfault" dmesg.log

Recursive Search

$ grep -rn "TODO" ./src/
./src/api/handler.go:42:  // TODO: add retry logic
./src/db/conn.go:17:      // TODO: use connection pool

Restrict to a file type with --include:

$ grep -rn --include="*.py" "import requests" .
./scripts/fetch.py:3:import requests
./api/client.py:1:import requests

Exclude a directory:

$ grep -rn --exclude-dir=".git" --exclude-dir="node_modules" "console.log" .

Extended Regex with -E

Match lines containing “foo” or “bar”:

$ grep -E "foo|bar" file.txt

One or more digits:

$ grep -E "[0-9]+" file.txt

Match IP addresses:

$ grep -E "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b" access.log

Anchors: ^ start of line, $ end:

$ grep "^ERROR" app.log        # lines starting with ERROR
$ grep "\.json$" filelist.txt  # lines ending with .json

Whole-Word and Whole-Line Matching

$ grep -w "log" file.txt     # matches "log" but not "logging" or "catalog"
$ grep -x "exact line" file.txt  # only lines that are exactly "exact line"

Printing Only the Match (-o)

Extract just the matching portion from each line:

$ grep -oE "[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+" access.log | sort | uniq -c | sort -rn
     342 192.168.1.1
      87 10.0.0.15
      12 172.16.0.3

This extracts IPs, counts occurrences, and sorts by frequency — all without awk.

Using grep in Scripts

grep -q exits silently with code 0 (match) or 1 (no match):

if grep -q "ERROR" app.log; then
  echo "Errors found, alerting oncall"
fi

-l returns only filenames — useful for batch processing:

$ grep -rl "deprecated_function" ./src/ | xargs sed -i 's/deprecated_function/new_function/g'

ripgrep (rg) — The Faster Alternative

Install:

$ brew install ripgrep    # macOS
$ sudo apt install ripgrep  # Ubuntu

rg is 5–10x faster than grep -r on codebases because it:

• Uses SIMD for matching
• Skips binary files automatically
• Respects .gitignore by default
• Searches in parallel across CPU cores

The interface mirrors grep in most ways:

$ rg "TODO" ./src/
src/api/handler.go:42:  // TODO: add retry logic
src/db/conn.go:17:      // TODO: use connection pool

rg-specific Advantages

Respects .gitignore — doesn’t search node_modules, dist, or .git by default. Override with --no-ignore.

File type filtering with -t:

$ rg -t py "import os"       # only .py files
$ rg -t js "console.log"     # only .js files
$ rg --type-list             # see all supported types

Fixed string (faster, no regex):

$ rg -F "function()" .

Count per file:

$ rg -c "ERROR" logs/
logs/app.log:14
logs/worker.log:3

Show only filenames:

$ rg -l "TODO" .

Multiline mode:

$ rg -U "foo\nbar" .

Replace in output (doesn’t modify files — use with sed or perl for that):

$ rg -r "new_name" "old_name" .

Common One-Liners

Find all files containing a function name:

$ grep -rl "handleRequest" ./src/

Count total lines of Python code (excluding blank lines and comments):

$ grep -r --include="*.py" -v "^[[:space:]]*#\|^[[:space:]]*$" . | wc -l

Find lines with two or more consecutive spaces:

$ grep -P "  +" file.txt

Check if a process is running:

$ pgrep -x nginx || grep -q nginx /proc/*/comm 2>/dev/null && echo "running"

Find all TODO and FIXME comments:

$ rg "TODO|FIXME|HACK|XXX" --color always | less -R

Search compressed logs:

$ zgrep "ERROR" /var/log/nginx/error.log.gz

grep Exit Codes

These are useful in shell scripts: grep -q "pattern" file && echo "found".

Conclusion

grep with -E, -n, -r, and -C covers most daily search needs. For code search, switch to ripgrep — it’s faster, .gitignore-aware, and has better defaults. The fundamentals — anchors, character classes, alternation — transfer directly between the two. Know how to extract just the match with -o, how to count with -c, and how to use exit codes in scripts, and you’ll have grep fully in your toolkit.

What is gRPC?

gRPC is a remote procedure call framework from Google. Instead of defining routes and JSON schemas, you define a service contract in a .proto file using Protocol Buffers, and the framework generates client and server code in your language of choice.

syntax = "proto3";

service UserService {
  rpc GetUser (GetUserRequest) returns (User);
  rpc ListUsers (ListUsersRequest) returns (stream User);
  rpc CreateUser (CreateUserRequest) returns (User);
}

message GetUserRequest {
  string user_id = 1;
}

message User {
  string id = 1;
  string name = 2;
  string email = 3;
  int64 created_at = 4;
}

Run protoc on this file and you get generated stubs in Python, Go, Java, TypeScript, or any other supported language. The client calls stub.GetUser(request) like a normal function call.

Side-by-Side Comparison

Performance

gRPC’s performance advantage comes from two sources:

1. Binary encoding — Protocol Buffers are more compact than JSON. A User object with 5 fields might be 200 bytes as JSON and 40 bytes as protobuf.
2. HTTP/2 multiplexing — multiple requests share one TCP connection, eliminating the head-of-line blocking and connection overhead of HTTP/1.1.

For high-frequency internal service calls — tens of thousands per second — this compounds into measurable throughput gains and latency reductions.

The Four Streaming Modes

This is gRPC’s biggest differentiator over REST:

service StreamingService {
  // classic request/response
  rpc UnaryCall (Request) returns (Response);

  // server sends a stream of responses
  rpc ServerStreaming (Request) returns (stream Response);

  // client sends a stream of requests
  rpc ClientStreaming (stream Request) returns (Response);

  // both sides stream simultaneously
  rpc BidirectionalStreaming (stream Request) returns (stream Response);
}

Bidirectional streaming enables real-time communication patterns that would require WebSockets over REST — chat, live data feeds, collaborative editing.

A Minimal gRPC Server in Python

import grpc
from concurrent import futures
import user_pb2
import user_pb2_grpc

class UserServicer(user_pb2_grpc.UserServiceServicer):
    def GetUser(self, request, context):
        # fetch from database
        return user_pb2.User(
            id=request.user_id,
            name="Mukul Kadel",
            email="mukul@example.com",
        )

    def ListUsers(self, request, context):
        users = fetch_users(limit=request.limit)
        for user in users:
            yield user_pb2.User(id=user.id, name=user.name, email=user.email)

def serve():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    user_pb2_grpc.add_UserServiceServicer_to_server(UserServicer(), server)
    server.add_insecure_port("[::]:50051")
    server.start()
    server.wait_for_termination()

The client:

channel = grpc.insecure_channel("localhost:50051")
stub = user_pb2_grpc.UserServiceStub(channel)

user = stub.GetUser(user_pb2.GetUserRequest(user_id="abc123"))
print(user.name)  # Mukul Kadel

for user in stub.ListUsers(user_pb2.ListUsersRequest(limit=100)):
    print(user.name)

Inspecting gRPC Without a Client

# like curl for gRPC
$ brew install grpcurl

# list services
$ grpcurl -plaintext localhost:50051 list

# call a method
$ grpcurl -plaintext \
  -d '{"user_id": "abc123"}' \
  localhost:50051 \
  UserService/GetUser

When to Choose REST

• Public APIs where external developers will integrate — JSON over HTTP is universally understood
• Browser-to-server communication without a proxy layer
• Simple CRUD with standard tooling (Swagger, Postman, curl)
• Small teams where the overhead of .proto files and code generation isn’t justified

When to Choose gRPC

• Internal microservices communicating at high frequency
• Streaming — live feeds, bidirectional channels, large file uploads chunked over a stream
• Multi-language systems where a shared .proto contract prevents schema drift
• Mobile clients where payload size and battery life matter

Conclusion

REST is the right default for most APIs — especially anything public-facing or browser-native. gRPC pays off when you’re building a mesh of internal services where performance, strong typing, and streaming matter more than ecosystem breadth. The good news is they’re not mutually exclusive: many systems use gRPC internally between services and REST externally for the public API, with a gateway layer translating between them.