In the world of DevOps, we don't like single points of failure. If you're running a NanoPi (like the R5S or R6S), you have a secret weapon: Internal eMMC storage. It’s faster, more resilient, and physically soldered to the board.

Here is the "battle-tested" guide to migrating your OS from the card to the chip.

1. The "Identify Your Target" Phase

Before you start throwing data around, you need to know who is who. In Linux, your storage devices aren't labeled "SD Card" or "Internal Drive"—they are blocks.

Most people reach for lsblk, but many lean OpenWrt builds don't include it. Instead, we go straight to the source:

cat /proc/partitions

The Discovery: You’ll see two main contenders. Usually, mmcblk0 is your boot source (the SD card) and mmcblk1 is the factory-fresh internal eMMC. Look at the sizes. If your SD card is 32GB and the eMMC is 30GB, you’ve found your targets.

2. The Great Migration (The "DD" Command)

We use the oldest tool in the shed: dd. It stands for "Data Duplicator," but many call it "Disk Destroyer" because if you swap the if (input) and of (output), you’ll wipe your work.

Run the clone:

dd if=/dev/mmcblk0 of=/dev/mmcblk1 bs=1M && sync

A Note on the "Silent Treatment": Don't panic if the cursor just blinks for 15 minutes. Most embedded versions of dd don't show a progress bar. It’s moving 30GB of data. Grab a coffee. If you’re anxious, open a second SSH window and run pgrep -l dd to make sure the heart is still beating.

3. The "No Space Left" Error (The Good Kind)

At the end, you might see: dd: error writing '/dev/mmcblk1': No space left on device.

In any other context, this is a failure. Here? It’s a victory. It means the eMMC was slightly smaller than the SD card, and dd filled every single available block. Since the actual OpenWrt partitions are at the very beginning of the drive, the OS is safe and sound on the new chip.

4. The Moment of Truth

The "Cut-over" is simple but nerve-wracking:

1. Run poweroff.

2. Pull the microSD card out. This is the most important step—if the card is in, the NanoPi will always prefer it over the internal storage.

3. Power it back on.

If that SSH prompt returns without the SD card in the slot, you’re officially running on the metal.

Pro-Tip for the Reader: After booting from eMMC, your "Free Space" might look smaller than expected because you cloned a fixed-size image. Your next task is to use parted or the LuCI web interface to expand the partition to fill the rest of your eMMC.

Happy Routing.

Someone on Twitter said they "lowkey want to understand the technology behind Google Authenticator" and I dropped a quick reply - explaining that it's basically TOTP: your device and the server share a secret key, both compute a code using HMAC-SHA1 and the current 30-second time window. No network calls. No "previous code." Same secret + same time slice = same 6-digit code.

That reply got some traction, and a few people DM me for a deeper breakdown. So here we are.

If you've ever wondered how your phone generates the exact same 6-digit code the server expects - with no internet request, no sync, nothing - this one's for you.

The Problem With Passwords

Passwords are static. Once someone has it, they have it forever - or until you change it. Even with a strong password, you're one phishing attack or database breach away from compromise.

Two-factor authentication fixes this by adding something that changes. But here's the catch - if your phone needs to call a server every time to get a new code, that's a point of failure. What happens when you're offline? On a plane? In a basement with no signal?

This is where TOTP comes in.

TOTP - Time-based One-Time Password

TOTP is defined in RFC 6238, but don't let the RFC scare you. The core idea is dead simple:

Both your phone and the server share a secret. They both know the current time. They both do the same math. They both get the same answer.

That's it. No network calls. No synchronization requests. Just two parties doing identical calculations independently.

The Setup - That QR Code You Scanned

When you enable 2FA on any service, they show you a QR code. That QR code contains a URL that looks something like this:

otpauth://totp/MyService:yusuf@yusadolat.me?secret=JBSWY3DPEHPK3PXP&issuer=MyService

The important part is the secret. This is a base32-encoded string that both your authenticator app and the server will store. This shared secret is the foundation of everything.

You scan it once. Your app saves it. The server saves it. They never exchange it again.

The Math - How Codes Get Generated

Every 30 seconds, both sides perform this calculation:

Step 1: Get the current time window

Take the current Unix timestamp and divide by 30. Floor it.

time_step = floor(current_unix_time / 30)

Right now, as I write this, the Unix timestamp is around 1733644800. Divided by 30, floored, gives us 57788160. This number changes every 30 seconds.

Step 2: Run HMAC-SHA1

Feed the time step and the shared secret into HMAC-SHA1:

hmac_result = HMAC-SHA1(secret, time_step)

This produces a 20-byte hash. It looks like random garbage, but it's deterministic - same inputs always give same outputs.

Step 3: Dynamic Truncation

20 bytes is too long for humans to type. So we extract 4 bytes from a specific position (determined by the last nibble of the hash), convert to an integer, and take modulo 1,000,000.

offset = hmac_result[19] & 0x0f
code = (hmac_result[offset:offset+4] & 0x7fffffff) % 1000000

Boom. You have your 6-digit code.

Why This Is Actually Clever

Think about what just happened:

1. No network needed - Your phone doesn't call anyone. The server doesn't push anything. Both just compute.

2. Codes expire automatically - Because time moves forward, old codes become useless. Even if someone shoulder-surfs your code, they have maybe 30 seconds to use it.

3. Can't predict future codes - Without the secret, you can't compute tomorrow's codes. The HMAC function is one-way.

4. Replay attacks fail - Use a code once, the server marks that time window as used. Try it again, rejected.

When Things Go Wrong

The system assumes both parties agree on what time it is. This is usually fine - your phone syncs with NTP servers, and servers have accurate clocks.

But I've seen people with phones that have "manual time" set, drifting by minutes. Their codes stop working and they have no idea why. The server is computing codes for 10:45:00, their phone is computing for 10:43:00. Different time windows, different codes.

Most implementations allow a small tolerance - they'll accept codes from one time window before or after. But drift too far and you're locked out.

The Recovery Code Situation

Those backup codes you're told to save somewhere? They're not TOTP. They're just long random strings stored in a database. Use one, it gets deleted. No time component, no algorithm - just a simple lookup.

Save them. Seriously. Losing access to your authenticator without backup codes is a special kind of pain.

Show Me The Code

Here's a minimal Python implementation to make this concrete:

import hmac
import hashlib
import struct
import time
import base64

def generate_totp(secret: str) -> str:
    # Decode the base32 secret
    key = base64.b32decode(secret.upper())

    # Get current time step (30-second window)
    time_step = int(time.time()) // 30

    # Pack as big-endian 8-byte integer
    time_bytes = struct.pack('>Q', time_step)

    # Compute HMAC-SHA1
    hmac_hash = hmac.new(key, time_bytes, hashlib.sha1).digest()

    # Dynamic truncation
    offset = hmac_hash[-1] & 0x0f
    code_int = struct.unpack('>I', hmac_hash[offset:offset+4])[0]
    code_int &= 0x7fffffff
    code = code_int % 1000000

    return f'{code:06d}'

# Test it
secret = 'JBSWY3DPEHPK3PXP'  # Example secret, This is what you add setup key on Google Auth
print(generate_totp(secret))

Want to see it work in real-time? Here's how to test:

1. Open Google Authenticator (or any TOTP app)

2. Tap the + button to add a new account

3. Select "Enter a setup key"

4. Enter any name (e.g., "TOTP Test")

5. For the key, enter: JBSWY3DPEHPK3PXP

6. Make sure it's set to Time-based

7. Save it

Now run the Python script. The 6-digit code it prints should match what's showing in your authenticator app. If you're a few seconds off, wait for the next 30-second window and try again.

Wrapping Up

There's no cloud magic happening when your authenticator generates codes. It's just math - the same math running independently on your device and the server, anchored to the same clock.

Understanding this changes how you think about 2FA. It's not some opaque security feature. It's a clever application of cryptographic primitives that's been working reliably for over a decade.

Next time you punch in those 6 digits, you'll know exactly what's happening behind the scenes.

If you found this useful, I write about DevOps, security, and cloud infrastructure. Connect with me on Twitter @Yusadolat.

Have you ever sat there waiting for your CodeBuild project to rebuild your entire Docker image... again? Even though you only changed a single line of code?

Yeah, me too. And it's frustrating.

Today, I'm going to show you how I reduced our Docker build times from ~7 minutes down to ~5 minutes (that's about 25-30% faster!) by implementing Amazon ECR as a persistent cache backend. This is based on an official AWS blog post, but I'll walk you through the practical implementation.

The Problem: Why Your Builds Are Slow

Here's the thing about AWS CodeBuild: every build runs in a completely fresh, isolated environment. That means:

• No build artifacts carry over between builds

• Every build starts from scratch

• CodeBuild's "local cache" is temporary and unreliable (works on a "best-effort" basis)

• If your builds happen at different times throughout the day, the local cache probably isn't helping you

So even if you only changed one line in your code, CodeBuild rebuilds every single Docker layer. Every. Single. Time.

The Solution: ECR Registry Cache Backend

The solution is surprisingly elegant: store your Docker layer cache persistently in Amazon ECR (Elastic Container Registry). Think of it as a separate "cache image" that lives alongside your actual application image.

Here's how it works:

1. First Build: Build from scratch, then export the cache to ECR as a separate image

2. Subsequent Builds: Import the cache from ECR, rebuild only what changed, export the updated cache back

The beauty? Your cache is always available, no matter when you trigger a build.

What You'll Need

Before we start, make sure you have:

• An existing AWS CodeBuild project that builds Docker images

• An ECR repository where your images are stored

• IAM permissions for your CodeBuild role to push/pull from ECR (if you can already push images, you're good!)

• About 10 minutes to implement this

Step-by-Step Implementation

Step 1: Understanding Your Current Buildspec

Your current buildspec probably looks something like this:

version: 0.2
phases:
  install:
    commands:
      - aws ecr get-login-password | docker login ...

  build:
    commands:
      - docker build -t myapp:latest .
      - docker tag myapp:latest $ECR_REPO:latest

  post_build:
    commands:
      - docker push $ECR_REPO:latest

This is the "basic" approach. Every build starts from zero.

[📸 IMAGE SUGGESTION: Split screen showing "Basic Build" vs "Cached Build" with layer rebuilding visualization]

Step 2: Add Cache Tag Variable

First, let's define a separate tag for our cache image. In your install phase, add:

install:
  commands:
    - CACHE_TAG=dev-cache  # or prod-cache, staging-cache, etc.
    - IMAGE_TAG=latest     # your actual app image tag

This creates a separate cache image (e.g., myapp:dev-cache) that's distinct from your application image (myapp:latest).

Step 3: Create the Buildx Builder

Here's the key part: Docker's default builder doesn't support registry cache backends. We need to create a new builder using buildx with the containerd driver.

Add this to your install phase:

install:
  commands:
    # ... your existing commands ...
    - docker buildx create --name containerd --driver=docker-container --driver-opt default-load=true --use || docker buildx use containerd

What's happening here?

• docker buildx create: Creates a new builder instance

• --driver=docker-container: Uses containerd (required for registry cache)

• --driver-opt default-load=true: Loads built images into local Docker (important!)

• || docker buildx use containerd: If the builder already exists, just switch to it

Step 4: Replace Your Docker Build Command

Now replace your regular docker build command with the new buildx version:

build:
  commands:
    - |
      docker buildx build \
        --builder=containerd \
        --cache-from type=registry,ref=$ECR_REPO:$CACHE_TAG \
        --cache-to type=registry,ref=$ECR_REPO:$CACHE_TAG,mode=max,image-manifest=true \
        -t $ECR_REPO:$IMAGE_TAG \
        --load \
        .

Let me break down what each flag does:

• --builder=containerd: Use the builder we just created

• --cache-from type=registry,ref=$ECR_REPO:$CACHE_TAG: Import cache from ECR

• --cache-to type=registry,ref=$ECR_REPO:$CACHE_TAG,mode=max,image-manifest=true: Export cache back to ECR

  • mode=max: Export all layers (recommended for best caching)

  • image-manifest=true: Required for ECR storage

• -t $ECR_REPO:$IMAGE_TAG: Tag your final image as usual

• --load: Load the built image into local Docker (so you can run it in post_build)

• .: Your Dockerfile location

Step 5: Complete Example Buildspec

Here's what a complete, production-ready buildspec looks like:

version: 0.2
phases:
  install:
    commands:
      - echo Logging in to Amazon ECR
      - aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com
      - CACHE_TAG=dev-cache
      - IMAGE_TAG=latest
      - ECR_REPO=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/myapp
      - docker buildx create --name containerd --driver=docker-container --driver-opt default-load=true --use || docker buildx use containerd

  build:
    commands:
      - echo Build started on `date`
      - |
        docker buildx build \
          --builder=containerd \
          --cache-from type=registry,ref=$ECR_REPO:$CACHE_TAG \
          --cache-to type=registry,ref=$ECR_REPO:$CACHE_TAG,mode=max,image-manifest=true \
          -t $ECR_REPO:$IMAGE_TAG \
          --load \
          .
      - docker tag $ECR_REPO:$IMAGE_TAG $ECR_REPO:latest

  post_build:
    commands:
      - echo Build completed on `date`
      - docker push $ECR_REPO:$IMAGE_TAG
      - docker push $ECR_REPO:latest

artifacts:
  files: 
    - imageDefinitions.json

Step 6: Update Your CodeBuild Project

You can update your buildspec in two ways:

Option 1: If your buildspec is in your repo Just commit the changes and push. CodeBuild will pick up the new buildspec automatically.

Option 2: If your buildspec is defined in CodeBuild Use the AWS CLI:

aws codebuild update-project --name your-project-name --cli-input-json file://buildspec.json

Or update it through the AWS Console: CodeBuild → Your Project → Edit → Buildspec

What to Expect: First Build vs Subsequent Builds

First Build (The Investment)

Your first build after implementing this will actually take slightly longer (maybe 30-60 seconds more). Don't panic! This is normal.

Here's what's happening:

1. Creating the buildx builder (~5-10 seconds)

2. Attempting to import cache (fails - no cache exists yet)

3. Building all layers from scratch

4. Exporting the cache to ECR (new step, adds ~20-40 seconds)

You'll see messages like:

=> importing cache manifest from $ECR_REPO:dev-cache
=> error: not found

This is expected! The cache doesn't exist yet.

Subsequent Builds (The Payoff)

This is where the magic happens. Your next builds will:

1. Successfully import the cache from ECR

2. Identify which layers haven't changed

3. Reuse cached layers (fast!)

4. Rebuild only the changed layers

5. Export the updated cache

Expected time savings:

• Before: 6-7 minutes (full rebuild every time)

• After: 5-5.5 minutes (25-30% faster!)

• Savings: 1-2 minutes per build

If you're doing 10 builds a day, that's 10-20 minutes saved daily. Over a month? That's 5-10 hours of compute time and costs saved.

Verifying It's Working

After your first build completes, check your ECR repository. You should now see two image tags:

1. Your application image (e.g., latest)

2. Your cache image (e.g., dev-cache)

The cache image will be roughly the same size as your application image - this is normal! It's storing all the layer information.

Troubleshooting Common Issues

Issue 1: "buildx: command not found"

Solution: Update your CodeBuild image to a newer version. Use aws/codebuild/standard:7.0 or later (or the ARM equivalent).

Issue 2: Cache Import Keeps Failing

Solution: Check your IAM permissions. Your CodeBuild role needs:

• ecr:BatchGetImage

• ecr:GetDownloadUrlForLayer

• ecr:BatchCheckLayerAvailability

• ecr:PutImage

• ecr:InitiateLayerUpload

• ecr:UploadLayerPart

• ecr:CompleteLayerUpload

Issue 3: Build Hangs at "exporting cache"

Solution: Make sure privilegedMode: true is enabled in your CodeBuild environment settings. This is required for Docker-in-Docker operations.

Advanced: Multi-Environment Setup

If you have multiple environments (dev, staging, prod), use different cache tags for each:

- CACHE_TAG=${ENVIRONMENT}-cache  # Results in: dev-cache, staging-cache, prod-cache

This way:

• Dev builds don't invalidate staging cache

• Each environment maintains its own optimized cache

• You can still share a base cache if needed

Cost Considerations

Storage Cost: You're now storing an additional cache image in ECR. At roughly the same size as your app image, this might add $0.10-0.50/month per repository depending on image size.

Compute Savings: Faster builds = less compute time. If you're saving 1-2 minutes per build and doing 10 builds/day, that's roughly 10-20 fewer compute hours per month. At ~$0.005/minute for BUILD_GENERAL1_SMALL, you could save $3-6/month.

Net Result: Typically a small net savings, plus the huge developer experience win of faster feedback loops.

Conclusion

By implementing ECR as a remote cache backend for your CodeBuild Docker builds, you get:

✅ Faster build times
✅ Persistent, reliable caching across all builds
✅ Better layer reuse with intelligent cache management
✅ Minimal code changes (just updating your buildspec)
✅ Cost savings from reduced compute time

The implementation is straightforward, and the benefits are immediate (after the first build). Give it a try on your next project!

References

• AWS Blog: Reduce Docker image build time using ECR as a remote cache

• Docker Buildx Documentation

• Docker Cache Backends Documentation

Got questions or run into issues? Drop a comment below - I'd love to hear about your experience implementing this!

After you complete this article, you will have a solid understanding of:

• Why deploying Next.js SSR on Cloudflare is different from traditional hosting

• The real difference between Cloudflare Pages with next-on-pages and OpenNext adapter

• How the free tier's 100,000 daily requests can handle production traffic

• Common deployment pitfalls that will waste hours of debugging

• Which adapter to choose for your specific use case

Have You Ever Seen This Error When Deploying to Cloudflare?

If you've tried deploying a Next.js app to Cloudflare Pages, you've probably encountered this frustrating message:

Error: Dynamic server usage: Page couldn't be rendered statically because it used `cookies`.

Or even worse:

Error: The edge runtime does not support Node.js 'fs' module.
You can use 'fs' module only in Node.js runtime.

And then you wonder: "But I thought Cloudflare supports Next.js SSR now?"

Let me clear up this confusion once and for all.

The Two Ways to Deploy Next.js on Cloudflare

There are two completely different approaches to deploying Next.js on Cloudflare, and choosing the wrong one will cause endless headaches.

Option 1: @cloudflare/next-on-pages (The Limited One)

This was the original way, and it comes with a massive limitation:

// Every server component needs this 👇
export const runtime = 'edge';

export default async function Page() {
  // ❌ This will fail!
  const fs = require('fs');

  // ❌ This will also fail!
  const bcrypt = require('bcrypt');

  // ✅ Only Web APIs work
  const response = await fetch('https://api.example.com');

  return <div>Limited to Edge Runtime</div>;
}

Option 2: @opennextjs/cloudflare (The Game Changer)

Released in 2024 and now in v1.0-beta, this adapter supports the Node.js runtime:

// No runtime declaration needed! 🎉

export default async function Page() {
  // ✅ This works now!
  const fs = require('fs');

  // ✅ This works too!
  const crypto = require('crypto');

  // ✅ Even database connections work
  const data = await prisma.user.findMany();

  return <div>Full Node.js support!</div>;
}

Setting Up Next.js SSR with OpenNext (The Right Way)

Let's deploy a real Next.js app with full SSR support on Cloudflare.

Option 1: Using Cloudflare CLI (Recommended)

# Create new Cloudflare project
npm create cloudflare@latest my-nextjs-app -- \
  --framework=next --platform=workers

# Deploy
npm run deploy

Ready to see this in action? Check out the complete working example with full source code, deployment configuration, and step-by-step setup instructions here

Migrating Your Existing Next.js App to Cloudflare

If you already have a Next.js app running on Vercel, AWS, or anywhere else, here's how to migrate it to Cloudflare.

Step 1: Check Your Current Setup

First, identify which features your app uses:

# Check your Next.js version
npm list next

# Look for these in your code:
grep -r "export const runtime" . # Edge runtime declarations
grep -r "getServerSideProps" .   # SSR pages
grep -r "getStaticProps" .        # SSG pages
grep -r "app/api" .               # API routes

Step 2: Choose Your Migration Path

If your app uses Vercel:

# Use Diverce for automatic migration!
npx diverce migrate

# This tool automatically:
# - Adds OpenNext to your project
# - Updates your configuration
# - Creates a PR with all changes

For manual migration:

# Install the OpenNext adapter
npm install -D @opennextjs/cloudflare wrangler

# Remove any Vercel-specific packages
npm uninstall @vercel/analytics @vercel/og

Step 3: Update Your Configuration

Remove edge runtime declarations:

// ❌ Remove these from all your files
export const runtime = 'edge';
export const dynamic = 'force-dynamic';

// ✅ OpenNext handles this automatically
// Just delete these lines!

Update your next.config.js:

// next.config.js
const { setupDevPlatform } = require('@cloudflare/next-on-pages/next-dev');

// ❌ Remove this if you have it
if (process.env.NODE_ENV === 'development') {
  await setupDevPlatform();
}

// ✅ Add this instead
import { initOpenNextCloudflareForDev } from "@opennextjs/cloudflare";
initOpenNextCloudflareForDev();

const nextConfig = {
  // Your existing config stays the same!
  images: {
    domains: ['example.com'],
  },
  // Remove any Vercel-specific settings
  // outputFileTracing: false, ❌
};

module.exports = nextConfig;

Step 4: Handle Platform-Specific Code

If you're using Vercel KV:

// ❌ Before (Vercel KV)
import { kv } from '@vercel/kv';
await kv.set('key', 'value');

// ✅ After (Cloudflare KV)
export async function GET(request: Request, env: Env) {
  await env.MY_KV.put('key', 'value');
  return new Response('Saved!');
}

If you're using Vercel Postgres:

// ❌ Before (Vercel Postgres)
import { sql } from '@vercel/postgres';
const { rows } = await sql`SELECT * FROM users`;

// ✅ After (Any PostgreSQL client works!)
import { Pool } from 'pg';
const pool = new Pool({
  connectionString: process.env.DATABASE_URL,
});
const { rows } = await pool.query('SELECT * FROM users');

If you're using Vercel Edge Config:

// ❌ Before (Vercel Edge Config)
import { get } from '@vercel/edge-config';
const value = await get('featureFlag');

// ✅ After (Cloudflare KV or D1)
export async function GET(request: Request, env: Env) {
  const value = await env.CONFIG_KV.get('featureFlag');
  return Response.json({ value });
}

Step 5: Update Environment Variables

Create a .dev.vars file for local development:

# .dev.vars (like .env.local but for Cloudflare)
DATABASE_URL=postgresql://user:pass@localhost:5432/mydb
NEXTAUTH_SECRET=your-secret-key
NEXT_PUBLIC_API_URL=https://api.example.com

Add bindings to wrangler.toml:
Before that
Optional: create the R2 bucket for the cache If you added R2 incremental cache in your open-next.config.ts.

Run this command to create it:

npx wrangler r2 bucket create your-bucket-name

name = "my-nextjs-app"
main = ".open-next/worker.js"
compatibility_date = "2025-07-27"
compatibility_flags = ["nodejs_compat"]


# Static assets configuration
[assets]
directory = ".open-next/assets"
binding = "ASSETS"


# R2 Buckets
[[r2_buckets]]
binding = "NEXT_INC_CACHE_R2_BUCKET"
bucket_name = "my-bucket-name"

Now let deploy :

//Run the command below
npm run deploy

If everything works well, your Next.js app will be live on Cloudflare's global network within seconds, accessible via a *.workers.dev URL or your custom domain, serving from 280+ locations worldwide with automatic SSL.

Common Migration Issues

Issue 1: Dynamic imports failing

// ❌ This might fail
const MyComponent = dynamic(() => import('./MyComponent'), {
  ssr: false
});

// ✅ Ensure proper configuration
const MyComponent = dynamic(
  () => import('./MyComponent'),
  { 
    ssr: false,
    loading: () => <div>Loading...</div>
  }
);

Issue 2: File system access

// ❌ This won't work in production
import fs from 'fs';
const data = fs.readFileSync('./data.json');

// ✅ Use static imports or fetch
import data from './data.json';
// OR
const response = await fetch('/data.json');
const data = await response.json();

Issue 3: Image optimization

// ✅ Next/Image works but needs configuration
// In next.config.js
module.exports = {
  images: {
    loader: 'custom',
    loaderFile: './image-loader.js',
  },
};

// image-loader.js
export default function cloudflareLoader({ src, width, quality }) {
  const params = [`width=${width}`];
  if (quality) {
    params.push(`quality=${quality}`);
  }
  const paramsString = params.join(',');
  return `/cdn-cgi/image/${paramsString}/${src}`;
}

Step 7: Test Your Migration

# 1. Build and preview locally
npm run preview

# 2. Check for errors
# Common errors and fixes:

# "Cannot find module 'fs'"
# → Remove file system operations

# "window is not defined"
# → Wrap in useEffect or check typeof window

# "Module not found: Can't resolve 'encoding'"
# → Add to externals in next.config.js

# 3. Test all your routes
curl http://localhost:8787/api/health
curl http://localhost:8787/dashboard

The Free Tier: More Powerful Than You Think

Cloudflare's free tier includes:

• 100,000 requests per day (resets at midnight UTC)

• 10ms CPU time per request (plenty for SSR)

• 128MB memory per Worker

• Unlimited static asset requests 🎉

Real-World Capacity Example

// Let's calculate what 100k requests means:

// Average SSR page: ~50ms total time (including I/O)
// CPU time used: ~5-10ms
// Memory used: ~30-50MB

// Daily capacity on free tier:
// - 100,000 page views
// - ~4,166 page views per hour
// - ~69 page views per minute

// That's enough for:
// - A blog with 50k daily visitors (2 pages each)
// - A SaaS dashboard with 10k daily active users
// - An e-commerce site with 20k daily shoppers

Advanced Features That Actually Work

1. API Routes with Full Node.js

// app/api/process/route.ts
import { createHash } from 'crypto';
import { headers } from 'next/headers';

export async function POST(request: Request) {
  const body = await request.json();

  // ✅ Node.js crypto works!
  const hash = createHash('sha256')
    .update(body.data)
    .digest('hex');

  // ✅ Headers manipulation
  const headersList = headers();
  const userAgent = headersList.get('user-agent');

  // ✅ Complex processing
  const result = await processDataWithNodeAPIs(body);

  return Response.json({ 
    hash, 
    processed: result,
    userAgent 
  });
}

2. Middleware That Scales

// middleware.ts
import { NextResponse } from 'next/server';
import type { NextRequest } from 'next/server';

export function middleware(request: NextRequest) {
  // Runs at the edge for EVERY request
  const country = request.geo?.country || 'US';

  // Add custom headers
  const response = NextResponse.next();
  response.headers.set('x-user-country', country);

  // Redirect based on geo
  if (country === 'CN' && request.nextUrl.pathname === '/') {
    return NextResponse.redirect(new URL('/cn', request.url));
  }

  return response;
}

export const config = {
  matcher: '/((?!api|_next/static|_next/image|favicon.ico).*)',
};

3. ISR That Actually Works

// app/blog/[slug]/page.tsx
export async function generateStaticParams() {
  // Pre-build these pages
  return [
    { slug: 'getting-started' },
    { slug: 'advanced-features' }
  ];
}

export default async function BlogPost({ params }: { params: { slug: string } }) {
  const post = await fetch(`https://api.blog.com/posts/${params.slug}`, {
    next: { revalidate: 3600 } // Revalidate every hour
  });

  return <article>{/* Your content */}</article>;
}

Production Deployment Checklist

Before deploying to production, ensure:

✅ nodejs_compat flag is set
✅ Environment variables are configured in Cloudflare dashboard
✅ R2 bucket is created for caching (optional)
✅ Custom domain is configured
✅ Preview deployments are working

Deploy Command

# Deploy to production
npm run deploy -- --env production

# Deploy preview
npm run deploy -- --env preview

When to Use Which Approach?

Use @cloudflare/next-on-pages when:

• Your app only uses Web APIs

• You need the absolute fastest cold starts

• You're building a simple marketing site

Use @opennextjs/cloudflare when:

• You need Node.js APIs (crypto, fs, etc.)

• You're using Prisma or other Node.js ORMs

• You have existing Next.js apps to migrate

• You want the full Next.js feature set

The Future is Here

With OpenNext adapter reaching v1.0, deploying production Next.js apps on Cloudflare is finally practical. You get:

• True SSR with full Node.js support

• Global edge deployment from 280+ locations

• Generous free tier for getting started

• Seamless scaling when you grow

Remember: You're not choosing between features and performance anymore. You can have both.

Was this article helpful for you? If so, let me know what you think in the comment section.

• What L1, L2, and L3 constructs actually are and when to use each

• Why AWS created three different abstraction levels (and the hidden benefits)

• How to avoid the most common CDK construct mistakes

• When to break the rules and mix construct levels

Have You Ever Been Confused by CDK Construct Levels?

If you've ever started learning AWS CDK, you've probably encountered code like this and wondered why there are so many ways to create the same S3 bucket:

// Wait, what? Three different ways to create a bucket? 
import { CfnBucket } from 'aws-cdk-lib/aws-s3'; 
import { Bucket } from 'aws-cdk-lib/aws-s3'; 
import { StaticWebsite } from '@aws-solutions-constructs/aws-s3-cloudfront';

// Which one should I use? 🤔

And then you see this error that makes you question everything:

Error: Cannot use property type 'BucketProps' with L1 construct 'CfnBucket'

"But they're both S3 buckets! Why can't I use the same properties?"

Let me help you understand these construct levels once and for all.

What Are CDK Constructs Anyway?

Think of CDK constructs as LEGO blocks for your cloud infrastructure. Just like LEGO has basic bricks, specialized pieces, and complete sets, CDK has three levels of constructs.

Level 1 (L1) Constructs: The Raw CloudFormation Experience

L1 constructs are the most basic building blocks. They start with Cfn (short for CloudFormation) and map directly to CloudFormation resources. No magic, no shortcuts.

import { CfnBucket } from 'aws-cdk-lib/aws-s3';

const bucket = new CfnBucket(this, 'MyL1Bucket', {
  bucketName: 'my-raw-bucket-2025',
  versioningConfiguration: {
    status: 'Enabled'
  },
  publicAccessBlockConfiguration: {
    blockPublicAcls: true,
    blockPublicPolicy: true,
    ignorePublicAcls: true,
    restrictPublicBuckets: true
  }
});

Notice how verbose this is? You have to configure EVERYTHING manually. It's like writing CloudFormation in TypeScript.

When Would You Ever Use L1 Constructs?

1. Brand New AWS Services - When AWS releases a new service, L1 support comes first

2. Debugging L2/L3 Issues - Sometimes you need to see what's really happening

3. Migrating from CloudFormation - Direct 1:1 mapping makes migration easier

4. Edge Cases - When you need a specific CloudFormation property not exposed in L2

Level 2 (L2) Constructs: The Sweet Spot

L2 constructs are what most developers use daily. They provide sensible defaults, helper methods, and hide complexity while still giving you control.

import { Bucket, BucketEncryption } from 'aws-cdk-lib/aws-s3';

const bucket = new Bucket(this, 'MyL2Bucket', {
  bucketName: 'my-friendly-bucket-2025',
  versioned: true,
  encryption: BucketEncryption.S3_MANAGED,
  removalPolicy: RemovalPolicy.DESTROY // Much cleaner!
});

// Look at these helper methods! 
bucket.grantRead(myLambdaFunction);
bucket.addLifecycleRule({
  expiration: Duration.days(90)
});

See the difference? L2 constructs:

• Use friendly property names (versioned vs versioningConfiguration)

• Provide helper methods (grantRead())

• Set security best practices by default

• Handle resource dependencies automatically

Level 3 (L3) Constructs: Complete Solutions

L3 constructs (also called patterns) are pre-built architectures for common use cases. They combine multiple resources into a working solution.

import { StaticWebsite } from '@aws-solutions-constructs/aws-s3-cloudfront';

const website = new StaticWebsite(this, 'MyWebsite', {
  websiteIndexDocument: 'index.html',
  websiteErrorDocument: 'error.html'
});

// That's it! You just created:
// - S3 bucket with proper website configuration
// - CloudFront distribution
// - Origin Access Identity
// - Proper IAM policies
// - HTTPS redirect
// - Security headers

With just a few lines, you get a production-ready static website setup that would take hundreds of lines in L1.

Common Mistakes That Will Drive You Crazy

Mistake #1: Mixing Property Types

// 🚫 This won't work!
const bucket = new CfnBucket(this, 'MyBucket', {
  encryption: BucketEncryption.S3_MANAGED // L2 property type
});

// ✅ Use the correct L1 property type
const bucket = new CfnBucket(this, 'MyBucket', {
  bucketEncryption: {
    serverSideEncryptionConfiguration: [{
      serverSideEncryptionByDefault: {
        sseAlgorithm: 'AES256'
      }
    }]
  }
});

Mistake #2: Assuming L3 Constructs Are Always Better

// Using L3 when you need specific customization
const website = new StaticWebsite(this, 'MyWebsite', {
  // Oh no! I can't set specific CloudFront behaviors
  // or custom cache policies here! 😱
});

// Sometimes L2 gives you more control
const bucket = new Bucket(this, 'WebBucket');
const distribution = new CloudFrontWebDistribution(this, 'MyDist', {
  // Full control over every setting
});

Mistake #3: Not Using Escape Hatches

What if you need to modify an L2 construct's underlying L1 resource?

const bucket = new Bucket(this, 'MyBucket');

// Access the L1 construct (escape hatch)
const cfnBucket = bucket.node.defaultChild as CfnBucket;

// Now you can set ANY CloudFormation property
cfnBucket.analyticsConfigurations = [{
  id: 'my-analytics',
  storageClassAnalysis: {
    dataExport: {
      destination: {
        bucketArn: 'arn:aws:s3:::my-analytics-bucket'
      }
    }
  }
}];

The Hidden Benefits of Each Level

L1 Benefits You Didn't Know About

1. Immediate AWS Feature Support - No waiting for CDK updates

2. CloudFormation Parity - Easy to convert existing templates

3. Learning Tool - Understand what L2 constructs do under the hood

L2 Benefits That Save Time

1. Automatic Security Defaults - Encryption enabled by default

2. Cross-Service Integration - grant* methods handle IAM for you

3. Type Safety - Catch errors at compile time, not deployment

L3 Benefits for Real Projects

1. Proven Architectures - AWS Solutions Constructs follow best practices

2. Compliance Ready - Many patterns are pre-validated for security

3. Rapid Prototyping - Get a working system in minutes

Creating Your Own L3 Construct

Here's a practical example of creating your own pattern:

import { Construct } from 'constructs';
import { Bucket, BucketEncryption } from 'aws-cdk-lib/aws-s3';
import { Function, Runtime, Code } from 'aws-cdk-lib/aws-lambda';
import * as path from 'path';

export class SecureDataProcessor extends Construct {
  public readonly bucket: Bucket;
  public readonly processor: Function;

  constructor(scope: Construct, id: string) {
    super(scope, id);

    // Create encrypted bucket
    this.bucket = new Bucket(this, 'DataBucket', {
      encryption: BucketEncryption.KMS_MANAGED,
      versioned: true,
      enforceSSL: true
    });

    // Create processing Lambda
    this.processor = new Function(this, 'Processor', {
      runtime: Runtime.NODEJS_18_X,
      handler: 'index.handler',
      code: Code.fromAsset(path.join(__dirname, 'lambda'))
    });

    // Wire them together
    this.bucket.grantRead(this.processor);
    this.bucket.addEventNotification(
      EventType.OBJECT_CREATED,
      new LambdaDestination(this.processor)
    );
  }
}

// Now anyone can use your pattern!
const dataProcessor = new SecureDataProcessor(this, 'MyProcessor');

When to Use Each Construct Level

Use L1 when:

• You need bleeding-edge AWS features

• Migrating from CloudFormation

• Debugging CDK issues

• You need a specific CloudFormation property

Use L2 when:

• Building most production applications

• You want security best practices by default

• You need to integrate multiple services

• You value developer productivity

Use L3 when:

• Implementing common patterns

• Rapid prototyping

• Enforcing organizational standards

• You don't need heavy customization

The Future of CDK Constructs

AWS is continuously improving CDK constructs. New services get L1 support immediately through CloudFormation, L2 constructs follow within weeks or months, and the community creates L3 patterns for common use cases.

Remember: There's no "wrong" construct level. Each serves a purpose, and experienced CDK developers often mix levels within the same application.

Was this article helpful for you? If so, kindly subscribe to my bimonthly newsletter.

In this article, we’ll explore RLS in PostgreSQL through a practical example involving pg_cron, a built-in job scheduling extension for PostgreSQL. We’ll walk through the benefits of this fine-grained security model, demonstrate how pg_cron leverages it, and highlight best practices to keep your database environment both secure and efficient.

What is Row-Level Security (RLS)?

At its core, Row-Level Security is a PostgreSQL feature that allows you to enforce access policies at the most granular level: the row. Instead of trusting your application code to handle security logic, RLS shifts that responsibility to the database engine itself. Each time a user queries a table, RLS policies determine which rows they can view or modify—automatically and behind the scenes.

Why RLS Matters

1. Multi-Tenant Isolation: Ideal for SaaS applications where multiple tenants share the same database.

2. Reduced Risk: Minimizes data leaks caused by application bugs or misconfigurations.

3. Cleaner Code: Moves security logic from the application layer to the database layer, making your code less cluttered.

4. Less Overhead: Users only see the rows they have access to, with no extra logic needed in queries or controllers.

A Real-World Example: pg_cron and RLS

To illustrate RLS, let’s look at pg_cron, PostgreSQL’s job scheduling extension. With pg_cron, you can schedule periodic tasks (like database backups or maintenance jobs) by storing job definitions inside dedicated tables.

pg_cron’s Key Tables

• cron.job: Stores scheduled job definitions (think of it like a cron schedule entry).

• cron.job_run_details: Stores execution history for those jobs.

By default, pg_cron uses RLS to ensure that each user can only manage or view the jobs they’ve created.

Default RLS Policies in pg_cron

Here’s a peek at how pg_cron’s built-in RLS policies look:

CREATE POLICY cron_job_policy ON cron.job 
    USING (username = CURRENT_USER);

CREATE POLICY cron_job_run_details_policy ON cron.job_run_details 
    USING (username = CURRENT_USER);

These policies effectively say: “Only show rows where username matches the currently logged-in user.” It’s a simple, yet powerful way to ensure user separation in a multi-tenant or multi-user environment.

Practical Implementation

Let’s walk through some hands-on steps to see how RLS and pg_cron work together.

1. Enabling Row-Level Security

By default, PostgreSQL requires you to enable RLS on a table before policies take effect. In pg_cron, this is often done for you, but if you ever need to do it manually:

ALTER TABLE cron.job ENABLE ROW LEVEL SECURITY;
ALTER TABLE cron.job_run_details ENABLE ROW LEVEL SECURITY;

2. Creating a Scheduled Job

When a user creates a job—say, a daily VACUUM ANALYZE—it automatically gets tagged with their identity. For example:

SELECT cron.schedule('daily-backup', '0 0 * * *', 'VACUUM ANALYZE');

The RLS policy ensures the job’s row is “owned” by the user who created it. When another user queries the cron.job table, they won’t see this entry.

3. Viewing Scheduled Jobs

Because of the RLS policy, each user sees only their own rows:

-- Logged in as User1:
SELECT * FROM cron.job;
-- Result: Only User1’s jobs

-- Logged in as User2:
SELECT * FROM cron.job;
-- Result: Only User2’s jobs

4. Administrator Access

What if you’re an admin and need to see every user’s job? You can create a policy that grants full visibility to superusers or a specific admin role:

CREATE POLICY admin_cron_job_policy ON cron.job 
    USING (
      username = CURRENT_USER 
      OR CURRENT_USER IN (SELECT rolname FROM pg_roles WHERE rolsuper)
    );

With this in place, admins can bypass the default policy and see all rows in cron.job.

Common Scenarios and Solutions

Scenario 1: Read-Only Access to All Jobs

You might have a monitoring role that needs to view all scheduled jobs but not modify them. Here’s how to give them read-only access:

-- Create a read-only policy for the monitor role:
CREATE POLICY monitor_cron_job_policy ON cron.job
    FOR SELECT
    TO monitor_role
    USING (true);

-- Grant SELECT permissions on the cron.job table:
GRANT SELECT ON cron.job TO monitor_role;

Scenario 2: Team-Based Access

In some organizations, teams need to share access to each other’s jobs while still isolating from other groups. You can implement a team-based policy, assuming a separate table (e.g., user_teams) stores team associations:

CREATE POLICY team_cron_job_policy ON cron.job
    USING (
      team_id = (
        SELECT team_id 
        FROM user_teams 
        WHERE username = CURRENT_USER
      )
    );

Best Practices

1. Test Your Policies Thoroughly: Run queries under different user roles to confirm that policies behave as intended.

2. Document Everything: Clear documentation on who can see and do what saves you headaches later.

3. Conduct Regular Audits: Periodically review logs and access patterns to ensure policies are still aligned with your security needs.

4. Include Policies in Backups: Policies are part of your schema. Make sure they’re included in any disaster recovery strategy.

Common Pitfalls

1. Performance Considerations: Very complex or large sets of RLS policies can affect query performance. Keep an eye on your query plans.

2. Overlapping Policies: Multiple policies can interact in unexpected ways. Always test for unintended overlaps.

3. Maintenance Overhead: More policies mean more complexity. Review them regularly to ensure they’re still necessary.

Monitoring and Troubleshooting

Viewing Active Policies

If you ever need a bird’s-eye view of all active RLS policies:

SELECT schemaname, tablename, policyname, roles, cmd, qual 
FROM pg_policies 
WHERE schemaname = 'cron';

Debugging Access Issues

• Check Permissions:

    SELECT has_table_privilege('username', 'cron.job', 'SELECT');
  

• Examine Query Plans:

    EXPLAIN (ANALYZE) SELECT * FROM cron.job;
  

  This helps you see if policies are being applied and how they affect performance.

Conclusion

Row-Level Security is a game-changer for multi-tenant or multi-user PostgreSQL databases. By integrating RLS with pg_cron, you get a firsthand look at how PostgreSQL enforces strict data isolation at the row level—automatically filtering out data that a user shouldn’t see.

Whether you’re managing a small startup or a large enterprise environment, RLS helps you sleep easier by ensuring each user’s data remains exactly where it should: out of sight for everyone else. Pair this with careful monitoring, thorough testing, and clear documentation, and you’ve got a powerful, secure setup that keeps your database environment running smoothly.

Feel free to explore these resources for more in-depth information. As always, happy coding and scheduling!

1. Log In to Your AWS Account

Head to the AWS Console and sign in as usual.

2. Go to Billing and Cost Management

You’ll find this under the main menu.

3. Preferences and Settings

Once there, look for an option labeled Payment Preferences.

4. Edit Payment Currency

Click Edit, then pick Nigerian Naira (NGN) from the dropdown.

5. Save Changes

Congrats, future invoices will be issued in Naira!

Why This Matters

• No More FX Fees – You’re not burning extra cash on currency conversions.

• Local Payment Ease – It’s simpler to manage local transactions and budgets.

• Aligns with the Lagos Local Zone – Perfect if you’re leveraging AWS’s local zone in Nigeria.

That’s it! It’s quick, it’s easy, and it saves you from fiddling with exchange rates. Now you can invest the difference in testing, automations, or that next big idea.

Peace to all, and happy building!

In this post, I’ll walk you through installing Nomad, spinning it up in a small environment, and running a workload to see it in action. By the end, you’ll have a solid hands-on feel for how to use Nomad. Let’s dive right in.

Why Nomad?

For me, Nomad offers a couple of killer advantages:

1. Simplicity: Nomad is a single, self-contained binary that can manage containers, VMs, and standalone applications. Configuration is straightforward and uses HCL (HashiCorp Configuration Language), which you might already know from Terraform.

2. Low Overhead: In contrast to something like Kubernetes, which demands multiple components (etcd, kube-scheduler, kube-apiserver, etc.), Nomad keeps the architecture lean, meaning fewer moving parts and less operational complexity.

3. Scale Without the Bloat: Just because it’s simple doesn’t mean it’s small-time. Nomad can run at massive scale. Start small on a single node, then grow into a cluster as your needs evolve.

4. Broad Workload Support: Containerized apps are the norm these days, but if you have legacy apps or specialized workloads, Nomad accommodates them too. This flexibility makes it easier to transition older systems into orchestrated environments without rewriting everything.

Setting Up Nomad Locally

Let’s talk about setting up a Nomad environment on your local machine for a quick test. I’ll assume you’re running on some flavor of Linux or macOS. If you’re on Windows, you can still follow along using WSL2 or a VM.

1. Download Nomad
   Head over to the official Nomad Releases page and download the appropriate binary. Extract it, move it to a directory in your PATH (like /usr/local/bin), and you’re good to go. For instance:

    wget https://releases.hashicorp.com/nomad/<version>/nomad_<version>_linux_amd64.zip
    unzip nomad_<version>_linux_amd64.zip
    sudo mv nomad /usr/local/bin/
    nomad version
   

   You’ll see Nomad’s version printed out if everything is correct.

2. Development Agent
   Nomad has a “dev” mode, which is a single-process setup that runs the server and client in one go—perfect for local testing. Simply run:

    nomad agent -dev
   

   This command starts Nomad in development mode and spawns a web UI on http://localhost:4646. If you navigate there, you’ll see the Nomad dashboard with your single node.

   

Nomad Architecture at a Glance

In a more robust setup, Nomad is typically deployed as a cluster of server nodes and client nodes:

• Server nodes handle scheduling decisions and maintain cluster state.

• Client nodes run workloads assigned to them by the servers.

But for now, dev mode is all we need. Later on, you could spin up a 3-node server cluster with as many clients as you want.

Running Your First Nomad Job

A Nomad “job” describes what you want to run, how many instances, resource constraints, etc. Jobs are written in HCL, so it’ll feel familiar if you’ve ever used Terraform. Let’s do a quick example by running a Docker-based web server.

Basic HCL Job File

Create a file called nginx.nomad:

job "nginx-web" {
  datacenters = ["dc1"]
  type        = "service"

  group "web-group" {
    count = 1

    task "web" {
      driver = "docker"

      config {
        image = "nginx:latest"
        ports = ["http"]
      }

      resources {
        cpu    = 100
        memory = 128
      }
    }

    network {
      port "http" {
        static = 8080
      }
    }
  }
}

Let’s break it down:

• job "nginx-web": Defines our job name and type. We’re calling it a “service” because it’s a long-running service.

• group "web-group": A group can contain multiple tasks that share resources and networking. Here, we only have one task.

• task "web": Tells Nomad to run an Nginx container. We specify the docker driver.

• network: Maps the container’s port to a static host port (8080 in this case) so you can access it on localhost:8080.

Run the Job

Launch it with:

nomad job run nginx.nomad

Nomad will parse the file, create the job, and schedule it on the local dev client. If all goes well, you’ll see output indicating the job has been placed.

Verify It’s Running

Head to your browser at http://localhost:4646 and click on “Jobs.” You should see nginx-web running. Now try http://localhost:8080 in your browser. Nginx’s default “Welcome” page means it’s working!

Scaling the Service

Nomad makes scaling super easy. Just update the count parameter in your job file. For instance, change it to:

count = 2

Then run:

nomad job run nginx.nomad

Nomad will place an additional instance of the container, though in dev mode you’re still on a single node, so you’ll have multiple containers on the same host. In a multi-node cluster, Nomad automatically figures out which clients have room.

Stopping or Updating the Job

If you want to stop the job, you can run:

nomad job stop nginx-web

For updates, just modify the HCL file (like changing the Docker image to a different version), then re-run nomad job run nginx.nomad. Nomad will handle rolling updates gracefully, spinning up new tasks before shutting down old ones (as long as you specify appropriate update stanzas).

Integrating with Other HashiCorp Tools

Because Nomad shares the same style of configuration as Terraform and the same developer DNA as Vault and Consul, it’s easy to create an entire stack:

• Consul for service discovery and dynamic DNS.

• Vault for secrets management.

• Terraform for provisioning the underlying infrastructure.

Nomad can automatically register services with Consul, making them discoverable to other services in your environment. Storing secrets in Vault means you can dynamically inject credentials into your Nomad jobs. It all plays nicely together.

Why I Use Nomad Over Alternatives

I’ve used Kubernetes for years, but Nomad is my go-to when:

1. Speed of Setup: Nomad dev mode is unbelievably quick. One binary, one command, done.

2. Fewer Dependencies: I don’t need etcd or a separate container runtime beyond Docker. Less to break, less to learn.

3. Flexibility: I can run Docker tasks, raw exec tasks, or even handle batch jobs and system workloads in a single cluster.

Don’t get me wrong Kubernetes excels in large, complex ecosystems. But if you want a more lightweight orchestrator or have a hybrid mix of containerized and legacy apps, Nomad’s a breath of fresh air.

Pro Tips (Anticipating What You Might Need Next)

1. High Availability: If you plan to run in production, spin up at least three Nomad server nodes. That ensures if one server goes down, the cluster can still schedule workloads.

2. Autopilot: Nomad’s built-in autopilot features let you automatically manage upgrades, Raft snapshots, and more to keep the cluster healthy.

3. Authentication and ACLs: In a multi-user setup, you can integrate Nomad’s ACL system to restrict who can submit jobs or read cluster data.

4. Plugins: There are driver plugins for everything from Docker to QEMU to AWS ECS tasks. You can run basically anything that can be launched from a command line or third-party tool.

5. Monitoring: Nomad exposes metrics that are easy to integrate with Prometheus, Grafana, or whatever your favorite monitoring stack is.

Final Thoughts

Nomad may look unassuming as a single binary, but don’t let that fool you. It’s a robust orchestrator that simplifies complex workload management. Whether you’re prototyping a new service, gradually migrating from manual server management, or just want to avoid the overhead of a full-fledged Kubernetes stack, Nomad can handle it.

Why not give it a shot in your own environment? If you’ve got that messy monolith or a small container workload, Nomad might be exactly the tool you need to keep everything running smoothly without drowning in complexity.

Sources

• Nomad Official Documentation

• Nomad GitHub Repository

Hope this helps you get started with Nomad. Let me know if you run into any snags or come up with a clever integration I’m always interested in new ways to push this awesome orchestrator.

As a DevOps engineer, I’m always looking for a cost-effective, reliable, and flexible way to prototype new ideas without overcommitting infrastructure resources. Sure, spinning up EC2 instances or provisioning dedicated hardware works, but when you want a low-power, low-cost sandbox, the Raspberry Pi is hard to beat. It’s an affordable, credit-card-sized computing powerhouse that helps me test concepts, automate environments, and even experiment with local AI without racking up unnecessary cloud fees or dealing with heavy metal servers.

In this post, I’ll share some real-world ways I integrate Raspberry Pi devices into my workflow. If you’ve never considered them as part of a professional DevOps toolkit, I hope this gives you a few reasons to start

What is a Raspberry Pi?

If you’ve never touched one, think of the Raspberry Pi as a tiny Linux-based computer board with just enough CPU, RAM, and storage to run a surprising range of workloads. It’s been wildly popular among hobbyists, educators, and professionals alike. Thanks to its Linux roots, you can tap into a massive ecosystem of software, scripting, containers, and automation tools that feel instantly familiar to anyone from a DevOps background.

Why Raspberry Pi Fits My Needs

As a DevOps engineer, I’ve got plenty of choices for running workloads. But the Pi hits a sweet spot when I need something quick, cheap, and on-prem:

• Cost-Effective: For the price of a mid-tier cloud instance running a few weeks, I can own a Pi outright and reuse it a million times over.

• Energy-Efficient: A Pi draws minimal power, so I can keep it running 24/7 without worrying about my electric bill.

• Exceptionally Versatile: It’s a lab in a box—CI/CD runners, IoT hubs, mini Kubernetes clusters, AI inferencing boxes, local proxies, you name it.

How I Use Raspberry Pi in My Workflow

1. Prototyping and Experimental Builds

When experimenting with a new microservice, pipeline, or integration, I often spin it up on a Pi first. This gives me a stable, always-on environment to validate code, run Docker containers, test APIs, and refine configurations. It’s a great way to ensure my code and infrastructure definitions hold up before I commit cloud spend.

2. Home Automation and IoT Management

I like to say that my home is my first “production” environment. Using a Pi as a hub, paired with something like Home Assistant I manage a network of sensors, lights, and other IoT devices. Not only is it fun, but it also lets me practice edge automation. This experience often translates back into my professional work, where edge computing scenarios are becoming more common.

3. Self-Hosted GitHub Actions Runners

If you’ve worked with GitHub Actions, you know that hosted runners can quickly rack up costs or queue times. By using a Pi as a self-hosted runner, I keep certain build and test pipelines local and cost-controlled. Best of all, I have full control over the environment and dependencies, making it easy to debug issues right in my home office.

4. Local AI Experiments

While you won’t train GPT-4 on a Raspberry Pi, it’s still possible to run smaller models like Google’s Gemma2 (2 billion parameters) for inference tasks. This is a great way to experiment with local AI workloads or test model-serving pipelines without relying on GPU-backed cloud instances. It’s not going to replace a beefy workstation, but it’s enough to poke around with models and APIs before deciding to scale up.

5. Network-Attached Storage (NAS) and Local File Serving

If I need a quick-and-dirty NAS solution, I can set up a Pi with Samba or OpenMediaVault, attach some external storage, and voilà: a lightweight NAS on my local network. It’s not enterprise-level, but it’s perfect for stashing logs, artifacts, or just sharing files among devices at home.

Why It Works for Me

Raspberry Pi devices are more than just cheap boards; they represent a frictionless approach to experimentation. Instead of spending hours setting up cloud VMs or maintaining bulky servers, I have a small fleet of Pis that act as a test bed for ideas. They let me:

• Quickly spin up and tear down environments on a budget.

• Learn and iterate with minimal risk.

• Scale horizontally by adding more boards when I need them.

• Develop intuition for edge, IoT, and ARM-based workloads.

Final Thoughts

The Raspberry Pi offers a unique blend of accessibility, affordability, and versatility. Whether I’m refining a new CI pipeline, tinkering with home automation, or trialing lightweight AI inference, the Pi is my go-to platform for hands-on exploration. It’s a genuine force multiplier that’s expanded the way I think about infrastructure and small-scale deployments.

What about you? How have you put Raspberry Pi to work? If you’ve got a unique use case or a clever hack, let me know. I’m always looking for fresh ways to push these tiny boards to their limits.

In the ever-evolving world of software development, testing methodologies play a crucial role in ensuring the quality and reliability of applications. Two prominent approaches that have gained significant traction in recent years are Test-Driven Development (TDD) and Behavior-Driven Development (BDD). While both methodologies share some common ground, they each bring unique perspectives to the testing process. This article delves into the intricacies of TDD and BDD, exploring their benefits, key differences, and how they can be effectively implemented in software projects.

Table of Contents

1. Understanding Test-Driven Development (TDD)

   • The TDD Process

   • Benefits of TDD

2. Exploring Behavior-Driven Development (BDD)

   • Key Features of BDD

   • BDD Scenario Examples

3. Comparing TDD and BDD

   • Shared Benefits

   • Focus and Approach

4. Implementing TDD and BDD in Your Projects

5. Conclusion

Understanding Test-Driven Development (TDD)

Test-Driven Development is a software development process that relies on the repetition of short development cycles. This methodology encourages simple designs and instills confidence in the code by ensuring that every piece of functionality is thoroughly tested.

The TDD Process

The TDD process follows a specific cycle:

1. Write a test for a new feature before implementing the code.

2. Run the new test to verify that it fails (as expected).

3. Write the minimum amount of code necessary to make the test pass.

4. Run all tests to ensure the new code passes without breaking existing functionality.

5. Refactor the code to improve its structure and remove any duplication.

6. Repeat the cycle for each new feature or functionality.

Benefits of TDD

• Encourages simple, modular designs

• Provides immediate feedback on code correctness

• Builds a comprehensive suite of unit tests

• Improves code quality and reduces bugs

• Facilitates easier refactoring and maintenance

Exploring Behavior-Driven Development (BDD)

Behavior-Driven Development is an agile software development process that extends the principles of TDD. BDD emphasizes collaboration among developers, quality assurance professionals, and business partners to create a shared understanding of how an application should behave.

Key Features of BDD

• Utilizes domain-specific scripting languages (DSLs)

• Defines user behavior in simple English

• Converts English descriptions into automated test scripts

• Focuses on the behavior of the application from an end-user perspective

BDD Scenario Examples

BDD often uses scenario-based descriptions to define expected behavior. For example:

Scenario: User adds an item to their shopping cart
  Given the user is on the product details page
  When the user selects a size "Medium"
  And the user clicks the "Add to Cart" button
  Then the item should be added to the user's shopping cart
  And the cart total should increase by 1
  And the user should see a confirmation message "Item added to cart"

Comparing TDD and BDD

While TDD and BDD share some common ground, they have distinct focuses and approaches:

Shared Benefits

Both TDD and BDD offer several advantages to development teams:

• Early detection of errors in requirements

• Improved communication between team members

• Reduced overall development costs

• Higher code quality and fewer bugs

Focus and Approach

• TDD focuses on the functionality of individual components

• BDD emphasizes the behavior of the application from a user's perspective

• TDD tests are typically written in the same programming language as the application

• BDD tests are often written in a more accessible, natural language format

Implementing TDD and BDD in Your Projects

To successfully implement TDD or BDD in your software projects:

1. Choose the appropriate methodology based on your project's needs and team structure

2. Invest in training and tools to support the chosen approach

3. Start small and gradually expand the use of TDD or BDD across your projects

4. Regularly review and refine your testing processes

5. Foster a culture of collaboration and continuous improvement

Conclusion

Both Test-Driven Development and Behavior-Driven Development offer valuable approaches to software testing and development. By understanding the strengths and differences of each methodology, development teams can make informed decisions about which approach best suits their projects. Whether you choose TDD, BDD, or a combination of both, implementing these methodologies can lead to higher quality software, improved team collaboration, and more satisfied end-users.

References:

1. Beck, K. (2002). Test-Driven Development: By Example. Addison-Wesley Professional.t

Understanding Service Discovery

At its core, service discovery facilitates the dynamic detection and interaction among services in a distributed ecosystem. The challenge lies in the fluid nature of these services, which may traverse across different environments, necessitating a flexible approach to maintain connectivity. Service discovery transcends the limitations of static configurations, allowing services to communicate based on logical identifiers rather than hard-coded network addresses.

The Role of Service Discovery in ECS

Amazon ECS simplifies service discovery by leveraging AWS Cloud Map, a fully managed service registry that automates the discovery of ECS services. Cloud Map enables your applications to discover resources by name, eliminating the need for manual IP management or service configuration. This abstraction not only enhances flexibility but also significantly reduces the overhead associated with deploying and managing microservices.

Implementing Service Discovery in ECS

To utilize service discovery in ECS, you begin by registering your services with AWS Cloud Map. This process involves creating a namespace, which serves as a container for all service instances. Within this namespace, you register service names that correspond to your ECS services. Each service can then be discovered through its logical name, streamlining the interaction between different components of your application.

Practical Example: Integrating Service Discovery with ECS

Consider a scenario where you have a microservices architecture with a front-end service needing to communicate with a back-end service. Instead of hardcoding the back-end service's IP address, you register both services with Cloud Map under a common namespace, say myapp.local. The back-end service registers itself with the name backend.myapp.local. The front-end service, needing to send a request to the back-end, queries Cloud Map for backend.myapp.local and receives the current IP address and port of the back-end service. This mechanism ensures that even if the back-end service is redeployed or its IP changes, the front-end can always discover and communicate with it without any manual intervention.

Best Practices for Service Discovery with ECS

• Automate Service Registration: Ensure your ECS services are automatically registered with Cloud Map upon deployment. This can be achieved through ECS task definitions or service configurations.

• Health Checks: Utilize Cloud Map's health checking capabilities to automatically remove unhealthy service instances from the registry. This ensures your applications always connect to operational instances.

• Security: Implement appropriate IAM policies to control access to the service discovery system, ensuring only authorized services can register or discover other services.

Conclusion

Service discovery is a cornerstone of modern distributed systems, ensuring applications remain resilient and adaptable to changing environments. By integrating AWS ECS with Cloud Map, developers can significantly simplify the discovery process, allowing services to dynamically interact regardless of their underlying infrastructure. This guide provides a foundation for leveraging service discovery within your ECS deployments, paving the way for more efficient and scalable applications.

Incorporating service discovery into your ECS strategy not only optimizes communication between services but also enhances overall application resilience. By following the practices outlined above, you can create a robust ecosystem where services seamlessly discover and interact with each other, driving efficiency and scalability across your deployments.

Thank you for reading this article! If you enjoyed it and would like to stay up to date on the latest technical articles and insights, I invite you to subscribe to my newsletter. By subscribing, you'll be the first to know when we publish new articles and you'll have access to exclusive content and resources.

Prerequisites

Before proceeding, you should have:

• Basic knowledge of JavaScript and Node.js

• Understanding of Express.js framework

• Familiarity with Docker and its basic commands

What is a Graceful Shutdown?

A graceful shutdown involves carefully handling the shutdown signal, completing the in-progress tasks, closing the active connections, and then finally allowing the application to terminate. This ensures that the system resources are properly freed and that the application does not exit while it's in the middle of important tasks.

Why is a Graceful Shutdown Important?

Handling shutdown signals in your applications allows you to manage resources properly, provide a better user experience, and help your system degrade more gracefully. Not handling these signals can lead to issues like data loss or corruption, incomplete transactions, resource leaks, and unexpected behavior.

Implementing Graceful Shutdown in Node.js

In this section, we'll walk through the code required to listen for shutdown signals in a Node.js application and how to perform cleanup tasks before allowing the application to exit.

Listening for Shutdown Signals

In Node.js, we can listen to process-level signals, such as SIGINT and SIGTERM. These signals are emitted when the process is requested to shut down, whether by manual user interruption (SIGINT from Ctrl+C) or system-level termination (SIGTERM from Docker or another process manager). To listen for these signals, we can use the process.on method and provide a callback function that will be executed when these signals are received.

process.on('SIGTERM', () => {
  console.log('SIGTERM signal received.');
});

process.on('SIGINT', () => {
  console.log('SIGINT signal received.');
});

Performing Cleanup

Once a shutdown signal is received, it's important to perform necessary cleanup tasks to close any open resources, finish transactions, and prepare the application for a graceful exit. This may involve closing database connections, completing any in-progress writes to file systems, or other application-specific cleanup. This cleanup code should be placed inside the callback function provided to process.on.

process.on('SIGTERM', () => {
  console.log('SIGTERM signal received.');
  // Perform cleanup tasks here
});

process.on('SIGINT', () => {
  console.log('SIGINT signal received.');
  // Perform cleanup tasks here
});

Remember to handle asynchronous cleanup tasks correctly. If a cleanup task is asynchronous (like closing a database connection), you'll need to handle it with async/await or Promises to ensure it completes before the process exits.

Exiting the Process

After performing the necessary cleanup tasks, we must manually terminate the Node.js process by calling process.exit(). This signals to the system (or Docker) that our application has finished shutting down. We can provide an exit code to this method; a code of 0 indicates a successful exit, while any other number indicates an error occurred. Typically, if we've handled everything correctly in our cleanup, we'll want to exit with a code of 0.

process.on('SIGTERM', () => {
  console.log('SIGTERM signal received.');
  // Perform cleanup tasks here

  process.exit(0);
});

process.on('SIGINT', () => {
  console.log('SIGINT signal received.');
  // Perform cleanup tasks here

  process.exit(0);
});

Remember, the goal of all this is to allow your application to exit gracefully when it receives a shutdown signal. This helps reduce the risk of data corruption, loss of data, and other issues associated with an abrupt termination.

Handling Shutdown in Express.js Application

Express.js applications, in particular, have some unique considerations when it comes to graceful shutdowns. This section discusses how to handle shutdown signals in an Express.js application, including how to stop the server from accepting new connections and how to ensure all existing connections are closed before shutdown.

Creating and Starting the Server

First, we need to create an Express.js application and start a server. Once the server is created, we can use it to close existing connections when we're ready to shut down the application.

import express from 'express';

const app = express();
const server = app.listen(3000, () => {
  console.log('Server listening on port 3000');
});

Listening for Shutdown Signals

Just like in a basic Node.js application, we need to listen for SIGINT and SIGTERM signals. We can use the process.on method to add listeners for these signals. Inside the callback function for each listener, we'll call server.close() to stop the server from accepting new connections and to begin the process of shutting down.

process.on('SIGTERM', () => {
  console.log('SIGTERM signal received.');
  server.close(() => {
    console.log('Closed out remaining connections');
    // Additional cleanup tasks go here
  });
});

process.on('SIGINT', () => {
  console.log('SIGINT signal received.');
  server.close(() => {
    console.log('Closed out remaining connections');
    // Additional cleanup tasks go here
  });
});

Closing Existing Connections

When server.close() is called, the server stops accepting new connections and waits for all existing connections to close. The function that we pass to server.close() will be called once all connections are closed. This is where we can perform any additional cleanup tasks that need to happen before the application shuts down.

Performing Additional Cleanup

Depending on the needs of your application, you may have additional cleanup tasks that need to happen when your application shuts down. For example, if you have a database connection, you should close it before your application exits. This cleanup code should be placed inside the callback function that you pass to server.close().

process.on('SIGTERM', () => {
  console.log('SIGTERM signal received.');
  server.close(() => {
    console.log('Closed out remaining connections');
    // Additional cleanup tasks go here, e.g., close database connection
    process.exit(0);
  });
});

process.on('SIGINT', () => {
  console.log('SIGINT signal received.');
  server.close(() => {
    console.log('Closed out remaining connections');
    // Additional cleanup tasks go here, e.g., close database connection
    process.exit(0);
  });
});

Handling Shutdown in a Dockerized Node.js Application

When running a Node.js application in a Docker container, there are additional considerations to take into account. This section discusses how Docker sends shutdown signals and how to ensure your Node.js application can handle them correctly.

Understanding Docker Shutdown Signals

When Docker is asked to stop a running container, it sends a SIGTERM signal to the main process running in the container. This is Docker's way of asking the process to shut down gracefully, by finishing what it's currently doing, cleaning up as needed, and then terminating.

However, if the process does not terminate within a certain period (10 seconds by default), Docker will then send a SIGKILL signal to forcibly terminate the process. This is akin to pulling the plug on the application - it won't have a chance to finish what it's doing or clean up.

This is why our Node.js application needs to listen for and handle the SIGTERM signal, as we discussed in previous sections. By handling SIGTERM, our application can ensure it shuts down gracefully when Docker asks it to stop.

Adjusting Docker's Grace Period

Sometimes, our application may need more than 10 seconds to shut down gracefully. For example, it might need to finish processing a long-running request, or it might need to wait for a database transaction to commit.

In such cases, we can tell Docker to wait longer before it sends the SIGKILL signal, by using the --stop-timeout option when we run our Docker container. This option takes several seconds as its argument.

For example, to start a Docker container and give it 30 seconds to shut down gracefully before forcibly killing it, we would use a command like this:

docker run --stop-timeout 30 my-nodejs-app

Keep in mind that while extending the stop timeout can help in some situations, it's not a panacea. If your application consistently takes a long time to shut down, it may be a sign that it needs to be optimized or refactored.

Conclusion

Handling shutdown signals in your Node.js applications allows you to manage resources properly, reduce potential data loss or corruption, provide a better user experience, and more. By understanding how to handle these signals, you can make your applications more robust and reliable, both in development and in production.

Why You Should Not Tolerate Flaky Tests

Flaky tests can cause several problems that affect the quality and reliability of your software. Here are some reasons why you should not tolerate flaky tests:

1. Flaky tests can make it difficult to identify real bugs: Flaky tests can mask real bugs in your code, making it difficult to identify and fix them.

2. Flaky tests can waste valuable time and resources: Flaky tests can consume valuable time and resources that could be better spent on other tasks.

3. Flaky tests can erode confidence in your tests: Flaky tests can erode confidence in your tests and make it difficult to trust the results.

4. Flaky tests can lead to false positives: Flaky tests can lead to false positives, which can cause unnecessary rework and delays.

What You Can Do to Fix or Delete Flaky Tests

Fixing or deleting flaky tests can help you avoid the problems caused by flaky tests. Here are some things you can do to fix or delete flaky tests:

1. Identify the root cause of the flakiness: To fix flaky tests, you need to identify the root cause of the flakiness. This can be done by analyzing the test results and identifying patterns.

2. Fix the root cause of the flakiness: Once you have identified the root cause of the flakiness, you can fix it. This may involve modifying the test code, the test environment, or the application code.

3. Delete the flaky tests: If you are unable to fix the root cause of the flakiness, you may need to delete the flaky tests. This can help you avoid the problems caused by flaky tests.

4. Prioritize fixing flaky tests: Fixing flaky tests should be a priority. You should allocate the necessary time and resources to fix or delete flaky tests.

Conclusion

Flaky tests can cause several problems that affect the quality and reliability of your software. To avoid these problems, you should not tolerate flaky tests and should fix or delete them as soon as possible. By identifying the root cause of the flakiness and fixing it or deleting the flaky tests, you can improve the reliability and quality of your software. Remember, a reliable and trustworthy test suite is crucial for the success of your DevOps pipeline.

We hope this article has been helpful in understanding why you should not tolerate flaky tests and what you can do to fix or delete them. If you have any questions or comments, feel free to leave them below.

Why Use Lambda Invocation?

AWS Lambda is a serverless compute service that allows you to run your code without provisioning or managing servers. Lambda invocation is a powerful feature that enables one Lambda function to call another, allowing you to offload tasks, process events in parallel, and set up complex workflows. In our case, invoking another Lambda function to train ML models enables the primary Lambda function to return a response immediately, ensuring that the main process doesn't get blocked waiting for the training job to finish.

How to Invoke a Lambda Function with Python

To invoke a Lambda function from another, you'll first need the AWS SDK for Python, Boto3. Make sure you have it installed by running:

pip install boto3

Next, you need to set up the necessary permissions to allow the parent Lambda function to call the child Lambda function. In the AWS Management Console, navigate to the IAM role assigned to the parent Lambda function and attach the AWSLambdaRole policy.

Now let's create a simple parent Lambda function that will invoke the child Lambda function. The parent function will use Boto3's invoke method to make an asynchronous call to the child function.

import boto3
import json

def lambda_handler(event, context):
    lambda_client = boto3.client('lambda')

    # Replace 'child-function-name' with the actual name of your child Lambda function
    response = lambda_client.invoke(
        FunctionName='child-function-name',
        InvocationType='Event',  # Asynchronous invocation
        Payload=json.dumps(event)
    )

    return {
        'statusCode': 200,
        'body': 'Model training job started.'
    }

Here's a sample child Lambda function that trains an ML model using the data passed from the parent function:

import json

def lambda_handler(event, context):
    # Load data from the event object sent by the parent Lambda function
    training_data = json.loads(event['body'])

    # Train your ML model here using the training_data
    # ...

    return {
        'statusCode': 200,
        'body': 'Model training completed.'
    }

In this example, the parent Lambda function sends the input event as a payload to the child function, which then extracts the data and uses it for training the ML model.

Conclusion

As demonstrated in this article, using AWS Lambda invocation to asynchronously call another Lambda function is a powerful technique that can solve complex use cases such as initiating ML model training without waiting for the process to complete. By leveraging this approach, you can achieve faster response times, better scalability, and an efficient serverless architecture for your applications. The Python examples provided serve as a solid foundation to build upon for your specific use case, allowing you to focus on the core logic of your application rather than infrastructure management.

One key aspect of using Terraform effectively is understanding the various available data sources. In this article, we'll explore what data sources are, how they work, and how you can use them in your Terraform code to create a more dynamic and flexible infrastructure.

What are Data Sources in Terraform?

Data sources in Terraform allow you to make use of external resources that are not part of your configuration. Information about a resource, such as its ID or IP address, can be retrieved from a data source and then used in your Terraform script.

Data sources are useful in a variety of scenarios. For example, you may want to create a security group in AWS that allows traffic from a specific IP address range. To do this, you need to know the public IP address of the machine that will be accessing the security group. You can use a data source to retrieve the current public IP address and then use that value in your security group configuration.

How do Data Sources Work in Terraform?

Data sources are defined in your Terraform code using a special block type called "data". The "data" block allows you to specify the type of data source you want to use, as well as any necessary parameters or filters.

Here's an example of a data source block that retrieves information about an AWS EC2 instance:

data "aws_instance" "example-server" {
  instance_id = "i-0123456789abcdef0"
}

In this example, the data source block has a type of "aws_instance" and a name of "example-server". The "instance_id" parameter specifies the ID of the EC2 instance that we want to retrieve information about.

Once you've defined a data source, you can reference it in your Terraform code using the syntax "data.<TYPE>.<NAME>.<ATTRIBUTE>". For example, to reference the public IP address of the EC2 instance in our previous example, we could use the following syntax:

resource "aws_security_group_rule" "example" {
  type        = "ingress"
  from_port   = 80
  to_port     = 80
  protocol    = "tcp"
  cidr_blocks = [data.aws_instance.example-server.public_ip]
}

In this example, we're creating an AWS security group rule that allows traffic on port 80 from the public IP address of the EC2 instance we retrieved using the data source.

Conclusion:

Data sources are a powerful feature of Terraform that allow you to retrieve information about existing resources and use that information in your configuration. With data sources, you can create a more dynamic and flexible infrastructure that can adapt to changes in your environment. By understanding how data sources work and how to use them effectively, you can take full advantage of Terraform's capabilities and create an infrastructure that is easier to manage and maintain over time.

In this article, we will focus on one aspect of observability in Kubernetes – Liveness Probes. Liveness probes are used to check if an application is running correctly. They are an important tool for making sure that applications running in containers are reliable and always available.

What are Liveness Probes?

Liveness Probes are a type of Kubernetes mechanism that allows you to monitor the health of a container running in a pod. The purpose of Liveness Probes is to detect when a container is no longer running as expected, and if necessary, restart it. This helps to ensure that the containers in a pod are always running and can respond to requests.

Liveness Probes are defined in a pod specification as a JSON or YAML file, and can be specified as either a HTTP request, a TCP socket connection, or a command executed inside the container. The Kubernetes control plane periodically performs the specified Liveness Probe, and if it fails, the control plane will restart the container.

How do Liveness Probes Work?

Liveness Probes work by sending requests to a container running in a pod to check its health status. The request can be either an HTTP request, a TCP socket connection, or a command executed inside the container. If the container is healthy, it will return a success status code. If the container is not healthy, it will return a failure status code, and the Kubernetes control plane will restart the container.

Examples of Using Liveness Probes

There are several types of liveness probes that can be used, including:

• HTTP requests

• TCP connections

• Command execution

Each type of probe has its own use case, and can be defined in the pod definition file.

HTTP Requests

HTTP requests are the most common type of liveness probe. They can be used to check if an application is responding to requests, and to ensure that the application is accessible.

To define an HTTP liveness probe in a pod definition file, you would add the following code:

livenessProbe:
  httpGet:
    path: /ping
    port: 3000
  initialDelaySeconds: 5
  timeoutSeconds: 1
  periodSeconds: 10
  successThreshold: 1
  failureThreshold: 3

In this example, the liveness probe will send an HTTP GET request to /ping on port 3000. The probe will be initiated after a delay of 5 seconds, and will time out after 1 second. The probe will run every 10 seconds and will be considered successful if it returns a response. If the probe fails 3 times in a row, the container will be restarted.

TCP Connections

TCP connections can be used to check if an application is listening on a specific port. They can be used to check if an application is running and to make sure the application can be reached.

To define a TCP liveness probe in a pod definition file, you would add the following code:

livenessProbe:
  tcpSocket:
    port: 3000
  initialDelaySeconds: 5
  timeoutSeconds: 1
  periodSeconds: 10
  successThreshold: 1
  failureThreshold: 3

In this example, the liveness probe will attempt to establish a TCP connection on port 3000. The probe will be initiated after a delay of 5 seconds, and will time out after 1 second. The probe will run every 10 seconds and will be considered successful if it is able to establish a connection. If the probe fails 3 times in a row, the container will be restarted.

Command Execution

Command execution can be used to check if an application is running properly by executing a command within the container. This is useful for checking the status of an application, and can be used to ensure that the application is running correctly.

livenessProbe:
      exec:
        command:
        - cat
        - /tmp/healthy
      initialDelaySeconds: 5
      periodSeconds: 5

In this example, the liveness probe executes the command cat /tmp/healthy in the target container. If the command succeeds, it returns 0, and the probes consider the container to be alive and healthy. If the command returns a non-zero value, it kills the container and restarts it.

Conclusion

Liveness probes are a key component of observability in Kubernetes, providing a mechanism to monitor the health of containers and ensure they are running as expected. By using Liveness Probes, you can make your Kubernetes cluster applications more reliable and available. Whether you choose to use HTTP Liveness Probes, TCP Socket Connection Liveness Probes, or Command Execution Liveness Probes, you can rest assured that your containers are being monitored and will be restarted if necessary to ensure they are always running and available to respond to requests.

Readiness probes are one of the key components of observability in Kubernetes, providing valuable information about the health and availability of your applications. They help to solve the problem of ensuring that only healthy containers receive traffic, and provide several advantages for your applications and infrastructure.

The Problem: Is my container ready to start accepting traffic?

In a large-scale production environment, it's critical to ensure that containers are healthy and ready to handle incoming requests. Without the proper tools and processes in place, it's difficult to determine the health of individual containers, leading to potential downtime and a negative impact on your users.

The Solution: Readiness Probes

Readiness probes in Kubernetes provide a solution to the problem of determining container health. They allow you to monitor the health of your containers in real time and determine if they're ready to handle incoming requests. Readiness probes are executed at regular intervals to ensure that containers are healthy and available, and if they fail, the Kubernetes scheduler will automatically stop sending traffic to that container.

What you get: Improved Reliability and Performance

By implementing readiness probes in your Kubernetes infrastructure, you can enjoy several advantages, including improved reliability and performance. With the ability to monitor the health of your containers in real-time, you can quickly identify and resolve any issues that may arise, reducing downtime and improving the overall availability of your applications. Additionally, by only sending traffic to healthy containers, you can improve the performance of your applications and provide a better experience for your users.

Setting up Readiness Probes in Kubernetes

To set up readiness probes in Kubernetes, you'll need to define a readiness probe as part of your container spec in the deployment configuration file. There are three types of probes you can use: HTTP, TCP, and Command.

Here's an example of how to set up an HTTP readiness probe in a Kubernetes deployment file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app-container
        image: my-app-image
        ports:
        - containerPort: 3000
        readinessProbe:
          httpGet:
            path: /ping
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 10

In this example, the readiness probe is an HTTP GET request to the /ping endpoint on port 3000. The probe will be executed every 10 seconds, with an initial delay of 5 seconds.

Here's an example of how to set up a TCP readiness probe in a Kubernetes deployment file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app-container
        image: my-app-image
        ports:
        - containerPort: 3000
        readinessProbe:
          tcpSocket:
            port: 3000
          initialDelaySeconds: 5
          periodSeconds: 10

In this example, the readiness probe is a TCP connection to port 3000. The probe will be executed every 10 seconds, with an initial delay of 5 seconds.

Here's an example of how to set up a Command readiness probe in a Kubernetes deployment file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app-container
        image: my-app-image
        ports:
        - containerPort: 3000
        readinessProbe:
          exec:
            command:
            - /bin/sh
            - -c
            - "curl -f http://localhost:3000/ping || exit 1"
          initialDelaySeconds: 5
          periodSeconds: 10

In this example, the readiness probe is a shell script that uses curl to make an HTTP request to the /ping endpoint on port 3000. The probe will be executed every 10 seconds, with an initial delay of 5 seconds.

If the curl command returns a non-zero exit code, it means the probe failed and the container is considered not ready.

Conclusion

Readiness probes are a crucial component of observability in Kubernetes, providing valuable information about the health and availability of your containers. By implementing readiness probes, you can improve the reliability and performance of your applications, ensuring that they are always available to handle incoming requests.

1. Terragrunt: This tool is a thin wrapper for Terraform that provides extra features, such as locking for Terraform state and the ability to keep your Terraform configurations DRY. With Terragrunt, you can organize your Terraform code into reusable modules, and reuse those modules across multiple projects. This helps to keep your Terraform code organized and reduces duplicated effort.

2. Terratest: This is a Go library that makes it easier to write automated tests for your Terraform code. Terratest provides utility functions for interacting with Terraform, as well as libraries for common infrastructure-as-code tools, such as AWS, GCP, and Kubernetes. With Terratest, you can write tests that check that your Terraform code creates the resources it should, that the resources are configured correctly, and that they can be destroyed.

3. Terraform-docs: This tool generates documentation for your Terraform modules in various formats, such as Markdown, HTML, and JSON. Terraform-docs parses your Terraform code and extracts documentation from comments, variable and output descriptions, and input/output examples. The tool then generates a table of contents with links to the relevant documentation for each module, which makes it easy to understand the purpose and usage of each module.

4. TFLint: This is a Terraform linter that checks for errors and best practices in your Terraform code. TFLint helps to catch common mistakes, such as variable name clashes, missing required variables, or invalid resource arguments. It also checks for compliance with best practices, such as naming conventions and resource ordering. By using TFLint, you can catch errors early on, which helps to improve the quality of your Terraform code.

5. Infracost : Infracost is an open-source tool that allows users to see the cost of running their infrastructure, such as AWS resources, in near real-time. It uses the AWS Price List API to determine the costs of resources, and can be integrated into CI/CD pipelines to provide cost feedback during the development process. This allows developers to make informed decisions about their infrastructure and optimize costs. Additionally, Infracost can be used to create alerts based on cost thresholds, so you can be notified when your infrastructure costs exceed a certain amount. This can be especially useful for teams that operate on a tight budget or need to manage costs closely.

In conclusion, the five tools discussed in this article can greatly enhance the development experience when working with Terraform. By incorporating these tools into your development process, you can increase productivity, and improve the reliability and maintainability of your infrastructure code. With these tools, you can supercharge your Terraform development and take your infrastructure as code to the next level.

There are two types of auto vacuum in Postgres: auto vacuum and auto analyze. Auto vacuum is responsible for removing dead tuples and keeping the database clean, while auto analyze gathers statistics about the database to help the planner make better query execution plans.

How Data is Deleted in Postgres data

To understand why auto vacuum is needed, it's important to understand how data is deleted in a Postgres database. When a row is deleted from a table, the space it occupied is not immediately reused. Instead, the row is marked as deleted and left in place. This is known as a "dead tuple." Over time, as more and more rows are deleted and marked as dead tuples, the table can become cluttered with unnecessary data that is taking up space and slowing down performance.

Auto vacuum is designed to identify and remove these dead tuples on a regular basis, helping to keep the database clean and efficient. It is typically run automatically by the database system, but it can also be triggered manually by a database administrator.

Here is an example of how auto vacuum works in a Postgres database:

Imagine you have a database table called "customers" that stores information about your company's customers. One day, you decide to delete a customer from the table because they have moved away. Instead of actually deleting the row from the table, Postgres marks the row as a dead tuple and leaves it in place.

Over time, as more and more rows are deleted and marked as dead tuples, the "customers" table may become cluttered with unnecessary data. This can cause issues such as increased disk space usage and slower query times.

To solve this problem, Postgres runs an auto vacuum process on the "customers" table. The process scans the table and identifies any dead tuples that need to be removed. It then removes these dead tuples, freeing up space and improving the performance of the table.

In conclusion, Postgres auto vacuum is an important maintenance process that helps keep a database running smoothly and efficiently. It helps to prevent bloat by removing unnecessary or outdated data, improving performance, and reducing disk space usage. While it is typically run automatically by the database system, it can also be triggered manually by a database administrator as needed.