Kustomize Configuration¶

Guide for using Kustomize to manage Kubernetes configurations and integrate with SOPS for secret management.

Overview¶

Kustomize provides declarative configuration management for Kubernetes resources, enabling environment-specific customizations without duplicating YAML files.

Kustomize Structure¶

Base and Overlays Pattern¶

apps/rciis/nucleus/
├── base/                    # Base configuration (future)
│   ├── kustomization.yaml
│   ├── deployment.yaml
│   └── service.yaml
├── local/                   # Local environment overlay
│   ├── kustomization.yaml
│   ├── values.yaml
│   └── extra/
├── testing/                 # Testing environment overlay
│   ├── kustomization.yaml
│   ├── values.yaml
│   └── extra/
└── staging/                 # Staging environment overlay
    ├── kustomization.yaml
    ├── values.yaml
    └── extra/

Kustomization Configuration¶

Basic Kustomization¶

# apps/rciis/nucleus/staging/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# Target namespace
namespace: nucleus

# Resources to include
resources:
- ../../../secrets/staging/nucleus/

# Secret generators
generators:
- secret-generator.yaml

# Configurations
configurations:
- extra/default.conf

# Transformers (KSOPS plugin)
transformers:
- ksops-transformer.yaml

# Image transformations
images:
- name: nucleus-image
  newName: harbor.devops.africa/rciis/nucleus
  newTag: v1.2.3

# Replica modifications
replicas:
- name: nucleus-deployment
  count: 2

# Strategic merge patches
patchesStrategicMerge:
- environment-patch.yaml

# JSON6902 patches
patchesJson6902:
- target:
    group: apps
    version: v1
    kind: Deployment
    name: nucleus-deployment
  path: patches/deployment-patch.yaml

KSOPS Integration¶

Secret Generator Configuration¶

# secret-generator.yaml
apiVersion: viaduct.ai/v1
kind: ksops
metadata:
  name: nucleus-secret-generator
  annotations:
    config.kubernetes.io/function: |
      exec:
        path: ksops
files:
- ../../../secrets/staging/nucleus/appsettings.yaml
- ../../../secrets/staging/nucleus/mssql-admin.yaml
- ../../../secrets/staging/nucleus/container-registry.yaml
- ../../../secrets/staging/nucleus/config.yaml

KSOPS Plugin Installation¶

# Install KSOPS plugin
curl -Lo ksops https://github.com/viaduct-ai/kustomize-sops/releases/latest/download/ksops_linux_amd64
chmod +x ksops

# Install to kustomize plugin directory
mkdir -p ~/.config/kustomize/plugin/viaduct.ai/v1/ksops/
mv ksops ~/.config/kustomize/plugin/viaduct.ai/v1/ksops/

# Verify installation
kustomize plugin list

Resource Transformations¶

Image Updates¶

# Update container images
images:
- name: nucleus-api
  newName: harbor.devops.africa/rciis/nucleus
  newTag: v1.2.3
- name: sidecar-proxy
  newName: envoyproxy/envoy
  newTag: v1.24.0

Replica Scaling¶

# Scale deployments
replicas:
- name: nucleus-deployment
  count: 3
- name: worker-deployment
  count: 5

Resource Patches¶

# patchesStrategicMerge
- |
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nucleus-deployment
  spec:
    template:
      spec:
        containers:
        - name: nucleus
          env:
          - name: ENVIRONMENT
            value: staging
          resources:
            limits:
              memory: 1Gi
              cpu: 500m

JSON Patches¶

# patches/deployment-patch.yaml
- op: replace
  path: /spec/template/spec/containers/0/image
  value: harbor.devops.africa/rciis/nucleus:v1.2.3
- op: add
  path: /spec/template/spec/containers/0/env/-
  value:
    name: NEW_ENV_VAR
    value: "staging-value"

Configuration Management¶

ConfigMap Generation¶

# Generate ConfigMaps from files
configMapGenerator:
- name: app-config
  files:
  - config.properties
  - logging.conf
- name: nginx-config
  files:
  - extra/default.conf

Secret Generation¶

# Generate secrets from literals
secretGenerator:
- name: app-secrets
  literals:
  - username=admin
  - password=secretvalue
  type: Opaque

Environment-Specific Configurations¶

Local Development¶

# local/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: nucleus

resources:
- ../../../secrets/local/nucleus/

generators:
- secret-generator.yaml

images:
- name: nucleus-image
  newName: nucleus
  newTag: local

replicas:
- name: nucleus-deployment
  count: 1

patchesStrategicMerge:
- |
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nucleus-deployment
  spec:
    template:
      spec:
        containers:
        - name: nucleus
          env:
          - name: ASPNETCORE_ENVIRONMENT
            value: Development
          resources:
            limits:
              memory: 512Mi
              cpu: 250m
            requests:
              memory: 256Mi
              cpu: 100m

Testing Environment¶

# testing/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: nucleus

resources:
- ../../../secrets/testing/nucleus/

generators:
- secret-generator.yaml

images:
- name: nucleus-image
  newName: harbor.devops.africa/rciis/nucleus
  newTag: testing

replicas:
- name: nucleus-deployment
  count: 1

patchesStrategicMerge:
- |
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nucleus-deployment
  spec:
    template:
      spec:
        containers:
        - name: nucleus
          env:
          - name: ASPNETCORE_ENVIRONMENT
            value: Testing

Production Environment¶

# production/kustomization.yaml (future)
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: nucleus

resources:
- ../../../secrets/production/nucleus/

generators:
- secret-generator.yaml

images:
- name: nucleus-image
  newName: harbor.devops.africa/rciis/nucleus
  newTag: v1.2.3

replicas:
- name: nucleus-deployment
  count: 3

patchesStrategicMerge:
- |
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nucleus-deployment
  spec:
    template:
      spec:
        containers:
        - name: nucleus
          env:
          - name: ASPNETCORE_ENVIRONMENT
            value: Production
          resources:
            limits:
              memory: 2Gi
              cpu: 1000m
            requests:
              memory: 1Gi
              cpu: 500m

Advanced Features¶

Custom Transformations¶

# Custom transformer
apiVersion: builtin
kind: PrefixSuffixTransformer
metadata:
  name: myPrefixSuffixTransformer
prefix: dev-
suffix: -v1
fieldSpecs:
- path: metadata/name
  kind: Deployment
- path: metadata/name
  kind: Service

Variable Substitution¶

# vars substitution
vars:
- name: DEPLOYMENT_NAME
  objref:
    kind: Deployment
    name: nucleus-deployment
    apiVersion: apps/v1
  fieldref:
    fieldpath: metadata.name
- name: SERVICE_NAME
  objref:
    kind: Service
    name: nucleus-service
    apiVersion: v1

Building and Testing¶

Build Commands¶

# Basic build
kustomize build apps/rciis/nucleus/staging/

# Build with plugins enabled
kustomize build --enable-alpha-plugins --enable-exec apps/rciis/nucleus/staging/

# Build and apply
kustomize build apps/rciis/nucleus/staging/ | kubectl apply -f -

# Dry run validation
kustomize build apps/rciis/nucleus/staging/ | kubectl apply --dry-run=client -f -

Testing Configurations¶

# Validate YAML syntax
kustomize build apps/rciis/nucleus/staging/ | yaml-validator

# Check resource differences
kustomize build apps/rciis/nucleus/staging/ > current.yaml
kustomize build apps/rciis/nucleus/testing/ > testing.yaml
diff current.yaml testing.yaml

# Validate against cluster
kustomize build apps/rciis/nucleus/staging/ | kubectl apply --dry-run=server -f -

CI/CD Integration¶

GitHub Actions¶

- name: Build Kustomize
  run: |
    kustomize build --enable-alpha-plugins --enable-exec apps/rciis/nucleus/staging/ > manifest.yaml

- name: Validate manifests
  run: |
    kubectl apply --dry-run=client -f manifest.yaml

- name: Deploy to staging
  run: |
    kubectl apply -f manifest.yaml

ArgoCD Integration¶

# ArgoCD Application with Kustomize
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: nucleus-staging
spec:
  source:
    repoURL: git@github.com:MagnaBC/rciis-devops.git
    targetRevision: master
    path: apps/rciis/nucleus/staging
    kustomize:
      buildOptions: "--enable-alpha-plugins --enable-exec"

Best Practices¶

Organization¶

Consistent Structure: Use standard directory layouts
Environment Separation: Clear environment boundaries
Resource Grouping: Group related resources logically
Documentation: Comment complex transformations

Configuration Management¶

DRY Principle: Avoid duplicating configurations
Environment Parity: Minimize environment differences
Secret Handling: Use KSOPS for sensitive data
Version Control: Track all configuration changes

Testing¶

Validation: Test all kustomizations before deployment
Dry Runs: Use dry-run mode for validation
Diff Analysis: Compare configurations between environments
Integration Testing: Test with actual clusters

Troubleshooting¶

Common Issues¶

# KSOPS plugin not found
export XDG_CONFIG_HOME=~/.config
echo $XDG_CONFIG_HOME/kustomize/plugin/viaduct.ai/v1/ksops/ksops

# SOPS decryption fails
export SOPS_AGE_KEY_FILE=~/.age/key.txt
sops --decrypt apps/rciis/secrets/staging/nucleus/appsettings.yaml

# Plugin execution fails
kustomize build --enable-alpha-plugins --enable-exec --load-restrictor=LoadRestrictionsNone apps/rciis/nucleus/staging/

Debug Commands¶

# Check kustomize version
kustomize version

# List available plugins
kustomize plugin list

# Verbose build output
kustomize build --enable-alpha-plugins --enable-exec apps/rciis/nucleus/staging/ --load-restrictor=LoadRestrictionsNone -v 10

For detailed Kustomize usage, refer to the Kustomize documentation.