Deployment

Production deployment guide for the Pixell Agent Runtime (PAR) including Docker containers, cloud platforms, Kubernetes orchestration, and monitoring setup.

Deployment Overview

PAR supports multiple deployment strategies for different environments and scale requirements:

• 🐳 Docker - Containerized deployment
• ☁️ Cloud Platforms - AWS, GCP, Azure
• ⚙️ Kubernetes - Container orchestration
• 🏢 On-Premises - Self-hosted infrastructure

Docker Deployment

Basic Docker Deployment

1. Create Dockerfile

# Dockerfile
FROM python:3.11-slim

# Set working directory
WORKDIR /app

# Install system dependencies
RUN apt-get update && apt-get install -y \
    gcc \
    g++ \
    && rm -rf /var/lib/apt/lists/*

# Copy requirements
COPY requirements.txt .

# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Copy application
COPY . .

# Create non-root user
RUN useradd -m -u 1000 par && chown -R par:par /app
USER par

# Expose ports
EXPOSE 8080 8081 9090

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
  CMD curl -f http://localhost:8080/health || exit 1

# Start PAR
CMD ["pixell-runtime", "--config", "par-config.yaml"]

2. Build and Run

# Build Docker image
docker build -t pixell-runtime:latest .

# Run Three-Surface mode
docker run -d \
  --name par-runtime \
  -p 8080:8080 \
  -p 8081:8081 \
  -p 9090:9090 \
  -v $(pwd)/agents:/app/agents \
  -v $(pwd)/logs:/app/logs \
  pixell-runtime:latest \
  --package /app/agents/my-agent.apkg

# Run Multi-Agent mode
docker run -d \
  --name par-platform \
  -p 8080:8080 \
  -p 8081:8081 \
  -p 9090:9090 \
  -v $(pwd)/agents:/app/agents \
  -v $(pwd)/data:/app/data \
  pixell-runtime:latest \
  --mode multi-agent

Docker Compose

1. Create docker-compose.yml

# docker-compose.yml
version: '3.8'

services:
  par-runtime:
    build: .
    container_name: par-runtime
    ports:
      - "8080:8080"
      - "8081:8081"
      - "9090:9090"
    volumes:
      - ./agents:/app/agents
      - ./logs:/app/logs
      - ./data:/app/data
    environment:
      - RUNTIME_MODE=three-surface
      - PORT=8080
      - ADMIN_PORT=9090
      - LOG_LEVEL=INFO
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

  # Optional: Add reverse proxy
  nginx:
    image: nginx:alpine
    container_name: par-nginx
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./ssl:/etc/nginx/ssl
    depends_on:
      - par-runtime
    restart: unless-stopped

  # Optional: Add monitoring
  prometheus:
    image: prom/prometheus:latest
    container_name: par-prometheus
    ports:
      - "9091:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    restart: unless-stopped

  grafana:
    image: grafana/grafana:latest
    container_name: par-grafana
    ports:
      - "3000:3000"
    volumes:
      - grafana-storage:/var/lib/grafana
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin
    restart: unless-stopped

volumes:
  grafana-storage:

2. Deploy with Docker Compose

# Start services
docker-compose up -d

# View logs
docker-compose logs -f par-runtime

# Scale services
docker-compose up -d --scale par-runtime=3

# Stop services
docker-compose down

Cloud Platform Deployment

AWS Deployment

1. AWS ECS with Fargate

# ecs-task-definition.json
{
  "family": "par-runtime",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "1024",
  "memory": "2048",
  "executionRoleArn": "arn:aws:iam::account:role/ecsTaskExecutionRole",
  "taskRoleArn": "arn:aws:iam::account:role/parTaskRole",
  "containerDefinitions": [
    {
      "name": "par-runtime",
      "image": "your-account.dkr.ecr.region.amazonaws.com/pixell-runtime:latest",
      "portMappings": [
        {
          "containerPort": 8080,
          "protocol": "tcp"
        },
        {
          "containerPort": 8081,
          "protocol": "tcp"
        },
        {
          "containerPort": 9090,
          "protocol": "tcp"
        }
      ],
      "environment": [
        {
          "name": "RUNTIME_MODE",
          "value": "multi-agent"
        },
        {
          "name": "PORT",
          "value": "8080"
        }
      ],
      "secrets": [
        {
          "name": "API_KEY",
          "valueFrom": "arn:aws:secretsmanager:region:account:secret:par/api-key"
        }
      ],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/par-runtime",
          "awslogs-region": "us-west-2",
          "awslogs-stream-prefix": "ecs"
        }
      },
      "healthCheck": {
        "command": ["CMD-SHELL", "curl -f http://localhost:8080/health || exit 1"],
        "interval": 30,
        "timeout": 5,
        "retries": 3,
        "startPeriod": 60
      }
    }
  ]
}

2. AWS EKS (Kubernetes)

# k8s-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: par-runtime
  labels:
    app: par-runtime
spec:
  replicas: 3
  selector:
    matchLabels:
      app: par-runtime
  template:
    metadata:
      labels:
        app: par-runtime
    spec:
      containers:
      - name: par-runtime
        image: your-account.dkr.ecr.region.amazonaws.com/pixell-runtime:latest
        ports:
        - containerPort: 8080
        - containerPort: 8081
        - containerPort: 9090
        env:
        - name: RUNTIME_MODE
          value: "multi-agent"
        - name: PORT
          value: "8080"
        - name: API_KEY
          valueFrom:
            secretKeyRef:
              name: par-secrets
              key: api-key
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: par-runtime-service
spec:
  selector:
    app: par-runtime
  ports:
  - name: http
    port: 80
    targetPort: 8080
  - name: grpc
    port: 8081
    targetPort: 8081
  - name: admin
    port: 9090
    targetPort: 9090
  type: LoadBalancer

Google Cloud Platform

1. Google Cloud Run

# cloud-run.yaml
apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: par-runtime
  annotations:
    run.googleapis.com/ingress: all
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/maxScale: "10"
        run.googleapis.com/cpu: "2"
        run.googleapis.com/memory: "4Gi"
    spec:
      containers:
      - image: gcr.io/your-project/pixell-runtime:latest
        ports:
        - containerPort: 8080
        env:
        - name: RUNTIME_MODE
          value: "multi-agent"
        - name: PORT
          value: "8080"
        resources:
          limits:
            cpu: "2"
            memory: "4Gi"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10

2. Google Kubernetes Engine

# Deploy to GKE
kubectl apply -f k8s-deployment.yaml

# Expose service
kubectl expose deployment par-runtime --type=LoadBalancer --port=80 --target-port=8080

# Check status
kubectl get pods
kubectl get services

Azure Deployment

1. Azure Container Instances

# azure-container-instance.yaml
apiVersion: 2021-07-01
location: eastus
name: par-runtime
properties:
  containers:
  - name: par-runtime
    properties:
      image: your-registry.azurecr.io/pixell-runtime:latest
      ports:
      - port: 8080
      - port: 8081
      - port: 9090
      environmentVariables:
      - name: RUNTIME_MODE
        value: "multi-agent"
      - name: PORT
        value: "8080"
      resources:
        requests:
          cpu: 2
          memoryInGb: 4
  osType: Linux
  restartPolicy: Always
  ipAddress:
    type: Public
    ports:
    - protocol: tcp
      port: 8080
    - protocol: tcp
      port: 8081
    - protocol: tcp
      port: 9090

2. Azure Kubernetes Service

# Create AKS cluster
az aks create --resource-group myResourceGroup --name par-cluster --node-count 3

# Get credentials
az aks get-credentials --resource-group myResourceGroup --name par-cluster

# Deploy PAR
kubectl apply -f k8s-deployment.yaml

Kubernetes Deployment

1. Namespace and ConfigMap

# k8s-namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: par-system
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: par-config
  namespace: par-system
data:
  par-config.yaml: |
    runtime:
      mode: multi-agent
      host: 0.0.0.0
      port: 8080
      admin_port: 9090
    logging:
      level: INFO
      format: json
    performance:
      max_workers: 4
      timeout: 30

2. Secrets

# k8s-secrets.yaml
apiVersion: v1
kind: Secret
metadata:
  name: par-secrets
  namespace: par-system
type: Opaque
data:
  api-key: <base64-encoded-api-key>
  jwt-secret: <base64-encoded-jwt-secret>

3. Deployment

# k8s-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: par-runtime
  namespace: par-system
spec:
  replicas: 3
  selector:
    matchLabels:
      app: par-runtime
  template:
    metadata:
      labels:
        app: par-runtime
    spec:
      containers:
      - name: par-runtime
        image: pixell-runtime:latest
        ports:
        - containerPort: 8080
          name: http
        - containerPort: 8081
          name: grpc
        - containerPort: 9090
          name: admin
        env:
        - name: RUNTIME_MODE
          value: "multi-agent"
        - name: PORT
          value: "8080"
        - name: API_KEY
          valueFrom:
            secretKeyRef:
              name: par-secrets
              key: api-key
        volumeMounts:
        - name: config
          mountPath: /app/config
        - name: data
          mountPath: /app/data
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5
      volumes:
      - name: config
        configMap:
          name: par-config
      - name: data
        persistentVolumeClaim:
          claimName: par-data-pvc

4. Service and Ingress

# k8s-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: par-runtime-service
  namespace: par-system
spec:
  selector:
    app: par-runtime
  ports:
  - name: http
    port: 80
    targetPort: 8080
  - name: grpc
    port: 8081
    targetPort: 8081
  - name: admin
    port: 9090
    targetPort: 9090
  type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: par-runtime-ingress
  namespace: par-system
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - par.yourdomain.com
    secretName: par-tls
  rules:
  - host: par.yourdomain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: par-runtime-service
            port:
              number: 80

5. Horizontal Pod Autoscaler

# k8s-hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: par-runtime-hpa
  namespace: par-system
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: par-runtime
  minReplicas: 3
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

Monitoring and Observability

1. Prometheus Configuration

# prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
- job_name: 'par-runtime'
  static_configs:
  - targets: ['par-runtime-service:9090']
  metrics_path: /metrics
  scrape_interval: 10s

2. Grafana Dashboard

{
  "dashboard": {
    "title": "PAR Runtime Dashboard",
    "panels": [
      {
        "title": "Request Rate",
        "type": "graph",
        "targets": [
          {
            "expr": "rate(par_requests_total[5m])",
            "legendFormat": "Requests/sec"
          }
        ]
      },
      {
        "title": "Response Time",
        "type": "graph",
        "targets": [
          {
            "expr": "histogram_quantile(0.95, rate(par_request_duration_seconds_bucket[5m]))",
            "legendFormat": "95th percentile"
          }
        ]
      },
      {
        "title": "Error Rate",
        "type": "graph",
        "targets": [
          {
            "expr": "rate(par_errors_total[5m])",
            "legendFormat": "Errors/sec"
          }
        ]
      }
    ]
  }
}

3. Alerting Rules

# alerting-rules.yml
groups:
- name: par-runtime
  rules:
  - alert: HighErrorRate
    expr: rate(par_errors_total[5m]) > 0.1
    for: 2m
    labels:
      severity: warning
    annotations:
      summary: "High error rate detected"
      description: "PAR runtime error rate is {{ $value }} errors/sec"
  
  - alert: HighResponseTime
    expr: histogram_quantile(0.95, rate(par_request_duration_seconds_bucket[5m])) > 2
    for: 5m
    labels:
      severity: critical
    annotations:
      summary: "High response time detected"
      description: "PAR runtime 95th percentile response time is {{ $value }}s"

Security Configuration

1. Network Policies

# k8s-network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: par-runtime-netpol
  namespace: par-system
spec:
  podSelector:
    matchLabels:
      app: par-runtime
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: ingress-nginx
    ports:
    - protocol: TCP
      port: 8080
  egress:
  - to: []
    ports:
    - protocol: TCP
      port: 53
    - protocol: UDP
      port: 53

2. Pod Security Policy

# k8s-psp.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: par-runtime-psp
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
  - ALL
  volumes:
  - 'configMap'
  - 'emptyDir'
  - 'projected'
  - 'secret'
  - 'downwardAPI'
  - 'persistentVolumeClaim'
  runAsUser:
    rule: 'MustRunAsNonRoot'
  seLinux:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'

Backup and Recovery

1. Data Backup

# Backup agent data
kubectl exec -n par-system deployment/par-runtime -- tar -czf /tmp/backup.tar.gz /app/data

# Copy backup
kubectl cp par-system/par-runtime-pod:/tmp/backup.tar.gz ./backup-$(date +%Y%m%d).tar.gz

# Restore data
kubectl cp ./backup-20240115.tar.gz par-system/par-runtime-pod:/tmp/restore.tar.gz
kubectl exec -n par-system deployment/par-runtime -- tar -xzf /tmp/restore.tar.gz -C /app/data

2. Configuration Backup

# Backup configuration
kubectl get configmap par-config -n par-system -o yaml > par-config-backup.yaml

# Restore configuration
kubectl apply -f par-config-backup.yaml

Troubleshooting

1. Common Issues

Pod Not Starting

# Check pod status
kubectl get pods -n par-system

# Check pod logs
kubectl logs -n par-system deployment/par-runtime

# Check pod events
kubectl describe pod -n par-system par-runtime-pod

Service Not Accessible

# Check service
kubectl get svc -n par-system

# Check endpoints
kubectl get endpoints -n par-system

# Test connectivity
kubectl exec -n par-system deployment/par-runtime -- curl localhost:8080/health

High Resource Usage

# Check resource usage
kubectl top pods -n par-system

# Check node resources
kubectl top nodes

# Scale down if needed
kubectl scale deployment par-runtime --replicas=1 -n par-system

2. Debug Commands

# Get all resources
kubectl get all -n par-system

# Check logs with timestamps
kubectl logs -n par-system deployment/par-runtime --timestamps

# Port forward for local access
kubectl port-forward -n par-system svc/par-runtime-service 8080:80

# Execute commands in pod
kubectl exec -n par-system deployment/par-runtime -- /bin/bash

Performance Optimization

1. Resource Limits

resources:
  requests:
    memory: "1Gi"
    cpu: "500m"
  limits:
    memory: "2Gi"
    cpu: "1000m"

2. Horizontal Scaling

# HPA configuration
minReplicas: 3
maxReplicas: 10
targetCPUUtilizationPercentage: 70

3. Vertical Scaling

# VPA configuration
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: par-runtime-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: par-runtime
  updatePolicy:
    updateMode: "Auto"

Next Steps

After deploying PAR:

1. Configuration - Advanced configuration options
2. REST Endpoints - HTTP API reference
3. gRPC Interface - A2A communication

---

Ready to configure? Check out Configuration to learn about advanced setup options!