monorepo

CrystalShards Autonomous Development

An autonomous agent that builds CrystalShards.org and CrystalDocs.org - a comprehensive Crystal language package registry and documentation platform.

Quick Start

# Set your credentials
export CLAUDE_CODE_OAUTH_TOKEN="your-claude-api-key"
export GITHUB_TOKEN="your-github-token"

# Launch the agent (uses kubernetes-dev-pod.yaml)
./remote.sh

# Watch it work
kubectl logs -f claude-agent -n claude

# Optional: Use custom settings
GIT_URL="https://github.com/yourfork/crystalshards-claude.git" \
GIT_BRANCH="feature/my-branch" \
POD_NAME="my-agent" \
./remote.sh

The script will:

  • Create namespace, PVC, secrets, and pod from kubernetes-dev-pod.yaml
  • Substitute your API keys automatically
  • Wait for pod readiness
  • Optionally tail logs

Philosophy

Following the "less is more" approach - a simple prompt drives continuous development with frequent commits.

What It Builds

  1. CrystalShards.org - Package registry like rubygems.org
  2. CrystalDocs.org - Documentation platform like docs.rs
  3. CrystalGigs.com - Job board for Crystal developers
  4. CrystalBits.org - Newsletter/blog platform

Features:

  • Full-text search, dependency graphs, download stats
  • Automated documentation generation with background workers
  • Everything runs in-cluster (no external cloud services)
  • Security-hardened with non-root containers and minimal capabilities
  • GKE Autopilot for automatic scaling and optimization

Files

  • PROMPT.md - Simple prompt that guides the agent
  • develop.sh - Continuous loop: prompt → claude → sleep → repeat
  • remote.sh - Launcher script using your environment variables
  • .agent/STATUS.md - Agent tracks its own progress
  • CLAUDE.md - Development guidelines

Architecture

/apps/crystalshards     - Main registry (Lucky app + Mosquito workers)
/apps/crystaldocs       - Documentation platform (Lucky app)
/apps/crystalgigs       - Job board (Lucky app)
/apps/crystalbits       - Newsletter/blog (Lucky app)
/terraform              - Infrastructure as Code (GKE, K8s resources)
  /modules/networking   - VPC, subnets, Cloud NAT
  /modules/cluster      - GKE Autopilot cluster
  /modules/operators    - Infrastructure operators
  /modules/ingress      - Traefik + external-dns
  /modules/applications - App-specific K8s resources
/.github/workflows      - CI/CD pipelines

In-Cluster Services

  • PostgreSQL: CloudNativePG operator (per-app databases)
  • Redis: Redis operator (shared cluster)
  • Object Storage: MinIO operator (packages & docs)
  • Background Jobs: Mosquito (Crystal job framework)
  • Ingress: Traefik with cert-manager
  • Monitoring: Prometheus + Grafana

Monitoring

# Watch logs
kubectl logs -f claude-agent -n claude

# Check pod status
kubectl describe pod claude-agent -n claude

# Get a shell
kubectl exec -it claude-agent -n claude -- bash

# Check PVC usage
kubectl get pvc -n claude

# Stop the agent and cleanup
kubectl delete -f kubernetes-dev-pod.yaml

Development Approach

The agent:

  1. Commits after EVERY file edit
  2. Self-regulates scope
  3. Tracks progress in .agent/STATUS.md
  4. Pushes to GitHub regularly
  5. Creates PRs when features are complete

Requirements

  • Kubernetes cluster with kubectl configured
  • GitHub token with repo permissions
  • Claude API key
  • envbuilder support (uses ghcr.io/coder/envbuilder:latest)
  • GitHub repository configured (see GITHUB_SETUP.md)

Cost Control

  • GKE Autopilot (pay only for running pods, automatic bin-packing)
  • In-cluster databases with CNPG (no Cloud SQL costs)
  • MinIO for object storage (no Cloud Storage costs)
  • Traefik ingress (no external load balancer costs per app)
  • Resource limits on all pods for predictable costs
  • Estimated monthly cost: $185-310

GitHub Setup Required

See GITHUB_SETUP.md for required:

  • Repository secrets
  • Service account setup
  • Branch protection rules
  • GitHub environments
  • Container registry access
Repository

monorepo

Owner
Statistic
  • 0
  • 0
  • 45
  • 0
  • 0
  • 12 days ago
  • August 25, 2025
License

Links
Synced at

Tue, 21 Oct 2025 22:22:56 GMT

Languages