hydra-tools
Hydra Tools
Components and replacements to make Hydra suck less
A collection of bridge tools that enhance Hydra (the Nix-based continuous integration system) with modern CI/CD workflow integrations.
What is this project?
Hydra Tools provides three essential bridge components that address key limitations in Hydra's integration with modern development workflows:
- 🔄 github-hydra-bridge: Webhook bridge from GitHub → Hydra
- 📊 hydra-github-bridge: Status bridge from Hydra → GitHub
- 📦 hydra-attic-bridge: Artifact bridge from Hydra → Attic binary cache
Problems Solved
The Hydra GitHub Integration Problem
Hydra is fundamentally designed as a poll-driven CI system, which creates several issues when integrating with GitHub:
- Rate Limiting: Hydra's GitHub plugin uses declarative jobsets created on a polling interval, leading to excessive GitHub API requests that quickly hit rate limits
- Delayed Feedback: Poll-based checks mean slower response times for PR status updates
- Resource Waste: Constant polling consumes unnecessary resources
- Poor Developer Experience: Developers don't get timely feedback on their PRs
Event-Driven Solution
Instead of modifying Hydra's Perl codebase, these tools provide clean Haskell-based bridges that:
- Enable event-driven CI through GitHub webhooks
- Provide real-time status updates back to GitHub
- Reduce API rate limiting by eliminating unnecessary polling
- Improve binary cache utilization through automatic artifact uploads
What Each Component Offers
🔄 github-hydra-bridge
GitHub → Hydra Webhook Bridge
Receives GitHub webhook events and translates them into Hydra API calls:
- Pull Request Events: Automatically creates/updates/removes Hydra jobsets for PRs
- Push Events: Triggers Hydra evaluations for branch updates
- Event-Driven: Eliminates the need for constant polling
- Rate Limit Friendly: Only makes API calls when necessary
Key Features:
- Handles PR lifecycle (opened, synchronized, closed)
- Configurable through environment variables
- PostgreSQL integration for state management
- Built-in webhook secret validation
📊 hydra-github-bridge
Hydra → GitHub Status Bridge
Monitors Hydra build events and reports status back to GitHub:
- Real-time Status Updates: GitHub Check Runs updated as builds progress
- Detailed Build Information: Links to Hydra build pages, logs, and artifacts
- Build Result Reporting: Success/failure status with detailed error information
- Dependency Tracking: Reports status for dependent builds
Key Features:
- PostgreSQL LISTEN/NOTIFY for real-time updates
- GitHub App integration with JWT authentication
- Comprehensive build log inclusion in GitHub checks
- Support for compressed build logs (bz2)
📦 hydra-attic-bridge
Hydra → Attic Binary Cache Bridge
Automatically uploads successful build artifacts to Attic binary cache:
- Automatic Upload: Successful builds are queued for upload to Attic
- Retry Logic: Failed uploads are retried with exponential backoff
- Database Triggers: Uses PostgreSQL triggers for reliable event capture
- Efficient Processing: Only uploads artifacts that haven't been uploaded
Key Features:
- PostgreSQL trigger-based event capture
- Configurable retry attempts and delays
- Support for both build steps and build outputs
- Environment-based configuration
Building and Usage
Prerequisites
- GHC 9.2+ and Cabal
- PostgreSQL database (for Hydra integration)
- Access to a Hydra instance
- GitHub App credentials (for GitHub integration)
- Attic instance (for binary cache integration)
Building
# Build all components
make all
# Build individual components
make github-hydra-bridge
make hydra-github-bridge
make hydra-attic-bridge
# Clean build artifacts
make clean
Using Nix
This project provides a complete Nix flake with packages and NixOS modules:
# Build with Nix
nix build .#github-hydra-bridge
nix build .#hydra-github-bridge
nix build .#hydra-attic-bridge
# Run directly
nix run .#github-hydra-bridge
NixOS Integration
NixOS modules are provided for easy deployment:
{
imports = [ inputs.hydra-tools.nixosModules.github-hydra-bridge ];
services.github-hydra-bridge = {
enable = true;
hydraHost = "hydra.example.com";
port = 8811;
# ... additional configuration
};
}
Configuration
Environment Variables
github-hydra-bridge
PORT
: Server port (default: 8080)KEY
: GitHub webhook secretHYDRA_HOST
: Hydra server hostnameHYDRA_DB*
: Database connection settings
hydra-github-bridge
HYDRA_HOST
: Hydra server hostnameHYDRA_STATE_DIR
: Directory for state filesGITHUB_APP_ID
: GitHub App IDGITHUB_USER_AGENT
: User agent for API requestsHYDRA_DB*
: Database connection settings
hydra-attic-bridge
ATTIC_HOST
: Attic server URLATTIC_CACHE
: Cache nameATTIC_TOKEN
: Authentication tokenHYDRA_DB*
: Database connection settings
Database Setup
For hydra-attic-bridge
, run the SQL setup:
psql -d hydra -f hydra-attic-bridge/setup.sql
Motivation
The primary motivation for this project is to bridge the gap between Hydra's design philosophy and modern development workflows. While Hydra is an excellent CI system for Nix projects, its polling-based approach doesn't align well with the event-driven nature of modern Git hosting platforms.
By providing these bridge components, we enable:
- Better Developer Experience: Immediate feedback on PRs through GitHub checks
- Resource Efficiency: Event-driven processing reduces unnecessary computation
- API Rate Limit Compliance: Eliminates excessive GitHub API calls
- Improved Caching: Automatic binary cache population improves build times
- Production Ready: Battle-tested components used in production environments
Contributing
This project is written in Haskell and uses Cabal for package management. The codebase is structured as multiple packages within a single repository for easy maintenance and shared dependencies.
License
This project is licensed under the BSD-3-Clause license. See the LICENSE files in individual component directories for details.
hydra-tools
- 3
- 5
- 0
- 0
- 0
- 10 days ago
- December 12, 2022
Wed, 01 Oct 2025 07:31:29 GMT