Shellmin

A fast, parallel CLI tool for managing Linux servers via SSH with extensible scripting support.

Shellmin provides a unified interface for managing multiple Linux servers simultaneously, with built-in support for common sysadmin tasks and a powerful plugin system for custom automation.

✨ Recent Updates (v0.1.0) ✨

• 🔒 Enhanced Security: Multi-layered command injection prevention, input sanitization, and audit logging
• ⚡ Performance Optimization: Atomic operations, pre-compiled regex patterns, and optimized memory usage
• 🔄 Modern Architecture: Crystal 1.x features, improved type safety, and modular design
• 📊 Advanced Monitoring: Built-in performance monitoring and health checks
• 🛡️ Security Hardening: Comprehensive validation, privilege escalation prevention, and secure defaults
• 🔧 Developer Experience: Improved error handling, better documentation, and CI/CD enhancements

Features

• Parallel Execution: Run commands across multiple servers simultaneously
• SSH Connection Management: Efficient connection pooling and reuse with health monitoring
• Extensible Plugin System: Support for bash, Python, and other scripting languages
• Built-in Modules: PostgreSQL, MariaDB, Docker, Nginx, Git, Security, Users, and more
• Server Inventory: Manage server configurations and SSH keys
• Script Automation: Create and execute custom scripts with environment variables
• Security Hardening: Built-in security auditing and compliance features
• Container Management: Full Docker container lifecycle management
• Web Server Management: Nginx configuration and site management
• Database Management: Comprehensive PostgreSQL and MariaDB support
• Performance Monitoring: Real-time operation timing and statistics
• Multi-format Configuration: Support for YAML, JSON, and legacy formats
• Advanced Logging: Structured logging with multiple formatters and severity levels

Installation

Prerequisites

- **Crystal 1.17.0+**: Modern Crystal programming language with type safety
- **SSH client**: Configured for password-less authentication (recommended)
- **Git**: For cloning the repository and version control
- **Make**: For build automation (optional but recommended)

Build from Source

.. code-block:: bash

# Clone the repository
git clone https://github.com/your-github-user/shellmin.git
cd shellmin

# Quick setup (recommended) - installs dependencies and builds
./setup.sh

# Or manual setup
shards install                    # Install Crystal dependencies
make build                       # Build the application
make test                        # Run test suite
crystal docs --output=docs       # Generate documentation

Install Binary

.. code-block:: bash

    # Copy the binary to your PATH
    sudo cp shellmin /usr/local/bin/
    sudo chmod +x /usr/local/bin/shellmin

Modern Features
----------------

Shellmin v0.1.0 includes significant improvements in security, performance, and developer experience:

**🔒 Security Enhancements**
- Multi-layered command injection prevention
- Input sanitization with null byte and control character filtering
- Path traversal and privilege escalation protection
- Comprehensive audit logging with structured formatters
- Secure default configurations

**⚡ Performance Optimizations**
- Atomic operations for thread-safe performance monitoring
- Pre-compiled regex patterns for optimal validation speed
- Connection pooling with health monitoring and automatic cleanup
- Memory-efficient string operations and reduced allocations
- Background fiber-based health monitoring

**🔧 Developer Experience**
- Modern Crystal 1.x features and idioms
- Enhanced error handling with rich context and severity levels
- Comprehensive type safety with nil inference fixes
- Improved documentation and API references
- Automated CI/CD with GitLab CI integration

**📊 Monitoring & Observability**
- Real-time performance monitoring and statistics
- Structured logging with multiple formatters
- Health check endpoints and uptime tracking
- Operation timing and bottleneck identification
- Memory usage and connection pool monitoring

Quick Start
-----------

1. Configure Hosts

First, copy the example configuration file and customize it:

.. code-block:: bash

# Copy the example configuration
mkdir -p ~/.config/shellmin
cp config.example.yaml ~/.config/shellmin/config.yaml

# Edit the configuration file with your server details
$EDITOR ~/.config/shellmin/config.yaml

Then add hosts using the CLI:

.. code-block:: bash

# Add your first host
shellmin host add --name web-server --host 192.168.1.100 --user admin

# Add more hosts
shellmin host add --name db-server --host 192.168.1.101 --user root
shellmin host add --name backup-server --host 192.168.1.102 --user ubuntu

# List configured hosts
shellmin host list

2. Basic Host Management

.. code-block:: bash

    # Check host status
    shellmin status --all

    # Update all hosts
    shellmin update --all --yes

    # Execute command on specific host
    shellmin host exec --host web-server "df -h"

    # Execute command on all hosts
    shellmin host exec --all "uptime"

3. Service Management
~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

    # Check service status
    shellmin service status nginx --all

    # Restart service on all servers
    shellmin service restart nginx --all

4. PostgreSQL Management

.. code-block:: bash

# Check PostgreSQL status
shellmin postgresql status --all

# Create database backup
shellmin postgresql backup --database myapp --server db-server

# Restore from backup
shellmin postgresql restore --database myapp --file /opt/backups/myapp_backup.sql --server db-server

5. Backup Management

.. code-block:: bash

    # Create backup of important directories
    shellmin backup create weekly-backup --paths /etc,/home,/var/www --all

    # List available backups
    shellmin backup list --server web-server

    # Schedule automated backups
    shellmin backup schedule daily-etc --paths /etc --time "0 2 * * *" --all

6. Ollama AI Management

.. code-block:: bash

# Install Ollama on servers
shellmin ollama install --all

# Pull a model
shellmin ollama pull llama2 --server ai-server

# Start Ollama service
shellmin ollama serve --all

 # List available models
 shellmin ollama models --server ai-server

7. Docker Container Management

.. code-block:: bash

    # Install Docker on all servers
    shellmin docker install --all

    # Check Docker status
    shellmin docker status --all

    # Run a container
    shellmin docker run nginx --name web-server --ports 80:80 --server web-01

    # List running containers
    shellmin docker ps --all

    # Stop all containers
    shellmin docker stop --all

    # Clean up Docker resources
    shellmin docker cleanup --all --images --volumes

8. Nginx Web Server Management

.. code-block:: bash

# Install Nginx
shellmin nginx install --all

# Check Nginx status
shellmin nginx status --all

# Create site configuration
shellmin nginx config create mysite --content "server { listen 80; server_name example.com; location / { proxy_pass http://localhost:3000; } }"

# Enable site
shellmin nginx config enable mysite

# Reload configuration
shellmin nginx reload --all

# View access logs
shellmin nginx logs --server web-01

9. Git Repository Management

.. code-block:: bash

    # Clone repository
    shellmin git clone https://github.com/user/myapp.git --directory /opt/apps --all

    # Pull latest changes
    shellmin git pull --directory /opt/apps/myapp --all

    # Check repository status
    shellmin git status --directory /opt/apps/myapp --all

    # Deploy application
    shellmin git deploy https://github.com/user/myapp.git --restart-service myapp

    # Manage branches
    shellmin git branch create feature-branch --directory /opt/apps/myapp
    shellmin git branch switch main --directory /opt/apps/myapp

10. MariaDB/MySQL Database Management

.. code-block:: bash

# Install MariaDB
shellmin mariadb install --all

# Check MariaDB status
shellmin mariadb status --all

# Create database backup
shellmin mariadb backup wordpress --user root --server db-server-01

# List databases
shellmin mariadb databases --user root

# Execute SQL query
shellmin mariadb query "SHOW TABLES" --database wordpress --user root

11. User and SSH Key Management

.. code-block:: bash

    # List system users
    shellmin users list --all

    # Create new user
    shellmin users create deploy --sudo --all

    # Add SSH key
    shellmin users ssh-keys add --key "ssh-rsa AAAAB3NzaC1yc..." --user deploy --all

    # Change user password
    shellmin users password deploy --all

    # Manage user groups
    shellmin users groups add --user deploy --group docker --all

12. Security Hardening and Compliance

.. code-block:: bash

# Perform security audit
shellmin security audit --comprehensive --all

# Apply security hardening
shellmin security harden --level standard --all

# Manage firewall
shellmin security firewall status --all
shellmin security firewall allow --port 443 --all
shellmin security firewall enable --all

# Security updates
shellmin security updates check --all
shellmin security updates install --all
shellmin security updates auto --all

# Monitor security events
shellmin security monitor --all

13. Host Management

.. code-block:: bash

    # Add a single host
    shellmin host add --name web-01 --host 192.168.1.100 --user admin

    # List configured hosts
    shellmin host list

    # Execute command on specific host
    shellmin host exec --host web-01 "uptime"

    # Execute command on all hosts
    shellmin host exec --all "df -h"

    # Remove a host
    shellmin host remove web-01

Bulk Host Import
^^^^^^^^^^^^^^^^

Import hundreds of hosts from various file formats:

**CSV Format:**
.. code-block:: bash

    # Create hosts.csv
    name,host,user,key_path
    web-01,192.168.1.100,admin,/home/admin/.ssh/web-key
    web-02,192.168.1.101,admin,/home/admin/.ssh/web-key
    db-01,192.168.1.200,postgres,/home/postgres/.ssh/db-key

    # Import CSV file
    shellmin host import hosts.csv --format csv

**JSON Format:**
.. code-block:: bash

    # Create hosts.json
    [
      {
        "name": "web-01",
        "host": "192.168.1.100",
        "user": "admin",
        "key_path": "/home/admin/.ssh/web-key"
      },
      {
        "name": "db-01",
        "host": "192.168.1.200",
        "user": "postgres"
      }
    ]

    # Import JSON file
    shellmin host import hosts.json --format json

**YAML Format:**
.. code-block:: bash

    # Create hosts.yaml
    - name: web-01
      host: 192.168.1.100
      user: admin
      key_path: /home/admin/.ssh/web-key
    - name: db-01
      host: 192.168.1.200
      user: postgres

    # Import YAML file (auto-detected)
    shellmin host import hosts.yaml

**Import Options:**
.. code-block:: bash

    # Dry run to preview import
    shellmin host import hosts.csv --dry-run

    # Skip validation during import
    shellmin host import hosts.csv --skip-validation

    # Auto-detect format from file extension
    shellmin host import hosts.json  # Automatically detects JSON

14. Firewalld Firewall Management

.. code-block:: bash

# Check firewalld status
shellmin firewalld status --all

# Manage zones
shellmin firewalld zones list --all
shellmin firewalld zones info --zone public --server web-01
shellmin firewalld zones create --zone dmz --server web-01

# Manage services
shellmin firewalld services add --service https --zone public --all
shellmin firewalld services remove --service telnet --zone public --server web-01
shellmin firewalld services list --zone public --all

# Manage ports
shellmin firewalld ports add --port 8080 --protocol tcp --zone public --all
shellmin firewalld ports remove --port 23 --protocol tcp --zone public --server web-01
shellmin firewalld ports list --zone public --all

Rich Rules Management ^^^^^^^^^^^^^^^^^^^^^

Advanced firewall rules with rich language:

.. code-block:: bash

# Allow SSH from specific IP
shellmin firewalld rich-rules add --rule "rule family=ipv4 source address=192.168.1.100 accept" --zone public --server web-01

# Rate limit connections
shellmin firewalld rich-rules add --rule "rule family=ipv4 source address=10.0.0.0/8 limit value=10/m accept" --zone public --all

# Port forwarding
shellmin firewalld rich-rules add --rule "rule family=ipv4 forward-port port=80 protocol=tcp to-port=8080" --zone public --server web-01

# List rich rules
shellmin firewalld rich-rules list --zone public --all

# Remove rich rule
shellmin firewalld rich-rules remove --rule "rule family=ipv4 source address=192.168.1.100 accept" --zone public --server web-01

Panic Mode: .. code-block:: bash

# Enable panic mode (block all traffic)
shellmin firewalld panic on --all

# Disable panic mode
shellmin firewalld panic off --all

# Reload firewall configuration
shellmin firewalld reload --all

15. Sudo Permissions Management

.. code-block:: bash

    # Grant passwordless sudo for specific commands
    shellmin sudo grant --user renich --commands "/usr/bin/systemctl restart nginx.service,/usr/bin/systemctl reload nginx.service" --passwordless --all

    # Grant sudo with password requirement
    shellmin sudo grant --user deploy --commands "/usr/bin/docker,/usr/bin/docker-compose" --server web-server-01

    # Use predefined templates
    shellmin sudo template web-admin --user renich --server web-01
    shellmin sudo template db-admin --user dba --all

    # List sudo rules
    shellmin sudo list --all
    shellmin sudo list --user renich --server web-01

    # Test sudo permissions
    shellmin sudo test --user renich --command "/usr/bin/systemctl restart nginx" --server web-01

    # Revoke sudo permissions
    shellmin sudo revoke --user renich --all-rules --all

16. Podman Container Management

.. code-block:: bash

# Check Podman system status
shellmin podman status --all

# List running containers
shellmin podman ps --all

# Run a container
shellmin podman run nginx --name web-server --ports 80:80 --detach

# Run with privileged access
shellmin podman run --privileged fedora --name privileged-container

# Stop containers
shellmin podman stop web-server
shellmin podman stop --all

# View container logs
shellmin podman logs web-server
shellmin podman logs web-server --follow

# Manage images
shellmin podman images list --all
shellmin podman images prune --all

# Build container image
shellmin podman build ./myapp --tag myapp:latest

# System management
shellmin podman system prune --all
shellmin podman system df --all

17. Package Management

.. code-block:: bash

    # Install packages (auto-detects package manager)
    shellmin package install nginx curl wget --all

    # Remove packages
    shellmin package remove apache2 --server web-01

    # Update system packages
    shellmin package update --all

    # Search for packages
    shellmin package search "python3" --server dev-01

    # List installed packages
    shellmin package list --all
    shellmin package list --pattern "python" --server dev-01

    # Show package information
    shellmin package info nginx --server web-01

Repository Management
^^^^^^^^^^^^^^^^^^^^^

.. code-block:: bash

    # List configured repositories
    shellmin package repo list --all

    # Add repository (Ubuntu/Debian)
    shellmin package repo add --repo "deb https://nginx.org/packages/ubuntu/ focal nginx" --key "https://nginx.org/keys/nginx_signing.key" --server web-01

    # Add repository (CentOS/RHEL)
    shellmin package repo add --repo "https://nginx.org/packages/centos/8/x86_64/" --server web-01

    # Remove repository
    shellmin package repo remove --repo nginx --server web-01

    # Update repository metadata
    shellmin package repo update --all

**Supported Package Managers:**
- **apt** (Ubuntu, Debian)
- **yum** (CentOS, RHEL 7)
- **dnf** (Fedora, RHEL 8+)
- **pacman** (Arch Linux)
- **zypper** (openSUSE)
- **apk** (Alpine Linux)

18. NetworkManager Management

.. code-block:: bash

# Check NetworkManager status
shellmin networkmanager status --all

# List network connections
shellmin networkmanager connections list --all

# Show network devices
shellmin networkmanager devices --all

# Manage WiFi connections
shellmin networkmanager wifi list --server laptop-01
shellmin networkmanager wifi connect --ssid "MyWiFi" --password "secret" --server laptop-01

# Create network connections
shellmin networkmanager create ethernet --name "eth0-static" --device eth0 --ip 192.168.1.100 --gateway 192.168.1.1 --dns "8.8.8.8,8.8.4.4" --server server-01

# Create WiFi connection
shellmin networkmanager create wifi --name "office-wifi" --ssid "OfficeWiFi" --password "office123" --server laptop-01

# Modify existing connections
shellmin networkmanager modify "eth0-static" --ip 192.168.1.101 --server server-01

# Enable/disable connections
shellmin networkmanager connections up "eth0-static" --server server-01
shellmin networkmanager connections down "eth0-static" --server server-01

# Delete connections
shellmin networkmanager delete "old-connection" --server server-01

Scripting System

Create Custom Scripts

.. code-block:: bash

    # Create a bash script template
    shellmin script create monitor-disk --language bash

    # Create a Python script template
    shellmin script create deploy-app --language python

Execute Scripts
~~~~~~~~~~~~~~~

.. code-block:: bash

    # Run script on specific server
    shellmin script run monitor-disk --server web-server

    # Run script on all servers
    shellmin script run deploy-app --all

    # Dry run to see what would be executed
    shellmin script run backup-logs --all --dry-run

Script Environment Variables

Scripts have access to these environment variables:

• SHELLMIN_HOST: Target server hostname/IP
• SHELLMIN_USER: SSH username
• SHELLMIN_SCRIPT_ARGS: Additional arguments passed to the script

Configuration

Shellmin stores configuration in ~/.shellmin/config.yml:

.. code-block:: yaml

servers: - name: web-server host: 192.168.1.100 user: admin key_path: ~/.ssh/web_server_key - name: db-server host: 192.168.1.101 user: root

Note: Configuration still uses 'servers' key for backward compatibility

The CLI commands use 'host' terminology for better clarity

Advanced Usage

Parallel Command Execution

.. code-block:: bash

    # Execute different commands on different hosts
    shellmin host exec --host web-server "nginx -t"
    shellmin host exec --host db-server "systemctl status postgresql"

Custom Modules
~~~~~~~~~~~~~~

Create your own modules by adding new Crystal files in ``src/shellmin/`` and registering them in ``src/shellmin.cr``.

SSH Key Management
~~~~~~~~~~~~~~~~~~

.. code-block:: bash

    # Use specific SSH key
    shellmin server add --name secure-server --host 10.0.0.1 --key ~/.ssh/production_key

    # Password authentication (less secure)
    # Configure SSH to use password auth, then use without --key option

Command Reference
-----------------

Global Options
~~~~~~~~~~~~~~

- ``-c, --config=CONFIG``: Path to configuration file
- ``-v, --verbose``: Enable verbose output
- ``-p, --parallel=PARALLEL``: Number of parallel connections

Main Commands
~~~~~~~~~~~~~

- ``status``: Check host status and basic information
- ``update``: Update system packages
- ``service``: Manage system services
- ``system``: System administration commands
- ``host``: Host management (add, remove, list, exec, import)
- ``script``: Script management and execution
- ``postgresql``: PostgreSQL database management
- ``mariadb``: MariaDB/MySQL database management
- ``docker``: Docker container management
- ``podman``: Podman container management (daemonless)
- ``nginx``: Nginx web server management
- ``git``: Git repository management and deployment
- ``backup``: Backup management and automation
- ``ollama``: Ollama AI model management
- ``users``: User and SSH key management
- ``security``: Security hardening and compliance
- ``sudo``: Sudo permissions management
- ``firewalld``: Firewalld firewall management
- ``networkmanager``: NetworkManager management using nmcli
- ``package``: System package and repository management

Examples
--------

Complete Host Setup Workflow

.. code-block:: bash

# Add new host
shellmin host add --name new-server --host 192.168.1.200 --user ubuntu

# Update and secure the host
shellmin update --host new-server --yes
shellmin host exec --host new-server "ufw enable"

# Install and configure PostgreSQL
shellmin postgresql install --server new-server
shellmin postgresql status --server new-server

# Set up automated backups
shellmin backup schedule daily-backup --server new-server --time "0 3 * * *"

Multi-Server Monitoring

.. code-block:: bash

    # Create monitoring script
    shellmin script create check-services --language bash

    # Edit the script to check your services
    # Then run it across all servers
    shellmin script run check-services --all

Development
-----------

Getting Started
~~~~~~~~~~~~~~~

.. code-block:: bash

    # Clone the repository
    git clone https://github.com/your-github-user/shellmin.git
    cd shellmin

    # Run the setup script (recommended)
    ./setup.sh

    # Or use make commands
    make setup    # Install dependencies
    make format   # Format code
    make test     # Run tests
    make build    # Build application

Available Make Targets
~~~~~~~~~~~~~~~~~~~~~~

- ``make build`` - Build the application
- ``make release`` - Build with release optimizations
- ``make test`` - Run test suite
- ``make format`` - Format code with crystal tool
- ``make docs`` - Generate documentation
- ``make clean`` - Clean build artifacts
- ``make install`` - Install to system (requires sudo)
- ``make lint`` - Run code linter
- ``make dev`` - Full development workflow

Project Structure
~~~~~~~~~~~~~~~~~

::

    src/
      shellmin.cr              # Main application entry point
      shellmin/
        connection_manager.cr  # SSH connection management
        utils.cr              # Shared utilities and helpers
        system.cr             # System administration commands
        host.cr               # Host inventory management
        script.cr             # Plugin/script system
        postgresql.cr         # PostgreSQL management
        mariadb.cr            # MariaDB/MySQL management
        docker.cr             # Docker container management
        podman.cr             # Podman container management
        nginx.cr              # Nginx web server management
        git.cr                # Git repository management
        backup.cr             # Backup management
        ollama.cr             # Ollama AI management
        users.cr              # User and SSH key management
        security.cr           # Security hardening and compliance
        sudo.cr               # Sudo permissions management
        firewalld.cr          # Firewalld firewall management
        networkmanager.cr     # NetworkManager management
        package.cr            # Package and repository management

Adding New Modules
~~~~~~~~~~~~~~~~~~

1. Create a new file in ``src/shellmin/``
2. Define your CLI commands using the Clim macro system
3. Register the module in ``src/shellmin.cr``
4. Add documentation to the README

Testing
~~~~~~~

.. code-block:: bash

    crystal spec

Contributing
------------

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

Documentation
--------------

Shellmin uses Crystal's built-in documentation generator to create comprehensive API documentation.

**Generate Documentation:**

.. code-block:: bash

    crystal docs --output=docs --project-name="Shellmin" --project-version="0.1.0"

**View Documentation:**

.. code-block:: bash

    # Serve locally
    ./serve_docs.sh

    # Or open docs/index.html in your browser

The documentation includes:
- Complete API reference for all modules
- CLI command documentation
- Code examples and usage patterns
- Interactive search functionality

Development
-----------

**Prerequisites:**
- Crystal 1.17.1 or later
- SSH client configured for key-based authentication

**Setup Development Environment:**

.. code-block:: bash

    git clone <repository-url>
    cd shellmin
    shards install
    crystal build --release

**Run Tests:**

.. code-block:: bash

    crystal spec

**Check Code Formatting:**

.. code-block:: bash

    crystal tool format --check
    # Auto-fix formatting issues:
    crystal tool format

**Generate Documentation:**

.. code-block:: bash

    crystal docs --output=docs --project-name="Shellmin" --project-version="0.1.0"

**Serve Documentation Locally:**

.. code-block:: bash

    ./serve_docs.sh

**Code Quality:**
- ✅ **Zero linting issues** with Ameba code quality tool
- ✅ **Type-safe** with comprehensive nil safety and inference fixes
- ✅ **Thread-safe** connection management with atomic operations
- ✅ **Security-hardened** with multi-layered validation and audit logging
- ✅ **Performance-optimized** with pre-compiled patterns and efficient algorithms
- ✅ **Modern Crystal 1.x** features and idioms throughout
- ✅ **Comprehensive test coverage** with automated CI/CD pipeline

**Continuous Integration:**
- 🔄 Automated testing on every push with 51 test cases
- 📝 Code formatting validation with `crystal tool format`
- 📚 Automated documentation generation with `crystal docs`
- 🔒 Security scanning and vulnerability assessment
- ⚡ Performance monitoring and benchmarking
- 🚀 Multi-stage build pipeline with release optimization
- 📊 Coverage reporting and quality metrics

Contributing
------------

1. Fork the repository
2. Create a feature branch: ``git checkout -b feature/your-feature``
3. Write tests for new functionality
4. Ensure all tests pass: ``crystal spec``
5. Check code formatting: ``crystal tool format --check``
6. Generate updated documentation: ``crystal docs``
7. Submit a merge request

**Code Style:**
- Follow Crystal naming conventions
- Add documentation comments for public APIs
- Include error handling for all operations
- Write comprehensive tests
- Ensure CI pipeline passes

License
-------

This project is licensed under the MIT License - see the LICENSE file for details.

Authors
-------

- `Renich Bon Ciric <https://github.com/your-github-user>`_ - creator and maintainer

**Contributors:**
- Community contributions welcome!