tokei-api

https://tokei.kojix2.net/

tokei-api is a web application that provides an API to retrieve source code from a specified Git repository, execute the tokei command, and return the results in JSON format. It also provides a web interface to visualize the code statistics.

Features

API

Core API

• POST /api/analyses - Analyzes the source code of a specified Git repository and returns the results in JSON

  {
    "url": "https://github.com/kojix2/tokei-api.git"
  }
  

• GET /api/analyses?url=... - Retrieve cached analysis result by repository URL

• GET /api/analyses/:id - Retrieves a specific analysis result with detailed information

• GET /api/analyses/:id/languages - Retrieves language statistics for a specific analysis

• GET /api/analyses/:id/badges/:type - Retrieves badge data for a specific analysis

GitHub-specific API

• GET /api/github/:owner/:repo - Analyzes a GitHub repository directly
• GET /api/github/:owner/:repo/languages - Retrieves language statistics for a GitHub repository
• GET /api/github/:owner/:repo/badges/:type - Retrieves badge data for a GitHub repository

Badge API

• GET /badge/github/:owner/:repo/:type - Simplified URL for retrieving badge data in shields.io compatible format

Available badge types: lines, language, languages, ratio

• GET /api/badge/:type?url=... - Retrieves badge data in shields.io compatible format for any Git repository

Web Interface

• Input repository URL on the home page for analysis
• Visualize results with graphs and tables
• View past analysis results
• Direct access to GitHub repositories via /github/:owner/:repo
• Badge integration for READMEs

Installation

Requirements

• Crystal 1.15.1 or higher
• tokei command
• Git
• SQLite

Setup

1. Clone the repository

   git clone https://github.com/kojix2/tokei-api.git
   cd tokei-api
   

2. Install dependencies

   shards install
   

3. Set environment variables

   cp .env.example .env
   # Edit the .env file to set cache and other settings
   

   Key environment variables:

   • CACHE_DB_PATH: SQLite cache database path
   • BASE_URL: Canonical public URL used for generated absolute links, OG tags, and badge URLs. Set this in production, for example https://tokei.kojix2.net.
   • TEMP_DIR: Directory for temporary git clones
   • CLONE_TIMEOUT_SECONDS: Timeout for git clone operations (default: 30)
   • RETENTION_DAYS: Number of days to retain analysis data (default: 7)

4. Prepare the database

   # Tables will be created automatically on first application startup
   

5. Start the application

   crystal run src/tokei-api.cr
   

Usage

API Usage Examples

# Analyze a repository
curl -X POST -H "Content-Type: application/json" -d '{"url":"https://github.com/kojix2/tokei-api.git"}' http://localhost:3000/api/analyses

# Analyze a GitHub repository directly
curl http://localhost:3000/api/github/kojix2/tokei-api

# Retrieve language statistics
curl http://localhost:3000/api/github/kojix2/tokei-api/languages

# Get badge data
curl http://localhost:3000/badge/github/kojix2/tokei-api/lines

Badge Integration

You can add badges to your README to showcase your code statistics:

[![Lines of Code](https://tokei.kojix2.net/badge/github/username/repo/lines)](https://tokei.kojix2.net/github/username/repo)
[![Top Language](https://tokei.kojix2.net/badge/github/username/repo/language)](https://tokei.kojix2.net/github/username/repo)
[![Languages](https://tokei.kojix2.net/badge/github/username/repo/languages)](https://tokei.kojix2.net/github/username/repo)
[![Code to Comment](https://tokei.kojix2.net/badge/github/username/repo/ratio)](https://tokei.kojix2.net/github/username/repo)

These badges are dynamic and will automatically reflect the latest analysis of your repository.

Web Interface

The web interface consists of several pages:

• Home page: Access http://localhost:3000 to enter a repository URL for analysis
• Results page: View detailed analysis with graphs and statistics
• API page: Documentation for the REST API endpoints
• Badges page: Information on how to integrate badges into your projects

For GitHub repositories, you can use the direct access URL format:

http://localhost:3000/github/owner/repo

For example: http://localhost:3000/github/kojix2/tokei-api

Development

# Run in development mode (auto-reload)
crystal run src/tokei-api.cr

Database Schema

The application uses a SQLite cache database with the following schema:

CREATE TABLE analyses (
  id TEXT PRIMARY KEY,
  repo_url TEXT NOT NULL,
  analyzed_at TEXT NOT NULL,
  result TEXT NOT NULL,
  total_lines INTEGER,
  total_code INTEGER,
  total_comments INTEGER,
  total_blanks INTEGER,
  top_language TEXT,
  top_language_lines INTEGER,
  language_count INTEGER,
  code_comment_ratio FLOAT
);

Deployment

Deploying to Koyeb

1. Create a Koyeb account
2. Create a new application
3. Connect your GitHub repository
4. Set environment variables. Set BASE_URL to the canonical public URL for the service. By default, CACHE_DB_PATH uses /tmp/tokei-api/tokei-api.sqlite3, so the SQLite cache is discarded when the instance is replaced.
5. Run a single instance when using SQLite file storage.
6. Execute deployment

Running with Docker

Running in a single container

# Build Docker image
docker build -t tokei-api .

# Run container
docker run -p 3000:3000 --env-file .env tokei-api

Running with Docker Compose (recommended)

Docker Compose starts the application with an ephemeral SQLite cache inside the container.

# Build and start containers
docker compose up -d

# Check logs
docker compose logs -f

# Stop containers
docker compose down

Cache path

The default cache path is /tmp/tokei-api/tokei-api.sqlite3. This is intentionally ephemeral for deployments where recomputing cached analyses is acceptable:

CACHE_DB_PATH=/tmp/tokei-api/tokei-api.sqlite3

Technology Stack

• Language: Crystal
• Framework: Kemal
• Database: SQLite
• Frontend: Bootstrap, Chart.js
• Other: tokei, Git

License

MIT

Contributing

1. Fork it (https://github.com/kojix2/tokei-api/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Author

• kojix2 - creator and maintainer