ETDAW - Extrinsic Time-Domain Audio-Codec Workbench

This project contains an abstract framework for brute-forcing the encoding of an audio file (extrinsically) for time-domain based audio-codecs.

The project can already be used as a high-quality encoder for the 1.2-bit SEA audio format with reasonable speed (or even 2.2-bit/3.2-bit SEA and QOA at slow speeds). This could already be useful for embedded projects, where compilation time is irrelevant in contrast to cpu speed and flash usage.

In addition to being used for encoding existing codecs with high quality, the framework is also used for developing new codecs. The generic, optimized brute-force encoder allows developing new codecs only by defining the decoding function.

Installation

Compiling the project requires the crystal compiler.
Once the project was cloned, the target to compile can be chosen using shards build.

git clone https://gitlab.com/BlobCodes/etdaw
cd etdaw
shards build --release --no-debug -Dexecution_context -Dpreview_mt

For other use-cases, see the development section.

Usage (SEA Encoding)

TLDR:

# 1. Convert audio file to raw PCM s16le

# Extract a single channel (here: Front Left)
ffmpeg -i '/some/audio/file.opus' -af "pan=mono|c0=FL" -f s16le -c:a pcm_s16le audio.raw
# Or:
# Convert to mono
ffmpeg -i '/some/audio/file.opus' -ac 1 -f s16le -c:a pcm_s16le audio.raw

# 2. Encode

bin/etdaw-enc encode -m rms -f sea1.2b -q 20 audio.raw

# 3. Compress to .sea format

bin/sea-pack sea1.2b audio-enc.raw audio.sea

# (4. Instantly play the decoded audio file, generated during the encoding)

# Pack PCM back into a WAV file
ffmpeg -f s16le -ar 44100 -ac 1 -i audio-dec.raw audio-dev.wav
# Play PCM file directly
ffplay -f s16le -ar 44100 -ch_layout mono audio-dec.raw

The SEA encoder only accepts a single raw mono s16le PCM file as its argument and creates three new files right next to it. For a file audio.raw, the following files are created:

• audio-enc.raw is the stream of raw SEA codepoints. The data returned would be usable as a single, endless SEA chunk without any headers or metadata (only scalefactors and residuals).

  All fields which are normally bit-packed are instead returned as a stream of 8-bit codepoints, where every 20th codepoint is in the 5-bit range (0-31, scalefactors and residual) and all other codepoints are in the 1-bit range (0-1, only residual).

  Additional work would be required to convert this file into a valid SEA file. However, the command etdaw-enc decode audio-enc.raw can be used to decode this file format.

• audio-dec.raw is the decoded raw mono s16le audio stream which a decoder would have produced.

The encoded file can be packed into the .sea file format using the bin/sea-pack script:

bin/sea-pack sea1.2b audio-enc.raw audio.sea

Recommended Encoding Options

Using the RMS metric is generally recommended for most use-cases (--metric RMS). It results in the quickest encodings, and is the most consistent accross the frequency spectrum.

For encoding in the SEA format, the lookahead count should always at least be equal to the scalefactor distance (20). For 1.2 Bit SEA, this is also the recommended encoding setting (--format sea1.2b --quality 20).

For 2.2 Bit SEA, using a lookahead of 20 would result in the encoder taking multiple HOURS to encode a few minutes of audio since it needs to compute 2^48 possible combinations for every single sample. If you want the maximum quality or the sample is very short, these settings can still be used (--format sea2.2b --quality 20).

If you want to encode 2.2 Bit SEA in a reasonable timeframe, you can use the chunked format which handles the scalefactor separately (--format chunked-sea2.2b --quality 4).

3.2 Bit SEA and QOA are only available in the chunked variant (--format chunked-sea3.2b --quality 4, --format chunked-qoa --quality 4).

Development

The project is primarily intended for developing and evaluating low-bitrate time-domain codecs, which mandates a modular structure.

Encoding mechanisms, quality evaluation and audio decoding are completely separated from each other.

Audio codecs are only defined by their decoding function, no encoder is required. Audio codecs receive a stream of codepoints (0-255), whose value range is defined by the codec itself and can change over time.

For every codepoint, an audio decoder returns an audio sample, which is compared by a comparator. A comparator is a cumulative quality evaluation function, which tells the encoder how well a stream of decoded samples matches the reference samples.

The encoder uses an optimized brute-force algorithm for finding the optimal codepoints for encoding a stream of samples. To improve quality, the encoder uses a lookahead. For every sample, the optimal combination in a window of codepoints is calculated, moving the window one sample at a time.

Additional encoding mechanisms, quality-improving comparators or decoding algorithms are always welcome additions.

Contributing

1. Fork it (https://gitlab.com/BlobCodes/etdaw/-/forks/new)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• David Keller - creator and maintainer