Util_Engine
This page documents the current utility surface that is backed by the source tree in this repository. For this module, the hand-written source of truth is:
include/util/
tests/unit/util/
cmakes/src/UTIL/UTILsrc.cmake
Generated API pages under API Reference remain useful for symbol lookup, but this guide is the recommended starting point for the maintained utility surface and the intended consumption boundaries.
Public Consumption Units
The current CMake targets split the utility layer into four consumer-facing units:
PDJE_UTIL base interface target for the utility include tree and the umbrella header path
PDJE_UTIL_DB aggregate database target that wires the SQLite, RocksDB, and Annoy utility backends
PDJE_UTIL_IMAGE_WEBP image/WebP helper target for the generic WebP writer path
PDJE_UTIL_IMAGE_WAVEFORM waveform image generation target that also compiles the STFT and backend loader implementation files
In other words, the utility layer is not roadmap-only. It is active code with headers, tests, and CMake consumption points in the current tree.
Header Boundaries
The utility surface is intentionally split between one umbrella header and a few direct-include headers.
Umbrella header:
#include "util/PDJE_Util.hpp"
This re-exports the currently maintained:
PDJE_UTIL::common::{StatusCode, Status, Result<T>, Result<void>}
database wrapper types and backend concepts
db::Database<Backend> as the key-value convenience alias
PDJE_UTIL::function::{clamp, slugify, EvalOptions, CacheContext}
Direct include headers still required for APIs that are not re-exported by the umbrella header:
util/function/image/WebpWriter.hpp
util/function/image/WaveformWebp.hpp
util/function/stft/STFT_Parallel.hpp
util/common/BackendLoader/OpenCL_Loader.hpp
util/common/BackendLoader/PDJE_Parallel_Runtime_Loader.hpp
Common Status And Result Types
The utility module uses one error transport pattern everywhere: StatusCode, Status, and Result<T>.
-
namespace PDJE_UTIL
-
namespace PDJE_UTIL
Current facts from the headers and tests:
StatusCode is the shared error taxonomy for utility APIs
Status stores code plus a human-readable message
Status::ok() is true only when code == StatusCode::ok
Result<T> stores either a value plus ok status, or a failure status
Result<void> keeps the same success/failure contract without a value
The header set and util status tests currently define and reference codes such as:
invalid_argument
not_found
type_mismatch
unsupported
io_error
closed
backend_error
out_of_range
internal_error
Typical usage looks like this:
#include "util/PDJE_Util.hpp"
using namespace PDJE_UTIL;
auto clamped = function::clamp(
{ .value = 2.5, .min_value = 0.0, .max_value = 1.0 });
if (!clamped.ok()) {
std::cerr << clamped.status().message << std::endl;
return;
}
std::cout << clamped.value() << std::endl;
Database Wrappers
The current utility DB layer is organized by access pattern, not by one global database class.
Wrapper split:
Wrapper |
Typical backend |
Current role |
|---|---|---|
db::Database<Backend> |
key-value backend |
convenience alias for db::keyvalue::KeyValueDatabase<Backend> |
db::keyvalue::KeyValueDatabase<Backend> |
db::backends::RocksDbBackend |
text/blob lookup, overwrite, erase, prefix scan |
db::relational::RelationalDatabase<Backend> |
db::backends::SqliteBackend |
SQL execution, queries, transactions |
db::nearest::NearestNeighborIndex<Backend> |
db::backends::AnnoyBackend |
embedding storage and nearest-neighbor search |
All three wrapper families follow the same high-level lifecycle:
create(config)
open(config)
use the opened object
close()
optionally destroy(config)
The shipped tests validate concrete behavior rather than only type presence:
RocksDB key-value: text writes, byte writes, overwrite, prefix list_keys(), erase, read-only rejection, and type mismatch reporting
SQLite relational: execute/query, parameter binding, blob transport, transaction rollback and commit, and read-only rejection
Annoy nearest-neighbor: upsert, get, search, erase, reopen/persistence, read-only rejection, and invalid embedding dimension handling
RocksDB key-value example:
#include "util/db/backends/RocksDbBackend.hpp"
#include "util/db/keyvalue/Database.hpp"
using Db = PDJE_UTIL::db::keyvalue::KeyValueDatabase<
PDJE_UTIL::db::backends::RocksDbBackend>;
PDJE_UTIL::db::backends::RocksDbConfig cfg{
.path = "tmp/example-rocks",
.open_options = { .create_if_missing = true }
};
auto opened = Db::open(cfg);
if (!opened.ok()) {
return;
}
auto db = std::move(opened.value());
(void)db.put_text("artist", "RLIOP913");
auto artist = db.get_text("artist");
if (artist.ok()) {
std::cout << artist.value() << std::endl;
}
SQLite relational example:
#include "util/db/backends/SqliteBackend.hpp"
#include "util/db/relational/Database.hpp"
using Db = PDJE_UTIL::db::relational::RelationalDatabase<
PDJE_UTIL::db::backends::SqliteBackend>;
PDJE_UTIL::db::backends::SqliteConfig cfg{
.path = "tmp/example.sqlite3",
.open_options = { .create_if_missing = true }
};
auto opened = Db::open(cfg);
if (!opened.ok()) {
return;
}
auto db = std::move(opened.value());
(void)db.execute(
"CREATE TABLE IF NOT EXISTS notes("
"id INTEGER PRIMARY KEY, lane TEXT NOT NULL);");
(void)db.begin_transaction();
(void)db.execute("INSERT INTO notes(id, lane) VALUES(?1, ?2);",
{ PDJE_UTIL::db::relational::Value{ std::int64_t{1} },
PDJE_UTIL::db::relational::Value{ PDJE_UTIL::db::Text{"left"} } });
(void)db.commit();
Annoy nearest-neighbor follows the same open/use/close shape, but its payloads are db::nearest::Item, SearchHit, and SearchOptions rather than text or SQL values.
Function Helpers
The function layer currently splits into small inline helpers plus larger image and signal-processing utilities.
Scalar And Text
PDJE_UTIL::function::clamp and slugify are the smallest maintained public helpers.
clamp(ClampArgs) clamps a floating-point value and returns Result<double>
slugify(SlugifyArgs, EvalOptions) normalizes text into a delimiter-based slug and rejects alphanumeric separators
EvalOptions currently carries an optional CacheContext*
CacheContext exists as a lightweight movable handle; the shipped inline helpers shown here do not require a populated cache context
Example:
#include "util/PDJE_Util.hpp"
auto slug = PDJE_UTIL::function::slugify(
{ .input = "Project DJ Engine", .separator = '-' });
if (slug.ok()) {
std::cout << slug.value() << std::endl; // "project-dj-engine"
}
Image Helpers
The image surface is split into a generic WebP writer path and a waveform generator path.
Generic WebP writer:
#include "util/function/image/WebpWriter.hpp"
Current public types in that header:
RasterPixelFormat
RasterImageView
EncodeWebpArgs
WriteWebpArgs
encode_webp(…)
write_webp(…)
Current source-backed behavior:
validates image dimensions, stride, and buffer layout
accepts gray8, gray_alpha8, rgb8, and rgba8 raster views
returns encoded WebP bytes from encode_webp(…)
writes encoded bytes to disk with write_webp(…)
validates compression_level in the [-1, 9] range before encoding
Minimal example:
#include "util/function/image/WebpWriter.hpp"
const std::vector<std::uint8_t> rgba{ 255, 0, 0, 255 };
auto encoded = PDJE_UTIL::function::image::encode_webp(
{ .image =
{
.pixels = rgba,
.width = 1,
.height = 1,
.stride = 0,
.pixel_format = PDJE_UTIL::function::image::RasterPixelFormat::rgba8,
} });
Waveform WebP generation:
#include "util/function/image/WaveformWebp.hpp"
Current public types in that header:
EncodedWebpBytes
ChannelWaveformWebps
WaveformWebpBatch
EncodeWaveformWebpArgs
encode_waveform_webps(…)
Current source-backed behavior and tests cover:
argument validation
interleaved PCM channel splitting
min/max aggregation per output column
incomplete chunk padding
transparent background output
deterministic output across single-worker, multi-worker, and auto-worker runs
Example:
#include "util/function/image/WaveformWebp.hpp"
std::vector<float> pcm{
1.0f, -1.0f, 0.25f, -0.25f,
-0.5f, 0.5f, 0.75f, -0.75f
};
auto waveform = PDJE_UTIL::function::image::encode_waveform_webps(
{ .pcm = pcm,
.channel_count = 2,
.y_pixels = 64,
.pcm_per_pixel = 256,
.x_pixels_per_image = 512,
.compression_level = 1,
.worker_thread_count = 0 });
if (!waveform.ok()) {
return;
}
const auto &batch = waveform.value();
// batch[channel][image] -> encoded WebP bytes
Signal / STFT Helpers
The maintained STFT surface lives outside util/PDJE_Util.hpp and requires a direct include:
#include "util/function/stft/STFT_Parallel.hpp"
The current public interfaces are:
PDJE_PARALLEL::BACKEND_T
PDJE_PARALLEL::Backend
PDJE_PARALLEL::STFT
PDJE_PARALLEL::WINDOW_LIST
PDJE_PARALLEL::POST_PROCESS
The current runtime model from the source tree is:
STFT always constructs a serial backend
STFT calls Backend::LoadBackend() and then reads the selected BACKEND_T through PrintBackendType()
if OpenCL loads successfully, STFT tries to construct OPENCL_STFT
if OpenCL startup or execution fails, STFT falls back to SERIAL
the current backend loader switches between OPENCL and SERIAL
BACKEND_T::METAL exists in the enum, but the shipped runtime loader does not currently select it
POST_PROCESS is the main post-chain contract:
to_bin
toPower
mel_scale
to_db
normalize_min_max
to_rgb
POST_PROCESS::check_values() currently applies the chain rules used by the implementation:
to_rgb implies normalize_min_max and mel_scale
mel_scale implies to_bin and toPower
The util tests currently validate:
serial backend stability across repeated calls
switching across cached FFT sizes
mel filter bank generation
mel, mel+db, and RGB reduction paths
OpenCL runtime shim caching and backend selection
CMRC resource packaging for STFT_MAIN.cl
Example:
#include "util/function/stft/STFT_Parallel.hpp"
std::vector<float> pcm(512, 0.0f);
PDJE_PARALLEL::STFT stft;
PDJE_PARALLEL::POST_PROCESS post;
post.mel_scale = true;
post.to_db = true;
post.check_values();
auto [real_out, imag_out] = stft.calculate(
pcm,
PDJE_PARALLEL::WINDOW_LIST::HANNING,
10,
0.5f,
post);
If you need to probe runtime availability directly, the loader entrypoint lives in:
#include "util/common/BackendLoader/OpenCL_Loader.hpp"
And the capability query is:
bool ready = PDJE_PARALLEL::EnsureOpenCLRuntimeLoaded();
AI Namespace
PDJE_UTIL::ai currently exists as a placeholder namespace only. The current header set does not define public AI utility functions inside it.