starxen/driftindex

By starxen

•Updated about 1 month ago

Search engine

Image

Developer tools

Web servers

Databases & storage

7.6K

Overview Tags

starxen/driftindex repository overview

⁠DriftIndex

A lightweight search server, written in D, that's compatible (currently in a narrow scope) with the Elasticsearch REST API. Built for testing in lightweight runtime environments — a drop-in replacement for full Elasticsearch wherever only a narrow subset of its API is needed (e.g. integration tests or a local dev environment).

Status: alpha. The API is not stable, and only a narrow, deliberately chosen subset of Elasticsearch queries is supported. DriftIndex is not optimized for performance, data persistence durability, or security — those are explicit non-goals. The one goal that matters is mirroring Elasticsearch's basic behavior as closely as possible while keeping RAM usage as low as possible, so it stays cheap to run alongside whatever it's supporting. Don't treat this as a full-fledged production replacement.

⁠Quick reference

Maintained by:

StarXen GitLab Repository⁠

Where to get help: the Docker Community Slack⁠, Server Fault⁠, Unix & Linux⁠, or Stack Overflow⁠

Where to file issues: https://gitlab.com/starxen/driftindex/-/issues⁠

Supported architectures: (more info⁠)

Distro	Architectures
`debian` / `debian-trixie`	`amd64` `arm64` `arm32v7` `riscv64`
`fedora` / `fedora-44`	`amd64` `arm64` `ppc64le`
`ubuntu` / `ubuntu-resolute`	`amd64` `arm64` `arm32v7` `riscv64` `ppc64le`
`alpine` / `alpine-3-24`	`amd64` `arm64` `loong64`
`raspios` / `raspios-trixie`	`arm64` `arm32v6`

Build Process Source Codes: You can find the source code for the build process at: GitLab Repository⁠

Published Image Artifact Details:

GitLab Pipelines Log⁠ — Contains the full build history.
Builds are triggered by new Tags⁠ (e.g. v1.0.0).

⁠Supported Tags and Respective Dockerfile Links

Docker image tags are derived from the git tag with the leading v stripped (e.g. git tag v0.1.0.A → image tag 0.1.0.A-debian-trixie); .A/.B still marks alpha/beta pre-release maturity.

.deb (Debian: amd64/arm64/riscv64/arm32v7; Ubuntu: amd64/arm64/riscv64/arm32v7/ppc64le; RaspiOS: arm64/arm32v6), .rpm (amd64/arm64/ppc64le — Fedora doesn't publish a riscv64 or 32-bit ARM base image), and .apk (amd64/arm64/loong64) packages are built alongside the image on every tag and published on the release page on GitLab⁠.

⁠How it works

Data is kept in memory (source/store.d), with a snapshot written to disk (source/snapshot.d) at most every ~30s — only if data changed since the last write — and always on process shutdown, so a container restart doesn't require re-indexing.
A minimal, custom HTTP/1.1 server (source/httpserver.d) — one connection per request, no keep-alive/chunked support.
Text analysis (source/analysis.d) mirrors a narrow subset of Elasticsearch analyzers: standard tokenizer, English/Polish stopwords, a Porter2 stemmer (English) and a simplified rule-based stemmer (Polish, an approximation of analysis-stempel), plus character n-grams.
The query engine (source/query.d) only supports a deliberately narrow set of query shapes: bool (filter/must/ should + minimum_should_match), term, nested, multi_match, match_all.
Bulk update actions (source/routes.d) merge the given doc into the existing document (_source), matching Elasticsearch's partial-update semantics; doc_as_upsert: true inserts the doc as a new document when the target id doesn't exist yet, otherwise a missing id returns not_found.

⁠How to use this image

docker run --rm -p 9200:9200 starxen/driftindex

This command pulls the image and starts DriftIndex listening on port 9200. It speaks a narrow subset of the Elasticsearch REST API, so most curl/client calls that only touch the endpoints below will work unmodified:

curl http://localhost:9200/

⁠Persisting data across restarts

Mount a volume at the snapshot path (default /data/driftindex.snapshot.json, see Environment Variables⁠):

docker run --rm -p 9200:9200 -v driftindex-data:/data starxen/driftindex

⁠Running fully in memory

Set DRIFTINDEX_IN_MEMORY=true (or in_memory=true in the config file) to disable the snapshot entirely — no writability check at startup, no periodic/shutdown snapshot writes, and no restore. Useful for ephemeral environments (e.g. integration tests) where a persistent volume isn't available or wanted. All data is lost when the process exits.

docker run --rm -p 9200:9200 -e DRIFTINDEX_IN_MEMORY=true starxen/driftindex

⁠Endpoints

Method	Path	Feature
GET	`/`	Cluster info (name, version, tagline)
PUT	`/{index}`	Create index (settings/mappings stored informationally)
DELETE	`/{index}`	Delete index
HEAD / GET	`/{index}`	Check whether an index exists
POST	`/_bulk`	Bulk index/create/update/delete (no default index given)
POST	`/{index}/_bulk`	Bulk index/create/update/delete within an index
POST	`/_refresh`	Refresh (no-op, returns acknowledgement)
POST	`/{index}/_refresh`	Refresh an index (no-op)
GET / POST	`/{index}/_search`	Search with pagination (`from`/`size`) and the query DSL engine
GET / POST	`/{index}/_count`	Count documents matching a query
GET	`/{index}/_doc/{id}`	Fetch a single document by id
DELETE	`/{index}/_doc/{id}`	Delete a single document by id

⁠Supported query DSL

Clause	Support
`match_all`	✅
`term`	✅ (only `visible`, `category.visible` fields)
`bool`	✅ (`filter`, `must`, `should`, `minimum_should_match`)
`nested`	✅ (`category`, `*.translationContents`)
`multi_match`	✅ (only nested inside `nested`; `.english`, `.polish`, `.ngram` fields)
others (`range`, `wildcard`, `fuzzy`, aggregations, etc.)	❌ not supported

⁠Persistence

Feature	Status
Snapshot to disk every ~30s (only if changed)	✅
Snapshot on graceful shutdown (SIGTERM/SIGINT)	✅
State restore from snapshot on startup	✅
Fully in-memory mode (`DRIFTINDEX_IN_MEMORY`)	✅
Durable transactional indexes / WAL	❌

⁠Running

dub run

⁠Unit tests

dub test --compiler=ldc2

dmd is not supported as a build/test compiler on all platforms (e.g. it segfaults on some Apple Silicon setups) — use ldc2 via --compiler=ldc2 instead.

⁠Environment Variables

Variable	Default	Description
`DRIFTINDEX_PORT`	`9200`	HTTP port
`DRIFTINDEX_BIND_ADDRESS`	`0.0.0.0`	Address to bind the HTTP server to
`DRIFTINDEX_SNAPSHOT_PATH`	`/data/driftindex.snapshot.json`	Snapshot file path
`DRIFTINDEX_IN_MEMORY`	`false`	Disable the snapshot entirely (see below)
`DRIFTINDEX_CONFIG_PATH`	`/etc/driftindex/config.conf`	Path to the config file (see below)

The Docker image also exposes read-only build metadata:

Variable	Description
`DRIFTINDEX_VERSION`	Installed version of DriftIndex
`IMAGE_BASE_DISTRO`	Base OS distribution: DEBIAN, FEDORA, UBUNTU, ALPINE, or RASPIOS
`IMAGE_BASE_VERSION`	Base OS version, such as 13, 44, 26.04, or 3.24
`IMAGE_BASE_VERSION_NAME`	Base OS version name, such as TRIXIE, 44, RESOLUTE, or 3.24

⁠Configuration file

For Linux service installs (e.g. a .deb package running under systemd), settings can also be provided via a config file at /etc/driftindex/config.conf (one key=value per line, # for comments):

port=9200
bind_address=0.0.0.0
snapshot_path=/data/driftindex.snapshot.json
in_memory=false

If both a config file and the corresponding environment variable are set, the environment variable wins — this keeps container deployments (which typically have no config file) working unchanged.

⁠More Information

⁠Backers

laserovsky.net⁠ for hosting GitLab Runners used for building and publishing Docker images.

⁠License

MIT — see LICENSE.md⁠.

Tag summary

Recent tags

Content type

Image

Digest

sha256:adf278318…

Size

30.4 MB

Last updated

about 1 month ago

docker pull starxen/driftindex