This package provides a Gerbil LevelDB driver using libleveldb.

You need to have libleveldb installed in your system.

$ sudo apt install libleveldb-dev

Install homebrew (if not already done):

$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

then install leveldb:

$ brew install leveldb

On ARM64 architectures (M1...) Homebrew installs its bits in /opt/homebrew. For some reason leveldb does not provide support for pkg-config. macOS users on ARM need to export CPPFLAGS and LDFLAGS for the compiler to find leveldb headers and library.

$ arch          ## arm64 
$ export CPPFLAGS=-I/opt/homebrew/include
$ export LDFLAGS=-L/opt/homebrew/lib

To install the package in your $GERBIL_PATH (~/.gerbil by default):

$ gerbil pkg install github.com/mighty-gerbils/gerbil-leveldb

Build the library:

$ ./build.ss
# or
$ make build

Run tests:

$ make test

The functional test (test/functional-test.ss) exercises all features including:

• Version query
• Options with all parameters (including max-file-size: and env:)
• Environment creation
• Basic put/get/delete operations
• Write batches and batch append
• Snapshots with point-in-time reads
• Iterators and in-leveldb iteration
• Database properties
• Approximate size queries
• Compaction, repair, and destroy

To use bindings from this package:

(import :clan/db/leveldb)

(import :clan/db/leveldb)

;; Open a database (creates if it doesn't exist)
(def db (leveldb-open "/path/to/mydb"))

;; Put a key-value pair
(leveldb-put db "mykey" "myvalue")

;; Get a value (returns u8vector or #f if not found)
(def val (leveldb-get db "mykey"))
(when val (displayln (bytes->string val)))

;; Delete a key
(leveldb-delete db "mykey")

;; Close the database
(leveldb-close db)

(leveldb-open name [opts]) -> leveldb

Opens a LevelDB database at the specified path. Creates the database if it doesn't exist (by default).

• name: Path to the database directory (string)
• opts: Optional options object created with leveldb-options

(leveldb-close db) -> void

Closes the database and releases resources.

(leveldb-put db key val [opts]) -> void

Stores a key-value pair in the database.

• key: Key as string or u8vector
• val: Value as string or u8vector
• opts: Optional write options

(leveldb-get db key [opts]) -> u8vector | #f

Retrieves a value by key. Returns #f if the key doesn't exist.

(leveldb-delete db key [opts]) -> void

Deletes a key from the database.

(leveldb-key? db key [opts]) -> boolean

Checks if a key exists in the database.

Write batches allow atomic updates of multiple keys.

(leveldb-writebatch) -> writebatch

Creates a new write batch.

(leveldb-writebatch-put batch key val) -> void

Adds a put operation to the batch.

(leveldb-writebatch-delete batch key) -> void

Adds a delete operation to the batch.

(leveldb-writebatch-append dest src) -> void

Appends all operations from src batch to dest batch. Useful for composing batches.

(def wb1 (leveldb-writebatch))
(def wb2 (leveldb-writebatch))
(leveldb-writebatch-put wb1 "a" "1")
(leveldb-writebatch-put wb2 "b" "2")
(leveldb-writebatch-append wb1 wb2)  ;; wb1 now has both operations
(leveldb-write db wb1)

(leveldb-writebatch-clear batch) -> void

Clears all operations from a write batch.

(leveldb-write db batch [opts]) -> void

Atomically applies all operations in the batch to the database.

Snapshots provide a consistent read-only view of the database at a point in time.

(leveldb-snapshot db) -> snapshot

Creates a snapshot of the current database state.

(leveldb-snapshot-release db snapshot) -> void

Releases a snapshot when no longer needed.

;; Create snapshot
(leveldb-put db "key" "value1")
(def snap (leveldb-snapshot db))

;; Modify after snapshot
(leveldb-put db "key" "value2")

;; Read current value
(leveldb-get db "key")  ;; => "value2"

;; Read from snapshot
(def snap-opts (leveldb-read-options snapshot: snap))
(leveldb-get db "key" snap-opts)  ;; => "value1"

;; Release when done
(leveldb-snapshot-release db snap)

Iterators allow sequential scanning of key-value pairs.

(leveldb-iterator db [opts]) -> iterator

Creates an iterator for the database.

(leveldb-iterator-close iter) -> void

Closes the iterator.

(leveldb-iterator-valid? iter) -> boolean

Returns #t if the iterator is positioned at a valid entry.

(leveldb-iterator-seek-first iter) -> void
(leveldb-iterator-seek-last iter) -> void

Positions the iterator at the first or last entry.

(leveldb-iterator-seek iter key) -> void

Positions the iterator at the first entry with key >= key.

(leveldb-iterator-next iter) -> void
(leveldb-iterator-prev iter) -> void

Moves to the next or previous entry.

(leveldb-iterator-key iter) -> u8vector
(leveldb-iterator-value iter) -> u8vector

Returns the key or value at the current position.

(in-leveldb db [start] [limit]) -> iterator
(in-leveldb-keys db [start] [limit]) -> iterator

Creates iterators compatible with Gerbil's for loops.

;; Iterate over all key-value pairs
(for ((values key val) (in-leveldb db))
  (displayln (bytes->string key) " = " (bytes->string val)))

;; Iterate over keys only
(for (key (in-leveldb-keys db))
  (displayln (bytes->string key)))

;; With range
(for ((values key val) (in-leveldb db "a" "z"))
  ...)

(leveldb-property db name) -> string | #f

Returns a database property value. Available properties include:

• "leveldb.stats" - Multi-line statistics
• "leveldb.sstables" - SSTable information
• "leveldb.approximate-memory-usage" - Memory usage estimate

(displayln (leveldb-property db "leveldb.stats"))

(leveldb-approximate-size db start-key end-key) -> integer

Returns the approximate size in bytes of the data in the given key range.

(leveldb-approximate-size db "a" "z")  ;; size of keys from "a" to "z"

(leveldb-compact-range db start-key end-key) -> void

Compacts the underlying storage for the given key range.

(leveldb-destroy-db name [opts]) -> void

Destroys (deletes) a database and all its files.

(leveldb-repair-db name [opts]) -> void

Attempts to repair a corrupted database.

(leveldb-options keyword-args ...) -> options

Creates database options. Keyword arguments:

(def opts (leveldb-options
            write-buffer-size: 67108864    ;; 64MB
            max-open-files: 1000
            bloom-filter-bits: 10
            max-file-size: 2097152))       ;; 2MB
(def db (leveldb-open "/path/to/db" opts))

(leveldb-default-options) -> options

Returns the default options (cached, reusable).

(leveldb-read-options keyword-args ...) -> read-options

Creates read options. Keyword arguments:

(leveldb-write-options keyword-args ...) -> write-options

Creates write options. Keyword arguments:

(leveldb-default-env) -> env

Returns the default environment. Can be passed to leveldb-options with env:.

(leveldb-env-test-directory env) -> string

Returns a path to a temporary test directory.

(leveldb-version) -> (values major minor)

Returns the LevelDB library version as two values.

(defvalues (major minor) (leveldb-version))
(displayln "LevelDB " major "." minor)  ;; e.g., "LevelDB 1.23"

All operations may raise LevelDBError on failure.

(leveldb-error? obj) -> boolean

Predicate for LevelDB errors.

(with-catch
  (lambda (e)
    (when (leveldb-error? e)
      (displayln "LevelDB error: " (error-message e))))
  (lambda ()
    (leveldb-open "/nonexistent/path")))

The following LevelDB C API features are not exposed due to FFI callback complexity:

• Custom comparators (leveldb_comparator_create)
• Custom filter policies (leveldb_filterpolicy_create)
• Write batch iteration (leveldb_writebatch_iterate)
• Custom environments and loggers

For custom key ordering, design your keys to sort correctly with the default byte-ordering comparator (e.g., use zero-padded numbers).

© 2017-2023 The Gerbil Core Team and contributors; License: LGPLv2.1 and Apache 2.0

Originally written by vyzo.