3

I would like to do a key -> value lookup for data that may not fit into memory (or simply avoid the overhead of having to load it an keep it in memory).

Does Emacs natively support any kind of database that can be used for fast lookups (like a hashtable) without having to first load everything into memory?

As an alternative I could hash the value myself, split it into buckets - then load a JSON file associated with that bucket (for example), but this seems like a weak solution to something that might already exist.

Drew
  • 75,699
  • 9
  • 109
  • 225
ideasman42
  • 8,375
  • 1
  • 28
  • 105
  • What did you find on melpa? – nega Dec 06 '22 at 15:38
  • 1
    I came across this very old [db](https://melpa.org/#/db) package on melpa. Also, starting from [Emacs 29](https://blog.phundrak.com/emacs-29-what-can-we-expect/?ref=upstract.com), SQLite will be included so a k/v store could be built on top of that. – g-gundam Dec 06 '22 at 17:41
  • Querying an external database seems the most practical given the goal of eliminating Emacs overhead. – lawlist Dec 06 '22 at 17:53
  • 1
    @g-gundam I wasn't aware of sqlite support, this could be made into an answer. – ideasman42 Dec 06 '22 at 21:53

1 Answers1

3

Soon... (maybe early 2023)

Starting from version 29, native SQLite support will be included in Emacs.

A key/value store could be built on top of this. There is already a library on MELPA called db which provides an API for key/value stores in Elisp. Its default store uses JSON files, but it was designed so that different backing stores could be used. For example, db-pg provides a PostgreSQL-backed key/value store. Perhaps a db-sqlite could be written. It might be time to breathe life back into db.

g-gundam
  • 1,096
  • 1
  • 3
  • 13