New in Python 3.13: SQLite support in dbm

python sqlite dbm

2024-10-07

Python 3.13 was released today, and it would be easy to overlook this:

"dbm.sqlite3: An SQLite backend for dbm. (Contributed by Raymond Hettinger and Erlend E. Aasland in gh-100414.)"

I don't think that many Python engineers know about the dbm module, which is a shame because it is a sharp tool. It's a tool for reading and writing to string databases, which has been around in some form since I first came across it in Python 1.5 (I am indeed that old).

dbm is easy to overlook, maybe due to it's fairly pedestrian subtitle: "Interfaces to Unix databases", or maybe because in previous versions of Python, it only supported some unglamorous databases: NDBM, it's cousin GDBM, and more recently the Berkeley DB. These are robust, reliable databases, but they have rather fallen out of favour. Readers who are familiar with those databases will know that such systems were the progenitor to something with a much sexier label: the NoSQL Database.

dbm provides a very simple API, which will feel familiar.

import dbm

# Open database, creating it if necessary.
with dbm.open("mydatabase", "c") as db:

    # Record some values
    db["mykey"] = "My Value"
    db["anotherkey"] = "another value"

    # Access your database like a dictionary
    print(f"mykey: {db.get("mykey")}")
    print(db.get("doesntexist", "Default value"))

    # Beware, however, storing a non-string value will raise an exception
    db["thiswillfail"] = 4

The API is clean, tiny, and memorable.

Some will see the inability to store anything but strings as a deal-breaker (which is understandable), but I find that using pickle or shelve is enough for these scenarios. The occasional cast to int is also perfectly bearable.

Now, with the release of Python 3.13, we can very easily use SQLite3 as our backend:

with dbm.sqlite3.open("database.sqlite", "c") as db:
    db["mydata"] = "Lorem ipsum..."

When I write code, I like to use the simplest possible datastore. In practice, I usually start with a JSON file in combination with json.loads and json.dumps, which is 'enough' for a lot of problems.

Having the option to now use SQLite via dbm is a nice further step; once created, the SQLite database can be manipulated with standard SQLite tools which are available just about everywhere. This is very useful as you mature from "I just need somewhere to store my data state" to "I have more sophisticated needs".

The dbm documentation and the dbm.sqlite3 documentation is a good next stop to learn the detail.