Skip to content

Latest commit

 

History

History
130 lines (81 loc) · 5.84 KB

README.md

File metadata and controls

130 lines (81 loc) · 5.84 KB
Raw

Taoensso open source
API | Wiki | Latest releases | Get support

Nippy

The fastest serialization library for Clojure

Clojure's rich data types are awesome. And its reader allows you to take your data just about anywhere. But the reader can be painfully slow when you've got a lot of data to crunch (like when you're serializing to a database).

Nippy is a mature, high-performance drop-in alternative to the reader.

It is used at scale by Carmine, Faraday, PigPen, Onyx, XTDB, Datalevin, and others.

Latest release/s

Main tests Graal tests

See here for earlier releases.

Why Nippy?

  • Small, simple pure-Clojure library
  • Terrific performance: the best for Clojure that I'm aware of
  • Comprehensive support for all standard data types
  • Easily extendable to custom data types
  • Robust test suite incl. coverage of every supported type
  • Mature and widely used in production for 12+ years
  • Optional auto fallback to Java Serializable for safe types
  • Optional auto fallback to Clojure Reader (including tagged literals)
  • Optional smart compression with LZ4 or Zstandard
  • Optional encryption with AES128
  • Tools for easy + robust integration into 3rd-party libraries, etc.
  • Powerful thaw transducer for flexible data inspection and transformation

Quick example

Nippy's super easy to use:

(require '[taoensso.nippy :as nippy])

;; Freeze any Clojure value
(nippy/freeze <my-value>) ; => Serialized byte[]

;; Thaw the byte[] to get back the original value:
(nippy/thaw (nippy/freeze <my-value>)) ; => <my-value>

See the wiki for more.

Operational considerations

Data longevity

Nippy is widely used to store long-lived data and promises (as always) that data serialized today should be readable by all future versions of Nippy.

But please note that the converse is not generally true:

  • Nippy vX should be able to read all data from Nippy vY<=X (backwards compatibility)
  • Nippy vX may/not be able to read all data from Nippy vY>X (forwards compatibility)

Rolling updates and rollback

From time to time, Nippy may introduce:

  • Support for serializing new types
  • Optimizations to the serialization of pre-existing types

To help ease rolling updates and to better support rollback, Nippy (since version v3.4) will always introduce such changes over two version releases:

  • Release 1: to add read support for the new types
  • Release 2: to add write support for the new types

Starting from v3.4, Nippy's release notes will always clearly indicate if a particular update sequence is recommended.

Stability of byte output

It has never been an objective of Nippy to offer predictable byte output, and I'd generally recommend against depending on specific byte output.

However, I know that a small minority of users do have specialized needs in this area.

So starting with Nippy v3.4, Nippy's release notes will always clearly indicate if any changes to byte output are expected.

Performance

Since its earliest versions, Nippy has consistently been the fastest serialization library for Clojure that I'm aware of. Latest results:

benchmarks-png

PRs welcome to include other alternatives in the benchmark suite!

Documentation

Funding

You can help support continued work on this project, thank you!! 🙏

License

Copyright © 2012-2024 Peter Taoussanis.
Licensed under EPL 1.0 (same as Clojure).