Fork me on GitHub

uPickle 4.0.2


uPickle (pronounced micro-pickle) is a lightweight JSON and binary (MessagePack) serialization library for Scala. It's key features are:

This documentation is meant as a thorough reference to this library. For a hands-on introduction, take a look at the following blog post:

Getting Started


"com.lihaoyi" %% "upickle" % "4.0.2" // SBT
ivy"com.lihaoyi::upickle:4.0.2" // Mill

And then you can immediately start writing and reading common Scala objects to strings: 

import upickle.default._

write(1)                          ==> "1"

write(Seq(1, 2, 3))               ==> "[1,2,3]"

read[Seq[Int]]("[1,2,3]")         ==> List(1, 2, 3)

write((1, "omg", true))           ==> """[1,"omg",true]"""

read[(Int, String, Boolean)]("""[1,"omg",true]""") ==> (1, "omg", true)

Or to compact byte arrays, using the MessagePack format: 

import upickle.default._

writeBinary(1)                          ==> Array(1)

writeBinary(Seq(1, 2, 3))               ==> Array(0x93.toByte, 1, 2, 3)

readBinary[Seq[Int]](Array[Byte](0x93.toByte, 1, 2, 3))  ==> List(1, 2, 3)

val serializedTuple = Array[Byte](0x93.toByte, 1, 0xa3.toByte, 111, 109, 103, 0xc3.toByte)

writeBinary((1, "omg", true))           ==> serializedTuple

readBinary[(Int, String, Boolean)](serializedTuple) ==> (1, "omg", true)

ScalaJS

For ScalaJS applications, use this dependencies instead:

"com.lihaoyi" %%% "upickle" % "4.0.2" // SBT
ivy"com.lihaoyi::upickle::4.0.2" // Mill

Scala Versions

uPickle supports Scala 2.12, 2.13 and 3.1+

Basics


Builtins

This is a non-comprehensive list of what the most commonly-used types pickle to using uPickle. To begin, let's import upickle 

import upickle.default._

Primitives

Booleans are serialized as JSON booleans 

write(true: Boolean)              ==> "true"
write(false: Boolean)             ==> "false"

Numbers are serialized as JSON numbers 

write(12: Int)                    ==> "12"
write(12: Short)                  ==> "12"
write(12: Byte)                   ==> "12"
write(Int.MaxValue)               ==> "2147483647"
write(Int.MinValue)               ==> "-2147483648"
write(12.5f: Float)               ==> "12.5"
write(12.5: Double)               ==> "12.5"

Except for Longs, which too large for Javascript. These are serialized as JSON Strings, keeping the interchange format compatible with the browser's own JSON parser, which provides the best performance in Scala.js 

write(12: Long)                   ==> "12"
write(4000000000000L: Long)       ==> "4000000000000"
// large longs are written as strings, to avoid floating point rounding
write(9223372036854775807L: Long) ==> "\"9223372036854775807\""

Special values of Doubles and Floats are also serialized as Strings 

write(1.0/0: Double)              ==> "\"Infinity\""
write(Float.PositiveInfinity)     ==> "\"Infinity\""
write(Float.NegativeInfinity)     ==> "\"-Infinity\""

Both Chars and Strings are serialized as Strings 

write('o')                        ==> "\"o\""
write("omg")                      ==> "\"omg\""

Collections

Arrays and most immutable collections are serialized as JSON lists 

write(Array.empty[Int])           ==> "[]"
write(Array(1, 2, 3))             ==> "[1,2,3]"

// You can pass in an `indent` parameter to format it nicely
write(Array.empty[Int], indent = 4)  ==> "[]"
write(Array(1, 2, 3), indent = 4)  ==>
  """[
    |    1,
    |    2,
    |    3
    |]""".stripMargin

write(Seq(1, 2, 3))               ==> "[1,2,3]"
write(Vector(1, 2, 3))            ==> "[1,2,3]"
write(List(1, 2, 3))              ==> "[1,2,3]"
import collection.immutable.SortedSet
write(SortedSet(1, 2, 3))         ==> "[1,2,3]"

Options are serialized as nullable values 

write(Some(1))                    ==> "1"
write(None)                       ==> "null"

Eithers are serialized as JSON lists with 2 elements, with the first element a number indicating Left (0) or Right (1), and the second containing the value. 

write(Left(42): Either[Int, String])     ==> "[0,42]"
write(Right("foo"): Either[Int, String]) ==> """[1,"foo"]"""

Tuples of all sizes (1-22) are serialized as heterogenous JSON lists 

write((1, "omg"))                 ==> """[1,"omg"]"""
write((1, "omg", true))           ==> """[1,"omg",true]"""

Maps with primitive keys are serialized into JSON dictionaries, while Map with complex keys are serialized into lists of 2-element tuples 

write(Map(1 -> 2, 3 -> 4))         ==> """{"1":2,"3":4}"""
write(Map("hello" -> "world"))     ==> """{"hello":"world"}"""
write(Map(Seq(1, 2) -> Seq(3, 4))) ==> """[[[1,2],[3,4]]]"""
write(Map.empty[Int, Int])         ==> """{}"""
write(Map(Seq.empty[Int] -> Seq.empty[Int])) ==> """[[[],[]]]"""

write(Map(Seq.empty[Int] -> Seq.empty[Int]), indent = 4) ==>
"""[
  |    [
  |        [],
  |        []
  |    ]
  |]""".stripMargin

write(Map.empty[Int, Int], indent = 4) ==> """{}"""

Case Classes

Case classes of sizes 1-64 are serialized as JSON dictionaries with the keys being the names of each field. To begin with, you need to define a serializer in the Case Class's companion object: 

import upickle.default.{ReadWriter => RW, macroRW}

After that, you can begin serializing that case class. 

case class Thing(myFieldA: Int, myFieldB: String)
object Thing{
  implicit val rw: RW[Thing] = macroRW
}
case class OptionThing(myFieldA: Option[Int] = None, myFieldB: Option[String] = None)
object OptionThing{
  implicit val rw: RW[OptionThing] = macroRW
}
case class Big(i: Int, b: Boolean, str: String, c: Char, t: Thing)
object Big{
  implicit val rw: RW[Big] = macroRW
}
import upickle._
write(Thing(1, "gg"))             ==> """{"myFieldA":1,"myFieldB":"gg"}"""
read[Thing]("""{"myFieldA":1,"myFieldB":"gg"}""") ==> Thing(1, "gg")
write(Big(1, true, "lol", 'Z', Thing(7, ""))) ==>
  """{"i":1,"b":true,"str":"lol","c":"Z","t":{"myFieldA":7,"myFieldB":""}}"""

write(Big(1, true, "lol", 'Z', Thing(7, "")), indent = 4) ==>
  """{
    |    "i": 1,
    |    "b": true,
    |    "str": "lol",
    |    "c": "Z",
    |    "t": {
    |        "myFieldA": 7,
    |        "myFieldB": ""
    |    }
    |}""".stripMargin

write(Big(1, true, "lol", 'Z', Thing(7, "")), indent = 4, sortKeys = true) ==>
  """{
    |    "b": true,
    |    "c": "Z",
    |    "i": 1,
    |    "str": "lol",
    |    "t": {
    |        "myFieldA": 7,
    |        "myFieldB": ""
    |    }
    |}""".stripMargin

Sealed hierarchies are serialized as tagged values, the serialized object tagged with the full name of the instance's class: 

sealed trait IntOrTuple
object IntOrTuple{
  implicit val rw: RW[IntOrTuple] = RW.merge(IntThing.rw, TupleThing.rw)
}
case class IntThing(i: Int) extends IntOrTuple
object IntThing{
  implicit val rw: RW[IntThing] = macroRW
}
case class TupleThing(name: String, t: (Int, Int)) extends IntOrTuple
object TupleThing{
  implicit val rw: RW[TupleThing] = macroRW
}
write(IntThing(1)) ==> """{"$type":"IntThing","i":1}"""

write(TupleThing("naeem", (1, 2))) ==>
  """{"$type":"TupleThing","name":"naeem","t":[1,2]}"""

// You can read tagged value without knowing its
// type in advance, just use type of the sealed trait
read[IntOrTuple]("""{"$type":"IntThing","i":1}""") ==> IntThing(1)

Serializability is recursive; you can serialize a type only if all its members are serializable. That means that collections, tuples and case-classes made only of serializable members are themselves serializable 

case class Foo(i: Int)
object Foo{
  implicit val rw: RW[Foo] = macroRW
}
case class Bar(name: String, foos: Seq[Foo])
object Bar{
  implicit val rw: RW[Bar] = macroRW
}
write((((1, 2), (3, 4)), ((5, 6), (7, 8)))) ==>
  """[[[1,2],[3,4]],[[5,6],[7,8]]]"""

write(Seq(Thing(1, "g"), Thing(2, "k"))) ==>
  """[{"myFieldA":1,"myFieldB":"g"},{"myFieldA":2,"myFieldB":"k"}]"""

write(Bar("bearrr", Seq(Foo(1), Foo(2), Foo(3)))) ==>
  """{"name":"bearrr","foos":[{"i":1},{"i":2},{"i":3}]}"""

Scala 3 Deriving

In Scala 3, you can use the derives keyword on standalone case classes, sealed trait hierarchies, and enums: 

case class Dog(name: String, age: Int) derives ReadWriter
upickle.default.write(Dog("Ball", 2)) ==> """{"name":"Ball","age":2}"""
upickle.default.read[Dog]("""{"name":"Ball","age":2}""") ==> Dog("Ball", 2)
sealed trait Animal derives ReadWriter
case class Person(name: String, address: String, age: Int = 20) extends Animal
case class Cat(name: String, owner: Person) extends Animal
case object Cthulu extends Animal
upickle.default.write(Person("Peter", "Ave 10")) ==>
  """{"$type":"Person","name":"Peter","address":"Ave 10"}"""

upickle.default.read[Animal]("""{"$type":"Person","name":"Peter","address":"Ave 10"}""") ==>
  Person("Peter", "Ave 10")

upickle.default.write(Cthulu) ==> "\"Cthulu\""
upickle.default.read[Animal]("\"Cthulu\"") ==> Cthulu
enum SimpleEnum derives ReadWriter:
  case A, B
upickle.default.write(SimpleEnum.A) ==> "\"A\""
upickle.default.read[SimpleEnum]("\"A\"") ==> SimpleEnum.A
enum ColorEnum(val rgb: Int) derives ReadWriter:
  case Red extends ColorEnum(0xFF0000)
  case Green extends ColorEnum(0x00FF00)
  case Blue extends ColorEnum(0x0000FF)
  case Mix(mix: Int) extends ColorEnum(mix)
  case Unknown(mix: Int) extends ColorEnum(0x000000)
Enclosing("test", SimpleEnum.A, Some(SimpleEnum.B)),
"""{"str":"test","simple1":"A","simple2":"B"}"""

Note that you only need to put the derives keyword on the enums or sealed traits, and not on all the individual case classes or case objects.

Also, note that for enums, the short un-qualified name is used for the type key, rather than the fully qualified name with package path. This is because unlike sealed traits, enums enforce that every case is in the same flat namespace, and thus the short name is enough to dis-ambiguate them. This behavior can be overriden with the @key("...") annotation in both cases, if a different type key is desired.

Read/Writing Other Things

Apart from reading & writing java.lang.Strings, allows you to easily read from alternate sources such as CharSequences, Array[Byte]s, java.io.Files and java.nio.file.Paths: 

import upickle.default._
val original = """{"myFieldA":1,"myFieldB":"gg"}"""
read[Thing](original) ==> Thing(1, "gg")
read[Thing](original: CharSequence) ==> Thing(1, "gg")
read[Thing](original.getBytes) ==> Thing(1, "gg")
import upickle.default._
val original = """{"myFieldA":1,"myFieldB":"gg"}"""

import java.nio.file.Files
val f = Files.createTempFile("", "")
Files.write(f, original.getBytes)

read[Thing](f) ==> Thing(1, "gg")
read[Thing](f.toFile) ==> Thing(1, "gg")

Reading from large files is automatically streamed so you do not read the entire file into memory. You can use writeTo to serialize your data to an arbitrary java.io.Writer/java.io.OutputStream: this can be streamed directly to files or over the network without having to accumulate the serialized JSON in memory.

Nulls

Nulls serialize into JSON nulls, as you would expect 
write(Bar(null, Seq(Foo(1), null, Foo(3)))) ==>
  """{"name":null,"foos":[{"i":1},null,{"i":3}]}"""

uPickle only throws exceptions on unpickling; if a pickler is properly defined, serializing a data structure to a String should never throw an exception.

All these examples can be similarly serialized to MessagePack-formatted binaries, in the same way: JSON booleans become MessagePack booleans, lists become MessagePack lists, and so on. Reading and writing MessagePack binary data is typically significantly faster than reading and writing JSON, and the serialized data is also significantly smaller.

Defaults

If a field is missing upon deserialization, uPickle uses the default value if one exists 

read[FooDefault]("{}")                ==> FooDefault(10, "lol")
read[FooDefault]("""{"i": 123}""")    ==> FooDefault(123,"lol")

If a field at serialization time has the same value as the default, uPickle leaves it out of the serialized blob 

write(FooDefault(i = 11, s = "lol"))  ==> """{"i":11}"""
write(FooDefault(i = 10, s = "lol"))  ==> """{}"""
write(FooDefault())                   ==> """{}"""

This allows you to make schema changes gradually, assuming you have already pickled some data and want to add new fields to the case classes you pickled. Simply give the new fields a default value (e.g. "" for Strings, or wrap it in an Option[T] and make the default None) and uPickle will happily read the old data, filling in the missing field using the default value.

You can enable serialization of default values a per-field or per-case-class basis using the @upickle.implicits.serializeDefaults(true) annotation on that field or case class, or globally on a Custom Configuration via override def serializeDefaults = true

Supported Types

Out of the box, uPickle supports writing and reading the following types:

Readability/writability is recursive: a container such as a Tuple or case class is only readable if all its contents are readable, and only writable if all its contents are writable. That means that you cannot serialize a List[Any], since uPickle doesn't provide a generic way of serializing Any. Case classes are only serializable up to 64 fields.

Case classes are de-serialized using the apply method on their companion objects. sealed hierarchies are serialized as tagged unions: whatever the serialization of the actual object, together with the partially-qualified name of its class in a "$type": "..." field, so the correct class in the sealed hierarchy can be reconstituted later.

That concludes the list of supported types. Anything else is not supported by default, but you can add support using Custom Picklers

Common Operations

The following common operations are available on any uPickle module, e.g. upickle.default or upickle.legacy: 

trait Api
    extends upickle.core.Types
    with implicits.Readers
    with implicits.Writers
    with implicits.CaseClassReadWriters
    with WebJson
    with JsReadWriters
    with MsgReadWriters
    with Annotator{

  private def maybeSortKeysTransform[T: Writer, V](t: T,
                                                   sortKeys: Boolean,
                                                   f: Visitor[_, V]): V = {
    BufferedValue.maybeSortKeysTransform(implicitly[Writer[T]], t, sortKeys, f)
  }
  /**
    * Reads the given MessagePack input into a Scala value
    */
  def readBinary[T: Reader](s: upack.Readable, trace: Boolean = false): T = {
    TraceVisitor.withTrace(trace, reader[T])(s.transform(_))
  }

  /**
    * Reads the given JSON input into a Scala value
    */
  def read[T: Reader](s: ujson.Readable, trace: Boolean = false): T = {
    TraceVisitor.withTrace(trace, reader[T])(s.transform(_))
  }

  def reader[T: Reader] = implicitly[Reader[T]]

  /**
    * Write the given Scala value as a JSON string
    */
  def write[T: Writer](t: T,
                       indent: Int = -1,
                       escapeUnicode: Boolean = false,
                       sortKeys: Boolean = false): String = {
    maybeSortKeysTransform(t, sortKeys, ujson.StringRenderer(indent, escapeUnicode)).toString
  }

  /**
    * Write the given Scala value as a MessagePack binary
    */
  def writeBinary[T: Writer](t: T,
                             sortKeys: Boolean = false): Array[Byte] = {
    maybeSortKeysTransform(t, sortKeys, new upack.MsgPackWriter(new ByteArrayOutputStream())).toByteArray
  }

  /**
    * Write the given Scala value as a JSON struct
    */
  def writeJs[T: Writer](t: T): ujson.Value = transform(t).to[ujson.Value]

  /**
    * Write the given Scala value as a MessagePack struct
    */
  def writeMsg[T: Writer](t: T): upack.Msg = transform(t).to[upack.Msg]

  /**
    * Write the given Scala value as a JSON string to the given Writer
    */
  def writeTo[T: Writer](t: T,
                         out: java.io.Writer,
                         indent: Int = -1,
                         escapeUnicode: Boolean = false,
                         sortKeys: Boolean = false): Unit = {
    maybeSortKeysTransform(t, sortKeys, new ujson.Renderer(out, indent = indent, escapeUnicode))
  }

  def writeToOutputStream[T: Writer](t: T,
                                     out: java.io.OutputStream,
                                     indent: Int = -1,
                                     escapeUnicode: Boolean = false,
                                     sortKeys: Boolean = false): Unit = {
    maybeSortKeysTransform(t, sortKeys, new ujson.BaseByteRenderer(out, indent = indent, escapeUnicode))
  }

  def writeToByteArray[T: Writer](t: T,
                                  indent: Int = -1,
                                  escapeUnicode: Boolean = false,
                                  sortKeys: Boolean = false): Array[Byte] = {
    val out = new java.io.ByteArrayOutputStream()
    writeToOutputStream(t, out, indent, escapeUnicode, sortKeys)
    out.toByteArray
  }

  /**
    * Write the given Scala value as a JSON string via a `geny.Writable`
    */
  def stream[T: Writer](t: T,
                        indent: Int = -1,
                        escapeUnicode: Boolean = false,
                        sortKeys: Boolean = false): geny.Writable = new geny.Writable{
    override def httpContentType = Some("application/json")
    def writeBytesTo(out: java.io.OutputStream) = {
      maybeSortKeysTransform(t, sortKeys, new ujson.BaseByteRenderer(out, indent = indent, escapeUnicode))
    }
  }

  /**
    * Write the given Scala value as a MessagePack binary to the given OutputStream
    */
  def writeBinaryTo[T: Writer](t: T,
                               out: java.io.OutputStream,
                               sortKeys: Boolean = false): Unit = {
    streamBinary[T](t, sortKeys = sortKeys).writeBytesTo(out)
  }

  def writeBinaryToByteArray[T: Writer](t: T,
                                        sortKeys: Boolean = false): Array[Byte] = {
    val out = new java.io.ByteArrayOutputStream()
    streamBinary[T](t, sortKeys = sortKeys).writeBytesTo(out)
    out.toByteArray
  }

  /**
    * Write the given Scala value as a MessagePack binary via a `geny.Writable`
    */
  def streamBinary[T: Writer](t: T, sortKeys: Boolean = false): geny.Writable = new geny.Writable{
    override def httpContentType = Some("application/octet-stream")
    def writeBytesTo(out: java.io.OutputStream) = maybeSortKeysTransform(t, sortKeys, new upack.MsgPackWriter(out))
  }
  
  def writer[T: Writer] = implicitly[Writer[T]]

  def readwriter[T: ReadWriter] = implicitly[ReadWriter[T]]

  case class transform[T: Writer](t: T) extends upack.Readable with ujson.Readable {
    def transform[V](f: Visitor[_, V]): V = writer[T].transform(t, f)
    def to[V](f: Visitor[_, V]): V = transform(f)
    def to[V](implicit f: Reader[V]): V = transform(f)
  }


  /**
   * Mark a `ReadWriter[T]` as something that can be used as a key in a JSON
   * dictionary, such that `Map[T, V]` serializes to `{"a": "b", "c": "d"}`
   * rather than `[["a", "b"], ["c", "d"]]`
   */
  def stringKeyRW[T](readwriter: ReadWriter[T]): ReadWriter[T] = {
    new ReadWriter.Delegate[T](readwriter) {
      override def isJsonDictKey = true
      def write0[R](out: Visitor[_, R], v: T): R = readwriter.write0(out, v)
    }
  }

  /**
   * Mark a `Writer[T]` as something that can be used as a key in a JSON
   * dictionary, such that `Map[T, V]` serializes to `{"a": "b", "c": "d"}`
   * rather than `[["a", "b"], ["c", "d"]]`
   */
  def stringKeyW[T](readwriter: Writer[T]): Writer[T] = new Writer[T]{
    override def isJsonDictKey = true
    def write0[R](out: Visitor[_, R], v: T): R = readwriter.write0(out, v)
  }

Customization


Custom Picklers

import upickle.default._
case class Wrap(i: Int)
implicit val fooReadWrite: ReadWriter[Wrap] =
  readwriter[Int].bimap[Wrap](_.i, Wrap(_))

write(Seq(Wrap(1), Wrap(10), Wrap(100))) ==> "[1,10,100]"
read[Seq[Wrap]]("[1,10,100]") ==> Seq(Wrap(1), Wrap(10), Wrap(100))

You can use the readwriter[T].bimap[V] function to create a pickler that reads/writes a type V, using the pickler for type T, by providing a conversion function between them.

The type you are .bimaping to doesn't need to be a case class, or be pickleable in any way, as long as the type you are .bimaping from is pickleable. The following example demonstrates using .bimap to define a serializer for non-case Scala class 

class CustomThing2(val i: Int, val s: String)
object CustomThing2 {
  implicit val rw: RW[CustomThing2] = upickle.default.readwriter[String].bimap[CustomThing2](
    x => s"${x.i} ${x.s}",
    str => {
      val Array(i, s) = str.split(" ", 2)
      new CustomThing2(i.toInt, s)
    }
  )
}

Note that when writing custom picklers, it is entirely up to you to get it right, e.g. making sure that an object that gets round-trip pickled/unpickled comes out the same as when it started.

Lastly, if you want more control over exactly how something is serialized, you can use readwriter[Js.Value].bimap to give yourself access to the raw JSON AST: 

import upickle.default._
case class Bar(i: Int, s: String)
implicit val fooReadWrite: ReadWriter[Bar] =
  readwriter[ujson.Value].bimap[Bar](
    x => ujson.Arr(x.s, x.i),
    json => new Bar(json(1).num.toInt, json(0).str)
  )

write(Bar(123, "abc")) ==> """["abc",123]"""
read[Bar]("""["abc",123]""") ==> Bar(123, "abc")

Custom Keys

uPickle allows you to specify the key that a field is serialized with via a @key annotation 

case class KeyBar(@upickle.implicits.key("hehehe") kekeke: Int)
object KeyBar{
  implicit val rw: RW[KeyBar] = macroRW
}
write(KeyBar(10))                     ==> """{"hehehe":10}"""
read[KeyBar]("""{"hehehe": 10}""")    ==> KeyBar(10)

Practically, this is useful if you want to rename the field within your Scala code while still maintaining backwards compatibility with previously-pickled objects. Simple rename the field and add a @key("...") with the old name so uPickle can continue to work with the old objects correctly.

You can also use @key to change the name used when pickling the case class itself. Normally case classes are pickled without their name, but an exception is made for members of sealed hierarchies which are tagged with their fully-qualified name. uPickle allows you to use @key to override what the class is tagged with: 

sealed trait A
object A{
  implicit val rw: RW[A] = RW.merge(B.rw, macroRW[C.type])
}
@upickle.implicits.key("Bee") case class B(i: Int) extends A
object B{
  implicit val rw: RW[B] = macroRW
}
case object C extends A
write(B(10))                          ==> """{"$type":"Bee","i":10}"""
read[B]("""{"$type":"Bee","i":10}""") ==> B(10)

This is useful in cases where:

You can also use @key to change the key used when pickling a member of a sealed hierarchy. The default key is $type, as seen above, but you can change it by annotating the sealed trait: 

@upickle.implicits.key("_tag")
sealed trait Tag
case class ATag(i: Int) extends Tag
object ATag {
  implicit val rw: RW[ATag] = macroRW
}
write(ATag(11)) ==>
  """{"_tag":"ATag","i":11}"""

read[ATag]("""{"_tag":"ATag","i":11}""") ==>
  ATag(11)

This is useful in cases where you need to override uPickle's default behavior, for example to preserve compatibility with another JSON library.

JSON Dictionary Formats

By default, serializing a Map[K, V] generates a nested array-of-arrays. This is because not all types K can be easily serialized into JSON strings, so keeping them as nested tuples preserves the structure of the serialized K values: 

import upickle.default._

case class FooId(x: Int)
implicit val fooRW: ReadWriter[FooId] = readwriter[Int].bimap[FooId](_.x, FooId(_))

write(FooId(123)) ==> "123"
read[FooId]("123") ==> FooId(123)

write(Map(FooId(123) -> "hello", FooId(456) -> "world")) ==>
  """[[123,"hello"],[456,"world"]]"""

read[Map[FooId, String]]("""[[123,"hello"],[456,"world"]]""") ==>
  Map(FooId(123) -> "hello", FooId(456) -> "world")

For types of K that you want to serialize to JSON strings in JSON dictionaries, you can wrap your ReadWriter[K] in a stringKeyRW. 

import upickle.default._

case class FooId(x: Int)
implicit val fooRW: ReadWriter[FooId] = stringKeyRW(readwriter[Int].bimap[FooId](_.x, FooId(_)))

write(FooId(123)) ==> "123"
read[FooId]("123") ==> FooId(123)

write(Map(FooId(123) -> "hello", FooId(456) -> "world")) ==>
  """{"123":"hello","456":"world"}"""

read[Map[FooId, String]]("""{"123":"hello","456":"world"}""") ==>
  Map(FooId(123) -> "hello", FooId(456) -> "world")

Note that this only works for types K which serialize to JSON primitives: numbers, strings, booleans, and so on. Types of K that serialize to complex structures like JSON arrays or dictionaries are unsupported for use a JSON dictionary keys.

Older versions of uPickle serialized almost all Map[K, V]s to nested arrays. Data already serialized in that format is forwards-compatible with the current implementation of uPickle, which can read both nested-json-arrays and json-dictionary formats without issue.

These subtleties around deciding between nested-json-array v.s. json-dictionary formats only apply to uPickle's JSON backend. When serializing to MessagePack using upickle.default.writeBinary or upickle.default.writeMsg, it always uses the dictionary-based format, since MessagePack does not have the restriction that dictionary keys can only be strings: 

import upickle.default._

write(Map(Seq(1) -> 1, Seq(1, 2) -> 3, Seq(1, 2, 3) -> 6)) ==> "[[[1],1],[[1,2],3],[[1,2,3],6]]"

read[Map[Seq[Int], Int]]("[[[1],1],[[1,2],3],[[1,2,3],6]]") ==>
  Map(Seq(1) -> 1, Seq(1, 2) -> 3, Seq(1, 2, 3) -> 6)

writeMsg(Map(Seq(1) -> 1, Seq(1, 2) -> 3, Seq(1, 2, 3) -> 6)) ==>
  upack.Obj(
    upack.Arr(upack.Int32(1)) -> upack.Int32(1),
    upack.Arr(upack.Int32(1), upack.Int32(2)) -> upack.Int32(3),
    upack.Arr(upack.Int32(1), upack.Int32(2), upack.Int32(3)) -> upack.Int32(6)
  )

readBinary[Map[Seq[Int], Int]](
  upack.Obj(
    upack.Arr(upack.Int32(1)) -> upack.Int32(1),
    upack.Arr(upack.Int32(1), upack.Int32(2)) -> upack.Int32(3),
    upack.Arr(upack.Int32(1), upack.Int32(2), upack.Int32(3)) -> upack.Int32(6)
  )
) ==>
  Map(Seq(1) -> 1, Seq(1, 2) -> 3, Seq(1, 2, 3) -> 6)

Other Per-Case-Class Annotations

The full list of annotations that can be applied to case classes and sealed traits is below: 

package upickle.implicits

import scala.annotation.StaticAnnotation

/**
 * Annotation for control over the strings used to serialize your data structure. Can
 * be applied in three ways:
 *
 * 1. To individual fields, in which case it overrides the JSON object fieldname
 *
 * 2. To `case class`es which are part of `sealed trait`s, where it overrides
 *    the value of the `"$type": "foo"` discriminator field
 *
 * 2. To `sealed trait`s themselves, where it overrides
 *    the key of the `"$type": "foo"` discriminator field
 */
class key(s: String) extends StaticAnnotation

/**
 * Annotation for fine-grained control of the `def serializeDefaults` configuration
 * on the upickle bundle; can be applied to individual fields or to entire `case class`es,
 * with finer-grained application taking precedence
 */
class serializeDefaults(s: Boolean) extends StaticAnnotation

/**
 * Annotation for fine-grained control of the `def allowUnknownKeys` configuration
 * on the upickle bundle; can be applied to individual `case class`es, taking precedence
 * over upickle pickler-level configuration
 */
class allowUnknownKeys(b: Boolean) extends StaticAnnotation

Custom Configuration

Often, there will be times that you want to customize something on a project-wide level. uPickle provides hooks in letting you subclass the upickle.Api trait to create your own bundles apart from the in-built upickle.default and upickle.legacy. These are listed below in the upickle.core.Config trait: 

trait Config {
  /**
   * Specifies the name of the field used to distinguish different `case class`es under
   * a `sealed trait`. Defaults to `$type`, but can be configured globally by overriding
   * [[tagName]], or on a per-`sealed trait` basis via the `@key` annotation
   */
  def tagName: String = Annotator.defaultTagKey

  /**
   * Whether to use the fully-qualified name of `case class`es and `case object`s which
   * are part of `sealed trait` hierarchies when serializing them and writing their `$type`
   * key. Defaults to `false`, so `$type` key uses the shortest partially-qualified name.
   * Can be set to `true` to use their fully-qualified name.
   */
  def objectTypeKeyWriteFullyQualified: Boolean = false

  /**
   * Whether or not to write `case class` keys which match their default values.
   * Defaults to `false`, allowing those keys to be omitted. Can be set to `true`
   * to always write field values even if they are equal to the default
   */
  def serializeDefaults: Boolean = false

  /**
   * Transform dictionary keys when writing `case class`es when reading. Can
   * be overriden to provide custom mappings between Scala field names and JSON
   * field names. Needs to be kept in sync with [[objectAttributeKeyWriteMap]]
   *
   * This customizes the mapping across all `case class`es fields handled by this
   * upickle instance. This can be customized on a field-by-field basis using the
   * [[upickle.implicits.key]] annotation on the `case class` field
   */
  def objectAttributeKeyReadMap(s: CharSequence): CharSequence = s

  /**
   * Map the name of JSON object fields to Scala `case class` fields during serialization.
   * Must be kept in sync with [[objectAttributeKeyReadMap]]
   */
  def objectAttributeKeyWriteMap(s: CharSequence): CharSequence = s

  /**
   * Transforms the value of the `$type` field when writing `sealed trait`s,
   * to allow custom mapping between the `case class` name and the `$type` field
   * in the generated JSON. Must be kept in sync with [[objectTypeKeyWriteMap]].
   *
   * * This customizes the mapping across all `case class`es fields handled by this
   * * upickle instance. This can be customized on a per-`sealed trait` basis using the
   * * [[upickle.implicits.key]] annotation on the `case class`
   */
  def objectTypeKeyReadMap(s: CharSequence): CharSequence = s

  /**
   * Map the name of Scala `case class` type names to JSON `$type` field value during
   * serialization. Must be kept in sync with [[objectTypeKeyReadMap]]
   */
  def objectTypeKeyWriteMap(s: CharSequence): CharSequence = s

  /**
   * Whether top-level `Some(t)`s and `None`s are serialized unboxed as `t` or
   * `null`, rather than `[t]` or `[]`. This is generally what people expect,
   * although it does cause issues where `Some(null)` when serialized and de-serialized
   * can become `None`. Can be disabled to use the boxed serialization format
   * as 0-or-1-element-arrays, presering round trip-ability at the expense of
   * un-intuitiveness and verbosity
   */
  def optionsAsNulls: Boolean = true

  /**
   * Configure whether you want upickle to skip unknown keys during de-serialization
   * of `case class`es. Can be overriden for the entire serializer via `override def`, and
   * further overriden for individual `case class`es via the annotation
   * `@upickle.implicits.allowUnknownKeys(b: Boolean)`
   */
  def allowUnknownKeys: Boolean = true
}

The following example demonstrates how to customize a bundle to automatically snake_case all dictionary keys. 
object SnakePickle extends upickle.AttributeTagged{
  def camelToSnake(s: String) = {
    s.replaceAll("([A-Z])","#$1").split('#').map(_.toLowerCase).mkString("_")
  }
  def snakeToCamel(s: String) = {
    val res = s.split("_", -1).map(x => s"${x(0).toUpper}${x.drop(1)}").mkString
    s"${s(0).toLower}${res.drop(1)}"
  }

  override def objectAttributeKeyReadMap(s: CharSequence) =
    snakeToCamel(s.toString)
  override def objectAttributeKeyWriteMap(s: CharSequence) =
    camelToSnake(s.toString)

  override def objectTypeKeyReadMap(s: CharSequence) =
    snakeToCamel(s.toString)
  override def objectTypeKeyWriteMap(s: CharSequence) =
    camelToSnake(s.toString)
}

// Default read-writing
upickle.default.write(Thing(1, "gg")) ==>
  """{"myFieldA":1,"myFieldB":"gg"}"""

upickle.default.read[Thing]("""{"myFieldA":1,"myFieldB":"gg"}""") ==>
  Thing(1, "gg")

implicit def thingRW: SnakePickle.ReadWriter[Thing] = SnakePickle.macroRW

// snake_case_keys read-writing
SnakePickle.write(Thing(1, "gg")) ==>
  """{"my_field_a":1,"my_field_b":"gg"}"""

SnakePickle.read[Thing]("""{"my_field_a":1,"my_field_b":"gg"}""") ==>
  Thing(1, "gg")

uPickle by default serializes Scala Nones to JSON nulls, and Some(t)s to unboxed JSON ts. This is generally what people want, but can cause issues at some edge cases (e.g. Some(null) can round trip and become None). You can use a more verbose serialization which converts Scala Nones to boxed JSON []]s, and Some(t)s to boxed JSON [t]s 

object BoxedOptionsPickler extends upickle.AttributeTagged {
  override def optionsAsNulls = false
}

You can also use a custom configuration to change how 64-bit Longs are handled. By default, small longs that can be represented exactly in 64-bit Doubles are written as raw numbers, while larger values (n > 2^53) are written as strings. This is to ensure the values are not truncated when the serialized JSON is then manipulated, e.g. by Javascript which truncates all large numbers to Doubles. If you wish to always write Longs as Strings, or always write them as numbers (at risk of truncation), you can do so as follows: 

upickle.default.write(123: Long) ==> "123"
upickle.default.write(Long.MaxValue) ==> "\"9223372036854775807\""

object StringLongs extends upickle.AttributeTagged{
  override implicit val LongWriter: Writer[Long] = new Writer[Long] {
    def write0[V](out: Visitor[_, V], v: Long) = out.visitString(v.toString, -1)
  }
}

StringLongs.write(123: Long) ==> "\"123\""
StringLongs.write(Long.MaxValue) ==> "\"9223372036854775807\""

object NumLongs extends upickle.AttributeTagged{
  override implicit val LongWriter: Writer[Long] = new Writer[Long] {
    def write0[V](out: Visitor[_, V], v: Long) = out.visitFloat64String(v.toString, -1)
  }
}

NumLongs.write(123: Long) ==> "123"
NumLongs.write(Long.MaxValue) ==> "9223372036854775807"

Limitations


uPickle doesn't currently support:

Most of these limitations are inherent in the fact that ScalaJS does not support reflection, and are unlikely to ever go away. In general, uPickle by default can serialize statically-typed, tree-shaped, immutable data structures. Anything more complex requires Custom Picklers

Manual Sealed Trait Picklers

Due to a bug in the Scala compiler SI-7046, automatic sealed trait pickling can fail unpredictably. This can be worked around by instead using the macroRW and merge methods to manually specify which sub-types of a sealed trait to consider when pickling: 

sealed trait TypedFoo
object TypedFoo{
  import upickle.default._
  implicit val readWriter: ReadWriter[TypedFoo] = ReadWriter.merge(
    macroRW[Bar], macroRW[Baz], macroRW[Quz],
  )

  case class Bar(i: Int) extends TypedFoo
  case class Baz(s: String) extends TypedFoo
  case class Quz(b: Boolean) extends TypedFoo
}

uJson


uJson is uPickle's JSON library, which can be used to easily manipulate JSON source and data structures without converting them into Scala case-classes. This all lives in the ujson package. Unlike many other Scala JSON libraries that come with their own zoo of new concepts, abstractions, and techniques, uJson has a simple & predictable JSON API that should be instantly familiar to anyone coming from scripting languages like Ruby, Python or Javascript.

uJson comes bundled with uPickle, or can be used stand-alone via the following package coordinates: 

libraryDependencies += "com.lihaoyi" %% "ujson" % "0.9.6"

Construction

You can use ujson to conveniently construct JSON blobs, either programmatically:


        val json0 = ujson.Arr(
          ujson.Obj("myFieldA" -> ujson.Num(1), "myFieldB" -> ujson.Str("g")),
          ujson.Obj("myFieldA" -> ujson.Num(2), "myFieldB" -> ujson.Str("k"))
        )

        val json = ujson.Arr( // The `ujson.Num` and `ujson.Str` calls are optional
          ujson.Obj("myFieldA" -> 1, "myFieldB" -> "g"),
          ujson.Obj("myFieldA" -> 2, "myFieldB" -> "k")
        )

        json0 ==> json

Or parsing them from strings, byte arrays or files: 

val str = """[{"myFieldA":1,"myFieldB":"g"},{"myFieldA":2,"myFieldB":"k"}]"""
val json = ujson.read(str)
json(0)("myFieldA").num   ==> 1
json(0)("myFieldB").str   ==> "g"
json(1)("myFieldA").num   ==> 2
json(1)("myFieldB").str   ==> "k"

ujson.write(json)         ==> str

ujson.Js ASTs are mutable, and can be modified before being re-serialized to strings: 

val str = """[{"myFieldA":1,"myFieldB":"g"},{"myFieldA":2,"myFieldB":"k"}]"""
val json: ujson.Value = ujson.read(str)

json.arr.remove(1)
json(0)("myFieldA") = 1337
json(0)("myFieldB") = json(0)("myFieldB").str + "lols"

You can also use the `_` shorthand syntax to update a JSON value in place, without having to duplicate the whole path: 

val str = """[{"myFieldA":1,"myFieldB":"g"},{"myFieldA":2,"myFieldB":"k"}]"""
val json: ujson.Value = ujson.read(str)

json(0)("myFieldA") = _.num + 100
json(1)("myFieldB") = _.str + "lol"

case classes or other Scala data structures can be converted to ujson.Js ASTs using upickle.default.writeJs, and the ujson.Js ASTs can be converted back using upickle.default.readJs or plain upickle.default.read: 

val data = Seq(Thing(1, "g"), Thing(2, "k"))
val json = upickle.default.writeJs(data)

json.arr.remove(1)
json(0)("myFieldA") = 1337

upickle.default.read[Seq[Thing]](json)   ==> Seq(Thing(1337, "g"))

JSON Utilities

uJson comes with some convenient utilities on the `ujson` package: 

package object ujson{
  def transform[T](t: Readable,
                   v: upickle.core.Visitor[_, T],
                   sortKeys: Boolean = false): T = {
    BufferedValue.maybeSortKeysTransform(Readable, t, sortKeys, v)
  }

  /**
    * Read the given JSON input as a JSON struct
    */
  def read(s: Readable, trace: Boolean = false): Value.Value =
    upickle.core.TraceVisitor.withTrace(trace, Value)(transform(s, _))

  def copy(t: Value.Value): Value.Value = transform(t, Value)

  /**
    * Write the given JSON struct as a JSON String
    */
  def write(t: Value.Value,
            indent: Int = -1,
            escapeUnicode: Boolean = false,
            sortKeys: Boolean = false): String = {
    val writer = new java.io.StringWriter
    writeTo(t, writer, indent, escapeUnicode, sortKeys)
    writer.toString
  }

  /**
    * Write the given JSON struct as a JSON String to the given Writer
    */
  def writeTo(t: Value.Value,
              out: java.io.Writer,
              indent: Int = -1,
              escapeUnicode: Boolean = false,
              sortKeys: Boolean = false): Unit = {
    transform(t, Renderer(out, indent, escapeUnicode), sortKeys)
  }

  def writeToOutputStream(t: Value.Value,
                          out: java.io.OutputStream,
                          indent: Int = -1,
                          escapeUnicode: Boolean = false,
                          sortKeys: Boolean = false): Unit = {
    transform(t, new BaseByteRenderer(out, indent, escapeUnicode), sortKeys)
  }

  def writeToByteArray(t: Value.Value,
                       indent: Int = -1,
                       escapeUnicode: Boolean = false,
                       sortKeys: Boolean = false): Array[Byte] = {
    val baos = new java.io.ByteArrayOutputStream
    writeToOutputStream(t, baos, indent, escapeUnicode, sortKeys)
    baos.toByteArray
  }

  /**
    * Parse the given JSON input, failing if it is invalid
    */
  def validate(s: Readable): Unit = transform(s, NoOpVisitor)
  /**
    * Parse the given JSON input and write it to a string with
    * the configured formatting
    */
  def reformat(s: Readable,
               indent: Int = -1,
               escapeUnicode: Boolean = false,
               sortKeys: Boolean = false): String = {
    val writer = new java.io.StringWriter()
    reformatTo(s, writer, indent, escapeUnicode, sortKeys)
    writer.toString
  }
  
  /**
    * Parse the given JSON input and write it to a string with
    * the configured formatting to the given Writer
    */
  def reformatTo(s: Readable,
                 out: java.io.Writer,
                 indent: Int = -1,
                 escapeUnicode: Boolean = false,
                 sortKeys: Boolean = false): Unit = {
    transform(s, Renderer(out, indent, escapeUnicode), sortKeys)
  }
  
  /**
    * Parse the given JSON input and write it to a string with
    * the configured formatting to the given Writer
    */
  def reformatToOutputStream(s: Readable,
                             out: java.io.OutputStream,
                             indent: Int = -1,
                             escapeUnicode: Boolean = false,
                             sortKeys: Boolean = false): Unit = {
    transform(s, new BaseByteRenderer(out, indent, escapeUnicode), sortKeys)
  }
  
  def reformatToByteArray(s: Readable,
                          indent: Int = -1,
                          escapeUnicode: Boolean = false,
                          sortKeys: Boolean = false): Array[Byte] = {
    val baos = new java.io.ByteArrayOutputStream
    reformatToOutputStream(s, baos, indent, escapeUnicode, sortKeys)
    baos.toByteArray
  }
}

Transformations

uJson allows you seamlessly convert between any of the following forms you may find your JSON in:

This is done using the ujson.transform(source, dest) function: 

// It can be used for parsing JSON into an AST
val exampleAst = ujson.Arr(1, 2, 3)

ujson.transform("[1, 2, 3]", Value) ==> exampleAst

// Rendering the AST to a string
ujson.transform(exampleAst, StringRenderer()).toString ==> "[1,2,3]"

// Or to a byte array
ujson.transform(exampleAst, BytesRenderer()).toByteArray ==> "[1,2,3]".getBytes

// Re-formatting JSON, either compacting it
ujson.transform("[1, 2, 3]", StringRenderer()).toString ==> "[1,2,3]"

// or indenting it
ujson.transform("[1, 2, 3]", StringRenderer(indent = 4)).toString ==>
  """[
    |    1,
    |    2,
    |    3
    |]""".stripMargin

// `transform` takes any `Transformable`, including byte arrays and files
ujson.transform("[1, 2, 3]".getBytes, StringRenderer()).toString ==> "[1,2,3]"

All transformations from A to B using ujson.transform happen in a direct fashion: there are no intermediate JSON ASTs being generated, and performance is generally very good.

You can use ujson.transform to validate JSON in a streaming fashion: 

test {
  ujson.transform("[1, 2, 3]", NoOpVisitor)

The normal upickle.default.read/write methods to serialize Scala data-types is just shorthand for ujson.transform, using a upickle.default.transform(foo) as the source or a upickle.default.reader[Foo] as the destination: 

ujson.transform("[1, 2, 3]", upickle.default.reader[Seq[Int]]) ==>
  Seq(1, 2, 3)

ujson.transform(upickle.default.transform(Seq(1, 2, 3)), StringRenderer()).toString ==>
  "[1,2,3]"

Other ASTs

uJson does not provide any other utilities are JSON that other libraries do: zippers, lenses, combinators, etc.. However, uJson can be used to seamlessly convert between the JSON AST of other libraries! This means if some other library provides a more convenient API for some kind of processing you need to do, you can easily parse to that library's AST, do whatever you need, and convert back after.

As mentioned earlier, conversions are fast and direct, and happen without creating intermediate JSON structures in the process. The following examples demonstrate how to use the conversion modules for Argonaut, Circe, Json4s, and Play Json.

Each example parses JSON from a string into that particular library's JSON AST, manipulates the AST using that library, un-pickles it into Scala data types, then serializes those data types first into that library's AST then back to a string.

Argonaut

Maven Coordinates 
libraryDependencies += "com.lihaoyi" %% "ujson-argonaut" % "0.9.6"
Usage 
val argJson: argonaut.Json = ArgonautJson(
  """["hello", "world"]"""
)

val updatedArgJson = argJson.withArray(_.map(_.withString(_.toUpperCase)))

val items: Seq[String] = ArgonautJson.transform(
  updatedArgJson,
  upickle.default.reader[Seq[String]]
)

items ==> Seq("HELLO", "WORLD")

val rewritten = upickle.default.transform(items).to(ArgonautJson)

val stringified = ArgonautJson.transform(rewritten, StringRenderer()).toString

stringified ==> """["HELLO","WORLD"]"""

Circe

Maven Coordinates 
libraryDependencies += "com.lihaoyi" %% "ujson-circe" % "0.9.6"
Usage 
val circeJson: io.circe.Json = CirceJson(
  """["hello", "world"]"""
)

val updatedCirceJson =
  circeJson.mapArray(_.map(x => x.mapString(_.toUpperCase)))

val items: Seq[String] = CirceJson.transform(
  updatedCirceJson,
  upickle.default.reader[Seq[String]]
)

items ==> Seq("HELLO", "WORLD")

val rewritten = upickle.default.transform(items).to(CirceJson)

val stringified = CirceJson.transform(rewritten, StringRenderer()).toString

stringified ==> """["HELLO","WORLD"]"""

Play-Json

Maven Coordinates 
libraryDependencies += "com.lihaoyi" %% "ujson-play" % "0.9.6"
Usage 
import play.api.libs.json._
val playJson: play.api.libs.json.JsValue = PlayJson(
  """["hello", "world"]"""
)

val updatedPlayJson = JsArray(
  for(v <- playJson.as[JsArray].value)
    yield JsString(v.as[String].toUpperCase())
)

val items: Seq[String] = PlayJson.transform(
  updatedPlayJson,
  upickle.default.reader[Seq[String]]
)

items ==> Seq("HELLO", "WORLD")

val rewritten = upickle.default.transform(items).to(PlayJson)

val stringified = PlayJson.transform(rewritten, StringRenderer()).toString

stringified ==> """["HELLO","WORLD"]"""

Json4s

Maven Coordinates 
libraryDependencies += "com.lihaoyi" %% "ujson-json4s" % "0.9.6"
Usage 
import org.json4s.JsonAST
val json4sJson: JsonAST.JValue = Json4sJson(
  """["hello", "world"]"""
)

val updatedJson4sJson = JsonAST.JArray(
  for(v <- json4sJson.children)
    yield JsonAST.JString(v.values.toString.toUpperCase())
)

val items: Seq[String] = Json4sJson.transform(
  updatedJson4sJson,
  upickle.default.reader[Seq[String]]
)

items ==> Seq("HELLO", "WORLD")

val rewritten = upickle.default.transform(items).to(Json4sJson)

val stringified = Json4sJson.transform(rewritten, StringRenderer()).toString

stringified ==> """["HELLO","WORLD"]"""

Cross-Library Conversions

uJson lets you convert between third-party ASTs efficiently and with minimal overhead: uJson converts one AST to the other directly and without any temporary compatibility data structures. The following example demonstrates how this is done: we parse a JSON string using Circe, perform some transformation, convert it to a Play-Json AST, perform more transformations, and finally serialize it back to a String and check that both transformations were applied: 

val circeJson: io.circe.Json = CirceJson(
  """["hello", "world"]"""
)

val updatedCirceJson =
  circeJson.mapArray(_.map(x => x.mapString(_.toUpperCase)))

import play.api.libs.json._

val playJson: play.api.libs.json.JsValue = CirceJson.transform(
  updatedCirceJson,
  PlayJson
)

val updatedPlayJson = JsArray(
  for(v <- playJson.as[JsArray].value)
    yield JsString(v.as[String].reverse)
)

val stringified = PlayJson.transform(updatedPlayJson, StringRenderer()).toString

stringified ==> """["OLLEH","DLROW"]"""

uPack


uPack is uPickle's MessagePack library, which can be used to easily manipulate MessagePack source and data structures without converting them into Scala case-classes. This all lives in the upack package.

uPack comes bundled with uPickle, or can be used stand-alone via the following package coordinates: 

libraryDependencies += "com.lihaoyi" %% "upack" % "0.9.6"

The following basic functions are provided in the upack package to let you read and write MessagePack structs: 

def transform[T](t: Readable, v: upickle.core.Visitor[_, T]) = t.transform(v)

/**
  * Read the given MessagePack input into a MessagePack struct
  */
def read(s: Readable, trace: Boolean = false): Msg = upickle.core.TraceVisitor.withTrace(trace, Msg)(transform(s, _))

def copy(t: Msg): Msg = transform(t, Msg)
/**
  * Write the given MessagePack struct as a binary
  */
def write(t: Msg): Array[Byte] = {
  transform(t, new MsgPackWriter()).toByteArray
}
/**
  * Write the given MessagePack struct as a binary to the given OutputStream
  */
def writeTo(t: Msg, out: java.io.OutputStream): Unit = {
  transform(t, new MsgPackWriter(out))
}
def writeToByteArray(t: Msg) = {
  val out = new ByteArrayOutputStream()
  transform(t, new MsgPackWriter(out))
  out.toByteArray
}
/**
  * Parse the given MessagePack input, failing if it is invalid
  */
def validate(s: Readable): Unit = transform(s, NoOpVisitor)

MessagePack structs are represented using the upack.Msg type. You can construct ad-hoc MessagePack structs using upack.Msg, and can similarly parse binary data into upack.Msg for ad-hoc querying and manipulation, without needing to bind it to Scala case classes or data types: 

val msg = upack.Arr(
  upack.Obj(upack.Str("myFieldA") -> upack.Int32(1), upack.Str("myFieldB") -> upack.Str("g")),
  upack.Obj(upack.Str("myFieldA") -> upack.Int32(2), upack.Str("myFieldB") -> upack.Str("k"))
)

val binary: Array[Byte] = upack.write(msg)

val read = upack.read(binary)
assert(msg == read)

You can read/write Scala values to upack.Msgs using readBinary/writeMsg: 

val big = Big(1, true, "lol", 'Z', Thing(7, ""))
val msg: upack.Msg = upickle.default.writeMsg(big)
upickle.default.readBinary[Big](msg) ==> big

Or include upack.Msgs inside Seqs, case-classes and other data structures when you read/write them: 

val msgSeq = Seq[upack.Msg](
  upack.Str("hello world"),
  upack.Arr(upack.Int32(1), upack.Int32(2))
)

val binary: Array[Byte] = upickle.default.writeBinary(msgSeq)

upickle.default.readBinary[Seq[upack.Msg]](binary) ==> msgSeq

You can also convert the uPack messages or binaries to ujson.Values via upack.transform. This can be handy to help debug what's going on in your binary message data: 

val msg = upack.Arr(
  upack.Obj(upack.Str("myFieldA") -> upack.Int32(1), upack.Str("myFieldB") -> upack.Str("g")),
  upack.Obj(upack.Str("myFieldA") -> upack.Int32(2), upack.Str("myFieldB") -> upack.Str("k"))
)

val binary: Array[Byte] = upack.write(msg)

// Can pretty-print starting from either the upack.Msg structs,
// or the raw binary data
upack.transform(msg, new ujson.StringRenderer()).toString ==>
  """[{"myFieldA":1,"myFieldB":"g"},{"myFieldA":2,"myFieldB":"k"}]"""

upack.transform(binary, new ujson.StringRenderer()).toString ==>
  """[{"myFieldA":1,"myFieldB":"g"},{"myFieldA":2,"myFieldB":"k"}]"""

// Some messagepack structs cannot be converted to valid JSON, e.g.
// they may have maps with non-string keys. These can still be pretty-printed:
val msg2 = upack.Obj(upack.Arr(upack.Int32(1), upack.Int32(2)) -> upack.Int32(1))
upack.transform(msg2, new ujson.StringRenderer()).toString ==> """{"[1,2]":1}"""

Note that such a conversion between MessagePack structs and JSON data is lossy: some MessagePack constructs, such as binary data, cannot be exactly represented in JSON and have to be converted to strings. Thus you should not rely on being able to round-trip data between JSON <-> MessagePack and getting the same thing back, although round tripping data between Scala-data-types <-> JSON and Scala-data-types <-> MessagePack should always work.

Some of the differences between the ways things are serialized in MessagePack and JSON include:

If you need to construct Scala case classes or other data types from your MessagePack binary data, you should directly use upickle.default.readBinary and upickle.default.writeBinary: these bypass the upack.Msg struct entirely for the optimal performance.

Performance


The uPickle has a small set of benchmarks in bench/ that tests reading and writing performance of a few common JSON libraries on a small, somewhat arbitrary workload. The numbers below show how many times each library could read/write a small data structure in 25 seconds (bigger numbers better). In some libraries, caching the serializers rather than re-generating them each read/write also improves performance: that effect can be seen in the (Cached) columns.

JVM Case Class Serialization Performance

uPickle runs 30-50% faster than Circe for reads/writes, and ~200% faster than play-json.

Library Reads Writes Reads (Cached) Write (Cached)
Play Json 2.9.2 331 296 361 309
Circe 0.13.0 517 504 526 502
upickle.default 1.3.0 (JSON Strings) 809 728 822 864
upickle.default 1.3.0 (JSON Array[Byte]) 761 706 774 830
upickle.default 1.3.0 (MsgPack Array[Byte]) 1652 1264 1743 1753

As you can see, uPickle's JSON serialization is pretty consistently ~50% faster than Circe for reads and writes, and 100-200% faster than Play-Json, depending on workload.

uPickle's binary MessagePack backend is then another 100% faster than uPickle JSON.

uPickle achieves this speed by avoiding the construction of an intermediate JSON AST: while most libraries parse from String -> AST -> CaseClass, uPickle parses input directly from String -> CaseClass. uPickle also provides a ujson.Js AST that you can use to manipulate arbitrary JSON, but ujson.Js plays no part in parsing things to case-classes and is purely for users who want to manipulate JSON.

JS Case Class Serialization Performance

While all libraries are much slower on Scala.js/Node.js than on the JVM, uPickle runs 4-5x as fast as Circe or Play-Json for reads and writes.

Library Reads Writes Reads (Cached) Write (Cached)
Play Json 2.9.2 64 81 66 86
Circe 0.13.0 98 121 99 99
upickle.default 1.3.0 (JSON String) 76 40 76 44
upickle.default 1.3.0 (JSON Array[Byte] 68 107 69 112
upickle.default.web 1.3.0 (JSON String) 349 327 353 465
upickle.default 1.3.0 (MsgPack Array[Byte]) 85 104 85 110

On Scala.js, uPickle's performance is comparable to othersl ike Play JSON or Circe. However, uPickle also exposes the upickle.default.web API, allowing you to use the JS runtime's built-in JSON parser to power serialization and deserialization. This ends up being 4-6x faster than the Scala-based JSON parsers of all the libraries compared (uPickle, Play-JSON, Circe)

uJson is a fork of Erik Osheim's excellent [Jawn](https://github.com/non/jawn) JSON library, and inherits a lot of it's performance from Erik's work.

Version History


4.0.2

4.0.1

4.0.0

3.3.1

3.3.0

3.2.0

3.1.4

3.1.3

3.1.2

3.1.1

3.1.0

3.0.0

2.0.0

1.6.0

1.5.0

1.4.3

  • MsgPackReader: properly increment index in ext #370

    • 1.4.2

    1.4.1

    1.4.0

    1.3.11

    1.3.8

    1.3.7

    1.3.0

    1.2.0

    0.9.6

    0.9.0

    0.8.0

    0.7.5

    0.7.1

    0.6.7

    0.6.6

    0.6.5

    0.6.4

    0.6.3

    0.6.2

    0.6.0

    0.5.1

    0.4.3

    0.4.1

    0.4.1

    0.4.0

    0.3.9

    0.3.8

    0.3.7

    0.3.6

    0.3.5

    0.3.4

    0.3.3

    0.3.2

    0.3.1

    0.3.0

    0.2.8

    0.2.7

    0.2.6

    0.2.5

    0.2.4

    0.2.3

    0.2.2

    0.2.1

    0.2.0

    0.1.7

    0.1.6

    0.1.5

    0.1.4

    0.1.3