Add documentation and examples (#158)

* added basic documentation with example * added more dependencies to docs * incorporated feedback by @thinkharderdev * fixed wording 'ZIO-Schema' -> 'ZIO Schema' * fixed naming of other zio libraries * fix sbt publish setting * added examples subproject * docs: fixed mdoc link and added version placeholder in overview/index.md * docs: incorporated suggestions by @TobiasPfeifer * scalafmt for build.sbt * incorporated suggestions by @thinkharderdev * Support for annotations on all types (#147) * Add addonation to Schema[A] interface and add as field in Schema.Sequence * Add annotation field to Schema.Transform * Add annotation field to Schema.Primitive * Add annotations field to Schema.Optional * Add annotations field to Schema.Fail * Add annotations field to Schema.Tuple * Add annotations field to EitherSchema * Add annotations field to Schema.Lazy * Add annotations field to Schema.Meta * Add remaining assertion equality checks to equalsSchema * Remove annotations from Record[A] (since it exists in Schema[A] already) and add annotations field to Schema.GenericRecord * Fix test case for schema generation on annotated ADT * Fix handling of annotations for Lazy (and thus failing tests) * Wherever we accept an annotations chunk in the constructor/method we default it to Chunk.empty * Add `def annotate(annotation: Any): Schema[A]` to Schema * Replace usage of Chunk.appended with :+ to fix 2.12.12 compilation * Provide direct support for Map (#142) * Provide direct support for Map Unit tests for json serdes Unit tests for protobuf serdes * run scalafmt * Rebased/reformatted * Rebased/reformatted, and added a minimum contribution guide * Run sbt `prepare test` instead of `fmt test` * Add defaultValue to Schema (#80) * feat(schema): add first draft defaultValue implementation for Schema * feat(schema): add missing test cases * feat(schema): change default value for BoolType to false * feat(schema): fold default values in Record type appropriately * fix(zio-schema): fix failing test * feat(zio-schema): add defaultValue implementation for MapSchema * patch(zio-schema): remove orElse for Scala 2.12 compat * first commit * [zio-schema-examples] sbt setup * [zio-schema-examples] readme init * [zio-schema-examples] example 1 * [zio-schema-examples] example 2 * [zio-schema-examples] example 3 (WIP) * [zio-schema-examples] example 3 * [zio-schema-examples] Example3: Don't share domain with other samples * [zio-schema-examples] Example4 transforming DTOs * [zio-schema-examples] Example5 diffing * [zio-schema-examples] example 4: fixed map/flatmap in example * [zio-schema-examples] example 4: another try to converting DTOs * [zio-schema-examples] example 6 reified optics stub * [zio-schema-examples] example 6 reified optics examples * [zio-schema-examples] example 1 + 2: schema fixes for newest zio-schema * [zio-schema-examples] Problem 7: Initial code * [zio-schema-examples] Example 6: adding employee now works * [zio-schema-examples] Example 7: added notes on performance requirements * [zio-schema-examples] Example 7: added parameter list to methods, fixed names * [zio-schema-examples] Example 7: added Person and Profile schema as implicits * [zio-schema-examples] Example 7: renamed method for viewing, added Runner * [zio-schema-examples] Example 7: sketched toDynamicValue method * [zio-schema-examples] Example 7: implemented toDynamicValue method * [zio-schema-examples] Example 7: primitive impl of decode using toDV * [zio-schema-examples] Example 7: produce multiple solutions, pick working one * [zio-schema-examples] Example 7: introduce QueryParam type alias and compile-function * [zio-schema-examples] Example 7: return static function on error case * [zio-schema-examples] Example 7: initial impl of Primitive(standardtype) * [zio-schema-examples] Example 7: implemented fail * [zio-schema-examples] Example 7: simple impl of lazy * [zio-schema-examples] Example 7: better lazy impl with `lazy val` * [zio-schema-examples] Example 7: impl of Meta * [zio-schema-examples] Example 7: impl of CaseClass1 * [zio-schema-examples] Example 7: impl of CaseClass2 * [zio-schema-examples] Example 7: impl of CaseClass3 * [zio-schema-examples] Example 7: extracted compiler to builder * [zio-schema-examples] Example 7: implemented Transform * [zio-schema-examples] * Example for an Encoder and Decoder using the DynamicValue API * Add scalafmt * Update zio-schema 0.1.2 * Update examples to account for annotations * revert local changes * moved sources to appropriate place * fixed example * [zio-schema-examples] reverted changes to build.sbt * [zio-schema-examples] removed additional build.sbt, fixed problems in example7, added explicit import in build.sbt * [zio-schema-examples] scalaFmt applied * Fix build for scala 2.12 and apply linter * Fix compilation errors for scala 2.12 * Unused imports Co-authored-by: Dominik Dorn <[email protected]> Co-authored-by: Alexander van Olst <[email protected]> Co-authored-by: Pierangelo Cecchetto <[email protected]> Co-authored-by: Maxwell Brown <[email protected]> Co-authored-by: calvinlfer <[email protected]>
zio · Dec 14, 2021 · e0c510d · e0c510d
1 parent c252e37
commit e0c510d
Show file tree

Hide file tree

Showing 18 changed files with 1,743 additions and 10 deletions.
diff --git a/README.md b/README.md
@@ -9,9 +9,9 @@ _ZIO Schema_ is a [ZIO](https://zio.dev)-based library for modeling the schema o
 With schema descriptions that can be automatically derived for case classes and sealed traits, _ZIO Schema_ provide powerful features for free:
 
  - Codecs for any supported protocol (JSON, protobuf, etc.), so data structures can be serialized and deserialized in a principled way
- - Diffing, patching, merging, and other generic-data-based operations (_TODO_)
- - Migration of data structures from one schema to another compatible schema (_TODO_)
- - Derivation of arbitrary type classes (`Eq`, `Show`, `Ord`, etc.) from the structure of the data (_TODO_)
+ - Diffing, patching, merging, and other generic-data-based operations
+ - Migration of data structures from one schema to another compatible schema
+ - Derivation of arbitrary type classes (`Eq`, `Show`, `Ord`, etc.) from the structure of the data
 
 When your data structures need to be serialized, deserialized, persisted, or transported across the wire, then _ZIO Schema_ lets you focus on data modeling and automatically tackle all the low-level, messy details for you.
 
@@ -22,7 +22,12 @@ _ZIO Schema_ is used by a growing number of ZIO libraries, including _ZIO Flow_,
 Add in your `build.sbt`:
 
 ```scala
-libraryDependencies += "dev.zio" %% "zio-schema" % "<version>"
+libraryDependencies ++= Seq(
+  "dev.zio" %% "zio-schema" % "<version>",
+  // Required for automatic generic derivation of schemas
+  "dev.zio" %% "zio-schema-derivation" % "<version>",
+  "org.scala-lang" % "scala-reflect"  % scalaVersion.value % "provided"
+)
 ```
 
 [Badge-CI]: https://github.com/zio/zio-schema/workflows/CI/badge.svg

diff --git a/build.sbt b/build.sbt
@@ -48,8 +48,8 @@ lazy val root = project
   .in(file("."))
   .settings(
     name := "zio-schema",
-    publish / skip := true,
-    unusedCompileDependenciesFilter -= moduleFilter("org.scala-js", "scalajs-library")
+    publish / skip := true
+//    unusedCompileDependenciesFilter -= moduleFilter("org.scala-js", "scalajs-library")
   )
   .aggregate(
     zioSchemaJVM,
@@ -62,6 +62,8 @@ lazy val root = project
     zioSchemaOpticsJVM,
     zioSchemaProtobufJS,
     zioSchemaProtobufJVM,
+    zioSchemaExamplesJS,
+    zioSchemaExamplesJVM,
     testsJVM,
     testsJS,
     zioSchemaZioTestJVM,
@@ -181,6 +183,23 @@ lazy val zioSchemaOpticsJS = zioSchemaOptics.js
 lazy val zioSchemaOpticsJVM = zioSchemaOptics.jvm
   .settings(Test / fork := true)
 
+lazy val zioSchemaExamples = crossProject(JSPlatform, JVMPlatform)
+  .in(file("zio-schema-examples"))
+  .settings(stdSettings("zio-schema-examples"))
+  .settings(crossScalaVersions -= Scala212)
+  .dependsOn(zioSchema, zioSchemaJson, zioSchemaProtobuf, zioSchemaOptics)
+  .settings(
+    publish / skip := true,
+    moduleName := "zio-schema-example",
+    scalacOptions -= "-Yno-imports",
+    scalacOptions -= "-Xfatal-warnings"
+  )
+
+lazy val zioSchemaExamplesJS = zioSchemaExamples.js
+  .settings(scalaJSUseMainModuleInitializer := true)
+
+lazy val zioSchemaExamplesJVM = zioSchemaExamples.jvm
+
 lazy val zioSchemaZioTest = crossProject(JSPlatform, JVMPlatform)
   .in(file("zio-schema-zio-test"))
   .dependsOn(zioSchema, zioSchemaDerivation, tests % "test->test")

diff --git a/docs/overview/first_schema.md b/docs/overview/first_schema.md
@@ -0,0 +1,203 @@
+---
+id: first_schema
+title: "Your first ZIO Schema"
+---
+## Your First ZIO Schema
+
+ZIO Schema provides macros to help you create `Schema`s out of your data types. But before using the macros,
+we should take a look at how to do this the manual way.
+
+### The Domain
+Like in [Overview](index.md), we define our example domain like this:
+
+```scala
+object Domain {
+  final case class Person(name: String, age: Int)
+
+  sealed trait PaymentMethod
+
+  object PaymentMethod {
+    final case class CreditCard(number: String, expirationMonth: Int, expirationYear: Int) extends PaymentMethod
+    final case class WireTransfer(accountNumber: String, bankCode: String) extends PaymentMethod
+  }
+
+  final case class Customer(person: Person, paymentMethod: PaymentMethod)
+
+}
+```
+
+### Manual construction of a Schema
+
+This part is similar to other libraries that you might know, e.g. for JSON processing.
+Basically, you create a `Schema` for every data type in your domain:
+
+```scala
+
+object ManualConstruction {
+  import zio.schema.Schema._
+  import Domain._
+  import Domain.PaymentMethod._
+
+  val schemaPerson: Schema[Person] = Schema.CaseClass2[String, Int, Person](
+    field1 = Schema.Field[String]("name", Schema.primitive[String]),
+    field2 = Schema.Field[Int]("age", Schema.primitive[Int]),
+    construct = (name, age) => Person(name, age),
+    extractField1 = p => p.name,
+    extractField2 = p => p.age
+  )
+
+  val schemaPaymentMethodWireTransfer: Schema[WireTransfer] = Schema.CaseClass2[String, String, WireTransfer](
+    field1 = Schema.Field[String]("accountNumber", Schema.primitive[String]),
+    field2 = Schema.Field[String]("bankCode", Schema.primitive[String]),
+    construct = (number, bankCode) => PaymentMethod.WireTransfer(number, bankCode),
+    extractField1 = p => p.accountNumber,
+    extractField2 = p => p.bankCode
+  )
+
+  val schemaPaymentMethodCreditCard: Schema[CreditCard] = Schema.CaseClass3[String, Int, Int, CreditCard](
+    field1 = Schema.Field[String]("number", Schema.primitive[String]),
+    field2 = Schema.Field[Int]("expirationMonth", Schema.primitive[Int]),
+    field3 = Schema.Field[Int]("expirationYear", Schema.primitive[Int]),
+    construct = (number, expirationMonth, expirationYear) => PaymentMethod.CreditCard(number, expirationMonth, expirationYear),
+    extractField1 = p => p.number,
+    extractField2 = p => p.expirationMonth,
+    extractField3 = p => p.expirationYear
+  )
+
+  val schemaPaymentMethod: Schema[PaymentMethod] = Schema.Enum2(
+    case1 = Case[PaymentMethod.CreditCard, PaymentMethod](
+      id = "CreditCard",
+      codec = schemaPaymentMethodCreditCard,
+      unsafeDeconstruct = pm => pm.asInstanceOf[PaymentMethod.CreditCard]
+    ),
+    case2 = Case[PaymentMethod.WireTransfer, PaymentMethod](
+      id = "WireTransfer",
+      codec = schemaPaymentMethodWireTransfer,
+      unsafeDeconstruct = pm => pm.asInstanceOf[PaymentMethod.WireTransfer]
+    )
+  )
+
+  val schemaCustomer: Schema[Customer] = Schema.CaseClass2[Person, PaymentMethod, Customer](
+    field1 = Schema.Field[Person]("person", schemaPerson),
+    field2 = Schema.Field[PaymentMethod]("paymentMethod", schemaPaymentMethod),
+    construct = (person, paymentMethod) => Customer(person, paymentMethod),
+    extractField1 = c => c.person,
+    extractField2 = c => c.paymentMethod
+  )
+}
+
+```
+
+### Macro derivation
+Using macros, the above code gets reduced to this:
+
+```scala
+
+object MacroConstruction  {
+  import Domain._
+
+  val schemaPerson: Schema[Person] = DeriveSchema.gen[Person]
+
+  val schemaPaymentMethod: Schema[PaymentMethod] = DeriveSchema.gen[PaymentMethod]
+
+  val schemaCustomer: Schema[Customer] = DeriveSchema.gen[Customer]
+}
+```
+
+## Applying it to our domain
+
+### Json example
+Lets put this all together in a small sample:
+```scala
+object JsonSample extends zio.App {
+  import zio.schema.codec.JsonCodec
+  import ManualConstruction._
+  import zio.stream.ZStream
+
+  override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] = for {
+    _ <- ZIO.unit
+    person = Person("Michelle", 32)
+    personToJsonTransducer = JsonCodec.encoder[Person](schemaPerson)
+    _ <- ZStream(person)
+      .transduce(personToJsonTransducer)
+      .transduce(ZTransducer.utf8Decode)
+      .foreach(ZIO.debug)
+  } yield ExitCode.success
+}
+```
+
+When we run this, we get our expected result printed out:
+```json
+{"name":"Michelle","age":32}
+```
+
+### Protobuf example
+
+```scala
+object ProtobufExample extends zio.App {
+  import zio.schema.codec.ProtobufCodec
+  import ManualConstruction._
+  import zio.stream.ZStream
+
+  override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] = for {
+    _ <- ZIO.unit
+    _ <- ZIO.debug("protobuf roundtrip")
+    person = Person("Michelle", 32)
+
+    personToProto = ProtobufCodec.encoder[Person](schemaPerson)
+    protoToPerson = ProtobufCodec.decoder[Person](schemaPerson)
+
+    newPerson <- ZStream(person)
+      .transduce(personToProto)
+      .transduce(protoToPerson)
+      .runHead
+      .some
+      .catchAll(error => ZIO.debug(error))
+    _ <- ZIO.debug("is old person the new person? " + (person == newPerson).toString)
+    _ <- ZIO.debug("old person: " + person)
+    _ <- ZIO.debug("new person: " + newPerson)
+  } yield ExitCode.success
+}
+```
+
+
+### Combining different encoders
+Let's take a look at a roundtrip converting an object to JSON and back, then converting it to a protobuf and back.
+This is a simple example, but it shows how to combine different encoders to achieve a roundtrip.
+
+```scala
+object CombiningExample extends zio.App {
+  import zio.schema.codec.JsonCodec
+  import zio.schema.codec.ProtobufCodec
+  import ManualConstruction._
+  import zio.stream.ZStream
+
+  override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] = for {
+    _ <- ZIO.unit
+    _ <- ZIO.debug("combining roundtrip")
+    person = Person("Michelle", 32)
+
+    personToJson = JsonCodec.encoder[Person](schemaPerson)
+    jsonToPerson = JsonCodec.decoder[Person](schemaPerson)
+
+    personToProto = ProtobufCodec.encoder[Person](schemaPerson)
+    protoToPerson = ProtobufCodec.decoder[Person](schemaPerson)
+
+    newPerson <- ZStream(person)
+      .tap(v => ZIO.debug("input object is: " + v))
+      .transduce(personToJson)
+      .transduce(jsonToPerson)
+      .tap(v => ZIO.debug("object after json roundtrip: " + v))
+      .transduce(personToProto)
+      .transduce(protoToPerson)
+      .tap(v => ZIO.debug("person after protobuf roundtrip: " + v))
+      .runHead
+      .some
+      .catchAll(error => ZIO.debug(error))
+    _ <- ZIO.debug("is old person the new person? " + (person == newPerson).toString)
+    _ <- ZIO.debug("old person: " + person)
+    _ <- ZIO.debug("new person: " + newPerson)
+  } yield ExitCode.success
+}
+```
+
diff --git a/docs/overview/index.md b/docs/overview/index.md
@@ -3,8 +3,55 @@ id: overview_index
 title: "Contents"
 ---
 
-TODO: List of contents available
-
 ## Installation
 
-TODO: Installation details
+Add in your `build.sbt`:
+
+```scala
+libraryDependencies += "dev.zio" %% "zio-schema" % "@RELEASE_VERSION@"
+libraryDependencies += "dev.zio" %% "zio-schema-json" % "@RELEASE_VERSION@"
+libraryDependencies += "dev.zio" %% "zio-schema-protobuf" % "@RELEASE_VERSION@"
+```
+
+## Purpose of ZIO Schema
+ZIO Schema allows you to create representations of your data types as values. 
+
+Once you have a representation of your data types, you can use it to 
+ - serialize and deserialize your types
+ - validate your types
+ - transform your types
+ - create instances of your types
+
+You can then use one of the various codecs (or create your own) to serialize and deserialize your types.
+
+Example of possible codecs are:
+ - CSV Codec
+ - JSON Codec (already available)
+ - Apache Avro Codec (in progress)
+ - Apache Thrift Codec (in progress)
+ - XML Codec
+ - YAML Codec
+ - Protobuf Codec (already available)
+ - QueryString Codec
+ - etc.
+
+Example use cases that are possible:
+ - Serializing and deserializing JSON
+ - Serializing and deserializing XML
+ - Validating JSON
+ - Validating XML
+ - Transforming JSON
+ - Transforming XML
+ - Transforming JSON to XML
+ - Transforming XML to JSON
+ - Creating diffs from arbitrary data structures
+ - Creating migrations / evolutions e.g. of Events used in Event-Sourcing
+ - Transformation pipelines, e.g. 
+   1. convert from protobuf to object, e.g. `PersonDTO`,
+   2. transform to another representation, e.g. `Person`,
+   3. validate
+   4. transform to JSON `JsonObject`
+   5. serialize to `String`
+
+
+