Document notifier tradeoffs (#1169)

Capturing insights from the [elixir forum][1] and from an Oban support email.

[1]: https://elixirforum.com/t/does-oban-cancel-job-2-stop-a-running-job-if-called-from-a-rolled-back-transaction/65010

guides/scaling.md

@@ -4,13 +4,14 @@
 
 Oban uses PubSub notifications for communication between nodes, like job inserts, pausing queues,
 resuming queues, and metrics for Web. The default notifier is `Oban.Notifiers.Postgres`, which
-sends all messages through the database. Postgres' notifications adds up at scale because each one
-requires a separate query.
-
-If you're clustered, switch to an alternative notifier like `Oban.Notifiers.PG`. That keeps
-notifications out of the db, reduces total queries, and allows larger messages. As long as you
-have a functional Distributed Erlang cluster, then it’s a single line change to your Oban
-config.
+sends all messages through the database. This provides transactional consistency, but Postgres'
+notifications adds up at scale because each one requires a separate query.
+
+If you're clustered, consider switching to an alternative notifier like `Oban.Notifiers.PG`. That
+keeps notifications out of the db, reduces total queries, and allows larger messages, with the
+tradeoff that notifications from within a database transaction may be sent even if the transaction
+is rolled back. As long as you have a functional Distributed Erlang cluster, switching is a
+single line change to your Oban config.
 
 ```diff
  config :my_app, Oban,

lib/oban/notifier.ex

@@ -13,7 +13,7 @@ defmodule Oban.Notifier do
   an implementation of the `Oban.Notifier` behaviour. 
 
   * `Oban.Notifiers.Postgres` — A Postgres notifier that uses `LISTEN/NOTIFY` to broadcast
-    messages. This is the default.
+    messages.
 
   * `Oban.Notifiers.PG` — A process groups notifier that relies on Distributed Erlang to broadcast
     messages.

lib/oban/notifiers/pg.ex

@@ -1,6 +1,8 @@
 defmodule Oban.Notifiers.PG do
   @moduledoc """
   A [PG (Process Groups)][pg] based notifier implementation that runs with Distributed Erlang.
+  This notifier scales much better than `Oban.Notifiers.Postgres` but lacks its transactional
+  guarantees.
 
   > #### Distributed Erlang {: .info}
   >

lib/oban/notifiers/postgres.ex

@@ -22,6 +22,11 @@ if Code.ensure_loaded?(Postgrex) do
 
     The notifications system is built on PostgreSQL's `LISTEN/NOTIFY` functionality. Notifications
     are only delivered **after a transaction completes** and are de-duplicated before publishing.
+    This means that notifications sent during a transaction will not be sent if the transaction is
+    rolled back, providing consistency; this is the only notifer which provides that guarantee.
+    However, it is not as scalable as other notifiers because because each notification requires a
+    separate query and notifications can't exceed 8kb.
+
     Typically, applications run Ecto in sandbox mode while testing, but sandbox mode wraps each test
     in a separate transaction that's rolled back after the test completes. That means the
     transaction is never committed, which prevents delivering any notifications.