feat: upgrade to main-0.22

INSTALL.md

@@ -6,7 +6,7 @@ The config file [`tinode.conf`](./server/tinode.conf) contains extensive instruc
 
 1. Visit the [Releases page](https://github.com/tinode/chat/releases/), choose the latest or otherwise the most suitable release. From the list of binaries download the one for your database and platform. Once the binary is downloaded, unpack it to a directory of your choosing, `cd` to that directory.
 
-2. Make sure your database is running. Make sure it's configured to accept connections from `localhost`. In case of MySQL, Tinode will try to connect as `root` without the password. See notes below (_Building from Source_, section 4) on how to configure Tinode to use a different user or a password. MySQL 5.7 or above is required. MySQL 5.6 or below **will not work**.
+2. Make sure your database is running. Make sure it's configured to accept connections from `localhost`. In case of MySQL, Tinode will try to connect as `root` without the password. In case of PostgreSQL, Tinode will try connect as `postgres` with the password `postgres`. See notes below (_Building from Source_, section 4) on how to configure Tinode to use a different user or a password. MySQL 5.7 or above is required. MySQL 5.6 or below **will not work**. PostgreSQL 13 or above is required. PostgreSQL 12 or below **will not work**.
 
 3. Run the database initializer `init-db` (or `init-db.exe` on Windows):
 	```
@@ -34,7 +34,8 @@ See [instructions](./docker/README.md)
 
 3. Make sure one of the following databases is installed and running:
  * MySQL 5.7 or above. MySQL 5.6 or below **will not work**.
- * MongoDB 4.0 or above.
+ * PostgreSQL 13 or above. PostgreSQL 12 or below **will not work**.
+ * MongoDB 4.2 or above.
  * RethinkDB.
 
 4. Fetch, build Tinode server and tinode-db database initializer:
@@ -43,6 +44,11 @@ See [instructions](./docker/README.md)
 	go install -tags mysql github.com/tinode/chat/server@latest
 	go install -tags mysql github.com/tinode/chat/tinode-db@latest
 	```
+	- **PostgreSQL**:
+	```
+	go install -tags postgres github.com/tinode/chat/server@latest
+	go install -tags postgres github.com/tinode/chat/tinode-db@latest
+	```
   - **MongoDB**:
 	```
 	go install -tags mongodb github.com/tinode/chat/server@latest
@@ -55,13 +61,13 @@ See [instructions](./docker/README.md)
 	```
   - **All** (bundle all of the above DB adapters):
 	```
-	go install -tags "mysql rethinkdb mongodb" github.com/tinode/chat/server@latest
-	go install -tags "mysql rethinkdb mongodb" github.com/tinode/chat/tinode-db@latest
+	go install -tags "mysql rethinkdb mongodb postgres" github.com/tinode/chat/server@latest
+	go install -tags "mysql rethinkdb mongodb postgres" github.com/tinode/chat/tinode-db@latest
 	```
 
     The steps above install Tinode binaries at `$GOPATH/bin/`, sorces and supporting files are located at `$GOPATH/pkg/mod/github.com/tinode/[email protected]/` where `X.XX.X` is the version you installed, such as `0.19.1`.
 
-    Note the required **`-tags rethinkdb`**, **`-tags mysql`** or **`-tags mongodb`** build option.
+    Note the required **`-tags rethinkdb`**, **`-tags mysql`**, **`-tags mongodb`** or **`-tags postgres`** build option.
 
     You may also optionally define `main.buildstamp` for the server by adding a build option, for instance, with a timestamp:
     ```
@@ -109,6 +115,10 @@ cd $GOPATH/pkg/mod/github.com/tinode/[email protected]
 	```
 	mysql.server start
 	```
+ - **PostgreSQL**: https://www.postgresql.org/docs/current/app-pg-ctl.html
+	```
+	pg_ctl start
+	```
  - **MongoDB**: https://docs.mongodb.com/manual/administration/install-community/
 MongoDB should run as single node replicaset. See https://docs.mongodb.com/manual/administration/replica-set-deployment/
 	```
@@ -202,6 +212,8 @@ Tinode does not provide ICE servers out of the box. You must install and configu
 
 Once you obtain the ICE TURN/STUN configuration from your service provider, add it to `tinode.conf` section `"webrtc"` - `"ice_servers"` (or `"ice_servers_file"`). Also change `"webrtc"` - `"enabled"` to `true`. An example configuration is provided in the `tinode.conf` for illustration only. IT WILL NOT FUNCTION because it uses dummy values instead of actual server addresses.
 
+You may find this information useful for choosing the servers: https://gist.github.com/yetithefoot/7592580
+
 
 ### Note on Running the Server in Background
 

README.md

@@ -1,10 +1,10 @@
 # Tinode Instant Messaging Server
 
-<img src="docs/logo.svg" align="left" width=128 height=128> Instant messaging server. Backend in pure [Go](http://golang.org) (license [GPL 3.0](http://www.gnu.org/licenses/gpl-3.0.en.html)), client-side binding in Java, Javascript, and Swift, as well as [gRPC](https://grpc.io/) client support for C++, C#, Go, Java, Node, PHP, Python, Ruby, Objective-C, etc. (license [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0)). Wire transport is JSON over websocket (long polling is also available) for custom bindings, or [protobuf](https://developers.google.com/protocol-buffers/) with gRPC. Persistent storage is any one of MySQL, MongoDB or [RethinkDB](http://rethinkdb.com/). Other databases can be supported by writing custom adapters.
+<img src="docs/logo.svg" align="left" width=128 height=128> Instant messaging server. Backend in pure [Go](http://golang.org) (license [GPL 3.0](http://www.gnu.org/licenses/gpl-3.0.en.html)), client-side binding in Java, Javascript, and Swift, as well as [gRPC](https://grpc.io/) client support for C++, C#, Go, Java, Node, PHP, Python, Ruby, Objective-C, etc. (license [Apache 2.0](http://www.apache.org/licenses/LICENSE-2.0)). Wire transport is JSON over websocket (long polling is also available) for custom bindings, or [protobuf](https://developers.google.com/protocol-buffers/) with gRPC.
 
 Tinode is *not* XMPP/Jabber. It is *not* compatible with XMPP. It's meant as a replacement for XMPP. On the surface, it's a lot like open source WhatsApp or Telegram.
 
-Version 0.20. This is beta-quality software: feature-complete and stable but probably with a few bugs or missing features. Follow [instructions](INSTALL.md) to install and run or use one of the cloud services below. Read [API documentation](docs/API.md).
+This is beta-quality software: feature-complete and stable but probably with a few bugs or missing features. Follow [instructions](INSTALL.md) to install and run or use one of the cloud services below. Read [API documentation](docs/API.md).
 
 <a href="https://apps.apple.com/us/app/tinode/id1483763538"><img src="docs/app-store.svg" height=36></a> <a href="https://play.google.com/store/apps/details?id=co.tinode.tindroidx"><img src="docs/play-store.svg" height=36></a> <a href="https://web.tinode.co/"><img src="docs/web-app.svg" height=36></a>
 
@@ -105,14 +105,18 @@ When you register a new account you are asked for an email address to send valid
   * Scriptable [command-line tool](tn-cli/) for server administration.
 * Performance, reliability and development:
   * Sharded clustering with failover.
-  * Storage and out of band transfer of large objects like images or document files using local file system or Amazon S3 (other storage systems can be supported with plugins).
+  * Storage and out of band transfer of large objects like images or document files using local file system or Amazon S3 (other storage systems can be supported with [media handlers](https://github.com/tinode/chat/blob/master/server/media/media.go#L21)).
   * JSON or [protobuf version 3](https://developers.google.com/protocol-buffers/) wire protocols.
   * Bindings for various programming languages:
     * Javascript with no external dependencies.
     * Java with dependencies on [Jackson](https://github.com/FasterXML/jackson) and [Java-Websocket](https://github.com/TooTallNate/Java-WebSocket). Suitable for Android but with no Android SDK dependencies.
     * Swift with no external dependencies.
     * C/C++, C#, Go, Python, PHP, Ruby and many other languages using [gRPC](https://grpc.io/docs/languages/).
-  * Choice of a database backend: MySQL, RethinkDB, MongoDB.
+  * Choice of a database backend. Other databases can be added with by writing [adapters](server/db/adapter.go).
+    * MySQL
+    * PostgreSQL
+    * MongoDB
+    * [RethinkDB](http://rethinkdb.com/)
 
 ### Planned
 
@@ -139,14 +143,15 @@ All client software has support for [internationalization](docs/translations.md)
 | Chinese (traditional) |   | &check; | &check; | &check; |
 | French | &check; | &check; | &check; |   |
 | German |   | &check; | &check; |   |
+| Hindi |   |   | &check; |   |
 | Korean |   | &check; | &check; |   |
 | Portugese | &check; |   | &check; |   |
 | Romanian |   | &check; |   |   |
 | Russian | &check; | &check; | &check; | &check; |
 | Spanish | &check; | &check; | &check; | &check; |
 | Vietnamese | &check; |   |   |   |
 
-More translations are [welcome](docs/translations.md). Particularly interested in Arabic, Vietnamese, Persian, Indonesian, Portuguese, Hindi, Bengali, Turkish.
+More translations are [welcome](docs/translations.md). In addition to languages listed above, particularly interested in Arabic, Bengali, Indonesian, Urdu, Japanese, Turkish, Vietnamese, Persian.
 
 ## Third-Party
 
@@ -200,10 +205,12 @@ Words 'chat' and 'instant messaging' in Chinese, Russian, Persian and a few othe
 * 인스턴트 메신저
 * پیام رسان فوری
 * تراسل فوري
+* فوری پیغام رسانی
 * Nhắn tin tức thời
 * anlık mesajlaşma sohbet
 * mensageiro instantâneo
 * pesan instan
 * mensajería instantánea
 * চ্যাট ইন্সট্যান্ট মেসেজিং
 * चैट त्वरित संदेश
+* তাৎক্ষণিক বার্তা আদান প্রদান

build-all.sh

@@ -14,7 +14,7 @@ goarc=( amd64 arm64 amd64 amd64 )
 buildCount=${#goplat[@]}
 
 # Supported database tags
-dbadapters=( mysql mongodb rethinkdb )
+dbadapters=( mysql mongodb rethinkdb postgres )
 dbtags=( ${dbadapters[@]} alldbs )
 
 for line in $@; do
@@ -72,7 +72,7 @@ then
   cp ./server/static/manifest.json ./releases/tmp/static
   cp ./server/static/service-worker.js ./releases/tmp/static
   # Create empty FCM client-side config.
-  touch ./releases/tmp/static/firebase-init.js
+  echo 'const FIREBASE_INIT = {};' > ./releases/tmp/static/firebase-init.js
 else
   echo "TinodeWeb not found, skipping"
 fi

docker-build.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-# Build Tinode docker linux/amd64 images. 
+# Build Tinode docker linux/amd64 images.
 # You may have to install buildx https://docs.docker.com/buildx/working-with-buildx/
 # if your build host and target architectures are different (e.g. building on a Mac
 # with Apple silicon).
@@ -30,7 +30,7 @@ if [ `uname -m` != 'x86_64' ]; then
   buildcmd='buildx build --platform=linux/amd64'
 fi
 
-dbtags=( mysql mongodb rethinkdb alldbs )
+dbtags=( mysql postgres mongodb rethinkdb alldbs )
 
 # Build an images for various DB backends
 for dbtag in "${dbtags[@]}"

docker-release.sh

@@ -31,7 +31,7 @@ if [[ ${ver[2]} != *"-"* ]]; then
   FULLRELEASE=1
 fi
 
-dbtags=( mysql mongodb rethinkdb alldbs )
+dbtags=( mysql postgres mongodb rethinkdb alldbs )
 
 # Read dockerhub login/password from a separate file
 source .dockerhub

docker/README.md

@@ -11,17 +11,19 @@ All images are available at https://hub.docker.com/r/tinode/
 
 3. Decide which database backend you want to use: RethinkDB, MySQL or MongoDB. Run the selected database container, attaching it to `tinode-net` network:
 
-	1. **RethinkDB**: If you've decided to use RethinkDB backend, run the official RethinkDB Docker container:
+	1. **MySQL**: If you've decided to use MySQL backend, run the official MySQL Docker container:
 	```
-	$ docker run --name rethinkdb --network tinode-net --restart always -d rethinkdb:2.3
+	$ docker run --name mysql --network tinode-net --restart always --env MYSQL_ALLOW_EMPTY_PASSWORD=yes -d mysql:5.7
 	```
-	See [instructions](https://hub.docker.com/_/rethinkdb/) for more options.
+	See [instructions](https://hub.docker.com/_/mysql/) for more options. MySQL 5.7 or above is required.
 
-	2. **MySQL**: If you've decided to use MySQL backend, run the official MySQL Docker container:
+	2. **PostgreSQL**: If you've decided to use PostgreSQL backend, run the official PostgreSQL Docker container:
 	```
-	$ docker run --name mysql --network tinode-net --restart always --env MYSQL_ALLOW_EMPTY_PASSWORD=yes -d mysql:5.7
+	$ docker run --name postgres --network tinode-net --restart always --env POSTGRES_PASSWORD=postgres -d postgres:13
 	```
-	See [instructions](https://hub.docker.com/_/mysql/) for more options. MySQL 5.7 or above is required.
+	See [instructions](https://hub.docker.com/_/postgres/) for more options. PostgresSQL 13 or above is required.
+
+	The name `rethinkdb`, `mysql`, `mongodb` or `postgres` in the `--name` assignment is important. It's used by other containers as a database's host name.
 
 	3. **MongoDB**: If you've decided to use MongoDB backend, run the official MongoDB Docker container and initialise it as single node replica set (you can change "rs0" if you wish):
    ```
@@ -34,25 +36,34 @@ All images are available at https://hub.docker.com/r/tinode/
    ```
 	See [instructions](https://hub.docker.com/_/mongo/) for more options. MongoDB 4.2 or above is required.
 
-	The name `rethinkdb`, `mysql` or `mongodb` in the `--name` assignment is important. It's used by other containers as a database's host name.
+	4. **RethinkDB**: If you've decided to use RethinkDB backend, run the official RethinkDB Docker container:
+	```
+	$ docker run --name rethinkdb --network tinode-net --restart always -d rethinkdb:2.3
+	```
+	See [instructions](https://hub.docker.com/_/rethinkdb/) for more options.
 
 4. Run the Tinode container for the appropriate database:
 
-	1. **RethinkDB**:
+	1. **MySQL**:
 	```
-	$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net tinode/tinode-rethinkdb:latest
+	$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net tinode/tinode-mysql:latest
 	```
 
-	2. **MySQL**:
+	2. **PostgreSQL**:
 	```
-	$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net tinode/tinode-mysql:latest
+	$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net tinode/tinode-postgresql:latest
 	```
 
 	3. **MongoDB**:
 	```
 	$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net tinode/tinode-mongodb:latest
 	```
 
+	4. **RethinkDB**:
+	```
+	$ docker run -p 6060:6060 -d --name tinode-srv --network tinode-net tinode/tinode-rethinkdb:latest
+	```
+
 	You can also run Tinode with the `tinode/tinode` image (which has all of the above DB adapters compiled in). You will need to specify the database adapter via `STORE_USE_ADAPTER` environment variable. E.g. for `mysql`, the command line will look like
 	```
 	$ docker run -p 6060:6060 -d -e STORE_USE_ADAPTER mysql --name tinode-srv --network tinode-net tinode/tinode:latest
@@ -64,9 +75,10 @@ All images are available at https://hub.docker.com/r/tinode/
 
 	You may replace `:latest` with a different tag. See all all available tags here:
 	 * [MySQL tags](https://hub.docker.com/r/tinode/tinode-mysql/tags/)
-	 * [RethinkDB tags](https://hub.docker.com/r/tinode/tinode-rethink/tags/)
+	 * [PostgreSQL tags](https://hub.docker.com/r/tinode/tinode-postgresql/tags/) (beta version)
 	 * [MongoDB tags](https://hub.docker.com/r/tinode/tinode-mongodb/tags/)
-	 * [All bundle tags](https://hub.docker.com/r/tinode/tinode/tags/) (comming soon)
+	 * [RethinkDB tags](https://hub.docker.com/r/tinode/tinode-rethink/tags/)
+	 * [All bundle tags](https://hub.docker.com/r/tinode/tinode/tags/)
 
 5. Test the installation by pointing your browser to [http://localhost:6060/](http://localhost:6060/).
 
@@ -189,6 +201,7 @@ You can specify the following environment variables when issuing `docker run` co
 | `MEDIA_HANDLER` | string | `fs` | Handler of large files, either `fs` or `s3` |
 | `MYSQL_DSN` | string | `'root@tcp(mysql)/tinode'` | MySQL [DSN](https://github.com/go-sql-driver/mysql#dsn-data-source-name). |
 | `PLUGIN_PYTHON_CHAT_BOT_ENABLED` | bool | `false` | Enable calling into the plugin provided by Python chatbot |
+| `POSTGRES_DSN` | string | `'postgresql://postgres:postgres@localhost:5432/tinode'` |  PostgreSQL [DSN](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING). |
 | `RESET_DB` | bool | `false` | Drop and recreate the database. |
 | `SAMPLE_DATA` | string |  _see comment →_ | File with sample data to load. Default `data.json` when resetting or generating new DB, none when upgrading. Use `` (empty string) to disable |
 | `SMTP_DOMAINS` | string |  | White list of email domains; when non-empty, accept registrations with emails from these domains only (email verification). |

docker/docker-compose/README.md

@@ -10,8 +10,11 @@ By default, this command starts up a mysql instance, Tinode server(s) and Tinode
 Tinode server(s) is(are) configured similar to [Tinode Demo/Sandbox](../../README.md#demosandbox) and
 maps its web port to the host's port 6060 (6061, 6062). Tinode exporter(s) serve(s) metrics for InfluxDB.
 
-Reference configuration for [RethinkDB 2.4.0](https://hub.docker.com/_/rethinkdb?tab=tags) and [MongoDB 4.2.3](https://hub.docker.com/_/mongo?tab=tags) is also available
-in the override files.
+Reference configuration for the following databases is also available in the override files:
+* [PostgreSQL 15.2](https://hub.docker.com/_/postgres/tags)
+* [MongoDB 4.2.3](https://hub.docker.com/_/mongo/tags)
+* [RethinkDB 2.4.2](https://hub.docker.com/_/rethinkdb/tags)
+
 
 ## Commands
 
@@ -20,12 +23,15 @@ To bring up the full stack, you can use the following commands:
 * MySql:
   - Single-instance setup: `docker-compose -f single-instance.yml up -d`
   - Cluster: `docker-compose -f cluster.yml up -d`
-* RethinkDb:
-  - Single-instance setup: `docker-compose -f single-instance.yml -f single-instance.rethinkdb.yml up -d`
-  - Cluster: `docker-compose -f cluster.yml -f cluster.rethinkdb.yml up -d`
+* PostgreSQL:
+  - Single-instance setup: `docker-compose -f single-instance.yml -f single-instance.postgres.yml up -d`
+  - Cluster: `docker-compose -f cluster.yml -f cluster.postgres.yml up -d`
 * MongoDb:
   - Single-instance setup: `docker-compose -f single-instance.yml -f single-instance.mongodb.yml up -d`
   - Cluster: `docker-compose -f cluster.yml -f cluster.mongodb.yml up -d`
+* RethinkDb:
+  - Single-instance setup: `docker-compose -f single-instance.yml -f single-instance.rethinkdb.yml up -d`
+  - Cluster: `docker-compose -f cluster.yml -f cluster.rethinkdb.yml up -d`
 
 You can run individual/separate components of the setup by providing their names to the `docker-compose` command.
 E.g. to start the Tinode server in the single-instance MySql setup,

docker/docker-compose/cluster.mongodb.yml

@@ -1,4 +1,4 @@
-version: '3.7'
+version: '3.8'
 
 x-mongodb-tinode-env-vars: &mongodb-tinode-env-vars
   "STORE_USE_ADAPTER": "mongodb"

docker/docker-compose/cluster.postgres.yml

@@ -0,0 +1,24 @@
+version: '3.8'
+
+x-postgres-tinode-env-vars: &postgres-tinode-env-vars
+  "STORE_USE_ADAPTER": "postgres"
+
+services:
+  db:
+    image: postgres:15.2
+    container_name: postgres
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready"]
+
+  tinode-0:
+    environment:
+      << : *postgres-tinode-env-vars
+      "WAIT_FOR": "postgres:5432"
+
+  tinode-1:
+    environment:
+      << : *postgres-tinode-env-vars
+
+  tinode-2:
+    environment:
+      << : *postgres-tinode-env-vars

docker/docker-compose/cluster.rethinkdb.yml

@@ -1,11 +1,11 @@
-version: '3.7'
+version: '3.8'
 
 x-rethinkdb-tinode-env-vars: &rethinkdb-tinode-env-vars
   "STORE_USE_ADAPTER": "rethinkdb"
 
 services:
   db:
-    image: rethinkdb:2.4.0
+    image: rethinkdb:2.4.2
     container_name: rethinkdb
     healthcheck:
       test: ["CMD", "curl -f http://localhost:8080/ || exit 1"]

docker/docker-compose/cluster.yml

@@ -4,7 +4,7 @@
 # * 3 Tinode servers
 # * 3 exporters
 
-version: '3.7'
+version: '3.8'
 
 # Base Tinode template.
 x-tinode:

docker/docker-compose/single-instance.mongodb.yml

@@ -1,4 +1,4 @@
-version: '3.7'
+version: '3.8'
 
 x-mongodb-tinode-env-vars: &mongodb-tinode-env-vars
   "STORE_USE_ADAPTER": "mongodb"

docker/docker-compose/single-instance.postgres.yml

@@ -0,0 +1,16 @@
+version: '3.8'
+
+x-postgres-tinode-env-vars: &postgres-tinode-env-vars
+  "STORE_USE_ADAPTER": "postgres"
+
+services:
+  db:
+    image: postgres:15.2
+    container_name: postgres
+    healthcheck:
+      test: ["CMD-SHELL", "pg_isready"]
+
+  tinode-0:
+    environment:
+      << : *postgres-tinode-env-vars
+      "WAIT_FOR": "postgres:5432"