About Otoroshi
-At the beginning of 2017, we had the need to create a new environment to be able to create new “digital” products very quickly in an agile fashion at MAIF. Naturally we turned to PaaS solutions and chose the excellent Clever-Cloud product to run our apps.
-We also chose that every feature team will have the freedom to choose its own technological stack to build its product. It was a nice move but it has also introduced some challenges in terms of homogeneity for traceability, security, logging, … because we did not want to force library usage in the products. We could have used something like Service Mesh Pattern but the deployement model of Clever-Cloud prevented us to do it.
-The right solution was to use a reverse proxy or some kind of API Gateway able to provide tracability, logging, security with apikeys, quotas, DNS as a service locator, etc. We needed something easy to use, with a human friendly UI, a nice API to extends its features, true hot reconfiguration, able to generate internal events for third party usage. A couple of solutions were available at that time, but not one seems to fit our needs, there was always something missing, too complicated for our needs or not playing well with Clever-Cloud deployment model.
-At some point, we tried to write a small prototype to explore what could be our dream reverse proxy. The design was very simple, there were some rough edges but every major feature needed was there waiting to be enhanced.
-Otoroshi was born and we decided to move ahead with our hairy monster :)
-Philosophy
-Every OSS product build at MAIF like Izanami follow a common philosophy.
--
-
- the services or API provided should be technology agnostic. -
- http first: http is the right answer to the previous quote -
- api First: The UI is just another client of the api. -
- secured: The services exposed need authentication for both humans or machines -
- event based: The services should expose a way to get notified of what happened inside. -
![](\"./img/architecture-1-bis.png\")
\n@@@\n\nOtoroshi tries to embrace our @ref:[global philosophy](./about.md#philosophy) by providing a full featured REST admin api, a gorgeous admin dashboard written in [React](https://reactjs.org/) that uses the api, by generating traffic events, alerts events, audit events that can be consumed by several channels. Otoroshi also supports a bunch of datastores to better match with different use cases.\n\n@@@ div { .centered-img }\n![](\"./img/architecture-2-bis.png\")
\n@@@\n"
- },
- {
- "name": "cli.md",
- "id": "/cli.md",
- "url": "/cli.html",
- "title": "Rust CLI",
- "content": "# Rust CLI\n\nOtoroshi provides a command line tool to command Otoroshi instances.\n\n## Download\n\nFirst, download the Otoroshi CLI depending on your operating system :\n\n```sh\ncurl -L -o otoroshicli https://dl.bintray.com/maif/binaries/linux-otoroshicli/1.4.4/otoroshicli\n# or if you use linux\ncurl -L -o otoroshicli https://dl.bintray.com/maif/binaries/mac-otoroshicli/1.4.4/otoroshicli\n# or if you use windows\ncurl -L -o otoroshicli.exe https://dl.bintray.com/maif/binaries/win-otoroshicli/1.4.4/otoroshicli.exe\n```\n\n## Usage\n\nThe documentation is not written yet, but you can explore the cli usage using the help flag `-h`\n\nYou can read more about it [here](https://github.com/MAIF/otoroshi/tree/master/clients/cli).\n\n```\n$ otoroshicli -h\n\notoroshicli 1.4.4\n ____ __________ ____ ____ _____ __ ______ ________ ____\n / __ \\/_ __/ __ \\/ __ \\/ __ \\/ ___// / / / _/ / ____/ / / _/\n / / / / / / / / / / /_/ / / / /\\__ \\/ /_/ // /_____/ / / / / /\n/ /_/ / / / / /_/ / _, _/ /_/ /___/ / __ // /_____/ /___/ /____/ /\n\\____/ /_/ \\____/_/ |_|\\____//____/_/ /_/___/ \\____/_____/___/\n\nA simple CLI to control the Otoroshi reverse proxy (https://maif.github.io/otoroshi).\nYou have to provide a $HOME/.otoroshicli.toml or a $PWD/otoroshicli.toml config file with\n'host', 'client_id', 'client_secret' values to access the Otoroshi instance you need.\n\nUSAGE:\n otoroshicli [FLAGS] [OPTIONS] [SUBCOMMAND]\n\nFLAGS:\n --debug Debug output\n -h, --help Prints help information\n -n, --no-colors Does not color output\n -V, --version Prints version information\n\nOPTIONS:\n --oto-client-id ![](\"../img/deploy-cc-jar-0.png\")
\n@@@\n\nThen choose what kind of app your want to create, for Otoroshi, choose `Java + Jar`\n\n@@@ div { .centered-img }\n![](\"../img/deploy-cc-jar-1.png\")
\n@@@\n\nNext, set up choose instance size and auto-scalling. Otoroshi can run on small instances, especially if you just want to test it.\n\n@@@ div { .centered-img }\n![](\"../img/deploy-cc-2.png\")
\n@@@\n\nFinally, choose a name for your app\n\n@@@ div { .centered-img }\n![](\"../img/deploy-cc-3.png\")
\n@@@\n\nNow you just need to customize environnment variables and add the custom build script as pre buid hook :\n\n`CC_PRE_BUILD_HOOK=./clevercloud/build.sh`\n\nat this point, you can also add other env. variables to configure Otoroshi like in [the example provided below](#example-of-clevercloud-env-variables)\n\n@@@ div { .centered-img }\n![](\"../img/deploy-cc-4-bis.png\")
\n@@@\n\nYou can also use expert mode :\n\n@@@ div { .centered-img }\n![](\"../img/deploy-cc-4.png\")
\n@@@\n\nNow, your app is ready, don't forget to add a custom domains name on the CleverCloud app matching the Otoroshi app domain. For instance if you used domain names in env. variables like `changeme`, `changeme-admin-internal-api`, `changeme-api` on the `cleverapps.io` domain, declare `changeme.cleverapps.io`, `changeme-api.cleverapps.io`, `changeme-admin-internal-api.cleverapps.io`.\n\nYou will find the login/password tuple for first login in the app. logs.\n\n## Build and deploy Otoroshi from its source code\n\nFirst, fork our project template on Github at https://github.com/MAIF/otoroshi-clevercloud-template.\n\nIf you want to customize the build script, edit `./clevercloud/build.sh`\n\nIf you want to customize the configuration file, edit `./clevercloud/prod.conf` or @ref:[use env. variables](../firstrun/env.md)\n\nCreate a new Clever Cloud app based on your fork.\n\n@@@ div { .centered-img }\n![](\"../img/deploy-cc-0.png\")
\n@@@\n\nThen, you need to choose what kind of app your want to create, for Otoroshi, choose `Java or Scala + Play 2`\n\n@@@ div { .centered-img }\n![](\"../img/deploy-cc-1.png\")
\n@@@\n\nThen, you will be asked to choose what kind of machine you want to use. `M` instances are a good choice but you can use a less powerful ones. You can also activate auto-scaling or multi-instances to provie high availibility.\n\n@@@ div { .centered-img }\n![](\"../img/datastores.png\")
\n@@@\n"
- },
- {
- "name": "env.md",
- "id": "/firstrun/env.md",
- "url": "/firstrun/env.html",
- "title": "Config. with ENVs",
- "content": "# Config. with ENVs\n\nNow that you know @ref:[how to configure Otoroshi with the config. file](./configfile.md) every property in the following block can be overriden by an environment variable (an env. variable is written like `${?ENV_VARIABLE}`).\n\n```\napp.storage = ${?APP_STORAGE}\napp.importFrom = ${?APP_IMPORT_FROM}\napp.domain = ${?APP_DOMAIN}\napp.rootScheme = ${?APP_ROOT_SCHEME}\napp.throttlingWindow = ${?THROTTLING_WINDOW}\napp.snowflake.seed = ${?INSTANCE_NUMBER}\napp.events.maxSize = ${?MAX_EVENTS_SIZE}\napp.backoffice.exposed = ${?APP_BACKOFFICE_EXPOSED}\napp.backoffice.subdomain = ${?APP_BACKOFFICE_SUBDOMAIN}\napp.backoffice.session.exp = ${?APP_BACKOFFICE_SESSION_EXP}\napp.privateapps.subdomain = ${?APP_PRIVATEAPPS_SUBDOMAIN}\napp.privateapps.session.exp = ${?APP_PRIVATEAPPS_SESSION_EXP}\napp.adminapi.exposed = ${?ADMIN_API_EXPOSED}\napp.adminapi.targetSubdomain = ${?ADMIN_API_TARGET_SUBDOMAIN}\napp.adminapi.exposedSubdomain = ${?ADMIN_API_EXPOSED_SUBDOMAIN}\napp.adminapi.defaultValues.backOfficeGroupId = ${?ADMIN_API_GROUP}\napp.adminapi.defaultValues.backOfficeApiKeyClientId = ${?ADMIN_API_CLIENT_ID}\napp.adminapi.defaultValues.backOfficeApiKeyClientSecret = ${?ADMIN_API_CLIENT_SECRET}\napp.adminapi.defaultValues.backOfficeServiceId = ${?ADMIN_API_SERVICE_ID}\napp.adminapi.proxy.https = ${?ADMIN_API_HTTPS}\napp.adminapi.proxy.local = ${?ADMIN_API_LOCAL}\napp.claim.sharedKey = ${?CLAIM_SHAREDKEY}\napp.webhooks.size = ${?WEBHOOK_SIZE}\napp.redis.host = ${?REDIS_HOST}\napp.redis.port = ${?REDIS_PORT}\napp.redis.password = ${?REDIS_PASSWORD}\napp.redis.windowSize = ${?REDIS_WINDOW_SIZE}\napp.redis.useScan = ${?REDIS_USE_SCAN}\napp.inmemory.windowSize = ${?INMEMORY_WINDOW_SIZE}\napp.leveldb.windowSize = ${?LEVELDB_WINDOW_SIZE}\napp.leveldb.path = ${?LEVELDB_PATH}\napp.cassandra.windowSize = ${?CASSANDRA_WINDOW_SIZE}\napp.cassandra.hosts = ${?CASSANDRA_HOSTS}\napp.cassandra.host = ${?CASSANDRA_HOST}\napp.cassandra.port = ${?CASSANDRA_PORT}\napp.elastic.url = ${?ELASTIC_URL}\napp.elastic.user = ${?ELASTIC_USER}\napp.elastic.password = ${?ELASTIC_PASSWORD}\napp.elastic.index = ${?ELASTIC_INDEX}\napp.elastic.type = ${?ELASTIC_TYPE}\napp.mongo.uri = ${?MONGO_URI}\nhttp.port = ${?PORT}\nhttps.port = ${?HTTPS_PORT}\nhttp2.enabled = ${?HTTP2_ENABLED}\nplay.http.secret.key = ${?PLAY_CRYPTO_SECRET}\nplay.http.session.secure = ${?SESSION_SECURE_ONLY}\nplay.http.session.maxAge = ${?SESSION_MAX_AGE}\nplay.http.session.domain = ${?SESSION_DOMAIN}\nplay.http.session.cookieName = ${?SESSION_NAME}\nplay.ws.play.ws.useragent=${?USER_AGENT}\n```\n"
- },
- {
- "name": "host.md",
- "id": "/firstrun/host.md",
- "url": "/firstrun/host.html",
- "title": "Setup your hosts",
- "content": "# Setup your hosts\n\nThe last thing you need to do is to add the following line to your `/etc/hosts` file. It is not actually needed to run Otoroshi, but it will be useful if you want to try the backoffice in your browser.\n\n```\n127.0.0.1 otoroshi-api.foo.bar otoroshi.foo.bar otoroshi-admin-internal-api.foo.bar privateapps.foo.bar\n```\n\nOf course, you have to change the values according to the setting you put in Otoroshi configuration\n\n* `app.domain` => `foo.bar`\n* `app.backoffice.subdomain` => `otoroshi`\n* `app.privateapps.subdomain` => `privateapps`\n* `app.adminapi.exposedSubdomain` => `otoroshi-api`\n* `app.adminapi.targetSubdomain` => `otoroshi-admin-internal-api`\n\nfor instance if you want to change the default domain and use something like `otoroshi.mydomain.org`, then start otoroshi like \n\n```sh\njava -Dapp.domain=mydomain.org -jar otoroshi.jar\n```\n\n@@@ warning\nOtoroshi cannot be accessed using `http://127.0.0.1:8080` or `http://localhost:8080` because Otoroshi uses Otoroshi to serve it's own UI and API. When otoroshi starts with an empty database, it will create a service descriptor for that using `app.domain` and the settings listed on this page and in the * [Config. with files page](./configfile.md) that serve Otoroshi API and UI on `http://otoroshi-api.${app.domain}` and `http://otoroshi.${app.domain}`.\nOnce the descriptor is saved in database, if you want to change `app.domain`, you'll have to edit the descriptor in the database or restart Otoroshi with an empty database.\n@@@\n"
- },
- {
- "name": "index.md",
- "id": "/firstrun/index.md",
- "url": "/firstrun/index.html",
- "title": "First run",
- "content": "# First run\n\nNow that you have your own distro of Otoroshi, it's time to run it. \n\nBut before doing so, you'll have to make some choices about some essential stuff in order to have your own customized version of Otoroshi.\n\nLet's start with the datastore\n\n\n@@@ index\n\n* [choose a datastore](./datastore.md)\n* [use custom config file](./configfile.md)\n* [use ENV](./env.md)\n* [initial state](./initialstate.md)\n* [Hosts](./host.md)\n* [Run](./run.md)\n\n@@@"
- },
- {
- "name": "initialstate.md",
- "id": "/firstrun/initialstate.md",
- "url": "/firstrun/initialstate.html",
- "title": "Import initial state",
- "content": "# Import initial state\n\nNow you are almost ready to run Otoroshi for the first time, but maybe you want to import data from previous Otoroshi installation in your current datastore.\n\nTo do that, you need to add the `app.importFrom` setting to the Otoroshi configuration (of `$APP_IMPORT_FROM` env).\n\nIt can be a file path or a URL\n\n## Example of export\n\n```json\n{\n \"config\": {\n \"lines\": [\"prod\"], \n \"limitConcurrentRequests\": true,\n \"maxConcurrentRequests\": 500,\n \"useCircuitBreakers\": true,\n \"apiReadOnly\": false,\n \"registerFromCleverHook\": false,\n \"u2fLoginOnly\": true,\n \"ipFiltering\": {\n \"whitelist\": [],\n \"blacklist\": []\n },\n \"throttlingQuota\": 100000,\n \"perIpThrottlingQuota\": 500,\n \"analyticsEventsUrl\": null,\n \"analyticsWebhooks\": [],\n \"alertsWebhooks\": [],\n \"alertsEmails\": [],\n \"endlessIpAddresses\": []\n },\n \"admins\": [],\n \"simpleAdmins\": [\n {\n \"username\": \"admin@otoroshi.io\",\n \"password\": \"xxxxxxxxxxxxxxxxx\",\n \"label\": \"Otoroshi Admin\",\n \"createdAt\": 1493971715708\n }\n ],\n \"serviceGroups\": [\n {\n \"id\": \"default\",\n \"name\": \"default-group\",\n \"description\": \"The default group\"\n },\n {\n \"id\": \"admin-api-group\",\n \"name\": \"Otoroshi Admin Api group\",\n \"description\": \"No description\"\n }\n ],\n \"apiKeys\": [\n {\n \"clientId\": \"admin-api-apikey-id\",\n \"clientSecret\": \"admin-api-apikey-secret\",\n \"clientName\": \"Otoroshi Backoffice ApiKey\",\n \"authorizedGroup\": \"admin-api-group\",\n \"enabled\": true,\n \"throttlingQuota\": 10000000,\n \"dailyQuota\": 10000000,\n \"monthlyQuota\": 10000000,\n \"metadata\": {}\n }\n ],\n \"serviceDescriptors\": [\n {\n \"id\": \"admin-api-service\",\n \"groupId\": \"admin-api-group\",\n \"name\": \"otoroshi-admin-api\",\n \"env\": \"prod\",\n \"domain\": \"foo.bar\",\n \"subdomain\": \"otoroshi-api\",\n \"targets\": [\n {\n \"host\": \"localhost:8080\",\n \"scheme\": \"http\"\n }\n ],\n \"root\": \"/\",\n \"enabled\": true,\n \"privateApp\": false,\n \"forceHttps\": false,\n \"maintenanceMode\": false,\n \"buildMode\": false,\n \"enforceSecureCommunication\": true,\n \"publicPatterns\": [],\n \"privatePatterns\": [],\n \"additionalHeaders\": {\n \"Host\": \"otoroshi-admin-internal-api.foo.bar\"\n },\n \"matchingHeaders\": {},\n \"ipFiltering\": {\n \"whitelist\": [],\n \"blacklist\": []\n },\n \"api\": {\n \"exposeApi\": false\n },\n \"healthCheck\": {\n \"enabled\": false,\n \"url\": \"/\"\n },\n \"metadata\": {}\n }\n ],\n \"errorTemplates\": []\n}\n```\n"
- },
- {
- "name": "run.md",
- "id": "/firstrun/run.md",
- "url": "/firstrun/run.html",
- "title": "Run Otoroshi",
- "content": "# Run Otoroshi\n\nNow you are ready to run Otoroshi. You can run the following command with some tweaks depending on the way you want to configure Otoroshi. If you want to pass a custom configuration file, use the `-Dconfig.file=/path/to/file.conf` flag in the following commands.\n\n## From .zip file\n\n```sh\nunzip otoroshi-dist.zip\ncd otoroshi-vx.x.x\n./bin/otoroshi\n```\n\n## From .jar file\n\nFor Java 8\n\n```sh\njava -jar otoroshi.jar\n```\n\nFor Java 9 and 10\n\n```sh\njava --add-modules java.xml.bind -jar otoroshi.jar\n```\n\n## From docker\n\n```sh\ndocker run -p \"8080:8080\" maif/otoroshi:1\n```\n\nYou can also pass useful args like :\n\n```\ndocker run -p \"8080:8080\" otoroshi -Dconfig.file=/usr/app/otoroshi/conf/otoroshi.conf -Dlogger.file=/usr/app/otoroshi/conf/otoroshi.xml\n```\n\nIf you want to provide your own config file, you can read @ref:[the documentation about config files](../firstrun/configfile.md).\n\nYou can also provide some ENV variable using the `--env` flag to customize your Otoroshi instance.\n\nThe list of possible env variables is available @ref:[here](../firstrun/env.md).\n\nYou can use a volume to provide configuration like :\n\n```sh\ndocker run -p \"8080:8080\" -v \"$(pwd):/usr/app/otoroshi/conf\" maif/otoroshi\n```\n\nYou can also use a volume if you choose to use `leveldb` datastore like :\n\n```sh\ndocker run -p \"8080:8080\" -v \"$(pwd)/leveldb:/usr/app/otoroshi/leveldb\" maif/otoroshi -Dapp.storage=leveldb\n```\n\nYou can also use a volume if you choose to use exports files :\n\n```sh\ndocker run -p \"8080:8080\" -v \"$(pwd):/usr/app/otoroshi/imports\" maif/otoroshi -Dapp.importFrom=/usr/app/otoroshi/imports/export.json\n```\n\n## Run examples\n\n```sh\n$ java \\\n -Xms2G \\\n -Xmx8G \\\n -Dhttp.port=8080 \\\n -Dapp.importFrom=/home/user/otoroshi.json \\\n -Dconfig.file=/home/user/otoroshi.conf \\\n -jar ./otoroshi.jar\n\n[warn] otoroshi-in-memory-datastores - Now using InMemory DataStores\n[warn] otoroshi-env - The main datastore seems to be empty, registering some basic services\n[warn] otoroshi-env - Importing from: /home/user/otoroshi.json\n[info] play.api.Play - Application started (Prod)\n[info] p.c.s.AkkaHttpServer - Listening for HTTP on /0:0:0:0:0:0:0:0:8080\n```\n\nIf you choose to start Otoroshi without importing existing data, Otoroshi will create a new admin user and print the login details in the log. When you will log into the admin dashboard, Otoroshi will ask you to create another account to avoid security issues.\n\n```sh\n$ java \\\n -Xms2G \\\n -Xmx8G \\\n -Dhttp.port=8080 \\\n -jar otoroshi.jar\n\n[warn] otoroshi-in-memory-datastores - Now using InMemory DataStores\n[warn] otoroshi-env - The main datastore seems to be empty, registering some basic services\n[warn] otoroshi-env - You can log into the Otoroshi admin console with the following credentials: admin@otoroshi.io / HHUsiF2UC3OPdmg0lGngEv3RrbIwWV5W\n[info] play.api.Play - Application started (Prod)\n[info] p.c.s.AkkaHttpServer - Listening for HTTP on /0:0:0:0:0:0:0:0:8080\n```\n"
- },
- {
- "name": "frombinaries.md",
- "id": "/getotoroshi/frombinaries.md",
- "url": "/getotoroshi/frombinaries.html",
- "title": "From binaries",
- "content": "# From binaries\n\nIf you want to download the last version of Otoroshi and its CLI, you can grab them from the release page of the Otoroshi github page :\n\nGo to https://github.com/MAIF/otoroshi/releases or get it from our bintray repository :\n\n* https://dl.bintray.com/maif/binaries/otoroshi.jar/1.4.4/otoroshi.jar\n* https://dl.bintray.com/maif/binaries/otoroshi-dist/1.4.4/otoroshi-dist.zip\n\nand get the last version of the `otoroshi-dist.zip` file or `otoroshi.jar` file\n"
- },
- {
- "name": "fromdocker.md",
- "id": "/getotoroshi/fromdocker.md",
- "url": "/getotoroshi/fromdocker.html",
- "title": "From docker",
- "content": "# From docker\n\nIf you're a Docker aficionado, Otoroshi is provided as a Docker image that your can pull directly from Official repos.\n\nfirst, fetch the last Docker image of Otoroshi :\n\n```sh\ndocker pull maif/otoroshi:1.4.4\n# or \ndocker pull maif/otoroshi:latest\n# or \ndocker pull maif/otoroshi:jdk8-1.4.4\n# or \ndocker pull maif/otoroshi:jdk9-1.4.4\n# or \ndocker pull maif/otoroshi:jdk10-1.4.4\n# or \ndocker pull maif/otoroshi:jdk11-1.4.4\n# or \ndocker pull maif/otoroshi:graal-1.4.4\n```"
- },
- {
- "name": "fromsources.md",
- "id": "/getotoroshi/fromsources.md",
- "url": "/getotoroshi/fromsources.html",
- "title": "From sources",
- "content": "# From sources\n\nto build Otoroshi from sources, you need the following tools :\n\n* git\n* JDK 8\n* SBT\n* node\n* yarn\n\nOnce you've installed all those tools, go to the [Otoroshi github page](https://github.com/MAIF/otoroshi) and clone the sources :\n\n```sh\ngit clone https://github.com/MAIF/otoroshi.git --depth=1\n```\n\nthen you need to run the `build.sh` script to build the documentation, the React UI and the server :\n\n```sh\nsh ./scripts/build.sh\n```\n\nand that's all, you can grab your Otoroshi package at `otoroshi/target/scala-2.12/otoroshi` or `otoroshi/target/universal/`.\n\nFor those who want to build only parts of Otoroshi, read the following.\n\n## Build the documentation only\n\nGo to the `documentation` folder and run :\n\n```sh\nsbt ';clean;paradox'\n```\n\nThe documentation is located at `documentation/target/paradox/site/main/`\n\n## Build the React UI\n\nGo to the `otoroshi/javascript` folder and run :\n\n```sh\nyarn install\nyarn build\n```\n\nYou will find the JS bundle at `otoroshi/public/javascripts/bundle/bundle.js`.\n\n## Build the Otoroshi server\n\nGo to the `otoroshi` folder and run :\n\n```sh\nsbt ';clean;compile;dist;assembly'\n```\n\nYou will find your Otoroshi package at `otoroshi/target/scala-2.12/otoroshi` or `otoroshi/target/universal/`.\n"
- },
- {
- "name": "index.md",
- "id": "/getotoroshi/index.md",
- "url": "/getotoroshi/index.html",
- "title": "Get Otoroshi",
- "content": "# Get Otoroshi\n\nThere are several ways to get Otoroshi to run it on your system.\n\nLet's start with a good old build from sources :)\n\n@@@ index\n\n* [from sources](./fromsources.md)\n* [from binaries](./frombinaries.md)\n* [from docker](./fromdocker.md)\n\n@@@"
- },
- {
- "name": "index.md",
- "id": "/index.md",
- "url": "/index.html",
- "title": "Otoroshi",
- "content": "# Otoroshi\n\n**Otoroshi** is a layer of lightweight api management on top of a modern http reverse proxy written in ![](\"https://github.com/MAIF/otoroshi/raw/master/resources/otoroshi-logo.png\"){kind=logo}
\n@@@\n\n## Installation\n\nYou can download the latest build of Otoroshi as a [fat jar](https://dl.bintray.com/maif/binaries/otoroshi.jar/1.4.4/otoroshi.jar), as a [zip package](https://dl.bintray.com/maif/binaries/otoroshi-dist/1.4.4/otoroshi-dist.zip) or as a @ref:[docker image](./getotoroshi/fromdocker.md).\n\nYou can install and run Otoroshi with this little bash snippet\n\n```sh\ncurl -L -o otoroshi.jar 'https://dl.bintray.com/maif/binaries/otoroshi.jar/1.4.4/otoroshi.jar'\n# run the following line if you want to use the admin UI in your browser\nsudo echo \"127.0.0.1 otoroshi-api.foo.bar otoroshi.foo.bar otoroshi-admin-internal-api.foo.bar\" >> /etc/hosts\n# Java 8\njava -jar otoroshi.jar\n# Java 9 and 10\njava --add-modules java.xml.bind -jar otoroshi.jar\n```\n\nor using docker\n\n```sh\ndocker run -p \"8080:8080\" maif/otoroshi:1.4.4\n```\n\nnow open your browser to ![](\"../img/push-to-elastic.png\")
\n@@@\n\n## Read events from Elasticsearch\n\nYou can use elastic search to store otoroshi events. To do this you have to configure the access to elasticsearch from `settings (cog icon) / Danger Zone` and expand the `Analytics: Elastic dashboard datasource (read)` section.\n\n@@@ div { .centered-img }\n![](\"../img/danger-zone-4-analytics.png\")
\n@@@\n\nHere you can configure the URL of the webhook and its headers if needed.\n\n## Push events to Kafka\n\nEvents can also be sent through a Kafka topic. Go to `settings (cog icon) / Danger Zone` and expand the `Analytics: Kafka` section.\n\n@@@ div { .centered-img }\n![](\"../img/danger-zone-5-kafka.png\")
\n@@@\n\nFill the form, default values for topic names are :\n\n* `otoroshi-alerts`\n* `otoroshi-analytics`\n* `otoroshi-audits`\n\n@@@ warning\nIf you use trustore/keystore to access your kafka instances, the paths should be absolute and refers to host paths.\n@@@\n"
- },
- {
- "name": "auth0.md",
- "id": "/integrations/auth0.md",
- "url": "/integrations/auth0.html",
- "title": "Auth0",
- "content": "# Auth0\n\nYou can use Auth0 to log into Otoroshi's backoffice and Otoroshi's private apps.\n\nGo to `settings (cog icon) / Danger Zone` and expand the `Backoffice Auth0 settings` and `Private apps Auth0 settings` sections.\n\n@@@ div { .centered-img }\n![](\"../img/danger-zone-8-auth0.png\")
\n@@@\n\nNow, you can fill the following fields :\n\n* `Client Id`\n* `Client Service`\n* `Domain`\n\nFor the `Callback URL` fields, use something like\n\n```\nhttps://otoroshi.foo.bar/backoffice/auth0/callback\nhttps://privateapps.foo.bar/privateapps/auth0/callback\n```\n\nOf course, you need to replace `otoroshi.foo.bar` and `privateapps.foo.bar` with your own domain and sub-domains. Don't forget to customize the callback URLs in your Auth0 backoffice too.\n\nNow if you logout, you will see the Auth0 option on the login screen\n\n@@@ div { .centered-img }\n![](\"../img/login-auth0.png\")
\n@@@\n"
- },
- {
- "name": "clevercloud.md",
- "id": "/integrations/clevercloud.md",
- "url": "/integrations/clevercloud.html",
- "title": "Clever Cloud",
- "content": "# Clever Cloud\n\nOtoroshi provides an integration with Clever Cloud to create easily services based on application deployed on your Clever Cloud account.\nGo to `settings (cog icon) / Danger Zone` and expand the `CleverCloud settings` section.\n\n@@@ div { .centered-img }\n![](\"../img/danger-zone-10-clevercloud.png\")
\n@@@\n\nFill the form with your CleverCloud credentials (https://www.clever-cloud.com/doc/clever-cloud-apis/cc-api/) and your CleverCloud `organization id`.\n\nOnce it's done, you will see a new menu in the side bar.\n\n@@@ div { .centered-img }\n![](\"../img/clevercloud-integration-1.png\")
\n@@@\n\nIf you click on it, you'll see a page listing all your apps deployed on Clever Cloud with buttons to create new services with the app as the target.\n\n@@@ div { .centered-img }\n![](\"../img/clevercloud-integration-2.png\")
\n@@@\n\nYou will also see a new button in the `Target` section of services to attach Clever Cloud applications as target for a service.\n\n@@@ div { .centered-img }\n![](\"../img/clevercloud-integration-3.png\")
\n@@@\n"
- },
- {
- "name": "index.md",
- "id": "/integrations/index.md",
- "url": "/integrations/index.html",
- "title": "Third party Integrations",
- "content": "# Third party Integrations\n\nOtoroshi provides some settings to interact with some third party systems.\n\n@@@ index\n\n* [Analytics](./analytics.md)\n* [Mailgun](./mailgun.md)\n* [StatsD / Datadog](./statsd.md)\n* [clevercloud](./clevercloud.md)\n\n@@@\n"
- },
- {
- "name": "mailgun.md",
- "id": "/integrations/mailgun.md",
- "url": "/integrations/mailgun.html",
- "title": "Mailgun",
- "content": "# Mailgun\n\nIf you want to receive Otoroshi alert by emails, you have to configure Otoroshi with your Mailgun credentials. Go to `settings (cog icon) / Danger Zone` and expand the `Mailgun settings` section.\n\n@@@ div { .centered-img }\n![](\"../img/danger-zone-9-mailgun.png\")
\n@@@\n\nFill the form with provided information on the `domain informations` page on Mailgun located at https://app.mailgun.com/app/domains/my.domain.\n\nThen, expand the `Alert settings` section and add email addresses separated by comma in the `Alert emails` field. **Don't forget to save.**\n\n@@@ div { .centered-img }\n![](\"../img/danger-zone-6-alerts.png\")
\n@@@\n"
- },
- {
- "name": "statsd.md",
- "id": "/integrations/statsd.md",
- "url": "/integrations/statsd.html",
- "title": "StatsD / Datadog",
- "content": "# StatsD / Datadog\n\nOtoroshi provides a StatsD integration to monitor some technical metrics across all your Otoroshi instances.\nGo to `settings (cog icon) / Danger Zone` and expand the `Statsd settings` section.\n\n@@@ div { .centered-img }\n![](\"../img/danger-zone-7-statsd.png\")
\n@@@\n\nAdd the host and port of the Statsd agent on your system.\nIf you're using Datadog, don't forget to check the `Datadog` switch.\n"
- },
- {
- "name": "quickstart.md",
- "id": "/quickstart.md",
- "url": "/quickstart.html",
- "title": "Try Otoroshi in 5 minutes",
- "content": "# Try Otoroshi in 5 minutes\n\nwhat you will need :\n\n* JDK 8\n* curl\n* 5 minutes of free time\n\nIf you don't/can't have these tools on your machine, you can start a sandboxed environment using here with the following command\n\n```sh\ndocker run -p \"8080:8080\" -it maif/otoroshi-demo bash\n```\n\nor you can also try Otoroshi online with Google Cloud Shell if you're a user of the Google Cloud Platform\n\n@@@ div { .centered-img }\n[](https://console.cloud.google.com/cloudshell/open?git_repo=https%3A%2F%2Fgithub.com%2Fmathieuancelin%2Fotoroshi-tutorial&page=shell&tutorial=tutorial.md)\n@@@\n\n## The elevator pitch\n\nOtoroshi is an awesome reverse proxy built with Scala that handles all the calls to and between your microservices without service locator and lets you change configuration dynamically at runtime.\n\n## I like sh but I really want to see the UI\n\nAs Otoroshi uses Otoroshi to serve its own admin UI and admin API, you won't be able to access the admin UI on `http://localhost:8080`. Otoroshi needs a domain name to know that you want to access the admin UI. By default, the admin UI is exposed on `http://otoroshi.foo.bar:8080`. Of course you can @ref:[configure](./firstrun/configfile.md#common-configuration) the domain of the subdomain. To configure access to the admin, just go to the [UI section of the quickstart](#what-about-the-ui).\n\n## Now some sh :)\n\n```sh\ncurl -L -o otoroshi.jar https://dl.bintray.com/maif/binaries/otoroshi.jar/1.4.4/otoroshi.jar\ncurl -L -o otoroshicli https://dl.bintray.com/maif/binaries/mac-otoroshicli/1.4.4/otoroshicli\n# or if you use linux\ncurl -L -o otoroshicli https://dl.bintray.com/maif/binaries/linux-otoroshicli/1.4.4/otoroshicli\n# or if you use windows\ncurl -L -o otoroshicli.exe https://dl.bintray.com/maif/binaries/win-otoroshicli/1.4.4/otoroshicli.exe\n\nchmod +x otoroshicli\n\n# Run the Otoroshi server on Java 8\njava -jar otoroshi.jar &\n\n# Run the Otoroshi server on Java 9 and 10\njava --add-modules java.xml.bind -jar otoroshi.jar &\n\n# Check if admin api works\n./otoroshicli services all\n./otoroshicli apikeys all\n./otoroshicli groups all\n\n# Create a service that will proxy call to https://freegeoip.net through http://ip.geo.com:8080\n./otoroshicli services create --group default --name geo-ip-api --env prod \\\n --domain geo.com --subdomain ip --root /json/ --target https://freegeoip.net \\\n --public-pattern '/.*' --no-force-https\n\n# Then test it\n./otoroshicli tryout call \"http://127.0.0.1:8080/\" -X GET -H 'Host: ip.geo.com'\n# works with curl -X GET -H 'Host: ip.geo.com' \"http://127.0.0.1:8080/\" | jqn\n\n# Run 3 new microservices in 3 new terminal processes\n./otoroshicli tryout serve 9901\n./otoroshicli tryout serve 9902\n./otoroshicli tryout serve 9903\n\n# Create a service that will loadbalance between these 3 microservices and serves them through\n# http://api.hello.com:8080\n./otoroshicli services create --group default --id hello-api --name hello-api \\\n --env prod --domain hello.com --subdomain api --root / \\\n --target \"http://127.0.0.1:9901\" \\\n --target \"http://127.0.0.1:9902\" \\\n --public-pattern '/.*' --no-force-https --client-retries 3\n\n# Then test it multiple time to observe loadbalancing\n# You can also define '127.0.0.1 api.hello.com' in your /etc/hosts file and test it in your browser\n./otoroshicli tryout call \"http://127.0.0.1:8080/\" -H 'Host: api.hello.com' -H 'Accept: application/json'\nsudo echo \"127.0.0.1 api.hello.com\" >> /etc/hosts\nopen \"http://api.hello.com:8080/\"\n\n# Then add a new target\n./otoroshicli services add-target hello-api --target=\"http://127.0.0.1:9903\"\n\n# Then test it multiple time to observe loadbalancing\n./otoroshicli tryout call \"http://127.0.0.1:8080/\" -H 'Host: api.hello.com' -H 'Accept: application/json'\n\n# Then kill one of the microservices and test it multiple time to observe loadbalancing\n./otoroshicli tryout call \"http://127.0.0.1:8080/\" -H 'Host: api.hello.com' -H 'Accept: application/json'\n\n# Then kill a second microservices and test it multiple time to observe loadbalancing\n./otoroshicli tryout call \"http://127.0.0.1:8080/\" -H 'Host: api.hello.com' -H 'Accept: application/json'\n\n# Then kill the last microservices and test it to observe connection error\n./otoroshicli tryout call \"http://127.0.0.1:8080/\" -H 'Host: api.hello.com' -H 'Accept: application/json'\n\n# Then delete your service\n./otoroshicli services delete hello-api\n```\n\n## What about the UI\n\nfirst you need to add the following line to your `/etc/hosts` file.\n\n```\n127.0.0.1 otoroshi-api.foo.bar otoroshi.foo.bar otoroshi-admin-internal-api.foo.bar privateapps.foo.bar\n```\n\nyou can use the following command\n\n```sh\nsudo echo \"127.0.0.1 otoroshi-api.foo.bar otoroshi.foo.bar otoroshi-admin-internal-api.foo.bar privateapps.foo.bar\" >> /etc/hosts\n```\n\nThen go to ![](\"../img/go-to-admins.png\")
\n@@@\n\nYou will see the list of registered admin users.\n\n@@@ div { .centered-img }\n![](\"../img/first-admins-screen.png\")
\n@@@\n\nNow, enter informations about the new admin you want to create.\n\n@@@ div { .centered-img }\n![](\"../img/create-admin.png\")
\n@@@\n\nClick on `Register Admin`.\n\n@@@ div { .centered-img }\n![](\"../img/admin-created-admin.png\")
\n@@@\n\nNow, you can discard the generated admin, confirm, then logout, login with the admin user you have just created and the danger popup will go away\n\n@@@ div { .centered-img }\n![](\"../img/home-page.png\")
\n@@@\n\n## Create admin user with U2F device login\n\nGo to `settings (cog icon) / Admins`.\n\n@@@ div { .centered-img }\n![](\"https://images-na.ssl-images-amazon.com/images/I/61hwQNWpSEL._SY542_.jpg\")
\n@@@\n\n@@@ warning\nTo be able to use FIDO U2F devices, Otoroshi must be served over https\n@@@\n\n## Discard admin user\n\nGo to `settings (cog icon) / Admins`, at the bottom of the page, you will see a list of admin users that you can discard. Just click on the `Discard User` button on the right side of the row and confirm that you actually want to discard an admin user.\n\n@@@ div { .centered-img }\n![](\"../img/discard-admin-user.png\")
\n@@@\n\n## Admin sessions management\n\nGo to `settings (cog icon) / Admins sessions`, you will see a list of active admin user sessions\n\n@@@ div { .centered-img }\n![](\"../img/admin-users-sessions.png\")
\n@@@\n\nYou can either discard single sessions one by one using the `Discard Session` on each targeted row of the list or discard all active sessions using the `Discard all sessions` button at the top of the page.\n"
- },
- {
- "name": "dangerzone.md",
- "id": "/setup/dangerzone.md",
- "url": "/setup/dangerzone.html",
- "title": "Configure the Danger zone",
- "content": "# Configure the Danger zone\n\nNow that you have an actual admin account, go to `setting (cog icon) / Danger Zone` in order to configure your Otoroshi instance.\n\n@@@ div { .centered-img }\n![](\"../img/go-to-danger-zone.png\")
\n@@@\n\n## Commons settings\n\nThis part allows you to configure various things :\n\n* `No Auth0 login` => allow you to disabled Auth0 login to the Otoroshi admin dashboard\n* `API read only` => disable `writes` on the Otoroshi admin api\n* `Use HTTP streaming` => use http streaming for each response. It should always be true\n* `Use circuit breakers` => allow usage of circuit breakers for each service\n* `Digitus medius` => change the character of endless HTTP responses from `0` to `🖕`\n* `Limit concurrent requests` => allow you to specify a max number of concurrent requests on an Otoroshi instance to avoid overloading\n* `Max concurrent requests` => max allowed number of concurrent requests on an Otoroshi instance to avoid overloading\n* `Max HTTP/1.0 response size` => max size of an HTTP/1.0 responses, because they are memory mapped\n* `Max local events` => number of events stored localy (alerts and audits)\n* `lines` => at least one (`prod`). for other, it will allow you to declare urls like `service.line.domain.tld`. For prod it will be `service.domain.tld`\n\n@@@ div { .centered-img }\n![](\"../img/danger-zone-1-commons.png\")
\n@@@\n\n## Whitelist / blacklist settings\n\nOtoroshi is capable of filtering request by ip address, allowing or blocking requests.\n\nOtoroshi also provides a fun feature called `Endless HTTP responses`. If you put an ip address in that field, then, for any http request on Otoroshi, every response will be 128 GB of `0`.\n\n@@@ div { .centered-img }\n![](\"../img/danger-zone-2-whitelist-blacklist.png\")
\n@@@\n\n@@@ note\nNote that you may provide ip address with wildcard like the following `42.42.*.42` or `42.42.42.*` or `42.42.*.*`\n@@@\n\n## Global throttling settings\n\nOtoroshi is capable of managing throttling at a global level. Here you can configure number of authorized requests per second on a single Otoroshi instance and the number of authorized request per second for a unique ip address.\n\n@@@ div { .centered-img }\n![](\"../img/danger-zone-3-throttling.png\")
\n@@@\n\n## Analytics settings\n\nOne on the major features of Otoroshi is being able of generating internal events. Those events are not stored in Otoroshi's datastore but can be sent using `WebHooks`. You can configure those `WebHooks` from the `Danger Zone`.\n\nOtoroshi is also capable of reading some analytics and displays it from another MAIF product called `Omoïkane`. As Omoikane is not publicly available yet, is capable of storing events in an [Elastic](https://www.elastic.co/) cluster. For more information about analytics and what it does, just go to the @ref:[detailed chapter](../integrations/analytics.md)\n\n## Kafka settings\n\nOne on the major features of Otoroshi is being able of generating internal events. These events are not stored in Otoroshi's datastore but can be sent using a [Kafka message broker](https://kafka.apache.org/). You can configure Kafka access from the `Danger Zone`.\n\nBy default, Otoroshi's alert events will be sent on `otoroshi-alerts` topic, Otoroshi's audit events will be sent on `otoroshi-audits` topic and Otoroshi's traffic events will be sent on `otoroshi-analytics` topic.\n\n@@@ warning\nKeystore and truststore paths are optional local path on the server hosting Otoroshi\n@@@\n\n@@@ div { .centered-img }\n![](\"../img/auth0-settings.png\")
\n@@@\n\n@@@ div { .centered-img }\n![](\"../img/danger-zone-11-bottom.png\")
\n@@@\n"
- },
- {
- "name": "index.md",
- "id": "/setup/index.md",
- "url": "/setup/index.html",
- "title": "Setup Otoroshi",
- "content": "# Setup Otoroshi\n\nNow that Otoroshi is running, you are ready to log into the Otoroshi admin dashboard and setup your instance. Just go to :\n\n![](\"../img/base-page.png\")
\n@@@\n\nnow click on the login button and you will see the login page\n\n@@@ div { .centered-img }\n![](\"../img/login-page.png\")
\n@@@\n\n@@@ warning\nUse the credentials generated in Otoroshi **logs** during **first run**.\n@@@\n\n@@@ div { .centered-img #first-login-example }\n![](\"../img/first-login.gif\")
\n@@@\n\n(of course, you can change this url dependending on the configuration you provided to Otoroshi).\n\nOnce logged in, the first screen you'll see should look like :\n\n@@@ div { .centered-img #first-login }\n![](\"../img/first-login.png\")
\n@@@\n\nAs you can see, Otoroshi is not really happy about you being logged with a generated admin account.\n\nBut we will fix that in the next chapter\n\n@@@ index\n\n* [create admins](./admin.md)\n* [configure danger zone](./dangerzone.md)\n\n@@@\n"
- },
- {
- "name": "toc.md",
- "id": "/toc.md",
- "url": "/toc.html",
- "title": "Table of contents",
- "content": "# Table of contents\n\n@@toc\n\n"
- },
- {
- "name": "clustering.md",
- "id": "/topics/clustering.md",
- "url": "/topics/clustering.html",
- "title": "Otoroshi clustering",
- "content": "# Otoroshi clustering\n\nOtoroshi can work as a cluster by default as you can spin many Otoroshi servers using the same datastore or datastore cluster. In that case any instance is capable of serving services, Otoroshi admin UI, Otoroshi admin API, etc.\n\nBut sometimes, this is not enough. So Otoroshi provides an additional clustering model named `Leader / Workers` where there is a leader cluster ([control plane](https://en.wikipedia.org/wiki/Control_plane)), composed of Otoroshi instances backed by a datastore like Redis, Cassandra or Mongo, that is in charge of all `writes` to the datastore through Otoroshi admin UI and API, and a worker cluster ([data plane](https://en.wikipedia.org/wiki/Forwarding_plane)) composed of horizontally scalable Otoroshi instances, backed by a super fast in memory datastore, with the sole purpose of routing traffic to your services based on data synced from the leader cluster. With this distributed Otoroshi version, you can reach your goals of high availability, scalability and security.\n\nOtoroshi clustering only uses http internally (right now) to make communications between leaders and workers instances so it is fully compatible with PaaS providers like [Clever-Cloud](https://www.clever-cloud.com/en/) that only provide one external port for http traffic.\n\n@@@ div { .centered-img }\n![](\"../img/cluster-6.png\")
\n\n*Fig. 1: Simplified view*\n@@@\n\n@@@ div { .centered-img }\n![](\"../img/cluster-5.jpg\")
\n\n*Fig. 2: Deployment view*\n@@@\n\n## Cluster configuration\n\n```hocon\notoroshi {\n cluster {\n mode = \"leader\" # can be \"off\", \"leader\", \"worker\"\n compression = 4 # compression of the data sent between leader cluster and worker cluster. From -1 (disabled) to 9\n leader {\n name = ${?CLUSTER_LEADER_NAME} # name of the instance, if none, it will be generated\n urls = [\"http://127.0.0.1:8080\"] # urls to contact the leader cluster\n host = \"otoroshi-api.foo.bar\" # host of the otoroshi api in the leader cluster\n clientId = \"apikey-id\" # otoroshi api client id\n clientSecret = \"secret\" # otoroshi api client secret\n cacheStateFor = 4000 # state is cached during (ms)\n }\n worker {\n name = ${?CLUSTER_WORKER_NAME} # name of the instance, if none, it will be generated\n retries = 3 # number of retries when calling leader cluster\n timeout = 2000 # timeout when calling leader cluster\n state {\n retries = ${otoroshi.cluster.worker.retries} # number of retries when calling leader cluster on state sync\n pollEvery = 10000 # interval of time (ms) between 2 state sync\n timeout = ${otoroshi.cluster.worker.timeout} # timeout when calling leader cluster on state sync\n }\n quotas {\n retries = ${otoroshi.cluster.worker.retries} # number of retries when calling leader cluster on quotas sync\n pushEvery = 2000 # interval of time (ms) between 2 quotas sync\n timeout = ${otoroshi.cluster.worker.timeout} # timeout when calling leader cluster on quotas sync\n }\n }\n }\n}\n```\n\nyou can also use many env. variables to configure Otoroshi cluster\n\n```hocon\notoroshi {\n cluster {\n mode = ${?CLUSTER_MODE}\n compression = ${?CLUSTER_COMPRESSION}\n leader {\n name = ${?CLUSTER_LEADER_NAME}\n host = ${?CLUSTER_LEADER_HOST}\n clientId = ${?CLUSTER_LEADER_CLIENT_ID}\n clientSecret = ${?CLUSTER_LEADER_CLIENT_SECRET}\n groupingBy = ${?CLUSTER_LEADER_GROUP_BY}\n cacheStateFor = ${?CLUSTER_LEADER_CACHE_STATE_FOR}\n stateDumpPath = ${?CLUSTER_LEADER_DUMP_PATH}\n }\n worker {\n name = ${?CLUSTER_WORKER_NAME}\n retries = ${?CLUSTER_WORKER_RETRIES}\n timeout = ${?CLUSTER_WORKER_TIMEOUT}\n state {\n retries = ${?CLUSTER_WORKER_STATE_RETRIES}\n pollEvery = ${?CLUSTER_WORKER_POLL_EVERY}\n timeout = ${?CLUSTER_WORKER_POLL_TIMEOUT}\n }\n quotas {\n retries = ${?CLUSTER_WORKER_QUOTAS_RETRIES}\n pushEvery = ${?CLUSTER_WORKER_PUSH_EVERY}\n timeout = ${?CLUSTER_WORKER_PUSH_TIMEOUT}\n }\n }\n }\n}\n```\n\n@@@ warning\nYou **should** use HTTPS exposition for the Otoroshi API that will be used for data sync as sensitive informations are exchanged between control plane and data plane.\n@@@\n\n@@@ warning\nYou **must** have the same cluster configuration on every Otoroshi instance (worker/leader) with only names and mode changed for each instance. Some things in leader/worker are computed using configuration of their counterpart worker/leader.\n@@@\n\n## Cluster UI\n\nOnce an Otoroshi instance is launcher as cluster Leader, a new row of live metrics tile will be available on the home page of Otoroshi admin UI.\n\n@@@ div { .centered-img }\n![](\"../img/cluster-3.png\")
\n@@@\n\nyou can also access a more detailed view of the cluster at `Settings (cog icon) / Cluster View`\n\n@@@ div { .centered-img }\n![](\"../img/cluster-4.png\")
\n@@@\n\n## Run examples\n\nfor leader \n\n```sh\njava -Dhttp.port=8091 -Dhttps.port=9091 -Dotoroshi.cluster.mode=leader -jar otoroshi.jar\n```\n\nfor worker\n\n```sh\njava -Dhttp.port=8092 -Dhttps.port=9092 -Dotoroshi.cluster.mode=worker \\\n -Dotoroshi.cluster.leader.urls.0=http://127.0.0.1:8091 -jar otoroshi.jar\n```\n"
- },
- {
- "name": "index.md",
- "id": "/topics/index.md",
- "url": "/topics/index.html",
- "title": "Detailed topics",
- "content": "# Detailed topics\n\nIn this sections, you will find informations about various topics supported by Otoroshi\n\n@@@ index\n\n* [Chaos engineering with the Snow Monkey](./snow-monkey.md)\n* [Service mesh](./service-mesh.md)\n* [JWT Tokens verification](./jwt-verifications.md)\n* [SSL/TLS termination with Otoroshi](./ssl.md)\n* [Mutual TLS with Otoroshi](./mtls.md)\n* [Otoroshi Clustering](./clustering.md)\n* [Requests trasnformation](./req-transformers.md)\n\n@@@\n"
- },
- {
- "name": "jwt-verifications.md",
- "id": "/topics/jwt-verifications.md",
- "url": "/topics/jwt-verifications.html",
- "title": "JWT Tokens verification",
- "content": "# JWT Tokens verification\n\nSometimes, it can be pretty useful to verify Jwt tokens coming from other provider on some services. Otoroshi provides a tool to do that per service. In the Service descriptor page, you can find a `Jwt token Verification` section dedicated to this topic.\n\n## Service descriptor local verifications\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-capture.png\")
\n@@@\n\nin this section you can select the type of verification you can choose if the verifier is local to the `Service descriptor` or reference a global one.\n\nYou can also enabled/disable jwt verification and activate strict mode. In strict mode, requests will be rejected if the jwt token is not found.\n\n### Jwt token location\n\nYou can use the `Source` selector to specify where the Jwt token can be found. \n\n* in a query string param\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-inquery.png\")
\n@@@\n\n* in a header\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-inheader.png\")
\n@@@\n\n* in a cookie\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-incookie.png\")
\n@@@\n\n### Jwt signing\n\nYou can use the `Algo.` selector to specify the signing algorithm to use to verifiy the token\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-signing-1.png\")
\n@@@\n\nyou can choose between\n\n* Hmac + SHA256\n* Hmac + SHA384\n* Hmac + SHA512\n* RSA + SHA256\n* RSA + SHA384\n* RSA + SHA512\n* Elliptic Curve + SHA256\n* Elliptic Curve + SHA384\n* Elliptic Curve + SHA512\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-signing-2.png\")
\n@@@\n\nYou can use syntax like `${env.MY_ENV_VAR}` or `${config.my.config.path}` to provide secret/keys values. \n\n\n### Just verify signature and fields value\n\nUsing the `Verif. strategy` selector, you can choose `Verify jwt token`. This will verify if the token is signed using the settings from `jwt signing` section and the value of the fields provided in `Verify token fields`. Then the token will be send to the target just like that.\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-verify.png\")
\n@@@\n\n### Re-sign the token\n\nUsing the `Verif. strategy` selector, you can choose `Verify and re-sign jwt token`. This will verify if the token is signed using the settings from `jwt signing` section and the value of the fields provided in `Verify token fields`. Then the token will be re-signed using the settings provided in `Re-sign algo` and will be send to the target.\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-resign.png\")
\n@@@\n\n### Transform the token\n\nUsing the `Verif. strategy` selector, you can choose `Verify, re-sign and transform jwt token`. This will verify if the token is signed using the settings from `jwt signing` section and the value of the fields provided in `Verify token fields`. Then the token will be re-signed using the settings provided in `Re-sign algo`. You can also change the location of the token using `Token location`, remove fields using `Remove token fields`, set fields value using `Set token fields` and even rename fields using `Rename token fields`.\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-transform.png\"){kind=spacer}
\n@@@\n\nYou can also use a mini expression language in `Set token fields`. You just have to add expressions in values like `${expression}`. Supported expressions are the following :\n\n* `${date}` => set the current date\n* `${date.format('dd/MM/yyyy')}` => set the current date formatted with the format you want\n* `${token.fieldName}` => get the value of the field named `fieldName`\n* `${token.fieldName.replace('a', 'b')}` => get the value of the field named `fieldName` and replace `a` with `b`\n* `${token.fieldName.replaceAll('[0-9]', '-')}` => get the value of the field named `fieldName` and replace digits with `-`\n\nyou can of course use multiple expressions in one field like `my-value-is-${date}-with${token.user}`\n\n## Global verifications\n\nYou can create global jwt verifiers and reference them in your services (from the `Type` selector). When you set the type of verification to `Reference to a global definition`, you can choose an existing global jwt verifier\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-global-ref.png\")
\n@@@\n\nTo create a global verifier, go to `Settings (cog icon) / Global Jwt Verifiers` and it will display the list of global verifiers.\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-global-verifiers.png\")
\n@@@\n\nyou can them create, edit or delete verifiers\n\n@@@ div { .centered-img }\n![](\"../img/jwt-verif-global-verifier.png\")
\n@@@\n\n"
- },
- {
- "name": "mtls.md",
- "id": "/topics/mtls.md",
- "url": "/topics/mtls.html",
- "title": "Mutual TLS with Otoroshi",
- "content": "# Mutual TLS with Otoroshi\n\nOtoroshi support mutual TLS out of the box. mTLS from client to Otoroshi and from Otoroshi to targets are supported. In this article we will see how to configure Otoroshi to use end-to-end mTLS. All code and files used in this articles can be found on the [Otoroshi github](https://github.com/MAIF/otoroshi/tree/master/demos/mtls)\n\n@@@ note { title=\"Experimental Feature\" }\nDynamic Mutual TLS is an experimental feature. It can change until it becomess an official feature\n@@@\n\n## End-to-end mTLS\n\nThe use case is the following :\n\n@@@ div { .centered-img }\n![](\"../img/mtls-arch-1.jpg\")
\n@@@\n\nfor this demo you will have to edit your `/etc/hosts` file to add the following entries\n\n```\n127.0.0.1 otoroshi-api.foo.bar otoroshi.foo.bar otoroshi-admin-internal-api.foo.bar privateapps.foo.bar \n127.0.0.1 api.backend.lol api.frontend.lol www.backend.lol www.frontend.lol validation.backend.lol\n```\n\n### Create certificates\n\nBut first we need to generate some certificates to make the demo work\n\n```sh\nmkdir mtls-demo\ncd mtls-demo\nmkdir ca\nmkdir server\nmkdir client\n\n# create a certificate authority key, use password as pass phrase\nopenssl genrsa -out ./ca/ca-backend.key 4096\n# remove pass phrase\nopenssl rsa -in ./ca/ca-backend.key -out ./ca/ca-backend.key\n# generate the certificate authority cert\nopenssl req -new -x509 -sha256 -days 730 -key ./ca/ca-backend.key -out ./ca/ca-backend.cer -subj \"/CN=MTLSB\"\n\n\n# create a certificate authority key, use password as pass phrase\nopenssl genrsa -out ./ca/ca-frontend.key 2048\n# remove pass phrase\nopenssl rsa -in ./ca/ca-frontend.key -out ./ca/ca-frontend.key\n# generate the certificate authority cert\nopenssl req -new -x509 -sha256 -days 730 -key ./ca/ca-frontend.key -out ./ca/ca-frontend.cer -subj \"/CN=MTLSF\"\n\n\n# now create the backend cert key, use password as pass phrase\nopenssl genrsa -out ./server/_.backend.lol.key 2048\n# remove pass phrase\nopenssl rsa -in ./server/_.backend.lol.key -out ./server/_.backend.lol.key\n# generate the csr for the certificate\nopenssl req -new -key ./server/_.backend.lol.key -sha256 -out ./server/_.backend.lol.csr -subj \"/CN=*.backend.lol\"\n# generate the certificate\nopenssl x509 -req -days 365 -sha256 -in ./server/_.backend.lol.csr -CA ./ca/ca-backend.cer -CAkey ./ca/ca-backend.key -set_serial 1 -out ./server/_.backend.lol.cer\n# verify the certificate, should output './server/_.backend.lol.cer: OK'\nopenssl verify -CAfile ./ca/ca-backend.cer ./server/_.backend.lol.cer\n\n\n# now create the frontend cert key, use password as pass phrase\nopenssl genrsa -out ./server/_.frontend.lol.key 2048\n# remove pass phrase\nopenssl rsa -in ./server/_.frontend.lol.key -out ./server/_.frontend.lol.key\n# generate the csr for the certificate\nopenssl req -new -key ./server/_.frontend.lol.key -sha256 -out ./server/_.frontend.lol.csr -subj \"/CN=*.frontend.lol\"\n# generate the certificate\nopenssl x509 -req -days 365 -sha256 -in ./server/_.frontend.lol.csr -CA ./ca/ca-frontend.cer -CAkey ./ca/ca-frontend.key -set_serial 1 -out ./server/_.frontend.lol.cer\n# verify the certificate, should output './server/_.frontend.lol.cer: OK'\nopenssl verify -CAfile ./ca/ca-frontend.cer ./server/_.frontend.lol.cer\n\n\n# now create the client cert key for backend, use password as pass phrase\nopenssl genrsa -out ./client/_.backend.lol.key 2048\n# remove pass phrase\nopenssl rsa -in ./client/_.backend.lol.key -out ./client/_.backend.lol.key\n# generate the csr for the certificate\nopenssl req -new -key ./client/_.backend.lol.key -out ./client/_.backend.lol.csr -subj \"/CN=*.backend.lol\"\n# generate the certificate\nopenssl x509 -req -days 365 -sha256 -in ./client/_.backend.lol.csr -CA ./ca/ca-backend.cer -CAkey ./ca/ca-backend.key -set_serial 2 -out ./client/_.backend.lol.cer\n# generate a pkcs12 version of the cert and key, use password as password\nopenssl pkcs12 -export -clcerts -in client/_.backend.lol.cer -inkey client/_.backend.lol.key -out client/_.backend.lol.p12\n\n\n# now create the client cert key for frontend, use password as pass phrase\nopenssl genrsa -out ./client/_.frontend.lol.key 2048\n# remove pass phrase\nopenssl rsa -in ./client/_.frontend.lol.key -out ./client/_.frontend.lol.key\n# generate the csr for the certificate\nopenssl req -new -key ./client/_.frontend.lol.key -out ./client/_.frontend.lol.csr -subj \"/CN=*.frontend.lol\"\n# generate the certificate\nopenssl x509 -req -days 365 -sha256 -in ./client/_.frontend.lol.csr -CA ./ca/ca-frontend.cer -CAkey ./ca/ca-frontend.key -set_serial 2 -out ./client/_.frontend.lol.cer\n# generate a pkcs12 version of the cert and key, use password as password\nopenssl pkcs12 -export -clcerts -in client/_.frontend.lol.cer -inkey client/_.frontend.lol.key -out client/_.frontend.lol.p12\n```\n\nonce it's done, you should have something like\n\n```sh\n$ tree\n.\n├── backend.js\n├── ca\n│ ├── ca-backend.cer\n│ ├── ca-backend.key\n│ ├── ca-frontend.cer\n│ └── ca-frontend.key\n├── client\n│ ├── _.backend.lol.cer\n│ ├── _.backend.lol.csr\n│ ├── _.backend.lol.key\n│ ├── _.backend.lol.p12\n│ ├── _.frontend.lol.cer\n│ ├── _.frontend.lol.csr\n│ ├── _.frontend.lol.key\n│ └── _.frontend.lol.p12\n└── server\n ├── _.backend.lol.cer\n ├── _.backend.lol.csr\n ├── _.backend.lol.key\n ├── _.frontend.lol.cer\n ├── _.frontend.lol.csr\n └── _.frontend.lol.key\n\n3 directories, 18 files\n```\n\n### The backend service \n\nnow, let's create a backend service using nodejs. Create a file named `backend.js`\n\n```sh\ntouch backend.js\n```\n\nand put the following content\n\n```js\nconst fs = require('fs'); \nconst https = require('https'); \n\nconst options = { \n key: fs.readFileSync('./server/_.backend.lol.key'), \n cert: fs.readFileSync('./server/_.backend.lol.cer'), \n ca: fs.readFileSync('./ca/ca-backend.cer'), \n}; \n\nhttps.createServer(options, (req, res) => { \n res.writeHead(200, {\n 'Content-Type': 'application/json'\n }); \n res.end(JSON.stringify({ message: 'Hello World!' }) + \"\\n\"); \n}).listen(8444);\n```\n\nto run the server, just do \n\n```sh\nnode ./backend.js\n```\n\nnow you can try your server with\n\n```sh\ncurl --cacert ./ca/ca-backend.cer https://api.backend.lol:8444/\n# will print {\"message\":\"Hello World!\"}\n```\n\nnow modify your backend server to ensure that the client provides a client certificate like:\n\n```js\nconst fs = require('fs'); \nconst https = require('https'); \n\nconst options = { \n key: fs.readFileSync('./server/_.backend.lol.key'), \n cert: fs.readFileSync('./server/_.backend.lol.cer'), \n ca: fs.readFileSync('./ca/ca-backend.cer'), \n requestCert: true, \n rejectUnauthorized: true\n}; \n\nhttps.createServer(options, (req, res) => { \n console.log('Client certificate CN: ', req.socket.getPeerCertificate().subject.CN);\n res.writeHead(200, {\n 'Content-Type': 'application/json'\n }); \n res.end(JSON.stringify({ message: 'Hello World!' }) + \"\\n\"); \n}).listen(8444);\n```\n\nyou can test your new server with\n\n```sh\ncurl --cacert ./ca/ca-backend.cer --cert-type pkcs12 --cert ./client/_.backend.lol.p12:password https://api.backend.lol:8444/\n# will print {\"message\":\"Hello World!\"}\n```\n\n### Otoroshi setup\n\nDownload the latest version of the Otoroshi jar and run it like\n\n```sh\njava -jar otoroshi.jar\n\n[info] otoroshi-env - Admin API exposed on http://otoroshi-api.foo.bar:8080\n[info] otoroshi-env - Admin UI exposed on http://otoroshi.foo.bar:8080\n[info] otoroshi-in-memory-datastores - Now using InMemory DataStores\n[info] otoroshi-env - The main datastore seems to be empty, registering some basic services\n[info] otoroshi-env - You can log into the Otoroshi admin console with the following credentials: admin@otoroshi.io / xxxxxxxxxxxx\n[info] play.api.Play - Application started (Prod)\n[info] p.c.s.AkkaHttpServer - Listening for HTTP on /0:0:0:0:0:0:0:0:8080\n[info] p.c.s.AkkaHttpServer - Listening for HTTPS on /0:0:0:0:0:0:0:0:8443\n[info] otoroshi-env - Generating a self signed SSL certificate for https://*.foo.bar ...\n```\n\nand log into otoroshi with the tuple `admin@otoroshi.io / xxxxxxxxxxxx` displayed in the logs. Once logged in, create a new public service exposed on `http://api.frontend.lol` that targets `ahc:https://api.backend.lol:8444/`.\n\n@@@ div { .centered-img }\n![](\"../img/mtls-service-1.png\")
\n@@@\n\nand test it\n\n```sh\ncurl http://api.frontend.lol:8080/\n# the following error should be returned: {\"Otoroshi-Error\":\"Something went wrong, you should try later. Thanks for your understanding.\"}\n```\n\n@@@ warning\nAs seen before, the target of the otoroshi service is `ahc:https://api.backend.lol:8444/`. `ahc:https://` is not a typo and is intended. This tells otoroshi to use its experimental `http client` with dynamic tls support to fetch this resource.\n@@@\n\nyou should get an error due to the fact that Otoroshi doesn't know about the server certificate or the client certificate expected by the server.\n\nWe have to add the client certificate for `https://api.backend.lol` to Otoroshi. Go to http://otoroshi.foo.bar:8080/bo/dashboard/certificates and create a new item. Copy and paste the content of `./client/_.backend.lol.cer` and `./client/_.backend.lol.key` respectively in `Certificate full chain` and `Certificate private key`.\n\n@@@ div { .centered-img }\n![](\"../img/mtls-cert-client-backend-1.png\")
\n@@@\n\nand retry the following curl command \n\n```sh\ncurl http://api.frontend.lol:8080/\n# the output should be: {\"message\":\"Hello World!\"}\n```\n\nnow we have to expose `https://api.frontend.lol:8443` using otoroshi. Go to http://otoroshi.foo.bar:8080/bo/dashboard/certificates and create a new item. Copy and paste the content of `./server/_.frontend.lol.cer` and `./server/_.frontend.lol.key` respectively in `Certificate full chain` and `Certificate private key`.\n\nand try the following command\n\n```sh\ncurl --cacert ./ca/ca-frontend.cer https://api.frontend.lol:8443/\n# the output should be: {\"message\":\"Hello World!\"}\n```\n\nnow we have to enforce the fact that we want client certificate for `api.frontend.lol`. To do that, we have to create a `Validation authority` in otoroshi and use it on the `api.frontend.lol` service. Go to http://otoroshi.foo.bar:8080/bo/dashboard/validation-authorities and create a new item. A validation authority is supposed to be a remote service that will say if the client certificate is valid. Here we don't really care if the certificate is valid or not, but we want to enforce the fact that there is a client certificate. So just check the `All cert. valid button`.\n\n@@@ div { .centered-img }\n![](\"../img/mtls-va-1.png\")
\n@@@\n\nnow go back on your `api.frontend.lol` service, in the `Validation authority` section and select the authority you just created.\n\n@@@ div { .centered-img }\n![](\"../img/mtls-va-ref-1.png\")
\n@@@\n\nnow if you retry \n\n```sh\ncurl --cacert ./ca/ca-frontend.cer https://api.frontend.lol:8443/\n# the output should be: {\"Otoroshi-Error\":\"You're not authorized here !\"}\n```\n\nyou should get an error because no client cert. is passed with the request. But if you pass the `./client/_.frontend.lol.p12` client cert in your curl call\n\n```sh\ncurl --cacert ./ca/ca-frontend.cer --cert-type pkcs12 --cert ./client/_.frontend.lol.p12:password https://api.frontend.lol:8443/\n# the output should be: {\"message\":\"Hello World!\"}\n```\n\n### End to end test\n\nNow we can try to write a small nodejs client that uses our client certificates. Create a `client.js` file with the following code\n\n```js\nconst fs = require('fs'); \nconst https = require('https'); \n\nprocess.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0;\n\nconst options = { \n hostname: 'api.frontend.lol', \n port: 8443, \n path: '/', \n method: 'GET', \n key: fs.readFileSync('./client/_.frontend.lol.key'), \n cert: fs.readFileSync('./client/_.frontend.lol.cer'), \n ca: fs.readFileSync('./ca/ca-frontend.cer'), \n}; \n\nconst req = https.request(options, (res) => { \n console.log('statusCode', res.statusCode);\n console.log('headers', res.headers);\n console.log('body:');\n res.on('data', (data) => { \n process.stdout.write(data); \n }); \n}); \n\nreq.end(); \n\nreq.on('error', (e) => { \n console.error(e); \n});\n```\n\nand run the following command\n\n```sh\n$ node client.js\n# statusCode 200\n# headers { date: 'Mon, 10 Dec 2018 16:01:11 GMT',\n# connection: 'close',\n# 'transfer-encoding': 'chunked',\n# 'content-type': 'application/json' }\n# body:\n# {\"message\":\"Hello World!\"}\n```\n\nAnd that's it \n\n## Validating client certificates based on user identity\n\n@@@ note { title=\"Experimental Feature\" }\nValidation authorities is an experimental feature. It can change until it becomess an official feature\n@@@\n\nThe use case is the following :\n\n@@@ div { .centered-img }\n![](\"../img/mtls-arch-2.jpg\")
\n@@@\n\nthe idea here is to provide a unique client certificate per device that can access Otoroshi and use a validation authority to check if the user is allowed to access the underlying app with a specific device.\n\n### Generate client certificates for devices\n\nTo do that we are going to create two client certificates, one per device (let say for a laptop and a desktop computer). We are going to use the device serial number as common name of the certificate to be able to identify the device behind the certificate.\n\n```sh\nopenssl genrsa -out ./client/device-1.key 2048\nopenssl rsa -in ./client/device-1.key -out ./client/device-1.key\nopenssl req -new -key ./client/device-1.key -out ./client/device-1.csr -subj \"/CN=mbp-123456789\"\nopenssl x509 -req -days 365 -sha256 -in ./client/device-1.csr -CA ./ca/ca-frontend.cer -CAkey ./ca/ca-frontend.key -set_serial 3 -out ./client/device-1\nopenssl pkcs12 -export -clcerts -in client/device-1 -inkey client/device-1.key -out client/device-1.p12\n\nopenssl genrsa -out ./client/device-2.key 2048\nopenssl rsa -in ./client/device-2.key -out ./client/device-2.key\nopenssl req -new -key ./client/device-2.key -out ./client/device-2.csr -subj \"/CN=nuc-987654321\"\nopenssl x509 -req -days 365 -sha256 -in ./client/device-2.csr -CA ./ca/ca-frontend.cer -CAkey ./ca/ca-frontend.key -set_serial 4 -out ./client/device-2\nopenssl pkcs12 -export -clcerts -in client/device-2 -inkey client/device-2.key -out client/device-2.p12\n```\n\n### Setup actual validation\n\nnow we are going to write an validation authority (with mTLS too) that is going to respond on `https://validation.backend.lol:8445`. The server has access to a list of apps, users and devices to check if everything is correct. In this implementation, the lists are hardcoded, but you can write your own implementation that will fetch data from your corporate LDAP, CA, etc. Create a `validation.js` file and add the following content. Don't forget to do `yarn add x509` before running the server with `node validation.js`\n\n```js\nconst fs = require('fs'); \nconst https = require('https'); \nconst x509 = require('x509');\n\n// list of knwon apps\nconst apps = [\n {\n \"id\": \"iogOIDH09EktFhydTp8xspGvdaBq961DUDr6MBBNwHO2EiBMlOdafGnImhbRGy8z\",\n \"name\": \"my-web-service\",\n \"description\": \"A service that says hello\",\n \"host\": \"www.frontend.lol\"\n }\n];\n\n// list of known users\nconst users = [\n {\n \"name\": \"Mathieu\",\n \"email\": \"mathieu@foo.bar\",\n \"appRights\": [\n {\n \"id\": \"iogOIDH09EktFhydTp8xspGvdaBq961DUDr6MBBNwHO2EiBMlOdafGnImhbRGy8z\",\n \"profile\": \"user\",\n \"forbidden\": false\n },\n {\n \"id\": \"PqgOIDH09EktFhydTp8xspGvdaBq961DUDr6MBBNwHO2EiBMlOdafGnImhbRGy8z\",\n \"profile\": \"none\",\n \"forbidden\": true\n },\n ],\n \"ownedDevices\": [\n \"mbp-123456789\",\n \"nuc-987654321\",\n ]\n }\n];\n\n// list of known devices\nconst devices = [\n {\n \"serialNumber\": \"mbp-123456789\",\n \"hardware\": \"Macbook Pro 2018 13 inc. with TouchBar, 2.6 GHz, 16 Gb\",\n \"acquiredAt\": \"2018-10-01\",\n },\n {\n \"serialNumber\": \"nuc-987654321\",\n \"hardware\": \"Intel NUC i7 3.0 GHz, 32 Gb\",\n \"acquiredAt\": \"2018-09-01\",\n },\n {\n \"serialNumber\": \"iphone-1234\",\n \"hardware\": \"Iphone XS, 256 Gb\",\n \"acquiredAt\": \"2018-12-01\",\n }\n];\n\nconst options = { \n key: fs.readFileSync('./server/_.backend.lol.key'), \n cert: fs.readFileSync('./server/_.backend.lol.cer'), \n ca: fs.readFileSync('./ca/ca-backend.cer'), \n requestCert: true, \n rejectUnauthorized: true\n}; \n\nfunction readBody(request) {\n return new Promise((success, failure) => {\n const body = [];\n request.on('data', (chunk) => {\n body.push(chunk);\n }).on('end', () => {\n const bodyStr = Buffer.concat(body).toString();\n success(JSON.parse(bodyStr));\n });\n });\n}\n\nfunction chainIsValid(chain) {\n // validate cert dates\n // validate cert against clr\n // validate whatever you want here\n return true;\n}\n\nfunction call(req, res) {\n readBody(req).then(body => {\n const service = body.service;\n const email = (body.user || { email: 'mathieu@foo.bar' }).email; // here, should not be null if used with an otoroshi auth. module\n // common name should be device serial number\n const commonName = x509.getSubject(body.chain).commonName\n // search for a known device\n const device = devices.filter(d => d.serialNumber === commonName)[0];\n // search for a known user\n const user = users.filter(d => d.email === email)[0];\n // search for a known application\n const app = apps.filter(d => d.id === service.id)[0];\n res.writeHead(200, { 'Content-Type': 'application/json' }); \n if (chainIsValid(body.chain.map(x509.parseCert)) && user && device && app) {\n // check if the user actually owns the device\n const userOwnsDevice = user.ownedDevices.filter(d => d === device.serialNumber)[0];\n // check if the user has rights to access the app\n const rights = user.appRights.filter(d => d.id === app.id)[0];\n const hasRightToUseApp = !rights.forbidden\n if (userOwnsDevice && hasRightToUseApp) {\n // yeah !!!!\n console.log(`Call from user \"${user.email}\" with device \"${device.hardware}\" on app \"${app.name}\" with profile \"${rights.profile}\" authorized`)\n res.end(JSON.stringify({ status: 'good', profile: rights.profile }) + \"\\n\"); \n } else {\n // nope !!! nope, nope nope\n console.log(`Call from user \"${user.email}\" with device \"${device.hardware}\" on app \"${app.name}\" unauthorized because user doesn't owns the hardware or has no rights`)\n res.end(JSON.stringify({ status: 'unauthorized' }) + \"\\n\"); \n }\n } else {\n console.log(`Call unauthorized`)\n res.end(JSON.stringify({ status: 'unauthorized' }) + \"\\n\"); \n }\n });\n}\n\nhttps.createServer(options, call).listen(8445);\n```\n\nthe corresponding authority validation can be created in Otoroshi like \n\n```json\n{\n \"id\": \"r7m8j31rh66hhdia3ormfm0wfevu1kvg0zgaxsp3oxb6ivf7fy8kvygmvnrlxv81\",\n \"name\": \"Actual validation authority\",\n \"description\": \"Actual validation authority\",\n \"url\": \"ahc:https://validation.backend.lol:8445\",\n \"host\": \"validation.backend.lol\",\n \"goodTtl\": 600000,\n \"badTtl\": 60000,\n \"method\": \"POST\",\n \"path\": \"/certificates/_validate\",\n \"timeout\": 10000,\n \"noCache\": false,\n \"alwaysValid\": false,\n \"headers\": {}\n}\n```\n\nbut you don't need to create it right now.\n\nTypically, a validation authority server is a server with a route on `POST /certificates/_validate` that accepts `application/json` and returns `application/json` with a body like\n\n```json\n{\n \"apikey\": nullable {\n \"clientId\": String,\n \"clientName\": String,\n \"authorizedGroup\": String,\n \"enabled\": Boolean,\n \"readOnly\": Boolean,\n \"allowClientIdOnly\": Boolean,\n \"throttlingQuota\": Long,\n \"dailyQuota\": Long,\n \"monthlyQuota\": Long,\n \"metadata\": Map[String, String]\n },\n \"user\": nullable {\n \"email\": String,\n \"name\": String,\n },\n \"service\": {\n \"id\": String,\n \"name\": String,\n \"groupId\": String,\n \"domain\": String,\n \"env\": String,\n \"subdomain\": String,\n \"root\": String,\n \"metadata\": String\n },\n \"chain\": PemFormattedCertificateChainString,\n \"fingerprints\": Array[String]\n}\n```\n\n\n### Setup Otoroshi\n\nYou can start Otoroshi and import data from the `state.json` file in the demo folder. The login tuple is `admin@otoroshi.io / password`. The `state.json` file contains everything you need for the demo, like certificates, service descriptors, auth. modules, etc ...\n\n```sh\njava -Dapp.importFrom=$(pwd)/state.json -Dapp.privateapps.port=8080 -jar otoroshi.jar\n\n[info] otoroshi-env - Admin API exposed on http://otoroshi-api.foo.bar:8080\n[info] otoroshi-env - Admin UI exposed on http://otoroshi.foo.bar:8080\n[info] otoroshi-in-memory-datastores - Now using InMemory DataStores\n[info] otoroshi-env - The main datastore seems to be empty, registering some basic services\n[info] otoroshi-env - Importing from: /pwd/state.json\n[info] play.api.Play - Application started (Prod)\n[info] otoroshi-env - Successful import !\n[info] p.c.s.AkkaHttpServer - Listening for HTTP on /0:0:0:0:0:0:0:0:8080\n[info] p.c.s.AkkaHttpServer - Listening for HTTPS on /0:0:0:0:0:0:0:0:8443\n```\n\n### Testing \n\nYou can test the service with curl like\n\n```sh\ncurl --cacert ./ca/ca-frontend.cer --cert-type pkcs12 --cert ./client/device-1.p12:password https://www.frontend.lol:8443/\n# output: ![](\"../img/mtls-ff-1.png\")
\n@@@\n\nNow restart firefox.\n\nNext, go to the `my-web-service` service in otoroshi (log in with `admin@otoroshi.io / password`) and activate `Enforce user login` in the Authentication section. It means that now, you'll have to log in when you'll go to https://www.frontend.lol:8443. With authentication activated on otoroshi, the user identity will be sent to the validation authority, so you can change the following line in the file `validation.js`\n\n```js\nconst email = (body.user || { email: 'mathieu@foo.bar' }).email; // here, should not be null if used with an otoroshi auth. module\n```\n\nto\n\n```js\nconst email = body.user.email;\n```\n\nThen, in Firefox, go to https://www.frontend.lol:8443/, firefox will ask which client certificate to use. Select the one you imported (in the process, maybe firefox will warn you that the certificate of the site is auto signed, just ignore it and continue ;) )\n\n@@@ div { .centered-img }\n![](\"../img/mtls-ff-2.png\")
\n@@@\n\nthen, you'll see a login screen from otoroshi. You can log in with `mathieu@foo.bar / password` and then you should see the hello world message.\n\n@@@ div { .centered-img }\n![](\"../img/mtls-ff-3.png\")
\n@@@\n\n### Going further with user authentication\n\nFor stronger user authentication, you can try to use an auth. module baked by a keycloak instance with yubikey as a strong second factor authentication instead of the basic auth. module we used previously in this article.\n"
- },
- {
- "name": "req-transformers.md",
- "id": "/topics/req-transformers.md",
- "url": "/topics/req-transformers.html",
- "title": "Request transformers",
- "content": "# Request transformers\n\nWhen everything has failed and you absolutely need a feature in Otoroshi to make your use case work, there is a solution. Request transformer is an experimental feature hidden behind a feature flag that allow you to code how Otoroshi should behave when receiving and rounting an http request. With request transformers, you can change request / response headers and request / response body to way you want.\n\n@@@ note { title=\"Experimental Feature\" }\nRequest transformers is an experimental feature. It can change until it becomess an official feature\n@@@\n\n@@@ warning { title=\"Use at your own risk\" }\nRequest transformers is a **very** powerful feature that allow you to do almost everything you want. But like any powerful feature, it comes with a price. The fact that you are responsible for how the request is transformed makes you responsible for all issues introduced by the transformer. If you block the current thread, introduce big latency, generate uncatched exceptions, etc. it's **your fault**. You have to ensure that your code is fast, non blocking, safe, etc. In any case, Otoroshi developers will not responsible for issues caused by a request transformer.\n\nAnd always remember this quote from ~~uncle Ben~~ Winston Churchill:\n\n![](\"../img/scripts-1.png\")
\n@@@\n\nwhen you write an instance of a transformer in the Otoroshi UI, do the following\n\n```scala\nimport akka.stream.Materializer\nimport env.Env\nimport models.{ApiKey, PrivateAppsUser, ServiceDescriptor}\nimport otoroshi.script._\nimport play.api.Logger\nimport play.api.mvc.{Result, Results}\nimport scala.util._\nimport scala.concurrent.{ExecutionContext, Future}\n\nclass MyTransformer extends RequestTransformer {\n\n val logger = Logger(\"my-transformer\")\n\n // implements the methods you want\n}\n\n// WARN: do not forget this line to provide a working instance of your transformer to Otoroshi\nnew MyTransformer()\n```\n\nYou can use the compile button to check if the script compiles, or code the transformer in your IDE (see next point).\n\nThen go to a service descriptor, scroll to the bottom of the page, and select your transformer in the list\n\n@@@ div { .centered-img }\n![](\"../img/scripts-2.png\")
\n@@@\n\n## Providing a transformer from Java classpath\n\nYou can write your own transformer using your favorite IDE. Just create an SBT project with the following dependencies. It can be quite handy to manage the source code like any other piece of code, and it avoid the compilation time for the script at Otoroshi startup.\n\n```scala\nlazy val root = (project in file(\".\")).\n settings(\n inThisBuild(List(\n organization := \"com.example\",\n scalaVersion := \"2.12.7\",\n version := \"0.1.0-SNAPSHOT\"\n )),\n name := \"request-transformer-example\",\n resolvers += Resolver.bintrayRepo(\"maif\", \"maven\"),\n libraryDependencies += \"fr.maif.otoroshi\" %% \"otoroshi\" % \"1.x.x\"\n )\n```\n\nWhen your code is ready, create a jar file \n\n```\nsbt package\n```\n\nand add the jar file to the Otoroshi classpath\n\n```sh\njava -Dotoroshi.scripts.enabled=true -cp \"/path/to/transformer.jar:$/path/to/otoroshi.jar\" play.core.server.ProdServerStart\n```\n\nthen, in your service descriptor, you can chose your transformer in the list. If you want to do it from the API, you have to defined the transformerRef using `cp:` prefix like \n\n```json\n{\n \"transformerRef\": \"cp:my.class.package.MyTransformer\"\n}\n```\n\n## Getting custom configuration from the Otoroshi config. file\n\nLet say you need to provide custom configuration values for a script, then you can customize a configuration file of Otoroshi\n\n```hocon\ninclude \"application.conf\"\n\notoroshi {\n scripts {\n enabled = true\n }\n}\n\nmy-transformer {\n env = \"prod\"\n maxRequestBodySize = 2048\n maxResponseBodySize = 2048\n}\n```\n\nthen start Otoroshi like\n\n```sh\njava -Dconfig.file=/path/to/custom.conf -jar otoroshi.jar\n```\n\nthen, in your transformer, you can write something like \n\n```scala\npackage com.example.otoroshi\n\nimport akka.stream.Materializer\nimport akka.stream.scaladsl._\nimport akka.util.ByteString\nimport env.Env\nimport models.{ApiKey, PrivateAppsUser, ServiceDescriptor}\nimport otoroshi.script._\nimport play.api.Logger\nimport play.api.mvc.{Result, Results}\nimport scala.util._\nimport scala.concurrent.{ExecutionContext, Future}\n\nclass BodyLengthLimiter extends RequestTransformer {\n\n override def transformResponseBody(\n snowflake: String,\n body: Source[ByteString, _],\n rawResponse: HttpResponse,\n otoroshiResponse: HttpResponse,\n desc: ServiceDescriptor,\n apiKey: Option[ApiKey],\n user: Option[PrivateAppsUser])(implicit env: Env, ec: ExecutionContext, mat: Materializer): Source[ByteString, _] = {\n val max = env.configuration.getOptional[Long](\"my-transformer.maxResponseBodySize\").getOrElse(Long.MaxValue)\n body.limitWeighted(max)(_.size)\n }\n\n override def transformRequestBody(\n snowflake: String,\n body: Source[ByteString, _],\n rawRequest: HttpRequest,\n otoroshiRequest: HttpRequest,\n desc: ServiceDescriptor,\n apiKey: Option[ApiKey],\n user: Option[PrivateAppsUser])(implicit env: Env, ec: ExecutionContext, mat: Materializer): Source[ByteString, _] = {\n val max = env.configuration.getOptional[Long](\"my-transformer.maxRequestBodySize\").getOrElse(Long.MaxValue)\n body.limitWeighted(max)(_.size)\n }\n}\n```\n\n## Using a library that is not embedded in Otoroshi\n\nJust use the `classpath` option when running Otoroshi\n\n```sh\njava -Dotoroshi.scripts.enabled=true -cp \"/path/to/library.jar:$/path/to/otoroshi.jar\" play.core.server.ProdServerStart\n```\n\nBe carefull as your library can conflict with other libraries used by Otoroshi and affect its stability\n"
- },
- {
- "name": "service-mesh.md",
- "id": "/topics/service-mesh.md",
- "url": "/topics/service-mesh.html",
- "title": "Service mesh with Otoroshi",
- "content": "# Service mesh with Otoroshi\n\n[Service mesh](http://philcalcado.com/2017/08/03/pattern_service_mesh.html) is a pattern that gained a lot of traction lately with tools like [Kubernetes](https://kubernetes.io/) and [Istio](https://istio.io/) using patterns like [sidecar container](https://kubernetes.io/blog/2015/06/the-distributed-system-toolkit-patterns/#example-1-sidecar-containers). It has the advantages to move the complexity of calling other services outside the application. For the application, it's just a local call and the proxy handles apikeys, throttling, quotas, resilience, tracing, etc ...\n\n## Architecture\n\nLet's say that we have 4 service that want to talk to each other. But we don't want to handle the complexity of calling those services with their own apikey, etc ...\n\n@@@ div { .centered-img }\n![](\"https://github.com/MAIF/otoroshi/raw/master/demos/service-mesh/calls.png\")
\n@@@\n\nSo we are going to build an infrastructure where each service has a sidecar Otoroshi that will handle the calls to other services\n\n@@@ div { .centered-img }\n![](\"https://github.com/MAIF/otoroshi/raw/master/demos/service-mesh/mesh.png\")
\n@@@\n\n## Configuration\n\nIn this infrastructure, each service has its own service descriptor in Otoroshi that points to another Otoroshi instance. The challenge here is to dynamically change the target of any service when the current call is supposed to access th actual app.\n\nTo do that, each Otoroshi instance (that is part of the same cluster and use the same database) will have a configuration slightly different configuration than the other ones. \n\n```hocon\nsidecar {\n serviceId = \"my-local-service-id\" # the id of the local sidecar app\n target = \"http://127.0.0.1:56876\" # the local service target, it should be on 127.0.0.1 but can be on another ip address\n from = \"127.0.0.1\" # the ip address authorized to actualy access the local app, it should be on 127.0.0.1 but can be on another ip address\n strict = true # use actual remote address or remote address/proxied remote address\n apikey {\n clientId = \"my-apikey-id\" # the apikey client id to access other services\n }\n}\n```\n\n## Configuration using env. variables\n\nYou can also only env. variables to configure sidecar Otoroshi instances \n\n```sh\nSIDECAR_SERVICE_ID=my-local-service-id\nSIDECAR_TARGET=http://127.0.0.1:56876\nSIDECAR_FROM=127.0.0.1\nSIDECAR_STRICT=true\nSIDECAR_APIKEY_CLIENT_ID=my-apikey-id\n```\n\n## Example\n\nYou can find a full example of a simple service mesh using Otoroshi [here](https://github.com/MAIF/otoroshi/tree/master/demos/service-mesh)."
- },
- {
- "name": "snow-monkey.md",
- "id": "/topics/snow-monkey.md",
- "url": "/topics/snow-monkey.html",
- "title": "Chaos engineering with the Snow Monkey",
- "content": "# Chaos engineering with the Snow Monkey\n\nNihonzaru (the Snow Monkey) is the chaos engineering tool provided by Otoroshi. You can access it at `Settings (cog icon) / Snow Monkey`.\n\n@@@ div { .centered-img }\n![](\"https://github.com/MAIF/otoroshi/raw/master/resources/nihonzaru-logo.png\"){kind=logo}
\n@@@\n\n## Chaos engineering\n\nOtoroshi offers some tools to introduce [chaos engineering](https://principlesofchaos.org/) in your everyday life. With chaos engineering, you will improve the resilience of your architecture by creating faults in production on running systems. With [Nihonzaru (the snow monkey)](https://en.wikipedia.org/wiki/Japanese_macaque) Otoroshi helps you to create faults on http request/response handled by Otoroshi. \n\n@@@ div { .centered-img }\n![](\"../img/snow-monkey.png\")
\n@@@\n\n## Settings\n\n@@@ div { .centered-img }\n![](\"../img/snow-monkey-settings.png\")
\n@@@\n\nThe snow monkey let you define a few settings to work properly :\n\n* **Include user facing apps.**: you want to create fault in production, but maybe you don't want your users to enjoy some nice snow monkey generated error pages. Using this switch let you include of not user facing apps (ui apps). Each service descriptor has a `User facing app switch` that will be used by the snow monkey.\n* **Dry run**: when dry run is enabled, outages will be registered and will generate events and alerts (in the otoroshi eventing system) but requests won't be actualy impacted. It's a good way to prepare applications to the snow monkey habits\n* **Outage strategy**: Either `AllServicesPerGroup` or `OneServicePerGroup`. It means that only one service per group or all services per groups will have `n` outages (see next bullet point) during the snow monkey working period\n* **Outages per day**: during snow monkey working period, each service per group or one service per group will have only `n` outages registered \n* **Working period**: the snow monkey only works during a working period. Here you can defined when it starts and when it stops\n* **Outage duration**: here you can defined the bounds for the random outage duration when an outage is created on a service\n* **Impacted groups**: here you can define a list of service groups impacted by the snow monkey. If none is specified, then all service groups will be impacted\n\n## Faults\n\nWith the snow monkey, you can generate four types of faults\n\n* **Large request fault**: Add trailing bytes at the end of the request body (if one)\n* **Large response fault**: Add trailing bytes at the end of the response body\n* **Latency injection fault**: Add random response latency between two bounds\n* **Bad response injection fault**: Create predefined responses with custom headers, body and status code\n\nEach fault let you define a ratio for impacted requests. If you specify a ratio of `0.2`, then 20% of the requests for the impacte service will be impacted by this fault\n\n@@@ div { .centered-img }\n![](\"../img/snow-monkey-faults.png\")
\n@@@\n\nThen you juste have to start the snow monkey and enjoy the show ;)\n\n@@@ div { .centered-img }\n![](\"../img/snow-monkey-start.png\")
\n@@@\n\n## Current outages\n\nIn the last section of the snow monkey page, you can see current outages (per service), when they started, their duration, etc ...\n\n@@@ div { .centered-img }\n![](\"../img/snow-monkey-outages.png\")
\n@@@"
- },
- {
- "name": "ssl.md",
- "id": "/topics/ssl.md",
- "url": "/topics/ssl.html",
- "title": "SSL/TLS termination with Otoroshi",
- "content": "# SSL/TLS termination with Otoroshi\n\nOtoroshi can be used as an SSL/TLS termination. It is enabled by default but you can customise HTTPS port with `https.port` config. and env. var `HTTPS_PORT`. You can create upload any certificate you want in the Otoroshi UI or using the API. Just go to `settings (cog icon) / SSL certificates`.\n\n@@@ note { title=\"Experimental Feature\" }\nDynamic SSL/TLS termination is an experimental feature. It can change until it becomess an official feature\n@@@\n\n@@@ note { title=\"TLS 1.3 support\" }\nOtoroshi does support TLS 1.3 when used in combination with JDK 11\n\n![](\"../img/tls13.png\")
\n@@@\n\n@@@ div { .centered-img }\n![](\"../img/ssl.png\")
\n@@@\n\nHere you can add your own certificates, your own CA and even create self signed certificates or certificates from CAs. You can enable auto renewal of thoses self signed certificates or certificates generated. Certificates have to be created with the certificate chain and the private key in PEM format with no password on the private key.\n\nYou can remove the password of a key with the following command\n\n```sh\nopenssl rsa -in keywithpassword.key -out keywithoutpassword.key\n```\n\n@@@ div { .centered-img }\n![](\"../img/ssl2.png\")
\n@@@\n\n"
- },
- {
- "name": "1-groups.md",
- "id": "/usage/1-groups.md",
- "url": "/usage/1-groups.html",
- "title": "Managing service groups",
- "content": "# Managing service groups\n\nGo to `settings (cog icon) / All service groups` to access the list of service groups.\n\n@@@ div { .centered-img }\n![](\"../img/settings-menu-groups.png\")
\n@@@\n\nAnd you should see the list of existing `Service groups`.\n\n@@@ div { .centered-img }\n![](\"../img/service-groups.png\")
\n@@@\n\nBut what is a `Service group` anyway ?\n\n## Otoroshi entities\n\nThere are 3 major entities at the core of Otoroshi :\n\n* **service groups**\n* service descriptors\n* api keys\n\n@@@ div { .centered-img }\n![](\"../img/models-group.png\")
\n@@@\n\nA `service group` is just some kind of logical container for `service descriptors`. A `service group` also has some `api keys` assigned that will be used to access all the `service descriptors` contained in the `service group`.\n\n## Create a service group\n\nA `service group` is a really simple structure with an `id`, a name and a description. To create a new one, just click on the `Add item` button.\n\n@@@ div { .centered-img }\n![](\"../img/service-groups-add.png\")
\n@@@\n\nmodify the name and the description of the group\n\n@@@ div { .centered-img }\n![](\"../img/service-groups-new.png\")
\n@@@\n\nand click on `Create group`\n\n@@@ div { .centered-img }\n![](\"../img/service-groups-create.png\")
\n@@@\n\nThen, you should find your brand new `Service group` in the list of `Service groups`\n\n@@@ div { .centered-img }\n![](\"../img/service-groups-created.png\")
\n@@@\n\n## Update a service\n\nTo update a `Service group`, just click on the edit button ![](\"../img/edit.png\")
of your `Service group`\n\n@@@ div { .centered-img }\n![](\"../img/service-groups-edit.png\")
\n@@@\n\nUpdate the name and description of the `Service group` and click on the `Update group` button to validate name update.\n\n@@@ div { .centered-img }\n![](\"../img/service-groups-update.png\")
\n@@@\n\n## Delete a service group\n\nTo delete a `Service group`, just click on the delete button ![](\"../img/delete.png\")
of your `Service group`\n\n@@@ div { .centered-img }\n![](\"../img/service-groups-delete.png\")
\n@@@\n\nFinally confirm the command\n\n@@@ div { .centered-img }\n![](\"../img/service-groups-delete-confirm.png\")
\n@@@\n"
- },
- {
- "name": "2-services.md",
- "id": "/usage/2-services.md",
- "url": "/usage/2-services.html",
- "title": "Managing services",
- "content": "# Managing services\n\nNow let's create services. Services or `service descriptor` let you declare how to proxy a call from a domain name to another domain name (or multiple domain names). Let's say you have an API exposed on `http://192.168.0.42` and I want to expose it on `https://my.api.foo`. Otoroshi will proxy all calls to `https://my.api.foo` and forward them to `http://192.168.0.42`. While doing that, it will also log everyhting, control accesses, etc.\n\n## Otoroshi entities\n\nThere are 3 major entities at the core of Otoroshi\n\n* service groups\n* **service descriptors**\n* api keys\n\n@@@ div { .centered-img }\n![](\"../img/models-service.png\")
\n@@@\n\nA `service descriptor` is contained in a `service group` and is allowed to be accessed by all the `api key`s authorized on the `service group`.\n\n## Create a service descriptor\n\nTo create a `service descriptor`, click on `Add service` on the Otoroshi sidebar. Then you will be asked to choose a name for the service what is the group of the service. You also have two buttons to create a new group and assign it to the service and create a new group with a name based on the service name.\n\nYou will have a serie of toggle buttons to\n\n* activate / deactivate a service\n* display maintenance page for a service\n* display contruction page for a service\n* enable otoroshi custom response headers containing request id, latency, etc \n* force https usage on the exposed service\n\nThen, you will be able to choose the URL that will be used to reach your new service on Otoroshi.\n\n@@@ div { .centered-img #service-flags }\n![](\"../img/service-flags-2.png\")
\n@@@\n\nIn the `service targets` section, you will be able to choose where the call will be forwarded. You can use multiple targets, in that case, Otoroshi will perform a round robin load balancing between the targets.\n\nYou can also specify a target root, if you say that the target root is `/foo/`, then any call to `https://my.api.foo` will call `http://192.168.0.42/foo/` and nay call to `https://my.api.foo/bar` will call `http://192.168.0.42/foo/bar`.\n\nIn the URL patterns section, you will be able to choose, URL by URL which is private and which is public. By default, all services are private and each call must provide an `api key`. But sometimes, you need to access a service publicly. In that case, you can provide patterns (regex) to make some or all URL public (for example with the pattern `/.*`). You also have a `private pattern` field to restrict public patterns.\n\n@@@ div { .centered-img #targets }\n![](\"../img/new-service-patterns.png\")
\n@@@\n\n### Otoroshi exchange protocol\n\nIf you enable secure communication for a given service, you will have to add a filter on the target application that will take the `Otoroshi-State` header and return it in a header named `Otoroshi-State-Resp`. Otoroshi is also sending a `JWT token`in a header named `Otoroshi-Claim` that the target app can validate.\n\n@@@ div { .centered-img }\n![](\"../img/exchange.png\")
\n@@@\n\nThe `Otoroshi-Claim` is a JWT token containing some informations about the service that is called and the client if available. \n\nBy default, the otoroshi jwt token is signed with the `app.claim.sharedKey` config property (or using the `$CLAIM_SHAREDKEY` env. variable) and uses the `HMAC512` signing algorythm. But it is possible to customize how the token is signed from the service descriptor page in the `Otoroshi exchange protocol` section. \n\n@@@ div { .centered-img }\n![](\"../img/sec-com-signing-bis.png\")
\n@@@\n\nusing another signing algo.\n\n@@@ div { .centered-img }\n![](\"../img/sec-com-signing-2-bis.png\")
\n@@@\n\nhere you can choose the signing algorithm and the secret/keys used. You can use syntax like `${env.MY_ENV_VAR}` or `${config.my.config.path}` to provide secret/keys values. \n\nFor example, for a service named `my-service` with a signing key `secret` with `HMAC512` signing algorythm, the basic JWT token that will be sent should look like the following\n\n```\neyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJzdWIiOiItLSIsImF1ZCI6Im15LXNlcnZpY2UiLCJpc3MiOiJPdG9yb3NoaSIsImV4cCI6MTUyMTQ0OTkwNiwiaWF0IjoxNTIxNDQ5ODc2LCJqdGkiOiI3MTAyNWNjMTktMmFjNy00Yjk3LTljYzctMWM0ODEzYmM1OTI0In0.mRcfuFVFPLUV1FWHyL6rLHIJIu0KEpBkKQCk5xh-_cBt9cb6uD6enynDU0H1X2VpW5-bFxWCy4U4V78CbAQv4g\n```\n\nif you decode it, the payload will look something like\n\n```json\n{\n \"sub\": \"apikey_client_id\",\n \"aud\": \"my-service\",\n \"iss\": \"Otoroshi\",\n \"exp\": 1521449906,\n \"iat\": 1521449876,\n \"jti\": \"71025cc19-2ac7-4b97-9cc7-1c4813bc5924\"\n}\n```\n\nIf you want to validate the `Otoroshi-Claim` on the target app side to ensure that the input requests only comes from `Otoroshi`, you will have to write an HTTP filter to do the job. For instance, if you want to write a filter to make sure that requests only comes from Otoroshi, you can write something like the following (using playframework 2.6).\n\nScala\n: @@snip [filter.scala](../snippets/filter.scala)\n\nJava\n: @@snip [filter.java](../snippets/filter.java)\n\n\n### Canary mode\n\nOtoroshi provides a feature called `Canary mode`. It lets you define new targets for a service, and route a percentage of the traffic on those targets. It's a good way to test a new version of a service before public release. As any client need to be routed to the same version of targets any time, Otoroshi will issue a special header and a cookie containing a `session id`. The header is named `Otoroshi-Canary-Id`.\n\n@@@ div { .centered-img }\n![](\"../img/new-service-canary.png\")
\n@@@\n\n### Service health check\n\nOtoroshi is also capable of checking the health of a service. You can define a URL that will be tested, and Otoroshi will ping that URL regularly. Will doing so, Otoroshi will pass a numeric value in a header named `Otoroshi-Health-Check-Logic-Test`. You can respond with a header named `Otoroshi-Health-Check-Logic-Test-Result` that contains the value of `Otoroshi-Health-Check-Logic-Test` + 42 to indicate that the service is working properly.\n\n@@@ div { .centered-img }\n![](\"../img/new-service-healthcheck.png\")
\n@@@\n\n### Service circuit breaker\n\nIn Otoroshi, each service has its own client settings with a circuit breaker and some retry capabilities. In the `Client settings` section, you will be able to customize the client's behavior.\n\n@@@ div { .centered-img }\n![](\"../img/new-service-client.png\")
\n@@@\n\n### Service settings\n\nYou can also provide some additionnal information about a given service, like an `Open API` descriptor, some metadata, a list of whitelisted/blacklisted ip addresses, etc.\n\nHere you can also define some headers that will be added to each request to the targets. And you will be able to define headers to route the call only if the defined header is present on the request.\n\n@@@ div { .centered-img #service-meta }\n![](\"../img/new-service-meta.png\")
\n@@@\n\n### Read only\n\nThe read only flag on a service descriptor means that this service can only be used with `HEAD`, `OPTIONS` and `GET` http verbs. You can also active the same flag on `ApiKey`s to be more specific on who cannot use write http verbs.\n\n### CORS \n\nIf you enabled this section, CORS will be automatically supported on the current service provider. The pre-flight request will be handled by Otoroshi. You can customize every CORS headers :\n\n@@@ div { .centered-img }\n![](\"../img/cors.png\")
\n@@@\n\n### Service authentication\n\nSee @ref:[Aauthentication](./9-auth.md)\n\n### Custom error templates\n\nFinally, you can define custom error templates that will be displayed when an error occurs when Otoroshi try to reach the target or when Otoroshi itself has an error. You can also define custom templates for maintenance and service pages.\n"
- },
- {
- "name": "3-apikeys.md",
- "id": "/usage/3-apikeys.md",
- "url": "/usage/3-apikeys.html",
- "title": "Managing API keys",
- "content": "# Managing API keys\n\nNow that you know how to create service groups and service descriptors, we will see how to create API keys.\n\n## Otoroshi entities\n\nThere are 3 major entities at the core of Otoroshi.\n\n* service groups\n* service descriptors\n* **api keys**\n\n@@@ div { .centered-img }\n![](\"../img/models-apikey.png\")
\n@@@\n\nAn `API key` related to a `service group` to allow you to access any `service descriptor` contained in a `service group`. You can, of course, create multiple `API key` for a given `service group`.\n\nIn the Otoroshi admin dashboard, we chose to access `API keys` from `service descriptors` only, but when you access `API keys` for a `service descriptor`, you actually access `API keys` for the `service group` containing the `service descriptor`.\n\n`API keys` can be provided to Otoroshi through :\n\n* `Otoroshi-Authorization: Basic $base64(client_id:client_secret)` header, in that case, the `Otoroshi-Authorization` header will **not** be sent to the target. `Basic ` is optional.\n* `Authorization: Basic $base64(client_id:client_secret)` header, in that case, the `Authorization` header **will** be sent to the target\n* `Otoroshi-Token: Bearer $jwt_token` where the JWT token has been signed with the `API key` client secret, in that case, the `Otoroshi-Token` header will **not** be sent to the target. `Bearer ` is optional.\n* `Authorization: Bearer $jwt_token` where the JWT token has been signed with the `API key` client secret, in that case, the `Authorization` header **will** be sent to the target\n* `Cookie: access_token=$jwt_token;` where the JWT token has been signed with the `API key` client secret, in that case, the cookie named `access_token` **will** be sent to the target\n* `Otoroshi-Client-Id` and `Otoroshi-Client-Secret` headers, in that case the `Otoroshi-Client-Id` and `Otoroshi-Client-Secret` headers will not be sent to the target.\n\n## List API keys for a service descriptor\n\nGo to a service descriptor using `All services` quick link in the sidebar or the search box.\n\n@@@ div { .centered-img }\n![](\"../img/sidebar-all-services.png\")
\n@@@\n\nSelect a `service descriptor`.\n\n@@@ div { .centered-img }\n![](\"../img/all-services.png\")
\n@@@\n\nClick on `API keys` in the sidebar\n\n@@@ div { .centered-img }\n![](\"../img/sidebar-apikeys.png\")
\n@@@\n\nYou should see the list of API keys for that `service descriptor`\n\n@@@ div { .centered-img }\n![](\"../img/apikeys-list.png\")
\n@@@\n\n## Create an API key for a service descriptor\n\n@@@ div { .centered-img }\n![](\"../img/add-apikey.png\")
\n@@@\n\nYou can add a name for your new API key, you can also change client's id and client's secret. You can also configure the throttling rate of the API key (calls per second), and the authorized number of call per day and per month. You may also activate or de-activate the api key from that screen.\n\nInformations about current quotas usage will be returned in response headers.\n\n* `Otoroshi-Daily-Calls-Remaining` : authorized calls remaining for this day\n* `Otoroshi-Monthly-Calls-Remaining` : authorized calls remaining for this month\n* `Otoroshi-Proxy-Latency` : latency induced by Otoroshi\n* `Otoroshi-Upstream-Latency` : latency between Otoroshi and target\n\n@@@ div { .centered-img #quotas }\n![](\"../img/create-apikey.png\")
\n@@@\n\n## Update an API key\n\nTo update an `API key`, just click on the edit button ![](\"../img/apikey-edit.png\")
\n@@@\n\nUpdate the name, secret, state and quotas (if needed) of the `API key` and click on the `Update API key` button\n\n@@@ div { .centered-img }\n![](\"../img/apikey-update.png\")
\n@@@\n\n## Delete an API key\n\nTo delete an `API key`, just click on the delete button ![](\"../img/apikey-delete.png\")
\n@@@\n\nand confirm the command\n\n@@@ div { .centered-img }\n![](\"../img/apikey-delete-confirm.png\")
\n@@@\n\n### Read only\n\nThe read only flag on an `ApiKey` this apikey can only use allowed services with `HEAD`, `OPTIONS` and `GET` http verbs.\n\n## Use a JWT token to pass an API key\n\nYou can use a JWT token to pass an API key to Otoroshi. \nYou can use `Otoroshi-Authorization: Bearer $jwt_token`, `Authorization: Bearer $jwt_token` header and `Cookie: access_token=$jwt_token;` to pass the JWT token.\nYou have to create a JWT token with a signing algorythm that can be `HS256` or `HS512`. Then you have to provide an `iss` claim with the value of your API key `clientId` and sign the JWT token with your API key `clientSecret`.\n\nFor example, with an API key like `clientId=abcdef` and `clientSecret=1234456789`, your JWT token should look like\n\n```json\n{\n \"alg\": \"HS256\",\n \"typ\": \"JWT\"\n}\n{\n \"iss\":\"abcdef\",\n \"name\": \"John Doe\",\n \"admin\": true\n}\n```\n\nin that case, when you sign the token with the secret of the API key `1234456789`, the signature will be `_eancnYCD3makSSox2v2xErjNYkRtcX558QiJGCbino`, resulting in a encoded JWT header like\n\n```\neyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.\neyJpc3MiOiJhYmNkZWYiLCJuYW1lIjoiSm9obiBEb2UiLCJhZG1pbiI6dHJ1ZX0.\n_eancnYCD3makSSox2v2xErjNYkRtcX558QiJGCbino\n```\n"
- },
- {
- "name": "4-monitor.md",
- "id": "/usage/4-monitor.md",
- "url": "/usage/4-monitor.html",
- "title": "Monitoring services",
- "content": "# Monitoring services\n\nOnce you have declared services, you can monitor them with Otoroshi.\n\n@@@ warning\nYou have to use [Elastic](https://www.elastic.co) to enable analytics features in Otoroshi\n@@@\n\nOnce you have setup @ref:[Otoroshi events push to an elastic cluster](../integrations/analytics.md) (through webhooks, kafka, or elastic integration) you can setup Otoroshi events read from an elastic cluster. Go to `settings (cog icon) / Danger Zone` and expand the `Analytics: Elastic dashboard datasource (read)` section.\n\n@@@ div { .centered-img }\n![](\"../img/service-healthcheck.png\")
\n@@@\n\n## Service live stats\n\nYou can also monitor live stats like total of served request, average response time, average overhead, etc. The live stats page can be accessed from the sidebar of the service page.\n\n@@@ div { .centered-img }\n![](\"../img/service-live-stats.png\")
\n@@@\n\n## Service analytics\n\nYou can also get some aggregated metrics. The analytics page can be accessed from the sidebar of the service page.\n\n@@@ div { .centered-img }\n![](\"../img/service-analytics.png\")
\n@@@\n"
- },
- {
- "name": "5-sessions.md",
- "id": "/usage/5-sessions.md",
- "url": "/usage/5-sessions.html",
- "title": "Managing sessions",
- "content": "# Managing sessions\n\nWith Otoroshi you can manage sessions of connected users and you can discard sessions whenever you want. Session last 24h by default and you can customize them with `app.backoffice.session.exp` and `app.privateapps.session.exp` @ref:[config keys](../firstrun/configfile.md)\n\n## Admin. sessions\n\nTo see last current admin session on Otoroshi from the UI, go to `settings (cog icon) / Admins sessions`. Here you can discard individual sessions or all sessions at once using `Discard session` and `Discard all sessions` buttons.\n\n@@@ div { .centered-img }\n![](\"../img/admin-sessions.png\")
\n@@@\n\n## Private apps. session\n\nTo see last current admin session on Otoroshi from the UI, go to `settings (cog icon) / Priv. apps sessions`. Here you can discard individual sessions or all sessions at once using `Discard session` and `Discard all sessions` buttons.\n\n@@@ div { .centered-img }\n![](\"../img/private-sessions.png\")
\n@@@\n"
- },
- {
- "name": "6-audit.md",
- "id": "/usage/6-audit.md",
- "url": "/usage/6-audit.html",
- "title": "Auditing Otoroshi",
- "content": "# Auditing Otoroshi\n\nWith Otoroshi, any admin action and any sucpicious/alert action is recorded. These records are stored in Otoroshi's datastore (only the last n records, defined by the `app.events.maxSize` @ref:[config key](../firstrun/configfile.md)). All the records can be send through the analytics mechanism (WebHook, Kafka, Elastic) for external and/or further usage. We recommand sending away those records for security reasons.\n\n@@@ warning\nYou have to use [Elastic](https://www.elastic.co) to enable analytics features in Otoroshi. See @ref:[Elastic setup section](../integrations/analytics.md)\n@@@\n\n## Audit trail\n\nTo see last `app.events.maxSize` admin actions on Otoroshi from the UI, go to `settings (cog icon) / Audit log`.\n\n@@@ div { .centered-img }\n![](\"../img/audit-log.png\")
\n@@@\n\n## Alerts\n\nTo see last `app.events.maxSize` alerts on Otoroshi from the UI, go to `settings (cog icon) / Alerts log`.\n\n@@@ div { .centered-img }\n![](\"../img/alerts-log.png\")
\n@@@\n\n## List of possible alerts\n\n```\nMaxConcurrentRequestReachedAlert\nCircuitBreakerOpenedAlert\nCircuitBreakerClosedAlert\nSessionDiscardedAlert\nSessionsDiscardedAlert\nPanicModeAlert\nOtoroshiExportAlert\nU2FAdminDeletedAlert\nBlackListedBackOfficeUserAlert\nAdminLoggedInAlert\nAdminFirstLogin\nAdminLoggedOutAlert\nDbResetAlert\nDangerZoneAccessAlert\nGlobalConfigModification\nRevokedApiKeyUsageAlert\nServiceGroupCreatedAlert\nServiceGroupUpdatedAlert\nServiceGroupDeletedAlert\nServiceCreatedAlert\nServiceUpdatedAlert\nServiceDeletedAlert\nApiKeyCreatedAlert\nApiKeyUpdatedAlert\nApiKeyDeletedAlert\n```\n"
- },
- {
- "name": "7-metrics.md",
- "id": "/usage/7-metrics.md",
- "url": "/usage/7-metrics.html",
- "title": "Otoroshi global metrics",
- "content": "# Otoroshi global metrics\n\nOtoroshi provide some global metrics about services usage. Go to `settings (cog icon) / Global Ananlytics`\n\n@@@ warning\nYou have to use [Elastic](https://www.elastic.co) to enable analytics features in Otoroshi. See @ref:[Elastic setup section](../integrations/analytics.md)\n@@@\n\n@@@ div { .centered-img }\n![](\"../img/global-analytics.png\")
\n@@@\n"
- },
- {
- "name": "8-importsexports.md",
- "id": "/usage/8-importsexports.md",
- "url": "/usage/8-importsexports.html",
- "title": "Import and export",
- "content": "# Import and export\n\nWith Otoroshi you can easily save the current state of the proxy and restore it later. Go to `settings (cog icon) / Danger Zone` and scroll to the bottom of the page\n\n## Full export\n\nClick on the `Full export` button.\n\n@@@ div { .centered-img }\n![](\"../img/full-export-1.png\")
\n@@@\n\nYour browser will start to download a JSON file containing the internal state of your Otoroshi cluster.\n\n@@@ div { .centered-img }\n![](\"../img/full-export-2.png\")
\n@@@\n\n## Full import\n\nIf you want to restore an export, go to `settings (cog icon) / Danger Zone` and scroll to the bottom of the page. Click on the `Recover from full export file` button\n\n@@@ div { .centered-img }\n![](\"../img/full-import-1.png\")
\n@@@\n\nChoose export file on your system.\n\n@@@ div { .centered-img }\n![](\"../img/full-import-1-bis.png\")
\n@@@\n\nClick on the `Flush datastore and import ...` button, confirm and you will be logged out.\n\n@@@ div { .centered-img }\n![](\"../img/full-import-2.png\")
\n@@@\n"
- },
- {
- "name": "9-auth.md",
- "id": "/usage/9-auth.md",
- "url": "/usage/9-auth.html",
- "title": "Authentication",
- "content": "# Authentication\n\nYou can create auth. configuration in Otoroshi. Just go to `settings (cog icon) / Global auth. configs`.\n\n## OAuth 2\n\nCreate a new `Generic oauth2 provider` config and customize the following informations:\n\n```json\n{\n \"clientId\": \"xxxx\",\n \"clientSecret\": \"xxxx\",\n \"authorizeUrl\": \"http://yourOAuthServer/oauth/authorize\",\n \"tokenUrl\": \"http://yourOAuthServer/oauth/token\",\n \"userInfoUrl\": \"http://yourOAuthServer/userinfo\",\n \"loginUrl\": \"http://yourOAuthServer/login\",\n \"logoutUrl\": \"http://yourOAuthServer/logout?redirectQueryParamName=${redirect}\",\n \"accessTokenField\": \"access_token\",\n \"nameField\": \"name\",\n \"emailField\": \"email\",\n \"callbackUrl\": \"http://privateapps.foo.bar/privateapps/generic/callback\"\n}\n```\n\nIf used for BackOffice authentication, the callback url should be `http://otoroshi.foo.bar/backoffice/auth0/callback`.\n\nFor `logoutUrl`, `redirectQueryParamName` is a parameter with a name specific to your OAuth2 provider (for example, in Auth0, this parameter is called `returnTo`, in Kecloak it is called `redirect_uri`).\n\nif you are using a [KeyCloak](https://www.keycloak.org/) server, you can configure it this way, assuming you are using the master realm and you created a new client with a client secret, callback urls set to `http://privateapps.foo.bar/*`.\n\n```json\n{\n \"clientId\": \"clientId\",\n \"clientSecret\": \"clientSecret\",\n \"authorizeUrl\": \"http://keycloakHost/auth/realms/master/protocol/openid-connect/auth\",\n \"tokenUrl\": \"http://keycloakHost/auth/realms/master/protocol/openid-connect/token\",\n \"userInfoUrl\": \"http://keycloakHost/auth/realms/master/protocol/openid-connect/userinfo\",\n \"loginUrl\": \"http://keycloakHost/auth/realms/master/protocol/openid-connect/auth\",\n \"logoutUrl\": \"http://keycloakHost/auth/realms/master/protocol/openid-connect/logout?redirect_uri=${redirect}\",\n \"accessTokenField\": \"access_token\",\n \"nameField\": \"name\",\n \"emailField\": \"email\",\n \"callbackUrl\": \"http://privateapps.foo.bar/privateapps/generic/callback\"\n}\n```\n\n## Ldap\n\nCreate a new `Ldap auth. provider` config and customize the following informations:\n\n```json\n{\n \"serverUrl\": \"ldap://ldap.forumsys.com:389\",\n \"searchBase\": \"dc=example,dc=com\",\n \"groupFilter\": \"ou=chemists\",\n \"searchFilter\": \"(mail=${username})\",\n \"adminUsername\": \"cn=read-only-admin,dc=example,dc=com\",\n \"adminPassword\": \"password\",\n \"nameField\": \"cn\",\n \"emailField\": \"mail\"\n}\n```\n\n## In Memory\n\nCreate a new `In memory auth. provider` config and then you will be able to create new users. To set the password, just click on the `Set password` button. It will generate a BCrypt hash of the password you typed.\n\n## Auth0\n\nCreate a new OAuth 2 config and add the following informations:\n\n```json\n{\n \"clientId\": \"yourAuth0ClientId\",\n \"clientSecret\": \"yourAuth0ClientSecret\",\n \"authorizeUrl\": \"https://yourAuth0Domain/authorize\",\n \"tokenUrl\": \"https://yourAuth0Domain/oauth/token\",\n \"userInfoUrl\": \"https://yourAuth0Domain/userinfo\",\n \"loginUrl\": \"https://yourAuth0Domain/authorize\",\n \"logoutUrl\": \"https://yourAuth0Domain/v2/logout?returnTo=${redirect}\",\n \"accessTokenField\": \"access_token\",\n \"nameField\": \"name\",\n \"emailField\": \"email\",\n \"otoroshiDataField\": \"app_metadata | otoroshi_data\",\n \"callbackUrl\": \"http://privateapps.foo.bar/privateapps/generic/callback\"\n}\n```\n\nIf you enable Otoroshi exchange protocol, the JWT xill have the following fields (all optional)\n\n* `email`\n* `name`\n* `picture`\n* `user_id`\n* `given_name`\n* `family_name`\n* `gender`\n* `locale`\n* `nickname`\n\nIn Auth0, the metadata is a flat object placed in the `profile / http://yourdomain/app_metadata / otoroshi_data`. You might need to write an Auth0 rule to copy app metadata under `http://yourdomain/app_metadata`, the `http://yourdomain/app_metadata` value is a config property `app.appMeta`. The rule could be something like the following\n\n```js\nfunction (user, context, callback) {\n var namespace = 'http://yourdomain/';\n context.idToken[namespace + 'user_id'] = user.user_id;\n context.idToken[namespace + 'user_metadata'] = user.user_metadata;\n context.idToken[namespace + 'app_metadata'] = user.app_metadata;\n callback(null, user, context);\n}\n```"
- },
- {
- "name": "index.md",
- "id": "/usage/index.md",
- "url": "/usage/index.html",
- "title": "Using Otoroshi",
- "content": "# Using Otoroshi\n\nNow we will see how to use Otoroshi for basic tasks that will be useful for your day to day work with Otoroshi.\n\n@@@ index\n\n* [create group](./1-groups.md)\n* [create service](./2-services.md)\n* [create API Keys](./3-apikeys.md)\n* [monitor service](./4-monitor.md)\n* [sessions management](./5-sessions.md)\n* [Audit trail and alerts](./6-audit.md)\n* [Global metrics](./7-metrics.md)\n* [Exports and imports](./8-importsexports.md)\n* [Authentication](./9-auth.md)\n\n@@@\n"
- },
- {
- "name": "videos.md",
- "id": "/videos.md",
- "url": "/videos.html",
- "title": "Video tutorials",
- "content": "# Video tutorials\n\n## Create a service restricted by an api key\n\nhere you will learn how to create a simple service that will proxy an existing api and how to secure it using an api key.\n\n\n\n## Use CLI to create load balanced services with retry\n\nhere you will learn how to create a service using the Otoroshi CLI and how to use load balancing with retry features of Otoroshi.\n\n"
- }
-]
\ No newline at end of file
diff --git a/docs/manual/content.json b/docs/manual/content.json
deleted file mode 100644
index 6a757843e9..0000000000
--- a/docs/manual/content.json
+++ /dev/null
@@ -1 +0,0 @@
-[{"name":"about.md","id":"/about.md","url":"/about.html","title":"About Otoroshi","content":"# About Otoroshi\n\nAt the beginning of 2017, we had the need to create a new environment to be able to create new \"digital\" products very quickly in an agile fashion at 
