Skip to content

4. Advanced Tips for Deployment

Jump to bottom
Ivan edited this page Nov 27, 2018 · 3 revisions

Use Docker volume for the data

For storing the data, it is advisable to use Docker volumes. For that, first create a volume using

docker volume create adata

When running the container, mount the volume to the destination /adampro/data where all the data is stored.

docker run ... --mount source=adata,destination=/adampro/data ...

With that you may create your entities and fill these with data. When deleting the container, the volume is kept and can be attached to a new container.

Expose ports

For accessing the underlying storage engines directly, expose the necessary ports. Note that not all ports might be available (depending on the container you are using).

  • 5890 for the ADAMpro grpc endpoint

  • 9099 for the ADAMpro Web UI

  • 4040 for the Apache Spark UI

  • 50010 50020 50070 50075 50090 for Apache Hadoop (HDFS)

  • 1988 for Apache Hadoop (MapReduce)

  • 8030 8031 8032 8033 8040 8042 8088 for Apache Hadoop (Yarn)

  • 5432 for PostgreSQL

  • 8983 for Apache Solr

  • 9042 for Apache Cassandra

  • 10088 for the Spark Notebook

  • 19999 for netdata

Adjust the available memory and number of threads

Adjust the environment variables ADAMPRO_EXECUTOR_MEMORY and ADAMPRO_DRIVER_MEMORY (for the default container and the hdfs container, set to 2g) or ADAMPRO_MEMORY (for the selfcontained container, set to 2g), respectively, denoting how much memory is allocated to ADAMpro and Apache Spark, and ADAMPRO_MASTER (set to local[4]), which sets the number of threads in the local deployment. Both parameters have very minimal default values.

Choose which storage engines to use

In the selfcontained container, you may choose which storage engines to start. You can do this by adjusting the following environmental variables:

  • ADAMPRO_START_POSTGRES (default: true)
  • ADAMPRO_START_CASSANDRA (default: true)
  • ADAMPRO_START_SOLR (default: true)
  • ADAMPRO_START_WEBUI (default: true)
  • ADAMPRO_START_NOTEBOOK (default: false)
  • ADAMPRO_START_NETDATA (default: true)

Graceful shutdown

Certain containers are configured to have a graceful shutdown. Use

docker stop --time 60 adampro

and give the container around 1 minute time to gracefully shutdown all services.

Clone this wiki locally