Docv1.0.1 (#1001)

Updates to documentation

README.md

@@ -21,8 +21,8 @@ We provide multiple options to get going with SnappyData. The easiest option is,
 You can simply get started by adding SnappyData as a package dependency. You can find more information on options for running SnappyData [here](docs/quickstart.md).
 
 ## Downloading and Installing SnappyData
-You can download and install the latest version of SnappyData from the [SnappyData Download Page](https://www.snappydata.io/download). 
-Refer to the [documentation](http://snappydatainc.github.io/snappydata/) for installation steps.
+You can download and install the latest version of SnappyData from the [SnappyData Download Page](https://www.snappydata.io/download).
+Refer to the [documentation](docs/install.md) for installation steps.
 
 If you would like to build SnappyData from source, refer to the [documentation on building from source](docs/install/building_from_source.md).
 
@@ -39,7 +39,9 @@ We monitor channels listed below for comments/questions.
 [Stackoverflow](http://stackoverflow.com/questions/tagged/snappydata) ![Stackoverflow](http://i.imgur.com/LPIdp12.png)    [Slack](http://snappydata-slackin.herokuapp.com/)![Slack](http://i.imgur.com/h3sc6GM.png)        [Gitter](https://gitter.im/SnappyDataInc/snappydata) ![Gitter](http://i.imgur.com/jNAJeOn.jpg)          [Mailing List](https://groups.google.com/forum/#!forum/snappydata-user) ![Mailing List](http://i.imgur.com/YomdH4s.png)             [Reddit](https://www.reddit.com/r/snappydata) ![Reddit](http://i.imgur.com/AB3cVtj.png)          [JIRA](https://jira.snappydata.io/projects/SNAP/issues) ![JIRA](http://i.imgur.com/E92zntA.png)
 
 ## Link with SnappyData Distribution
+
 **Using Maven Dependency**
+
 SnappyData artifacts are hosted in Maven Central. You can add a Maven dependency with the following coordinates:
 
 ```
@@ -51,8 +53,10 @@ groupId: io.snappydata
 artifactId: snappydata-cluster_2.11
 version: 1.0.1
 ```
-**Using sbt**
-If you are using sbt, add this line to your **build.sbt** for core SnappyData artifacts:
+
+**Using SBT Dependency**
+
+If you are using SBT, add this line to your **build.sbt** for core SnappyData artifacts:
 
 `libraryDependencies += "io.snappydata" % "snappydata-core_2.11" % "1.0.1"`
 
@@ -62,6 +66,17 @@ For additions related to SnappyData cluster, use:
 
 You can find more specific SnappyData artifacts [here](http://mvnrepository.com/artifact/io.snappydata)
 
+**Note:** If your project fails when resolving the above dependency (that is, it fails to download javax.ws.rs#javax.ws.rs-api;2.1), it may be due an issue with its pom file. </br> As a workaround, you can add the below code to your **build.sbt**:
+
+```
+val workaround = {
+  sys.props += "packaging.type" -> "jar"
+  ()
+}
+```
+
+For more details, refer [https://github.com/sbt/sbt/issues/3618](https://github.com/sbt/sbt/issues/3618).
+
 ## Ad Analytics using SnappyData
 Here is a stream + Transactions + Analytics use case example to illustrate the SQL as well as the Spark programming approaches in SnappyData - [Ad Analytics code example](https://github.com/SnappyDataInc/snappy-poc). Here is a [screencast](https://www.youtube.com/watch?v=bXofwFtmHjE) that showcases many useful features of SnappyData. The example also goes through a benchmark comparing SnappyData to a Hybrid in-memory database and Cassandra.
 

docs/best_practices/ha_considerations.md

@@ -1,20 +1,19 @@
 <a id="cores"></a>
-# HA Considerations
-
+# High Availability (HA) Considerations
 <a id="ha-consideration"></a>
 
-High availability options are available for all the SnappyData components. 
+High availability options are available for all the SnappyData components.
 
-**Lead HA** </br> 
+**Lead** </br>
 SnappyData supports secondary lead nodes. If the primary lead becomes unavailable, one of  the secondary lead nodes takes over immediately. 
 Setting up the secondary lead node is highly recommended because the system cannot function if the lead node is unavailable. Currently, the queries and jobs that are executing when the primary lead becomes unavailable, are not re-tried and have to be resubmitted.
 
-**Locator**</br>  
+**Locator**</br>
 SnappyData supports multiple locators in the cluster for high availability. 
 It is recommended to set up multiple locators (ideally two) as, if a locator becomes unavailable, the cluster continues to be available. New members can however not join the cluster.</br>
 With multiple locators, clients notice nothing and the fail over recovery is completely transparent.
 
-**DataServer**</br> 
+**DataServer**</br>
 SnappyData supports redundant copies of data for fault tolerance. A table can be configured to store redundant copies of the data.  So, if a server is unavailable, and if there is a redundant copy available on some other server, the tasks are automatically retried on those servers. This is totally transparent to the user. 
 However, the redundant copies double the memory requirements. If there are no redundant copies and a server with some data goes down, the execution of the queries fail and PartitionOfflineException is reported. The execution does not begin until that server is available again. 
 

docs/configuration.md

@@ -13,16 +13,18 @@ The following topics are covered in this section:
 	- [Configuring Locators](configuring_cluster/configuring_cluster.md#locator)
 
 	- [Configuring Leads](configuring_cluster/configuring_cluster.md#lead)
+
+	- [Configuring Data Servers](configuring_cluster/configuring_cluster.md#dataserver)
 
 	- [Configuring SnappyData Smart Connector](configuring_cluster/configuring_cluster.md#configure-smart-connector)
 
-* [Environment Settings](configuring_cluster/configuring_cluster.md#environment)
+	- [Environment Settings](configuring_cluster/configuring_cluster.md#environment)
 
- 	- [Hadoop Provided Settings](configuring_cluster/configuring_cluster.md#hadoop-setting)
+	 	- [Hadoop Provided Settings](configuring_cluster/configuring_cluster.md#hadoop-setting)
 
- 	- [SnappyData Command Line Utility](configuring_cluster/configuring_cluster.md#command-line)
+	 	- [SnappyData Command Line Utility](configuring_cluster/configuring_cluster.md#command-line)
 
-* [Logging](configuring_cluster/configuring_cluster.md#logging)
+	- [Logging](configuring_cluster/configuring_cluster.md#logging)
 
 * [SSL Setup for Client-Server](configuring_cluster/ssl_setup.md)
 

docs/configuring_cluster/available_scripts.md

@@ -0,0 +1,36 @@
+| Name | Description |
+|--------|--------|
+|snappy-lead.sh| |
+|snappy-leads.sh| | 
+|snappy-locator.sh| |
+|snappy-locators.sh| |
+|snappy-server.sh| |
+|snappy-start-all.sh| |
+|snappy-status-all.sh| |
+|snappy-stop-all.sh| |
+|collect-debug-artifacts.sh| Spark|
+|start-all.sh| Spark|
+|start-history-server.sh| Spark|
+|start-master.sh|  Spark Only|
+|start-slave.sh| Spark|
+|start-slaves.sh| Spark|
+|start-thriftserver.sh|Spark |
+|stop-all.sh| Spark|
+|stop-master.sh| Spark|
+|stop-slave.sh| |
+|stop-slaves.sh| |
+
+
+
+
+
+| Name | Description |
+|--------|--------|
+|snappy|        |
+|snappy-job.sh|        |
+|snappy-sql|        |
+|pyspark |Spark|
+|Spark-shell|Spark        |
+|Spark-sql -- Spark|        |
+|Spark-submit |	Spark        |
+

docs/configuring_cluster/configuring_cluster.md

@@ -6,8 +6,9 @@ To do so, you can copy the existing template files **servers.template**, **locat
 These files should contain the hostnames of the nodes (one per line) where you intend to start the member. You can modify the properties to configure individual members.
 
 !!! Tip:
-	For system properties (set in the conf/leads, conf/servers and conf/locators file), -D and -XX: can be used. All other JVM properties need the `-J` prefix.
+	- For system properties (set in the conf/lead, conf/servers and conf/locators file), -D and -XX: can be used. All other JVM properties need the `-J` prefix.
 
+    - Instead of starting the SnappyData cluster, you can [start](../howto/start_snappy_cluster.md) and [stop](../howto/stop_snappy_cluster.md) individual components on a system locally.
 
 <a id="locator"></a>
 ## Configuring Locators
@@ -34,7 +35,7 @@ Refer to the [SnappyData properties](property_description.md) for the complete l
 |-----|-----|
 |-bind-address|IP address on which the locator is bound. The default behavior is to bind to all local addresses.|
 |-classpath|Location of user classes required by the SnappyData Server.</br>This path is appended to the current classpath.|
-|-dir|The working directory of the server that contains the SnappyData Server status file and the default location for the log file, persistent files, data dictionary, and so forth (defaults to the current directory). </br>If you want your data to be stored on separate disks (and not the default location), [create separate diskstores](../reference/sql_reference/create-diskstore.md) and specify the directories where you want the data to be stored.|
+|-dir|The working directory of the server that contains the SnappyData Server status file and the default location for the log file, persistent files, data dictionary, and so forth (defaults to the current directory).| 
 |-heap-size|<a id="heap-size"></a> Sets the maximum heap size for the Java VM, using SnappyData default resource manager settings. </br>For example, -heap-size=1024m. </br>If you use the `-heap-size` option, by default SnappyData sets the critical-heap-percentage to 90% of the heap size, and the `eviction-heap-percentage` to 81% of the `critical-heap-percentage`. </br>SnappyData also sets resource management properties for eviction and garbage collection if they are supported by the JVM. |
 |-J|JVM option passed to the spawned SnappyData server JVM. </br>For example, use -J-Xmx1024m to set the JVM heap to 1GB.|
 |-locators|List of locators as comma-separated host:port values used to communicate with running locators in the system and thus discover other peers of the distributed system. </br>The list must include all locators in use and must be configured consistently for every member of the distributed system.|
@@ -59,6 +60,9 @@ Lead Nodes primarily runs the SnappyData managed Spark driver. There is one prim
 
 Create the configuration file (**leads**) for leads in the **<_SnappyData_home_>/conf** directory.
 
+!!!Note:
+	In the **conf/spark-env.sh** file, set the `SPARK_PUBLIC_DNS` property to the public DNS name of the lead node. This enables the Member Logs to be displayed correctly to users accessing SnappyData Pulse from outside the network.
+
 ### List of Lead Properties
 Refer to the [SnappyData properties](property_description.md) for the complete list of SnappyData properties.
 
@@ -158,20 +162,6 @@ To do this you need to put an entry in $SNAPPY-HOME/conf/spark-env.sh as below:
 ```no-highlight
 export SPARK_DIST_CLASSPATH=$($OTHER_HADOOP_HOME/bin/hadoop classpath)
 ```
-<a id="command-line"></a>
-### SnappyData Command Line Utility
-
-Instead of starting SnappyData cluster using the `snappy-start-all.sh` script, individual components can be configured, started and stopped on a system locally using these commands.
-
-```no-highlight
-$ bin/snappy locator start  -dir=/node-a/locator1 
-$ bin/snappy server start  -dir=/node-b/server1  -locators=localhost[10334] -heap-size=16g 
-$ bin/snappy leader start  -dir=/node-c/lead1  -locators=localhost[10334] -spark.executor.cores=32
-
-$ bin/snappy locator stop -dir=/node-a/locator1
-$ bin/snappy server stop -dir=/node-b/server1
-$ bin/snappy leader stop -dir=/node-c/lead1
-```
 
 <a id="logging"></a>
 ## Logging