Merge pull request #1 from nsjames/nsjames-updates

minor updates, restructuring, and image copy

native/30_history/01_intro-to-hyperion-full-history.md → native/07_node-operation/30_history/01_intro-to-hyperion-full-history.md

@@ -1,50 +1,62 @@
 ---
-title: Introduction to Hyperion Full History
+title: Introduction to Hyperion
 contributors:
-  - { name: Ross Dold, github: https://github.com/eosphere }
+  - { name: Ross Dold (EOSphere), github: eosphere }
 ---
 
-_“Hyperion is a full history solution for indexing, storing and retrieving Antelope blockchain’s historical data.”_
+Hyperion is a full history solution for indexing, storing and retrieving Antelope blockchain’s historical data.
+It was built by EOS RIO to be an enterprise grade, performant and highly scalable Antelope History Solution. Their  [documentation](https://hyperion.docs.eosrio.io/)  is excellent and certainly a worthwhile starting point, this Technical How To series will cover some of their same content and add operational nuances from a practical stand point and EOSphere's experience.
 
-Hyperion was built by EOS RIO to be an enterprise grade, performant and highly scalable Antelope History Solution. Their  [documentation](https://hyperion.docs.eosrio.io/)  is excellent and certainly a worthwhile starting point, this Technical How To series will cover some of their same content and add operational nuances from a practical stand point and our experience.
+[Learn more about EOS RIO Hyperion](https://eosrio.io/hyperion/)
 
-## Hyperion Software Components
+![image](/images/hyperion.png)
+
+## Components
 
 The Hyperion Full History service is a collection of purpose built EOS RIO software and industry standard applications. The eight primary building blocks are the following:
 
-**EOS RIO Hyperion Indexer and API**\
-The  **Indexer**  processes data sourced from an Antelope Leap software State-History (SHIP) node and enables it to be indexed in Elasticsearch. The Hyperion Indexer also makes use of the Antelope Binary to JSON conversion functionality using ABI’s called  [abieos](https://github.com/EOSIO/abieos). Deserialisation performance is greatly improved by using abieos C++ code through EOS RIO’s own NPM package  [**node-abieos**](https://github.com/eosrio/node-abieos)  that provides a Node.js native binding.
+#### EOS RIO Hyperion Indexer and API
+
+The  **Indexer**  processes data sourced from an Antelope Leap software State-History (SHIP) node and enables it to be indexed in Elasticsearch. The Hyperion Indexer also makes use of the Antelope Binary to JSON conversion functionality using ABI’s called  [abieos](https://github.com/AntelopeIO/abieos). Deserialisation performance is greatly improved by using abieos C++ code through EOS RIO’s own NPM package  [**node-abieos**](https://github.com/eosrio/node-abieos)  that provides a Node.js native binding.
 
 The  **API**  is the front end for client queries, it responds to V2 or legacy V1 requests and finds data for these responses by directly querying the Elasticsearch cluster.
 
-**Antelope Leap Software State-History (SHIP) Node**\
+#### Antelope Leap Software State-History (SHIP) Node
+
 The State-History plugin is used by nodeos to capture historical data about the blockchain state and store this data in an externally readable flat file format. This readable file is accessed by the Hyperion Indexer.
 
-**RabbitMQ**\
+#### RabbitMQ
+
 [RabbitMQ](https://www.rabbitmq.com/)  is an open source message broker that is used by Hyperion to queue messages and transport data during the multiple stages of indexing to Elasticsearch.
 
-**Redis**\
+#### Redis
+
 [Redis](https://redis.io/)  is an in-memory data structure store and is used by Hyperion as a predictive temporary database cache for HTTP API client queries and as a Indexer transaction cache.
 
-**Node.js**\
+#### Node.js
+
 The Hyperion indexer and API are  [Node.js](https://nodejs.org/en/)  applications and of course then use Node.js as an open-sourced back-end JavaScript runtime environment.
 
-**PM2**\
+#### PM2
+
 [PM2](https://pm2.keymetrics.io/)  is a process manager for Node.js and used to launch and run the Hyperion Indexer and API.
 
-**Elasticsearch Cluster**\
+#### Elasticsearch Cluster
+
 [Elasticsearch](https://www.elastic.co/)  is a search engine based on the Lucene library, it is used by Hyperion to store and retrieve all indexed data in highly performant schema-free JSON document format.
 
-**Kibana**\
+#### Kibana
+
 [Kibana](https://www.elastic.co/kibana/)  is a component of the Elastic Stack, a dashboard that enables visualising data and simplified operation and insight of an Elasticsearch cluster. All Hyperion Indexed data resides in the Elasticsearch database, Kibana gives a direct view of this data and the health of the Elasticsearch cluster.
 
 ## Hyperion Topology
 
 The Topology of your Hyperion deployment depends on your history services requirement and the network you intend to index. Whether it’s Public/Private, Mainnet/Testnet or Full/Partial History.
 
-This article will discuss EOS Mainnet with Full History, in relation to what currently works in the EOSphere Public Service Offerings. Testnets and Private networks generally have far lower topology and resource requirements.
+This guide will discuss EOS Mainnet with Full History. Testnets and Private networks generally have far lower topology and resource requirements.
+
+**EOS Mainnet**
 
-**EOS Mainnet**\
 EOSphere originally started with a single server running all Hyperion Software Components except for the EOS State-History Node. However a challenge was discovered in relation to Elasticsearch JVM heap size when the EOS network utilisation grew and our API became well used.
 
 JVM Heap size is the amount of memory allocated to the Java Virtual Machine of an Elasticsearch node, the more heap available the more cache memory available for indexing and search operations. If it’s too low Hyperion Indexing will be slow and search queries will be very latent. If the JVM heap size is more than 32GB (usually lower than this) on an Elasticsearch node, the threshold for compressed ordinary object pointers (OOP) will be exceeded and JVM will stop using compression. This will be exceptionally inefficient in regards to memory management and the node will consume vastly more memory.

native/30_history/02_build-hyperion-software-components.md → native/07_node-operation/30_history/02_build-hyperion-software-components.md

@@ -1,28 +1,16 @@
 ---
-title: Build Hyperion Software Components
+title: Build Hyperion Components
 contributors:
-  - { name: Ross Dold, github: https://github.com/eosphere }
+  - { name: Ross Dold (EOSphere), github: eosphere }
 ---
 
-Following on from our Introduction to Hyperion Full History article, this next guide in this series will walk through the process of building each of the Hyperion Software Components.
-
-EOS RIO have an excellent  [Hyperion Documentation Repository](https://hyperion.docs.eosrio.io/)  as well as an  **installation script**  for all components, however in this article a  **manual build**  process will be covered.
-
-Once again this Technical How To series will cover some of the EOS RIO same content and will add operational nuances from a practical stand point and our experience.
-
-[Learn more about EOS RIO Hyperion](https://eosrio.io/hyperion/)
-
-![image](https://github.com/eosphere/Antelope-Technical-How-To/assets/12730423/34e760f0-a199-4061-8fdf-ab47eda2c8d7)
-
-# Build Hyperion Software Components
-
 The Hyperion Full History service is a collection of  **eight**  purpose built EOS RIO software and industry standard applications.
 
-This walk through will install all components excluding the SHIP node on a single Ubuntu 22.04 server, please reference  [Introduction to Hyperion Full History](https://github.com/eosphere/Antelope-Technical-How-To/blob/main/articles/hyperion/intro-to-hyperion-full-history.md) for infrastructure suggestions.
+This walk through will install all components excluding the SHIP node on a single Ubuntu 22.04 server, please reference  [Introduction to Hyperion Full History](./01_intro-to-hyperion-full-history.md) for infrastructure suggestions.
 
 The process for building each of these primary building blocks is covered below:
 
-## **State-History (SHIP) Node**
+## State-History (SHIP) Node
 
 The Hyperion deployment requires access to a fully sync’d State-History Node, the current SHIP recommend version is Leap `v5.0.2`.
 
@@ -32,7 +20,7 @@ To install the latest RabbitMQ currently  `3.13.0`  be sure to check their lates
 
 The summary process is below:
 
-```
+```bash
 > sudo apt update
 
 > sudo apt-get install curl gnupg apt-transport-https -y
@@ -79,7 +67,7 @@ EOF
 
 Our current Hyperion deployments are running on the latest Redis stable version  `v7.2.4`  which is built as below:
 
-```
+```bash
 > sudo apt install lsb-release curl gpg
 
 #Redis Signing Key#  
@@ -99,7 +87,7 @@ Our current Hyperion deployments are running on the latest Redis stable version
 
 Hyperion requires Node.js v18 , our current Hyperion deployments are running the current LTS  `v18.19.1`  which is built below:
 
-```
+```bash
 #Download and import the Nodesource GPG key#
 > sudo apt update
 
@@ -127,7 +115,7 @@ Hyperion requires Node.js v18 , our current Hyperion deployments are running the
 
 The latest public version is  `5.3.1`  and is built as below:
 
-```
+```bash
 > sudo apt update
 
 #Install PM2#  
@@ -141,7 +129,7 @@ The latest public version is  `5.3.1`  and is built as below:
 
 Currently most of our Hyperion deployments are using Elasticsearch  `8.5-12.x`  with great results, however the current recommended Elasticsearch version is  `8.12.2`  which I expect will work just as well or better. Build the latest Elasticsearch  `8.x`  as below:
 
-```
+```bash
 > sudo apt update
 
 > sudo apt install apt-transport-https
@@ -164,7 +152,7 @@ Currently most of our Hyperion deployments are using Elasticsearch  `8.5-12.x`
 
 The utilised Kibana version should be paired with the installed Elasticsearch version, the process below will install the current version:
 
-```
+```bash
 > sudo apt update
 
 > sudo apt-get install apt-transport-https
@@ -187,7 +175,7 @@ Currently (March 2024) the most robust and production ready version of Hyperion
 
 Build Hyperion from  `main`  as below:
 
-```
+```bash
 > git clone https://github.com/eosrio/hyperion-history-api.git
 
 > cd hyperion-history-api
@@ -201,4 +189,4 @@ Build Hyperion from  `main`  as below:
 
 After all Hyperion Software Components are built and provisioned you can now proceed to configuration.
 
-The next  **Hyperion Full History**  guide will walk through the technical configuration of each component.
+The next guide will walk through the technical configuration of each component.