Skip to content

Commit

Permalink
Merge pull request #1487 from steemit/master
Browse files Browse the repository at this point in the history
Steem 0.19.2
  • Loading branch information
Michael Vandeberg authored Aug 31, 2017
2 parents 88d9098 + 76ae3a3 commit 07ccb8b
Show file tree
Hide file tree
Showing 24 changed files with 655 additions and 259 deletions.
7 changes: 7 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,13 @@ if( SKIP_BY_TX_ID )
SET( CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -DSKIP_BY_TX_ID" )
endif()

OPTION( STEEM_STATIC_BUILD "Build steemd as a static library (ON or OFF)" OFF )
if( STEEM_STATIC_BUILD AND ( ( MSVC AND NOT MINGW ) OR APPLE ) )
MESSAGE( STATUS "Statuc build is not available on Windows or OS X" )
SET( STEEM_STATIC_BUILD OFF )
endif()
MESSAGE( STATUS "STEEM_STATIC_BUILD: ${STEEM_STATIC_BUILD}" )

IF( WIN32 )
SET(BOOST_ROOT $ENV{BOOST_ROOT})
set(Boost_USE_MULTITHREADED ON)
Expand Down
7 changes: 6 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ FROM phusion/baseimage:0.9.19

#ARG STEEMD_BLOCKCHAIN=https://example.com/steemd-blockchain.tbz2

ARG STEEM_STATIC_BUILD=ON
ENV STEEM_STATIC_BUILD ${STEEM_STATIC_BUILD}

ENV LANG=en_US.UTF-8

RUN \
Expand Down Expand Up @@ -92,8 +95,9 @@ RUN \
-DCMAKE_BUILD_TYPE=Release \
-DLOW_MEMORY_NODE=ON \
-DCLEAR_VOTES=ON \
-DSKIP_BY_TX_ID=ON \
-DSKIP_BY_TX_ID=OFF \
-DBUILD_STEEM_TESTNET=OFF \
-DSTEEM_STATIC_BUILD=${STEEM_STATIC_BUILD} \
.. \
&& \
make -j$(nproc) && \
Expand All @@ -116,6 +120,7 @@ RUN \
-DCLEAR_VOTES=OFF \
-DSKIP_BY_TX_ID=ON \
-DBUILD_STEEM_TESTNET=OFF \
-DSTEEM_STATIC_BUILD=${STEEM_STATIC_BUILD} \
.. \
&& \
make -j$(nproc) && \
Expand Down
30 changes: 15 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
# Introducing Steem (beta)

Steem is an experimental Delegated Proof of Stake blockchain with an unproven consensus
algorithm.
Steem is a Delegated Proof of Stake blockchain that uses a "Proof of Brain" social consnensus algorithm for token allocation.

- Currency symbol STEEM
- 1.000 STEEM block reward at launch
- Currency symbol STEEM.
- 10% APR inflation narrowing to 1% APR over 20 years.
- 75% of inflation to "Proof of Brain" social consensus algorithm.
- 15% of inflation to stake holders.
- 10% of inflation to block producers.

# Public Announcement & Discussion

Expand All @@ -23,15 +24,19 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.

# Blockchain consensus rules
# Whitepaper

Rather than attempt to describe the rules of the blockchain, it is up to
each individual to inspect the code to understand the consensus rules.
You can read the Steem Whitepaper at [Steem.io/SteemWhitePaper.pdf](https://steem.io/SteemWhitePaper.pdf).

# Quickstart

Just want to get up and running quickly? Try deploying a prebuilt
dockerized container. Both common binary types are included.
Just want to get up and running quickly? We have pre-build docker images for your convenience. More details are in our [quickstart guide](https://github.com/steemit/steem/blob/master/doc/exchangequickstart.md).

# Building

We strongly recommend using one of our pre-built Docker images or using Docker to build Steem. Both of these processes are described in the [quickstart guide](https://github.com/steemit/steem/blob/master/doc/exchangequickstart.md).

If you would still like to build from source, we do have [build instructions](https://github.com/steemit/steem/blob/master/doc/building.md) for Linux (Ubuntu LTS) and macOS X.

## Dockerized p2p Node

Expand Down Expand Up @@ -82,19 +87,14 @@ This same file is baked into the docker images and can be overridden by
setting `STEEMD_SEED_NODES` in the container environment at `docker run`
time to a whitespace delimited list of seed nodes (with port).

# Building

See [doc/building.md](doc/building.md) for detailed build instructions, including
compile-time options, and specific commands for Linux (Ubuntu LTS) or macOS X.

# Testing

See [doc/testing.md](doc/testing.md) for test build targets and info
on how to use lcov to check code test coverage.

# System Requirements

For a full web node, you need at least 55GB of space available. Steemd uses a memory mapped file which currently holds 36GB of data and by default is set to use up to 40GB. The block log of the blockchain itself is a little over 10GB. It's highly recommended to run steemd on a fast disk such as an SSD or by placing the shared memory files in a ramdisk and using the `--shard-file-dir=/path` command line option to specify where. At least 16GB of memory is required for a full web node. Seed nodes (p2p mode) can run with as little as 4GB of memory. Any CPU with decent single core performance should be sufficient.
For a full web node, you need at least 110GB of disk space available. Steemd uses a memory mapped file which currently holds 56GB of data and by default is set to use up to 80GB. The block log of the blockchain itself is a little over 27GB. It's highly recommended to run steemd on a fast disk such as an SSD or by placing the shared memory files in a ramdisk and using the `--shared-file-dir=/path` command line option to specify where. At least 16GB of memory is required for a full web node. Seed nodes (p2p mode) can run with as little as 4GB of memory with a 16 GB state file. Any CPU with decent single core performance should be sufficient. Steemd is constantly growing. As of August 2017, these numbers were accurate, but you may find you need more disk space to run a full node. We are also constantly working on optimizing Steem's use of disk space.

On Linux use the following Virtual Memory configuration for the initial sync and subsequent replays. It is not needed for normal operation.

Expand Down
4 changes: 2 additions & 2 deletions contrib/fullnode.config.ini
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Set larger shared-file-size than default
shared-file-size = 48G
shared-file-size = 80G

# Set an API to be publicly available, may be specified multiple times
public-api = database_api login_api account_by_key_api network_broadcast_api tag_api follow_api market_history_api raw_block_api
Expand All @@ -13,7 +13,7 @@ bcd-trigger = [[0,10],[85,300]]
# Defines a range of accounts to track as a json pair ["from","to"] [from,to]
# track-account-range =

history-whitelist-ops = transfer_operation transfer_to_vesting_operation withdraw_vesting_operation interest_operation transfer_to_savings_operation transfer_from_savings_operation cancel_transfer_from_savings_operation escrow_transfer_operation escrow_approve_operation escrow_dispute_operation escrow_release_operation fill_convert_request_operation fill_order_operation claim_reward_balance_operation author_reward_operation curation_reward_operation fill_vesting_withdraw_operation fill_transfer_from_savings_operation delegate_vesting_shares_operation return_vesting_delegation_operation comment_benefactor_reward_operation
history-whitelist-ops = transfer_operation transfer_to_vesting_operation withdraw_vesting_operation interest_operation transfer_to_savings_operation transfer_from_savings_operation cancel_transfer_from_savings_operation escrow_transfer_operation escrow_approve_operation escrow_dispute_operation escrow_release_operation convert_operation fill_convert_request_operation fill_order_operation claim_reward_balance_operation author_reward_operation curation_reward_operation fill_vesting_withdraw_operation fill_transfer_from_savings_operation delegate_vesting_shares_operation return_vesting_delegation_operation comment_benefactor_reward_operation

# Ignore posting operations, only track transfers and account updates
# filter-posting-ops =
Expand Down
7 changes: 6 additions & 1 deletion contrib/paas-sv-run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@ if [[ ! $? -eq 0 ]]; then
if [[ -e /tmp/core.$SAVED_PID ]]; then
gdb --batch --quiet -ex "thread apply all bt full" -ex "quit" /usr/local/steemd-full/bin/steemd /tmp/core.$SAVED_PID >> /tmp/stacktrace
STACKTRACE=`cat /tmp/stacktrace`
echo NOTIFYALERT! steemdsync stacktrace from coredump: $STACKTRACE
echo NOTIFYALERT! steemdsync stacktrace from coredump:
for ((i=0;i<${#STACKTRACE};i+=120)); do
echo "${STACKTRACE:i:120}"
done
CORE_FILE_NAME=coredump-`date '+%Y%m%d-%H%M%S'`.$SAVED_PID
aws s3 cp /tmp/core.$SAVED_PID s3://$S3_BUCKET/$CORE_FILE_NAME
fi
RUN_SV_PID=`pgrep -f /etc/service/steemd`
kill -9 $RUN_SV_PID
Expand Down
7 changes: 6 additions & 1 deletion contrib/sync-sv-run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,12 @@ if [[ ! $? -eq 0 ]]; then
if [[ -e /tmp/core.$SAVED_PID ]]; then
gdb --batch --quiet -ex "thread apply all bt full" -ex "quit" /usr/local/steemd-full/bin/steemd /tmp/core.$SAVED_PID >> /tmp/stacktrace
STACKTRACE=`cat /tmp/stacktrace`
echo NOTIFYALERT! steemdsync stacktrace from coredump: $STACKTRACE
echo NOTIFYALERT! steemdsync stacktrace from coredump:
for ((i=0;i<${#STACKTRACE};i+=120)); do
echo "${STACKTRACE:i:120}"
done
CORE_FILE_NAME=coredump-`date '+%Y%m%d-%H%M%S'`.$SAVED_PID
aws s3 cp /tmp/core.$SAVED_PID s3://$S3_BUCKET/$CORE_FILE_NAME
fi
RUN_SV_PID=`pgrep -f /etc/service/steemd`
kill -9 $RUN_SV_PID
Expand Down
115 changes: 115 additions & 0 deletions doc/example_config.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# this will need to increase depending on which plugins/apis you use
# 16 GB should be sufficient for a consensus node
shared-file-size = 16G

# Endpoint for P2P node to listen on
# p2p-endpoint =

# Maxmimum number of incoming connections on P2P endpoint
# p2p-max-connections =

# P2P nodes to connect to on startup (may specify multiple times)
# seed-node =

# Pairs of [BLOCK_NUM,BLOCK_ID] that should be enforced as checkpoints.
# checkpoint =

# Endpoint for websocket RPC to listen on
# rpc-endpoint =

# Endpoint for TLS websocket RPC to listen on
# rpc-tls-endpoint =

# The TLS certificate file for this server
# server-pem =

# Password for this certificate
# server-pem-password =

# Block signing key to use for init witnesses, overrides genesis file
# dbg-init-key =

# API user specification, may be specified multiple times
# api-user =

# Set an API to be publicly available, may be specified multiple times
public-api = database_api login_api

# Plugin(s) to enable, may be specified multiple times
enable-plugin = account_history

# JSON list of [nblocks,nseconds] pairs, see doc/bcd-trigger.md
bcd-trigger = [[0,10],[85,300]]

# Defines a range of accounts to track as a json pair ["from","to"] [from,to]
track-account-range = ["exchange_account", "exchange_account"]

# Ignore posting operations, only track transfers and account updates
# filter-posting-ops =

# Database edits to apply on startup (may specify multiple times)
# edit-script =

# RPC endpoint of a trusted validating node (required)
# trusted-node =

# Set the maximum size of cached feed for an account
# follow-max-feed-size = 500

# Track market history by grouping orders into buckets of equal size measured in seconds specified as a JSON array of numbers
# bucket-size = [15,60,300,3600,86400]

# How far back in time to track history for each bucket size, measured in the number of buckets (default: 5760)
# history-per-size = 5760

# Defines a range of accounts to private messages to/from as a json pair ["from","to"] [from,to)
# pm-account-range =

# Enable block production, even if the chain is stale.
# enable-stale-production = false

# Percent of witnesses (0-99) that must be participating in order to produce blocks
# required-participation = false

# name of witness controlled by this node (e.g. initwitness )
# witness =

# name of miner and its private key (e.g. ["account","WIF PRIVATE KEY"] )
# miner =

# Number of threads to use for proof of work mining
# mining-threads =

# WIF PRIVATE KEY to be used by one or more witnesses or miners
# private-key =

# Account creation fee to be voted on upon successful POW - Minimum fee is 100.000 STEEM (written as 100000)
# miner-account-creation-fee =

# Maximum block size (in bytes) to be voted on upon successful POW - Max block size must be between 128 KB and 750 MB
# miner-maximum-block-size =

# SBD interest rate to be vote on upon successful POW - Default interest rate is 10% (written as 1000)
# miner-sbd-interest-rate =

# declare an appender named "stderr" that writes messages to the console
[log.console_appender.stderr]
stream=std_error

# declare an appender named "p2p" that writes messages to p2p.log
[log.file_appender.p2p]
filename=logs/p2p/p2p.log
# filename can be absolute or relative to this config file

# route any messages logged to the default logger to the "stderr" logger we
# declared above, if they are info level are higher
[logger.default]
level=info
appenders=stderr

# route messages sent to the "p2p" logger to stderr too
[logger.p2p]
level=info
appenders=stderr


95 changes: 95 additions & 0 deletions doc/exchangequickstart.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
Exchange Quickstart
-------------------

System Requirements: A minimum of 16GB of RAM and at least 50GB of fast local SSD storage. STEEM is one of the most active blockchains in the world and handles an incredibly large amount of transactions per second, as such, it requires fast storage to run efficiently.

We recommend using docker to both build and run STEEM for exchanges. Docker is the world's leading containerization platform and using it guarantees that your build and run environment is identical to what our developers use. You can still build from source and you can keep both blockchain data and wallet data outside of the docker container. The instructions below will show you how to do this in just a few easy steps.

### Install docker and git (if not already installed)

On Ubuntu 16.04+:
```
apt-get update && apt-get install git docker.io -y
```

On other distributions you can install docker with the native package manager or with the script from get.docker.com:
```
curl -fsSL get.docker.com -o get-docker.sh
sh get-docker.sh
```

### Clone the steem repo

Pull in the steem repo from the official source on github and then change into the directory that's created for it.
```
git clone https://github.com/steemit/steem
cd steem
```

### Build the image from source with docker

Docker isn't just for downloading already built images, it can be used to build from source the same way you would otherwise build. By doing this you ensure that your build environment is identical to what we use to develop the software. Use the below command to start the build:

```
docker build -t=steemit/steem .
```

Don't forget the `.` at the end of the line which indicates the build target is in the current directory.

This will build everything including running our full suite of tests during the build process. It will anywhere from thirty minutes to a couple hours depending on how fast your equipment is.

When the build completes you will see a message indicating that it is 'successfully built'.

### Using our official Docker images without building from source

If you'd like to use our already pre-built official binary images, it's as simple as downloading it from the Dockerhub registry with only one command:

```
docker pull steemit/steem
```

### Running a binary build without a Docker container

If you build with Docker but do not want to run steemd from within a docker container, you can stop here with this step and instead extract the binary from the container with the commands below. If you are going to run steemd with docker (recommended method), skip this step altogether. We're simply providing an option for everyone's use-case. Our binaries are built mostly static, only dynamically linking to linux kernel libraries. We have tested and confirmed binaries built in Docker work on Ubuntu and Fedora and will likely work on many other Linux distrubutions. Building the image yourself or pulling one of our pre-built images both work.

To extract the binary you need to start a container and then copy the file from it.

```
docker run -d --name steemd-exchange steemit/steem
docker cp steemd-exchange:/usr/local/steemd-default/bin/steemd /local/path/to/steemd
docker cp steemd-exchange:/usr/local/steemd-default/bin/cli_wallet /local/path/to/cli_wallet
docker stop steemd-exchange
```

For your convenience, we have provided a provided an [example\_config](doc/example\_config.ini) that we expect should be sufficient to run your exchange node. Be sure to rename it to simply `config.ini`.

### Create directories to store blockchain and wallet data outside of Docker

For re-usability, you can create directories to store blockchain and wallet data and easily link them inside your docker container.

```
mkdir blockchain
mkdir steemwallet
```

### Run the container

The below command will start a daemonized instance opening ports for p2p and RPC while linking the directories we created for blockchain and wallet data inside the container. Fill in `TRACK_ACCOUNT` with the name of your exchange account that you want to follow. The `-v` flags are how you map directories outside of the container to the inside, you list the path to the directories you created earlier before the `:` for each `-v` flag. The restart policy ensures that the container will automatically restart even if your system is restarted.

```
docker run -d --name steemd-exchange --env TRACK_ACCOUNT=nameofaccount -p 2001:2001 -p 8090:8090 -v /path/to/steemwallet:/var/steemwallet -v /path/to/blockchain:/var/lib/steemd/blockchain --restart always steemit/steem
```

You can see that the container is running with the `docker ps` command.

To follow along with the logs, use `docker logs -f`.

Initial syncing will take between 6 and 48 hours depending on your equipment, faster storage devices will take less time and be more efficient. Subsequent restarts will not take as long.

### Running the cli_wallet

The command below will run the cli_wallet from inside the running container while mapping the `wallet.json` to the directory you created for it on the host.

```
docker exec -it steemd-exchange /usr/local/steemd-default/bin/cli_wallet -w /var/steemwallet/wallet.json
```
8 changes: 8 additions & 0 deletions libraries/app/application.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
#include <fc/rpc/api_connection.hpp>
#include <fc/rpc/websocket_api.hpp>
#include <fc/network/resolve.hpp>
#include <fc/stacktrace.hpp>
#include <fc/string.hpp>

#include <boost/algorithm/string.hpp>
Expand Down Expand Up @@ -252,6 +253,12 @@ namespace detail {

void startup()
{ try {
if( _options->at("backtrace").as<string>() == "yes" )
{
fc::print_stacktrace_on_segfault();
ilog( "Backtrace on segfault is enabled" );
}

_shared_file_size = fc::parse_size( _options->at( "shared-file-size" ).as< string >() );
ilog( "shared_file_size is ${n} bytes", ("n", _shared_file_size) );
bool read_only = _options->count( "read-only" );
Expand Down Expand Up @@ -984,6 +991,7 @@ void application::set_program_options(boost::program_options::options_descriptio
("enable-plugin", bpo::value< vector<string> >()->composing()->default_value(default_plugins, str_default_plugins), "Plugin(s) to enable, may be specified multiple times")
("max-block-age", bpo::value< int32_t >()->default_value(200), "Maximum age of head block when broadcasting tx via API")
("flush", bpo::value< uint32_t >()->default_value(100000), "Flush shared memory file to disk this many blocks")
("backtrace", bpo::value<string>()->default_value("yes"), "Whether to print backtrace on SIGSEGV")
;
command_line_options.add(configuration_file_options);
command_line_options.add_options()
Expand Down
Loading

0 comments on commit 07ccb8b

Please sign in to comment.