Release 2.5.1.0

The release introduces a rework of the mempool management and propagation, a small miner optimization, and improvements of the syncing process, especially end game syncing and syncing of a large number of disconnected intervals.

• Transactions are not rejected when the mempool limit is reached; instead, the least valuable transactions are dropped from the memory pool;
• picking transactions for a block is made faster;
• the peers for gossip are chosen from the entire network (some preference still goes to the better-rated peers) to avoid propagation issues when the rating network becomes too clustered; blocks are also sent to the trusted peers (specified on startup);
• the default number of peers transactions are gossiped to is reduced to speed up propagation;
• the node does not only search for confirmed transactions when searching for missing transactions for a block;
• freshly synced block and transaction headers are stored in RocksDB now;
• a bug is fixed where already known transactions would only be immediately ignored on repeated submission with 5 seconds since the previous submission. At the same time, the identifiers are not kept in memory indefinitely, preventing unnecessary albeit slow increase of the memory footprint;
• the headers are always stored in the disk cache (the size is controlled by the disk_cache_size_mb parameter) to make v1 transactions with data quickly available for download before their data is picked up from the disk pool and indexed for mining;
• the redundant signature validation during mempool filtering is removed;
• the miner is accelerated ~5-10% by removing one extra in-memory chunk copy per mined chunk;
• upgraded erlang-rocksdb to 1.7.0; RocksDB 6.25.3;
• made various improvements to the syncing process;
• fixed tracking of the disk_pool_chunks_count metric;
• improved the disk pool processing in the presence of many repeated data_roots throughout the weave.

API changes

• The node will reject a v2 transaction with the attached data if the node fails to reconstruct its data_root from the data. It does not pose consensus issues because 1) nodes strip data before gossip 2) the data field is ignored when a node fetches a missing transaction itself. The motivation is to reduce the likelihood of not uploading the data by mistake, e.g., when a new client library uses this way of uploading but does not split the data accordingly.

Upgrade instructions

N.2.5.1.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. arweave-2.5.1.0.linux-x86_64.tar.gz works at least on Ubuntu 20 and CentOS 8. To run on Ubuntu 18, grab arweave-2.5.1.0.ubuntu18-x86_64.tar.gz. CentOS 7 is not supported.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.5.1.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not on the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 or 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.5.1.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at [email protected].

Release 2.5.0.0

This release introduces a hard fork that activates at height 812970, approximately 2021-11-17 14:00 UTC. You will need to make sure you have upgraded your miner before this time to connect to the network.

This protocol upgrade includes a number of improvements, laying the groundwork for the switch to HDD-dominated mining in 2.6. After the fork, SPoRA solution candidates have to be "packed" such that they can be identified as part of a unique replica. Additionally, the protocol will now create some entropy on every mining attempt. Miners have to carry it on throughout the process, further strengthening the requirement to keep data close to the computation power.

Note for the miners: the upgrade does not make hard drives the most cost-efficient medium (we plan to make this transition in the next major release).

Note for the miners: the new mining algorithm operating with the packed data and entropy does not take effect immediately after the fork but gradually replaces the existing algorithm over time.

Note for the users: starting from the fork block, the protocol would no longer recognize new data that is not split in a particular way. Specifically, all chunk sizes have to equal 256 * 1024 bytes unless they meet one of the three requirements:

• the chunk is the last or the only chunk of its transaction;
• the chunk is the second last chunk of its transaction, and together with the last chunk, they exceed 256 KiB;
• the chunk size is larger than its Merkle proof (data_path).

The change is compatible with the major ecosystem clients, like arweave-js, arweave-stream-tx, or goar.

Note that it would still be possible to submit a transaction with a data_root computed from a wrong split, but the chunks not meeting the criteria mentioned above won't be accepted by the nodes.

Furthermore, every TX requires a fee covering the smallest multiple of 256 KiB bigger than or equal to its data_size after the fork.

The difficulty-based price adjustment is re-enabled in 2.5. The price is recalculated every 50 blocks; however, we introduce a one-time change cap to eliminate the effects of short-term difficulty and token price fluctuations.

Furthermore, an accelerated difficulty adjustment is used at the fork block, similar to the mechanism used at 2.4, because a significant network difficulty drop is possible. However, the transition is gradual, so we expect a relatively small difficulty change. The node will migrate the historical data and pack the incoming chunks to have them ready for mining.

Finally, the floating-point math is no longer present in the protocol after the fork, simplifying alternative clients' development.

Upgrade instructions

N.2.5.0.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. arweave-2.5.0.0.linux-x86_64.tar.gz works at least on Ubuntu 20 and CentOS 8. To run on Ubuntu 18, grab arweave-2.5.0.0.ubuntu18-x86_64.tar.gz. CentOS 7 is not supported.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.5.0.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not on the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 or 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.5.0.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at [email protected].

Release 2.4.4.0

The release aims to eliminate congestion in the transaction propagation queues. Nodes now monitor eachother's mempools, eliminating a significant amount of the communication during transaction distribution.

Additionally, the upgrade speeds up the downloading of missing transactions from new blocks (an event which has increased in frequency during the transaction distribution issues).

Also, fixes #304.

Upgrade instructions

N.2.4.4.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. arweave-2.4.4.0.linux-x86_64.tar.gz works at least on Ubuntu 20 and CentOS 8. To run on Ubuntu 18 grab arweave-2.4.4.0.ubuntu18-x86_64.tar.gz. CentOS 7 is not supported.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.4.4.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not in the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 and execute the following command:

git fetch --all --tags && git checkout -f N.2.4.4.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at [email protected].

Release 2.4.3.0

The release introduces a new reliable way to maintain the storage under the limited disk space, fixes a bug where the node would accumulate redundant chunks in the disk pool, improves transaction propagation, and contains a few smaller performance improvements.

Upgrade instructions

N.2.4.3.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. arweave-2.4.3.0.linux-x86_64.tar.gz works at least on Ubuntu 20 and CentOS 8. To run on Ubuntu 18 grab arweave-2.4.3.0.ubuntu18-x86_64.tar.gz. CentOS 7 is not supported.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.4.3.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not in the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 or 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.4.3.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at [email protected].

Release 2.4.2.0

The release features a significant syncing speedup, various other improvements, and changes made in preparation for the new iteration of the mining algorithm.

Among the prepared but not yet activated protocol upgrades is removing floating-point arithmetic from the protocol - likely the most significant obstacle on the path to alternative node implementations.

Please, note you can run the new release with existing data, but you cannot run the previous versions of the miner from the same data_dir after running the latest release.

Other changes include:

• Synchronization is made much faster by removing the bottleneck added in 2.1.
• A header_sync_jobs command line and config file parameter is introduced for speeding up chain headers download.
• The joining process is made faster.
• A new synchronization scheme is implemented improving data discovery after a significant amount of data is synced.
• The requirement for the peers specified on startup to be available is dropped.
• v1 transactions with more than 100 bytes of data are deprioritized over all other transactions. Such transactions saturate network links and fill up blocks much quicker. Users are expected to submit v2 transactions to upload data.
• Orphaned transactions are returned to the mempool now. Very roughly, about 0.5% of all submitted transactions used to be dropped from the weave after a reorg. The measure will eliminate almost all of such cases.
• A bug is fixed where the peer list update may get stuck (the ar_bridge_update_peers_process_is_stuck log).
• A bug is fixed where the node may store and report as synced but not serve some chunks after shutdown.
• A bug is fixed where a node might lose a chunk after a chain reorg.

Upgrade instructions

N.2.4.2.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.4.2.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not in the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 and execute the following command:

git fetch --all --tags && git checkout -f N.2.4.2.0.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at [email protected].

Release 2.4.1.0

The release's main goal is to improve the efficiency of the new SPoRA miner.

The changes include:

• blob storage for chunks optimized for random reads tailored specifically to the miner;
• the recall byte computation - the only mining operation on the Erlang side that takes ~100 us per chunk - is re-implemented in C;
• Erlang VM tuning;
• a more streamlined and configurable mining server.

Upgrade note

There is no automatic migration to the new storage, only the new data is written there. You can still mine with existing data; however, you need to sync from scratch to achieve the largest performance gain. If you run the new version with existing storage, run it with enable search_in_rocksdb_when_mining.

See the new version of the mining guide for more details on tuning the node. Also, the new storage contains sparse files - you need to account for it if you copy your data from one place to another. You can find more information about it in the guide.

Other improvements

• Transaction processing performance is improved a bit in response to increased transaction volumes.
• The rocksdb and chunk_storage folders can reside on their own disks now, and the node would still track the amount of available space. These folders contain all the data required during mining. E.g., a fast nvme SSD drive can be mounted to these folders while the rest of the data may be on a cheaper HDD disk.
• A few issues with the peering logic are fixed (no more ar_bridge_update_peers_process_is_stuck logs).
• An improvement in the block propagation logic is made where nodes re-send a block to some of the best peers who did not know the previous block at the time of the first attempt, usually when two successive blocks are mined close in time. Previously, those peers had to ask for the missing blocks themselves.
• Serving HTML over /tx/[id]/data.html and /[id] can be optionally disabled via the disable serve_html_data command line setting.
• The joining procedure is made more reliable to make it easier to join when the recent blocks are big.
• The broken disk_space setting is fixed.

Upgrade instructions

N.2.4.1.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.4.1.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, execute the following command to upgrade:

git fetch --all --tags && git checkout -f N.2.4.1.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at [email protected].

Release 2.4.0.0

This release introduces a hard fork that activates at height 633720, approximately 2021-02-24 11:50 UTC. You will need to make sure you have upgraded your miner before this time in order to connect to the 2.4 network.

The release introduces SPoRA - the new consensus mechanism aiming to promote further replication of the dataset, and reduce the energy consumed by the network by trade hashing expenditure for throughput of data access links. See more about the design here.

The estimated USD to AR rate is fixed in the release version due to the high uncertainty about the difficulty change at the fork block. A follow-up hard fork will re-enable the dynamic difficulty-based price estimation. Thereby the /price endpoint would now return the precise amount to pay, not a pessimistic estimation of it.

It may take significantly longer than usual to mine the fork block, leading to disruption in usage of the network for this period.

HTTP API changes

• The "reward_pool", "block_size", and "weave_size" block fields are served as JSON strings starting from the fork block.

Transaction verification changes

• The "target" field must be empty when no tokens are transferred. For transfer transactions, the size of the "target" field must be 32 bytes.
• A bug is fixed where the nodes would not verify some valid RSA PSS signatures.

New features and improvements

• A new logging system is introduced. Logs are now written to "logs/[email protected].*" and rotated automatically. The messages are one-liners.
• The "polling" command line option is an integer now - the polling frequency in seconds.
• The memory footprint is reduced.

The release also includes various performance improvements.

Bug fixes

• A bug was fixed where the miner won't log the "[Stage 3/3]" message about the confirmed mined blocks.

Upgrade instructions

N.2.4.0.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.4.0.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, execute the following command to upgrade:

git fetch --all --tags && git checkout -f N.2.4.0.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at [email protected].

Release 2.3.0.0

This release introduces a hard fork that activates at height 591140, approximately 2020-12-21 11:00 UTC. You will need to make sure you have upgraded your miner before this time in order to connect to the 2.3 network.
Most of the work put into this release is preparation for SPoRA - the new consensus mechanism.

New features and improvements

• The difficulty of mining on alternative recall chunks grows linearly as a function of the alternative sequence number. It follows the pattern applied in SPoRA. Link.
• A sync_jobs configuration parameter is added to control the intensity of data syncing. The node now runs two jobs by default. Consider increasing it on the new nodes and decreasing once you replicate the desired amount of data.
• Block application is reworked. The new version maintains a cache of uncle blocks to efficiently validate the work component of a block for the fast and DoS-resistant block propagation in the network. The update is especially relevant for validating blocks that are not on top of the longest chain currently known to the node.
• The data of v1 transactions is moved to the new index. The planned consensus mechanism incentivizes fast random access to past data. Therefore, it is crucial to have quick access to the historical v1 data.
• The benchmark is made more representative by running a node. Run it via ./bin/benchmark.
• The node can run under limited disk space now. Once there is little available space left, the node stops syncing data and begins cleaning up older headers to free up space for the new block and transaction headers.
• The transaction blacklist is reworked. The new blacklist is more scalable, supports v2 data, allows to fetch blacklisted transactions from external sources, and automatically removes already stored blacklisted data. Additionally, the change introduces a whitelist.
• The v2 chunk data is migrated from the individual files to RocksDB, see the commit for more details.

Bug fixes

• A bug is fixed where the node would never attempt to sync a historical transaction if it crashed right after syncing the corresponding block header.
• A bug where the node would not send new blocks to the configured webhooks is fixed.
• The broken requests_per_minute flag works again.

Upgrade instructions

N.2.3.0.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.3.0.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, execute the following command to upgrade:

git fetch --all --tags && git checkout -f N.2.3.0.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at [email protected].

Release 2.2.0.0

This release introduces a hard fork that activates at height 552180, approximately 2020-10-21 13:00 UTC. You will need to make sure you have upgraded your miner before this time in order to connect to the 2.2 network.

The core of this release is an update to the protocol that allows to efficiently incorporate an
arbitrarily big number of wallets in the weave. The changes makes block processing times
grow very slowly and only up to a hard cap with the growing number of wallets. For details,
see the commit description.

The release also changes the wallet generation fee. Starting from the block 552180, an extra fee for a new wallet is 0.1$ according to the AR to USD exchange rate estimated by the protocol at the given block.

Additionally, the release includes a significant improvement of the internal state management,
decreasing request latencies and memory footprint and increasing the hashing power. See the commit fore more info.

A new GET /wallet_list/[root_hash]/[addr]/balance endpoint is added. The endpoint allows to lookup wallet balances with a certain number of confirmations. root_hash is the Merkle root of the wallet tree corresponding to a block (the wallet_list field), addr is a wallet address.

Finally, a change is introduced to make nodes that missed transactions included in a block catch up faster.

Upgrade instructions

N.2.2.0.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.2.0.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, execute the following command to upgrade:

git fetch --all --tags && git checkout -f N.2.2.0.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at [email protected].

Release 2.1.0.2

This release provides a few patches and improvements.

• A randomx_bulk_hashing_iterations configuration parameter is introduced - tweaking it allows increasing the mining performance on powerful CPUs significantly.
• The memory pool is now persisted before shutdown and loaded on startup so pending transactions do not get lost on restart increasing the joining speed and decreasing the chance the data attached to v2 transaction headers gets lost when poorly seeded.
• The data attached to v2 transaction headers is propagated to the trusted peers to increase the initial seeding coverage.
• A bug is fixed where RocksDB may not have sufficient time to shut down.
• A bug is fixed, which would delay the network joining by 30 seconds.
• An improvement is made, which relaxes the bandwidth requirements on network join.

Upgrade instructions

N.2.1.0.2 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.1.0.2.linux-x86_64.tar.gz

The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start <arguments>. See the mining guide for more details.
If your OS/platform architecture is not in the list, check README for how to build the miner from source.

If you want to run the miner from the existing Git folder, execute the following command to upgrade:

git fetch --all --tags && git checkout -f N.2.1.0.2

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at [email protected].