v2.0.0rc

A candidate for the stable derecho (v2.0.0) release. Because we have added many features since v0.9.2, we skip release number v1.

Major improvements since v0.9.2

New Features

• Enabled external client API.
• Revived code for using ibverbs API with flow control.
• Derecho can now be configured with a "restart leader" distinct from its normal leader, and if the enable_backup_restart_leaders option is True, it can also use a list of multiple restart leaders in priority order.
• Added a build script.
• The type T in Replicated is now aware of the subgroup ID it belongs to.
• Move the ObjectStore out to the new Cascade project.

Bugfixes

• Fixed the completion queue overrun issue.
• Avoided relying on ibv_wc::wr_id (or fi_cq_err_entry::op_context) to determine which remote node failed when a request posted to a queue pair fails. Now uses timeout logic to detect the failure.
• Refactored p2p_connection code.
• Stored Persistent state into a file named by SHA256 hash string instead of a type string, which might be longer than filename length limitation.

Dependencies

• Moved to spdlog v1.3.1
• Moved to libfabric v1.7.0

Known Issues

• An update to a subgroup type with Persistent fields is acknowledged (by derecho::rpc::QueryResults<>) when all shard members delivered the update. Persistence is processed in the background off the critical path. Applications must use local_persistentce_callback or global_persistence_callback in group constructor to make sure when the updates are persisted locally or globally.

 /**
  * Bundles together a set of callback functions for message delivery events.
  * These will be invoked by MulticastGroup or ViewManager to hand control back
  * to the client if it wants to implement custom logic to respond to each
  * message's arrival. (Note, this is a client-facing constructor argument,
  * not an internal data structure).
  */
 struct CallbackSet {
     message_callback_t global_stability_callback;
     persistence_callback_t local_persistence_callback = nullptr;
     persistence_callback_t global_persistence_callback = nullptr;
 };
 

• A Derecho node cannot join multiple subgroups so far. We plan to add this feature, aka overlapping subgroup, in the near future.
• Slow startup (It takes several seconds for a Derecho application to start).