Bkushigian/issue14 Serialize Tree/Card Configurations

.gitignore

@@ -1,2 +1,3 @@
 /target
 Cargo.lock
+.vscode

Cargo.toml

@@ -1,19 +1,22 @@
 [package]
 name = "postflop-solver"
-version = "0.1.0"
-authors = ["Wataru Inariba"]
+version = "0.1.1"
+authors = ["Wataru Inariba", "Ben Kushigian"]
 edition = "2021"
 description = "An open-source postflop solver for Texas hold'em poker"
 documentation = "https://b-inary.github.io/postflop_solver/postflop_solver/"
-repository = "https://github.com/b-inary/postflop-solver"
+repository = "https://github.com/bkushigian/postflop-solver"
 license = "AGPL-3.0-or-later"
 
 [dependencies]
+clap = { version = "4.5", features = ["derive"] }
 bincode = { version = "2.0.0-rc.3", optional = true }
 once_cell = "1.18.0"
 rayon = { version = "1.8.0", optional = true }
 regex = "1.9.6"
 zstd = { version = "0.12.4", optional = true, default-features = false }
+serde = { version = "1.0", features = ["derive"] }
+serde_json = "1.0"
 
 [features]
 default = ["bincode", "rayon"]

DESIGN.md

@@ -0,0 +1,200 @@
+# Design
+
+This document is a description, as far as I understand it, of the inner design
+of the solver and PostFlopGame. This is a working document for me to get my
+bearings.
+
+## PostFlopGame
+
+### Build/Allocate/Initialize
+
+To set up a `PostFlopGame` we need to **create a  `PostFlopGame` instance**, 
+**allocate global storage and `PostFlopNode`s**, and **initialize the
+`PostFlopNode` child/parent relationship**. This is done in several steps.
+
+We begin by creating a `PostFlopGame` instance.
+
+```rust
+let mut game = PostFlopGame::with_config(card_config, action_tree).unwrap();
+```
+
+A `PostFlopGame` requires an
+`ActionTree` which describes all possible actions and lines (no runout
+information), and a `CardConfig`, which describes player ranges and
+flop/turn/river data.
+
+Once we have created a `PostFlopGame` instance we need to allocate the following
+memory and initialize its values:
+
++ `game.node_arena`
++ `game.storage1`
++ `game.storage2`
++ `game.storage_ip`
++ `game.storage_chance`
+
+These fields are not allocated/initialized at the same time:
+
++ `game.node_arena` is allocated and initialized via `with_config()` (i.e., when
+  we created our `PostFlopGame`),
++ other storage is allocated via `game.allocate_memory()`.
+
+#### Allocating and Initializing `node_arena`
+
+We constructed a `PostFlopGame` by calling
+`PostFlopGame::with_config(card_config, action_tree)`, which under the hood
+actually calls:
+
+```rust
+    let mut game = Self::new();
+    game.update_config(card_config, action_tree)?;
+```
+
+`PostFlopGame::update_config` sets up configuration data, sanity checks things
+are correct, and then calls `self.init_root()`.
+
+`init_root` is responsible for:
+
+1. Counting number of `PostFlopNode`s to be allocated (`self.nodes_per_street`),
+   broken up by flop, turn, and river
+2. Allocating `PostFlopNode`s in the `node_arena` field
+3. Clearing storage: `self.clear_storage()` sets each storage item to a new
+   `Vec`
+4. Invoking `build_tree_recursive` which initializes each node's child/parent
+   relationship via `child_offset` (through calls to `push_actions` and
+   `push_chances`).
+
+Each `PostFlopNode` points to node-specific data (e.g., strategies and
+cfregrets) that is located inside of `PostFlopGame.storage*` fields (which is
+currently unallocated) via similarly named fields `PostFlopNode.storage*`.
+
+Additionally, each node points to the children offset with `children_offset`,
+which records where in `node_arena` relative to the current node that node's
+children begin. We allocate this memory via:
+
+```rust
+game.allocate_memory(false);  // pass `true` to use compressed memory
+```
+
+This allocates the following memory:
+
++ `self.storage1`
++ `self.storage2`
++ `self.storage3`
++ `self.storage_chance`
+
+Next, `allocate_memory()` calls `allocate_memory_nodes(&mut self)`, which
+iterates through each node in `node_arena` and sets storage pointers.
+
+After `allocate_memory` returns we still need to set `child_offset`s.
+
+### Storage
+
+There are several fields marked as `// global storage` in `game::mod::PostFlopGame`:
+
+```rust
+    // global storage
+    // `storage*` are used as a global storage and are referenced by `PostFlopNode::storage*`.
+    // Methods like `PostFlopNode::strategy` define how the storage is used.
+    node_arena: Vec<MutexLike<PostFlopNode>>,
+    storage1: Vec<u8>,
+    storage2: Vec<u8>,
+    storage_ip: Vec<u8>,
+    storage_chance: Vec<u8>,
+    locking_strategy: BTreeMap<usize, Vec<f32>>,
+```
+
+These are referenced from `PostFlopNode`:
+
+```rust
+    storage1: *mut u8, // strategy
+    storage2: *mut u8, // regrets or cfvalues
+    storage3: *mut u8, // IP cfvalues
+```
+
++ `storage1` seems to store the strategy
++ `storage2` seems to store regrets/cfvalues, and
++ `storage3` stores IP's cf values (does that make `storage2` store OOP's cfvalues?)
+
+Storage is a byte vector `Vec<u8>`, and these store floating point values.
+
+> [!IMPORTANT]
+> Why are these stored as `Vec<u8>`s? Is this for swapping between
+> `f16` and `f32`s?
+
+Some storage is allocated in `game::base::allocate_memory`:
+
+```rust
+    let storage_bytes = (num_bytes * self.num_storage) as usize;
+    let storage_ip_bytes = (num_bytes * self.num_storage_ip) as usize;
+    let storage_chance_bytes = (num_bytes * self.num_storage_chance) as usize;
+
+    self.storage1 = vec![0; storage_bytes];
+    self.storage2 = vec![0; storage_bytes];
+    self.storage_ip = vec![0; storage_ip_bytes];
+    self.storage_chance = vec![0; storage_chance_bytes];
+```
+
+`node_arena` is allocated in `game::base::init_root()`:
+
+```rust
+    let num_nodes = self.count_nodes_per_street();
+    let total_num_nodes = num_nodes[0] + num_nodes[1] + num_nodes[2];
+
+    if total_num_nodes > u32::MAX as u64
+        || mem::size_of::<PostFlopNode>() as u64 * total_num_nodes > isize::MAX as u64
+    {
+        return Err("Too many nodes".to_string());
+    }
+
+    self.num_nodes = num_nodes;
+    self.node_arena = (0..total_num_nodes)
+        .map(|_| MutexLike::new(PostFlopNode::default()))
+        .collect::<Vec<_>>();
+    self.clear_storage();
+```
+
+`locking_strategy` maps node indexes (`PostFlopGame::node_index`) to a locked
+strategy.  `locking_strategy` is initialized to an empty `BTreeMap<usize,
+Vec<f32>>` by deriving Default. It is inserted into via
+`PostFlopGame::lock_current_strategy`
+
+### Serialization/Deserialization
+
+Serialization relies on the `bincode` library's `Encode` and `Decode`. We can set
+the `target_storage_mode` to allow for a non-full save. For instance,
+
+```rust
+game.set_target_storage_mode(BoardState::Turn);
+```
+
+will ensure that when `game` is encoded, it will only save Flop and Turn data.
+When a serialized tree is deserialized, if it is a partial save (e.g., a Turn
+save) you will not be able to navigate to unsaved streets.
+
+Several things break when we deserialize a partial save:
+
++ `node_arena` is only partially populated
++ `node.children()` points to raw data when `node` points to an street that is
+  not serialized (e.g., a chance node before the river for a Turn save).
+
+### Allocating `node_arena`
+
+We want to first allocate nodes for `node_arena`, and then run some form of
+`build_tree_recursive`. This assumes that `node_arena` is already allocated, and
+recursively visits children of nodes and modifies them to 
+
+### Data Coupling/Relations/Invariants
+
++ A node is locked IFF it is contained in the game's locking_strategy
++ `PostFlopGame.node_arena` is pointed to by `PostFlopNode.children_offset`. For
+  instance, this is the basic definition of the `PostFlopNode.children()`
+  function:
+
+  ```rust
+    slice::from_raw_parts(
+        self_ptr.add(self.children_offset as usize),
+        self.num_children as usize,
+    )
+  ```
+
+  We get a pointer to `self` and add children offset.

README.md

@@ -1,9 +1,12 @@
 # postflop-solver
 
+This is a fork of [b-inary's postflop solver] that I will be maintaining.
+
 > [!IMPORTANT]
 > **As of October 2023, I have started developing a poker solver as a business and have decided to suspend development of this open-source project. See [this issue] for more information.**
 
 [this issue]: https://github.com/b-inary/postflop-solver/issues/46
+[b-inary's postflop solver]: https://github.com/b-inary/postflop-solver
 
 ---
 

examples/batch_solve.rs

@@ -0,0 +1,118 @@
+use std::path::{Path, PathBuf};
+
+use clap::Parser;
+use postflop_solver::{cards_from_str, solve, ActionTree, CardConfig, PostFlopGame, TreeConfig};
+
+/// Simple program to greet a person
+#[derive(Parser, Debug)]
+#[command(version, about, long_about = None)]
+struct Args {
+    /// Path to configuration file
+    #[arg(required = true)]
+    config: String,
+
+    /// Boards to run on
+    #[arg(short, long)]
+    boards: Option<Vec<String>>,
+
+    /// File with boards to run on
+    #[arg(short, long)]
+    boards_file: Option<String>,
+
+    /// Directory to output solves to
+    #[arg(short, long, default_value = ".")]
+    dir: String,
+
+    /// Max number of iterations to run
+    #[arg(short = 'n', long, default_value = "1000")]
+    max_iterations: u32,
+
+    /// Default exploitability as ratio of pot. Defaults to 0.2 (20% of pot),
+    /// but for accurate solves we recommend choosing a lower value.
+    #[arg(short = 'e', long, default_value = "0.2")]
+    exploitability: f32,
+}
+
+fn main() -> Result<(), String> {
+    let args = Args::parse();
+
+    let config = std::fs::read_to_string(args.config).expect("Unable to read in config");
+
+    let boards = if let Some(boards) = args.boards {
+        boards
+    } else {
+        let boards_files = args
+            .boards_file
+            .expect("Must specify boards or boards_file");
+        let boards_contents =
+            std::fs::read_to_string(boards_files).expect("Unable to read boards_file");
+        boards_contents
+            .lines()
+            .map(|s| s.to_string())
+            .collect::<Vec<String>>()
+    };
+    let configs_json: serde_json::Value =
+        serde_json::from_str(&config).expect("Unable to parse config");
+    let configs_map = configs_json.as_object().expect("Expected a json object");
+
+    let card_config = configs_map.get("card_config").unwrap();
+    let card_config: CardConfig = serde_json::from_value(card_config.clone()).unwrap();
+
+    let tree_config = configs_map.get("tree_config").unwrap();
+    let tree_config: TreeConfig = serde_json::from_value(tree_config.clone()).unwrap();
+
+    // Create output directory if needed. Check if ".pfs" files exist, and if so abort
+    let dir = PathBuf::from(args.dir);
+    setup_output_directory(&dir)?;
+    ensure_no_conflicts_in_output_dir(&dir, &boards)?;
+
+    for board in &boards {
+        let cards =
+            cards_from_str(&board).expect(format!("Couldn't parse board {}", board).as_str());
+
+        let mut game = PostFlopGame::with_config(
+            card_config.with_cards(cards).unwrap(),
+            ActionTree::new(tree_config.clone()).unwrap(),
+        )
+        .unwrap();
+
+        game.allocate_memory(false);
+
+        let max_num_iterations = args.max_iterations;
+        let target_exploitability = game.tree_config().starting_pot as f32 * args.exploitability;
+        solve(&mut game, max_num_iterations, target_exploitability, true);
+    }
+    Ok(())
+}
+
+fn setup_output_directory(dir: &Path) -> Result<(), String> {
+    if dir.exists() {
+        if !dir.is_dir() {
+            panic!(
+                "output directory {} exists but is not a directory",
+                dir.to_str().unwrap()
+            );
+        }
+        Ok(())
+    } else {
+        std::fs::create_dir_all(&dir).map_err(|_| "Couldn't create dir".to_string())
+    }
+}
+
+fn ensure_no_conflicts_in_output_dir(dir: &Path, boards: &[String]) -> Result<(), String> {
+    for board in boards {
+        // create board file name
+        let board_file_name = board
+            .chars()
+            .filter(|c| !c.is_whitespace())
+            .collect::<String>();
+        let board_path = dir.join(board_file_name).with_extension("pfs");
+        if board_path.exists() {
+            return Err(format!(
+                "board path {} already exists",
+                board_path.to_string_lossy()
+            ));
+        }
+    }
+    Ok(())
+}