README.md

AA228-CS238-Student

AA228/CS238: Decision Making under Uncertainty, Autumn 2024, Stanford University.

This repository provides starter code and data for Projects 1 and 2.

Project 0:

Project 1: Bayesian Structure Learning

LaTeX Overleaf template: click the link, go to "Menu", and "Copy Project" (make sure you're signed into Overleaf). Note this is an optional template, you're free to use your own (or not even LaTeX).

project1/
├── data                    # CSV data files to apply structured learning
│   ├── small.csv               # Titanic dataset¹
│   ├── medium.csv              # Wine dataset²
│   └── large.csv               # Secret dataset
├── example                 # Helpful examples
│   ├── example.gph             # Example graph (3 parents, 2 children with 1 parent each and 1 child with 3 parents)
│   ├── example.csv             # Example data generated from "example.gph"
│   ├── example.score           # Bayesian score of the "examples.gph" given the data "examples.csv"
│   ├── examples.pdf            # Visualized "examples.gph" as a TikZ graph
│   └── titanicexample.pdf      # Simple example network using "small.csv"
├── project1.jl             # Starter code in Julia (optional, meant to help)
└── project1.py             # Starter code in Python (optional, meant to help)

¹https://cran.r-project.org/web/packages/titanic/titanic.pdf
²https://archive.ics.uci.edu/ml/datasets/Wine+Quality

Notes

• The starter code is there to help, but you're free to use any language.
• Use example.gph to validate your Bayesian scoring algorithm, not your structure learning algorithm.

Graph Plotting

Here are some resources for plotting graphs in Julia, Python, and MATLAB.

• Julia:
  • Graphs.jl
  • TikzGraphs.jl
  • GraphPlot.jl
  • GraphRecipes.jl
• Python:
  • NetworkX
• MATLAB:
  • GraphPlot

Julia Examples

TikzGraphs.jl

using Graphs  # for DiGraph and add_edge!
using TikzGraphs   # for TikZ plot output

# An example [Chvatal Graph](https://en.wikipedia.org/wiki/Chv%C3%A1tal_graph)
g = smallgraph(:chvatal) 

# Create notional names for the nodes
node_names = ["A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L"]

# Create TikZ plot with node labels
p = plot(g, node_names)

# Save as PDF (using TikzPictures)
using TikzPictures # to save TikZ as PDF
save(PDF("chvatal_tikz.pdf"), p)

GraphPlot.jl

using Graphs
using GraphPlot

g = smallgraph(:chvatal)
node_names = ["A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L"]

p = gplot(g; nodelabel=node_names)

# Save using Compose
using Compose, Cairo, Fontconfig
draw(PDF("chvatal_graphplot.pdf", 16cm, 16cm), p)

GraphRecipes.jl

using Graphs
using Plots
using GraphRecipes


g = smallgraph(:chvatal)
node_names = ["A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L"]

p = graphplot(g; names=node_names, node_size=0.2)

savefig(p, "chvatal_graphrecipes.pdf")

Project 2: Reinforcement Learning

LaTeX Overleaf template: click the link, go to "Menu", and "Copy Project" (make sure you're signed into Overleaf). Note this is an optional template, you're free to use your own (or not even LaTeX).

project2/
└── data                      # CSV data files of (s,a,r,sp)
    ├── small.csv                 # 10x10 grid world
    ├── medium.csv                # MountainCarContinuous-v0
    └── large.csv                 # Secret RL problem

Note: no starter code provided for Project 2.

Contact

Please post on Ed with any questions regarding this code, the data, and the projects in general. We'd be happy to help!