Templates for data i/o

[smmaurer]

It will be helpful to have a set of templates for loading data into Orca and saving outputs back to disk. Like the model step templates, these will be implemented as Orca steps: running a data loading step will register a table, running a data output step will save a table to disk, etc.

Loading data: urbansim_templates.data.LoadTable()

Earlier discussion in Issue #66.

• register table from local csv or hdf file (PR [0.2.dev0] Template for loading data #93)
• register Pandana network

How should we handle single columns of data?

Saving data: urbansim_templates.data.SaveTable()

• save table to local csv or hdf (allow subset of columns) (PR [0.2.dev2] Template for saving data #95)
• save table to authenticated cloud storage?

Registering data locations

Would it makes sense to create a stand-alone object to represent "data locations" as well? Data locations would be things like local directory paths, authenticated cloud storage locations, etc.

Representing this separately would allow multiple data i/o steps to refer to a single abstract location that could be easily swapped out. It would also support a data output workflow where a new directory is generated for each model run.

On the other hand, it might be difficult in practice to separate this from the code for reading and writing tables -- probably need to try it out to make sure it's a good idea.

• object to represent data locations?

Registering merge relationships

Earlier discussion in Issue #78 about Orca broadcasts vs. implicit join keys. Include some kind of template-based backward compatibility with Orca broadcasts? Probably not.

• validation of implicit join keys (PR [0.2.dev0] Template for loading data #93)
• utility to merge list of tables/columns based on implicit join keys
• confirm none of the model step templates rely on broadcasts