-
Notifications
You must be signed in to change notification settings - Fork 370
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add documentation for JLD2 I/O #3365
base: main
Are you sure you want to change the base?
Changes from all commits
844235e
8f32a2f
f5ef5d3
aff1770
67ec270
8b1bcba
09735db
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
@@ -113,6 +113,56 @@ As you can see the code required to transform `iris` into a proper input to the | |||||||||||||||||||||||||||
format is not easy. Therefore CSV.jl is the preferred package to write CSV files | ||||||||||||||||||||||||||||
for data stored in data frames. | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
## JLD2 Files | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
JLD2 is a HDF5-compatible file format that allows reading and writing data frames. | ||||||||||||||||||||||||||||
A valuable feature of JLD2 format is that it preserves custom column types of the stored data frame. | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
The `save` and `load` functions, provided by FileIO.jl, allow to read/write a data frame from/to a JLD2 file. | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
A data frame can be saved as a JLD2 file output.jld2 using | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
Comment on lines
+123
to
+124
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
If you have not used the FileIO and JLD2 packages before then you may need to install it first: | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
```julia | ||||||||||||||||||||||||||||
using Pkg; | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
Pkg.add("FileIO") | ||||||||||||||||||||||||||||
Pkg.add("JLD2") | ||||||||||||||||||||||||||||
Comment on lines
+128
to
+129
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This will typically be faster:
Suggested change
|
||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
The FileIO and JLD2 functions are not loaded automatically and must be imported into the session. | ||||||||||||||||||||||||||||
```julia | ||||||||||||||||||||||||||||
using FileIO | ||||||||||||||||||||||||||||
alex-s-gardner marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||||||||||||||||||||||||
using JLD2 | ||||||||||||||||||||||||||||
Comment on lines
+134
to
+135
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
We can now create a simple data frame and save it as a jld2 file using `save`. `save` | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
accepts an AbstractDict yielding the key/value pairs, where the key is a string representing | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
the name of the dataset and the value represents its contents: | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
```julia | ||||||||||||||||||||||||||||
df = DataFrame(x=1, y=2) | ||||||||||||||||||||||||||||
save("output.jld2", Dict("df" => df)) | ||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
A jld2 file can be read in using `load`. If `load` is called with a single dataset name, | ||||||||||||||||||||||||||||
load returns the contents of that dataset from the file: | ||||||||||||||||||||||||||||
Comment on lines
+147
to
+148
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
```julia | ||||||||||||||||||||||||||||
df = load("output.jld2", "df") | ||||||||||||||||||||||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
1×2 DataFrame | ||||||||||||||||||||||||||||
Row │ x y | ||||||||||||||||||||||||||||
│ Int64 Int64 | ||||||||||||||||||||||||||||
─────┼────────────── | ||||||||||||||||||||||||||||
1 │ 1 2 | ||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
If `load` is called with a filename argument only, the load function loads all datasets | ||||||||||||||||||||||||||||
from the given file into a Dict: | ||||||||||||||||||||||||||||
Comment on lines
+158
to
+159
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
```julia | ||||||||||||||||||||||||||||
df = load("output.jld2") | ||||||||||||||||||||||||||||
Dict{String, Any} with 1 entry: | ||||||||||||||||||||||||||||
"df" => 1×2 DataFrame… | ||||||||||||||||||||||||||||
Comment on lines
+161
to
+163
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||||||||||||||||||||||||
``` | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
## Other formats | ||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||
Other data formats are supported for reading and writing in the following packages | ||||||||||||||||||||||||||||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.