Skip to content

Commit

Permalink
Ensure Pluto-defined structs have a stable hash (#44)
Browse files Browse the repository at this point in the history
A bug in detecting anonymous types meant that Pluto workspace defined structs generated a different hash id each time the cell was updated. This ensures a stable ID for pluto-defined structs, and improves type-name validation.
  • Loading branch information
haberdashPI authored Jan 10, 2024
1 parent 1447160 commit 198c17f
Show file tree
Hide file tree
Showing 17 changed files with 105 additions and 4 deletions.
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "StableHashTraits"
uuid = "c5dd0088-6c3f-4803-b00e-f31a60c170fa"
authors = ["Beacon Biosignals, Inc."]
version = "1.1.3"
version = "1.1.4"

[deps]
Compat = "34da2185-b29b-5c13-b0c7-acf172513d20"
Expand Down
17 changes: 15 additions & 2 deletions src/StableHashTraits.jl
Original file line number Diff line number Diff line change
Expand Up @@ -424,12 +424,25 @@ function cleanup_name(str)
# later versions of julia, they do not
str = replace(str, "AbstractVector{T} where T" => "AbstractVector")
str = replace(str, "AbstractMatrix{T} where T" => "AbstractMatrix")
# cleanup pluto workspace names

# handle pluto symbols

# TODO: eventually, when we create hash version 3 (which will generate strings from
# scratch rather than leveraging `string(T)`), we should handle pluto symbols by
# checking `is_inside_pluto` as defined here
# https://github.com/JuliaPluto/PlutoHooks.jl/blob/f6bc0a3962a700257641c3449db344cf0ddeae1d/src/notebook.jl#L89-L98

# NOTE: in more recent julia versions (>= 1.8) the values are surrounded by `var`
# qualifiers
str = replace(str, r"var\"workspace#[0-9]+\"" => "PlutoWorkspace")
str = replace(str, r"workspace#[0-9]+" => "PlutoWorkspace")
return str
end

function validate_name(str)
if occursin(r"\.#[^.]*$", str)
throw(ArgumentError("Anonymous types (those containing `#`) cannot be hashed to a reliable value"))
if occursin("#", str)
throw(ArgumentError("Anonymous types (those containing `#`) cannot be hashed to a reliable value: found type $str"))
end
return str
end
Expand Down
1 change: 1 addition & 0 deletions test/Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
CRC32c = "8bf52ea8-c179-5cab-976a-9e18b702a9bc"
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"
Pluto = "c3e4b0f8-55cb-11ea-2926-15256bba5781"
ReferenceTests = "324d217c-45ce-50fc-942e-d289b448e8cf"
SHA = "ea8e919c-243c-51af-8825-aaa63cd721ce"
Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
Expand Down
1 change: 1 addition & 0 deletions test/references/pluto01_1_crc.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
e2568b25890efb3e569dc44cc24409249a2d2b1126bbf80e9b82fac526eb99db
1 change: 1 addition & 0 deletions test/references/pluto01_1_sha1.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
e2568b25890efb3e569dc44cc24409249a2d2b1126bbf80e9b82fac526eb99db
1 change: 1 addition & 0 deletions test/references/pluto01_1_sha256.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
e2568b25890efb3e569dc44cc24409249a2d2b1126bbf80e9b82fac526eb99db
1 change: 1 addition & 0 deletions test/references/pluto01_2_crc32c.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
e2568b25890efb3e569dc44cc24409249a2d2b1126bbf80e9b82fac526eb99db
1 change: 1 addition & 0 deletions test/references/pluto01_2_sha1.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
e2568b25890efb3e569dc44cc24409249a2d2b1126bbf80e9b82fac526eb99db
1 change: 1 addition & 0 deletions test/references/pluto01_2_sha256.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
e2568b25890efb3e569dc44cc24409249a2d2b1126bbf80e9b82fac526eb99db
1 change: 1 addition & 0 deletions test/references/pluto02_1_crc.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c50004f5a3abd3c6c98926f354b08eb2557188086db60d661f3a0b7b7503fa7f
1 change: 1 addition & 0 deletions test/references/pluto02_1_sha1.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c50004f5a3abd3c6c98926f354b08eb2557188086db60d661f3a0b7b7503fa7f
1 change: 1 addition & 0 deletions test/references/pluto02_1_sha256.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c50004f5a3abd3c6c98926f354b08eb2557188086db60d661f3a0b7b7503fa7f
1 change: 1 addition & 0 deletions test/references/pluto02_2_crc32c.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c50004f5a3abd3c6c98926f354b08eb2557188086db60d661f3a0b7b7503fa7f
1 change: 1 addition & 0 deletions test/references/pluto02_2_sha1.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c50004f5a3abd3c6c98926f354b08eb2557188086db60d661f3a0b7b7503fa7f
1 change: 1 addition & 0 deletions test/references/pluto02_2_sha256.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
c50004f5a3abd3c6c98926f354b08eb2557188086db60d661f3a0b7b7503fa7f
76 changes: 75 additions & 1 deletion test/runtests.jl
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,8 @@ include("setup_tests.jl")
@test_reference("references/ref20_$(V)_$(nameof(hashfn)).txt",
bytes2hex_(test_hash(==("test"))))
end

# verifies that transform can be called recursively

@testset "FnHash" begin
@test test_hash(GoodTransform(2)) == test_hash(GoodTransform("-0.2"))
@test test_hash(GoodTransform(3)) != test_hash(GoodTransform("-0.2"))
Expand Down Expand Up @@ -202,6 +202,80 @@ include("setup_tests.jl")
@test_throws ArgumentError test_hash(BadHashMethod())
end

@testset "Pluto-defined strucst are stable" begin
notebook_project_dir = joinpath(@__DIR__, "..")
@info "Notebook project: $notebook_project_dir"

notebook_str = """
### A Pluto.jl notebook ###
# v0.19.36
using Markdown
using InteractiveUtils
# ╔═╡ 3592b099-9c96-4939-94b8-7ef2614b0955
import Pkg
# ╔═╡ 72871656-ae6e-11ee-2b23-251ac2aa38a3
begin
Pkg.activate("$notebook_project_dir")
using StableHashTraits
end
# ╔═╡ 1c505fa8-75fa-4ed2-8c3f-43e28135b55d
begin
bytes2hex_(x::Number) = x
bytes2hex_(x) = bytes2hex(x)
end
# ╔═╡ b449d8e9-7ede-4171-a5ab-044c338ebae2
struct MyStruct end
# ╔═╡ 1e683f1d-f5f6-4064-970c-1facabcf61cc
stable_hash(MyStruct()) |> bytes2hex_
# ╔═╡ f8f3a7a4-544f-456f-ac63-5b5ce91a071a
stable_hash((a=MyStruct, b=(c=MyStruct(), d=2))) |> bytes2hex_
# ╔═╡ Cell order:
# ╠═3592b099-9c96-4939-94b8-7ef2614b0955
# ╠═72871656-ae6e-11ee-2b23-251ac2aa38a3
# ╠═1c505fa8-75fa-4ed2-8c3f-43e28135b55d
# ╠═b449d8e9-7ede-4171-a5ab-044c338ebae2
# ╠═1e683f1d-f5f6-4064-970c-1facabcf61cc
# ╠═f8f3a7a4-544f-456f-ac63-5b5ce91a071a
"""

server = Pluto.ServerSession()
server.options.evaluation.workspace_use_distributed = false
olddir = pwd()
nb = mktempdir() do dir
path = joinpath(dir, "notebook.pluto.jl")
write(path, notebook_str)
nb = Pluto.load_notebook(path)
Pluto.update_run!(server, nb, nb.cells)
return nb
end
# pluto changes pwd
cd(olddir)

# NOTE: V refers to the hash version currently in loose
# its the `for` loop at the top of this file
if nb.cells[5].output.body isa Dict
throw(Error("Failed notebook eval: $(nb.cells[5].output.body[:msg])"))
else
@test_reference("references/pluto01_$(V)_$(nameof(hashfn)).txt",
strip(nb.cells[5].output.body, '"'))
end

if nb.cells[6].output.body isa Dict
throw(Error("Failed notebook eval: $(nb.cells[6].output.body[:msg])"))
else
@test_reference("references/pluto02_$(V)_$(nameof(hashfn)).txt",
strip(nb.cells[6].output.body, '"'))
end
end

if V > 2 && hashfn == sha256
@testset "Hash-invariance to buffer size" begin
data = (rand(Int8, 2), rand(Int8, 2))
Expand Down
1 change: 1 addition & 0 deletions test/setup_tests.jl
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ using CRC32c
using DataFrames
using Tables
using AWSS3
using Pluto

struct TestType
a::Any
Expand Down

2 comments on commit 198c17f

@haberdashPI
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JuliaRegistrator
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration pull request created: JuliaRegistries/General/98604

Tip: Release Notes

Did you know you can add release notes too? Just add markdown formatted text underneath the comment after the text
"Release notes:" and it will be added to the registry PR, and if TagBot is installed it will also be added to the
release that TagBot creates. i.e.

@JuliaRegistrator register

Release notes:

## Breaking changes

- blah

To add them here just re-invoke and the PR will be updated.

Tagging

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:

git tag -a v1.1.4 -m "<description of version>" 198c17f8c06c1e7d278c1d09c7aa97941f89bccc
git push origin v1.1.4

Please sign in to comment.