Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reconciler: Fix Status JSON marshalling #63

Merged
merged 1 commit into from
Oct 31, 2024
Merged

Conversation

joamaki
Copy link
Contributor

@joamaki joamaki commented Oct 31, 2024

43587d4 broke JSON marshalling of Status by introducing a separate statusJSON type and Unmarshal{JSON,YAML} methods, but didn't introduce the Marshal{JSON,YAML} methods, which caused dictionary key mismatches as Status struct was used to marshal instead of statusJSON.

Fix this by removing the whole statusJSON construct and add json and yaml tags to Status. There is no need for filling in the id field when unmarshalling Status as this is completely internal to the reconciler.

Fixes: 43587d4 ("reconciler: Implement tests using scripttest")

43587d4 broke JSON marshalling of Status by introducing a separate statusJSON
type and Unmarshal{JSON,YAML} methods, but didn't introduce the Marshal{JSON,YAML}
methods, which caused dictionary key mismatches as Status struct was used to marshal
instead of statusJSON.

Fix this by removing the whole statusJSON construct and add json and yaml tags to Status.
There is no need for filling in the `id` field when unmarshalling Status as this is completely
internal to the reconciler.

Fixes: 43587d4 ("reconciler: Implement tests using scripttest")

Signed-off-by: Jussi Maki <[email protected]>
@joamaki joamaki requested a review from gandro October 31, 2024 13:02
@joamaki joamaki requested a review from a team as a code owner October 31, 2024 13:02
@joamaki joamaki requested review from bimmlerd and removed request for a team October 31, 2024 13:02
Copy link

$ make test
go: downloading golang.org/x/sys v0.17.0
go: downloading golang.org/x/tools v0.17.0
go: downloading golang.org/x/exp v0.0.0-20240119083558-1b970713d09a
go: downloading github.com/fsnotify/fsnotify v1.7.0
go: downloading github.com/sagikazarmark/slog-shim v0.1.0
go: downloading github.com/spf13/afero v1.11.0
go: downloading github.com/spf13/cast v1.6.0
go: downloading github.com/subosito/gotenv v1.6.0
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading gopkg.in/ini.v1 v1.67.0
go: downloading github.com/magiconair/properties v1.8.7
go: downloading github.com/pelletier/go-toml/v2 v2.1.0
go: downloading golang.org/x/text v0.14.0
	github.com/cilium/statedb/reconciler/benchmark		coverage: 0.0% of statements
	github.com/cilium/statedb/reconciler/example		coverage: 0.0% of statements
ok  	github.com/cilium/statedb	27.192s	coverage: 82.6% of statements
ok  	github.com/cilium/statedb/index	0.005s	coverage: 28.7% of statements
ok  	github.com/cilium/statedb/internal	0.011s	coverage: 46.7% of statements
ok  	github.com/cilium/statedb/part	2.748s	coverage: 82.9% of statements
ok  	github.com/cilium/statedb/reconciler	0.208s	coverage: 89.0% of statements
-----
$ make bench
go test ./... -bench . -benchmem -test.run xxx
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkDB_WriteTxn_1-4                    	  456190	      2668 ns/op	    374779 objects/sec	    2816 B/op	      32 allocs/op
BenchmarkDB_WriteTxn_10-4                   	 1228448	       976.1 ns/op	   1024510 objects/sec	     743 B/op	      10 allocs/op
BenchmarkDB_WriteTxn_100-4                  	 1534062	       772.0 ns/op	   1295375 objects/sec	     600 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_1000-4                 	 1502379	       799.4 ns/op	   1250874 objects/sec	     550 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_100_SecondaryIndex-4   	  495452	      2395 ns/op	    417503 objects/sec	    1506 B/op	      37 allocs/op
BenchmarkDB_Modify-4                        	    1290	    926772 ns/op	   1079016 objects/sec	  774911 B/op	    8462 allocs/op
BenchmarkDB_GetInsert-4                     	    1206	    999472 ns/op	   1000529 objects/sec	  757856 B/op	    8462 allocs/op
BenchmarkDB_RandomInsert-4                  	    2365	    497197 ns/op	   2011275 objects/sec	  402298 B/op	    7098 allocs/op
BenchmarkDB_RandomReplace-4                 	     320	   3724302 ns/op	    268507 objects/sec	 2355675 B/op	   48570 allocs/op
BenchmarkDB_SequentialInsert-4              	    1549	    798087 ns/op	   1252996 objects/sec	  550137 B/op	    7284 allocs/op
BenchmarkDB_Changes_Baseline-4              	    1273	    931399 ns/op	   1073656 objects/sec	  553118 B/op	   10252 allocs/op
BenchmarkDB_Changes-4                       	     709	   1700181 ns/op	    588173 objects/sec	  993389 B/op	   14494 allocs/op
BenchmarkDB_RandomLookup-4                  	   21186	     55434 ns/op	  18039350 objects/sec	     160 B/op	       1 allocs/op
BenchmarkDB_SequentialLookup-4              	   27534	     43723 ns/op	  22871143 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_Prefix_SecondaryIndex-4         	    7612	    147099 ns/op	   6798135 objects/sec	   93894 B/op	    1044 allocs/op
BenchmarkDB_FullIteration_All-4             	     898	   1506652 ns/op	  66372448 objects/sec	     480 B/op	      12 allocs/op
BenchmarkDB_FullIteration_Get-4             	     218	   5418292 ns/op	  18456040 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_PropagationDelay-4              	  488198	      2343 ns/op	        20.00 50th_µs	        24.00 90th_µs	        89.00 99th_µs	    1590 B/op	      24 allocs/op
PASS
ok  	github.com/cilium/statedb	27.919s
PASS
ok  	github.com/cilium/statedb/index	0.003s
PASS
ok  	github.com/cilium/statedb/internal	0.004s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/part
cpu: AMD EPYC 7763 64-Core Processor                
Benchmark_Insert_RootOnlyWatch-4    	    8647	    131383 ns/op	   7611316 objects/sec	  104169 B/op	    2041 allocs/op
Benchmark_Insert-4                  	    6363	    181171 ns/op	   5519650 objects/sec	  219068 B/op	    3065 allocs/op
Benchmark_Modify-4                  	    8197	    142662 ns/op	   7009574 objects/sec	  212720 B/op	    1205 allocs/op
Benchmark_GetInsert-4               	    6745	    171610 ns/op	   5827187 objects/sec	  212568 B/op	    1204 allocs/op
Benchmark_Replace-4                 	27011292	        45.17 ns/op	  22136409 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Replace_RootOnlyWatch-4   	27062319	        44.72 ns/op	  22359010 objects/sec	       0 B/op	       0 allocs/op
Benchmark_txn_1-4                   	 3065887	       392.7 ns/op	   2546460 objects/sec	     448 B/op	       7 allocs/op
Benchmark_txn_10-4                  	 7500230	       163.3 ns/op	   6123331 objects/sec	     154 B/op	       2 allocs/op
Benchmark_txn_100-4                 	 7666651	       143.6 ns/op	   6965668 objects/sec	     224 B/op	       2 allocs/op
Benchmark_txn_1000-4                	 7297327	       161.8 ns/op	   6178731 objects/sec	     216 B/op	       2 allocs/op
Benchmark_txn_delete_1-4            	 3203024	       373.3 ns/op	   2678699 objects/sec	     856 B/op	       6 allocs/op
Benchmark_txn_delete_10-4           	 8405701	       142.2 ns/op	   7029922 objects/sec	     132 B/op	       1 allocs/op
Benchmark_txn_delete_100-4          	10378332	       114.4 ns/op	   8745034 objects/sec	      60 B/op	       1 allocs/op
Benchmark_txn_delete_1000-4         	10940889	       108.2 ns/op	   9246180 objects/sec	      26 B/op	       1 allocs/op
Benchmark_Get-4                     	   38112	     31602 ns/op	  31643748 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterate-4                 	  173079	      6878 ns/op	 145393494 objects/sec	      80 B/op	       3 allocs/op
Benchmark_Hashmap_Insert-4          	   15936	     75372 ns/op	  13267613 objects/sec	   86554 B/op	      64 allocs/op
Benchmark_Hashmap_Get_Uint64-4      	  154456	      7860 ns/op	 127221244 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Hashmap_Get_Bytes-4       	  147664	      8065 ns/op	 123990909 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Uint64Map_Random-4        	    1376	    860406 ns/op	   1162243 items/sec	 2698779 B/op	    9016 allocs/op
Benchmark_Uint64Map_Sequential-4    	    1501	    797123 ns/op	   1254513 items/sec	 2492405 B/op	    9749 allocs/op
PASS
ok  	github.com/cilium/statedb/part	28.229s
PASS
ok  	github.com/cilium/statedb/reconciler	0.004s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go run ./reconciler/benchmark -quiet
1000000 objects reconciled in 3.00 seconds (batch size 1000)
Throughput 333567.11 objects per second
Allocated 6011335 objects, 424784kB bytes, 541360kB bytes still in use

Copy link
Member

@bimmlerd bimmlerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Kind StatusKind
UpdatedAt time.Time
Error string
Kind StatusKind `json:"kind" yaml:"kind"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you even need these struct tags?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not necessarily. My thinking is that this makes the struct nicer to use in scripttest.

@joamaki joamaki merged commit 132edd6 into main Oct 31, 2024
1 check passed
@joamaki joamaki deleted the pr/joamaki/fix-status-json branch October 31, 2024 13:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants