Any equivalency to torch.nn.Parameter?

[liuyxpp]

I've asked in Discourse but without luck. Sorry to ask it here. I cannot find any doc about this.

BTW, I am trying to reproduce the SegRNN Pytorch model using Lux.jl, which uses torch.nn.Parameter. The source code of SegRNN implementation is here: https://github.com/hughxx/tsf-new-paper-taste/blob/master/models/SegRNN.py.

If there is no equivalent to torch.nn.Parameter, can we still implement SegRNN in Lux.jl? Thanks!

[avik-pal]

Pytorch needs nn.Parameter since it needs to differentiate "tracked" and non-"tracked" arrays. If you want a strict equivalence, Tracker.TrackedArray does the same thing.

Regarding how to do it in Lux, you would want to create a custom layer and define initialparameters https://lux.csail.mit.edu/dev/manual/interface#layer-interface which returns a pos_emb and other such things. Then using Zygote you can train them.

[liuyxpp]

That's neat! I will give it a try.