Skip to content

Commit

Permalink
added some links to roadmap
Browse files Browse the repository at this point in the history
  • Loading branch information
MartinuzziFrancesco committed Dec 8, 2024
1 parent cc0823b commit 4a04279
Showing 1 changed file with 13 additions and 6 deletions.
19 changes: 13 additions & 6 deletions docs/src/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,21 @@
This page documents some planned work for RecurrentLayers.jl.
Future work for this library includes additional cells such as:

- Unitary recurrent neural networks
- Linear recurrent neural networks
- Quasi recurrent neural networks
- FastRNNs and FastGRUs (current focus) [arxiv](https://arxiv.org/abs/1901.02358)
- Unitary recurrent neural networks [arxiv](https://arxiv.org/abs/1611.00035)
- Modern recurrent neural networks such as [LRU](https://arxiv.org/abs/2303.06349)
and [minLSTM/minGRU](https://arxiv.org/abs/2410.01201)
- Quasi recurrent neural networks [arxiv](https://arxiv.org/abs/1611.01576)

Additionally, some cell-independent architectures are also planned,
that expand the ability of recurrent architectures and could theoretically take
any cell:

- Clockwork rnns
- Segment rnn
- Fast-Slow rnns
- Clockwork rnns [arxiv](https://arxiv.org/abs/1402.3511)
- Phased rnns [arxiv](https://arxiv.org/abs/1610.09513)
- Segment rnn [arxiv](https://arxiv.org/abs/2308.11200)
- Fast-Slow rnns [arxiv](https://arxiv.org/abs/1705.08639)

An implementation of these ideally would be,
for example `FastSlow(RNNCell, input_size => hidden_size)`.
More details on this soon!

0 comments on commit 4a04279

Please sign in to comment.