BlinkDL/RWKV-LM
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
Growth
- Last 24 hours
- +2
- stars
- Last 7 days
- +11
- stars
- Last 30 days
- +55
- stars
- Total stars
- 14,750
- 14.7k a month ago
- Forks
- 1,023
- 0 contributors
Star history
Last 12 weeks · weekly cumulative stars · computed from public GitHub star events
Rank in its topics
| Topic | Today | This week | This month |
|---|---|---|---|
| Python | – | – | #594of 639 |
| LLM | – | – | #418of 431 |
| Machine learning | – | – | #150of 170 |
README
RWKV-LM
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
Upstar hasn’t fetched this README yet. Read it on GitHub.