TKintegers K5: 3.21× prefill and 3.10× decode throughput vs TorchAO FP8 on NVIDIA L4 [R] [P]
I built TKintegers, a balanced-ternary weight representation, converter, and GPU inference implementation.
Reddit - r/ML2 min read
1 story
I built TKintegers, a balanced-ternary weight representation, converter, and GPU inference implementation.
Showing 1 of 1 stories
You're all caught up.
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts