1
Show HN: KV-psi, using Linux PSI to to trim an LLM KV cache
利用Linux系统压力信息动态修剪大模型KV缓存，专为边缘设备优化内存使用
I thought it'd be interesting to use Linux PSI (Pressure Stall Information) for an LLM runtime to trim the KV cache. This is mainly useful imo for edg…