Issues

• kv cache: integrate nixl for kv cache transfer

  vectorch-ai/ScaleLLM#457 · Filter by author guocuimi opened on May 1, 2025

• kv cache: cache aware router

  vectorch-ai/ScaleLLM#456 · Filter by author guocuimi opened on May 1, 2025

• kv cache: fp8/int8 kv cache support

  vectorch-ai/ScaleLLM#455 · Filter by author guocuimi opened on May 1, 2025

• model: add qwen new models

  vectorch-ai/ScaleLLM#454 · Filter by author guocuimi opened on May 1, 2025

• model: add deepseek new models

  vectorch-ai/ScaleLLM#453 · Filter by author guocuimi opened on May 1, 2025

• kernel: tile scheduling

  vectorch-ai/ScaleLLM#452 · Filter by author guocuimi opened on May 1, 2025

• kernel: attention kernel for sm_75/70

  vectorch-ai/ScaleLLM#451 · Filter by author guocuimi opened on May 1, 2025

• feat: disaggregate Prefill and Decoding

  vectorch-ai/ScaleLLM#450 · Filter by author guocuimi opened on May 1, 2025

• kernel: deepep integration

  vectorch-ai/ScaleLLM#449 · Filter by author guocuimi opened on May 1, 2025

• kernel: grouped gemm for MOE quantization

  vectorch-ai/ScaleLLM#448 · Filter by author guocuimi opened on May 1, 2025

• kernel: attention kernel for sm_120

  vectorch-ai/ScaleLLM#447 · Filter by author guocuimi opened on May 1, 2025

• kernel: attention kernel for sm_100/101

  vectorch-ai/ScaleLLM#446 · Filter by author guocuimi opened on May 1, 2025

• kernel: attention kernel for sm_90

  vectorch-ai/ScaleLLM#445 · Filter by author guocuimi opened on May 1, 2025

• feat: EP support for Deepseek models

  vectorch-ai/ScaleLLM#444 · Filter by author guocuimi opened on May 1, 2025

• kernel: Grouped GEMM for MOE

  vectorch-ai/ScaleLLM#443 · Filter by author guocuimi opened on May 1, 2025

• ScaleAttention: a custom CUDA kernel, optimized for inference.

  performanceroadmap
  vectorch-ai/ScaleLLM#356 · Filter by author guocuimi opened on Jan 1, 2025

• RuntimeError: Timed out

  vectorch-ai/ScaleLLM#310 · Filter by author spongxin opened on Aug 16, 2024
  1

• Is there any plans to support Int8 weight quant ?

  vectorch-ai/ScaleLLM#276 · Filter by author sitabulaixizawaluduo opened on Jul 18, 2024
  2

• [Issue] Qwen-14B-Chat init fail and performance issue.

  vectorch-ai/ScaleLLM#275 · Filter by author liutongxuan opened on Jul 16, 2024
  2

• LoRA: QLoRA/S-LoRA: Serving thousands of LoRA adapters

  vectorch-ai/ScaleLLM#166 · Filter by author guocuimi opened on Apr 28, 2024

• Introducing the Mamba model

  vectorch-ai/ScaleLLM#165 · Filter by author guocuimi opened on Apr 28, 2024

• Introducing a ring attention mechanism for handling long contexts

  vectorch-ai/ScaleLLM#164 · Filter by author guocuimi opened on Apr 28, 2024

• Quantization: Supporting FP8 for both models and KV caches

  vectorch-ai/ScaleLLM#163 · Filter by author guocuimi opened on Apr 28, 2024

• Enhancing documentation for improved usability

  vectorch-ai/ScaleLLM#162 · Filter by author guocuimi opened on Apr 28, 2024
  

• Exploring other chips such as TPU, etc.

  backlog
  vectorch-ai/ScaleLLM#160 · Filter by author guocuimi opened on Apr 28, 2024