Pinned Loading

1. mini-llm-inference-engine mini-llm-inference-engine Public

   A pure Transformer inference engine written in C++ from scratch, with no dependencies on PyTorch, TensorFlow, or any other ML framework. It implements the entire modern LLM architecture (RoPE, RMSN…

   C++ 9 1

2. hardware-aware-llm-runtime hardware-aware-llm-runtime Public

   Hardware-calibrated LLM inference performance model using Roofline theory and analytical batch optimization.

   C++

3. llm-runtime-simulator llm-runtime-simulator Public

   Systems-level simulation of LLM serving runtime including paged KV cache, priority scheduling, SLA modeling, and adaptive QoS control.

   C++

4. llm-serving-scheduler-engine llm-serving-scheduler-engine Public

   Discrete-event simulation engine modeling LLM serving under stochastic load: dynamic batching, SLA-aware admission control, and horizontally scalable multi-GPU (M/M/k) architecture.

   C++

5. sm75-tensorcore-microkernel sm75-tensorcore-microkernel Public

   Instruction-level Tensor Core micro-kernel engineering on SM75 (RTX 2070), including PTX manual emission, shared-memory staging, and hardware-aware auto-tuning.

   Cuda

6. llm-serving-trace-replay llm-serving-trace-replay Public

   Trace-driven discrete-event LLM serving simulator with SLA-aware GPU cluster sizing and prefill/decode resource optimization.

   Python