Project Description
The main goal of this project is to evaluate the performance of vLLM (https://github.com/vllm-project/vllm) for LLM inference tasks. In order to achieve this, the student should read some related papers (The first one should be this -> https://arxiv.org/abs/2309.06180) and get her/his hand dirty with vLLM installations.
The ultimate aim is to create an experiment testbed where users can execute parametric and repeatable experiments for trying out different LLMs and reporting their performance.
Project Description
The main goal of this project is to evaluate the performance of vLLM (https://github.com/vllm-project/vllm) for LLM inference tasks. In order to achieve this, the student should read some related papers (The first one should be this -> https://arxiv.org/abs/2309.06180) and get her/his hand dirty with vLLM installations.
The ultimate aim is to create an experiment testbed where users can execute parametric and repeatable experiments for trying out different LLMs and reporting their performance.