Project Description
As of today, there are a wide variety of AI coding frameworks available. These frameworks come with certain features and promises, however very few studies address their performance and usability in a systematic and objective manner. In this project student(s) are expected come up with an experiment design where these frameworks can be executed against pre-defined coding tasks and their performances can be evaluated. Deciding on what type of coding tasks to include (write a class, refactor a huge open source project, design and implement a new feature on an existing code base etc) and the performance evaluation metrics are the key pre-requisite tasks that needs to be done in the initial phases of the project.
If time permits comparing the capabilities of AI coding assistants with human users would be a nice-to-have outcome of this project.
Project Description
As of today, there are a wide variety of AI coding frameworks available. These frameworks come with certain features and promises, however very few studies address their performance and usability in a systematic and objective manner. In this project student(s) are expected come up with an experiment design where these frameworks can be executed against pre-defined coding tasks and their performances can be evaluated. Deciding on what type of coding tasks to include (write a class, refactor a huge open source project, design and implement a new feature on an existing code base etc) and the performance evaluation metrics are the key pre-requisite tasks that needs to be done in the initial phases of the project.
If time permits comparing the capabilities of AI coding assistants with human users would be a nice-to-have outcome of this project.