Detailed description of the requested feature
After the puzzletron prunes the model, we want a repair phase that will run distillation from the teacher (original model, but possibly any other model) to the pruned model
Timeline
Work in progress. Requires "general" support of heterogeneous models in Mcore. The idea is to patch the mbridge-mcore model creation using the block_configs from anymodel.
This also requires some updates (PRs) to MBridge code as well
Detailed description of the requested feature
After the puzzletron prunes the model, we want a repair phase that will run distillation from the teacher (original model, but possibly any other model) to the pruned model
Timeline
Work in progress. Requires "general" support of heterogeneous models in Mcore. The idea is to patch the mbridge-mcore model creation using the block_configs from anymodel.
This also requires some updates (PRs) to MBridge code as well