Could your please give a detailed explanation for the "rank classification"?

[A11en0]

Hi, thanks for your excellent job. I've read the paper the reviewed the code. I've encounter some issues as outlined following:

1. I'd appreciate a detailed explanation of how the "rank classification" is implemented. Could you please provide clarification on the code found at this link?

2. I'm curious about how the "rank classification" process influences the final results. Is it feasible to employ a direct generation approach, such as generating the label words and matching them against the true answer, as an alternative method?

[dptam]

1. The code listed is for training, where we do not use rank classification. Instead, we use the unlikelihood loss that tries to increase the loss for the incorrect labels (i.e. minimizing their probability). This is why there is a negative sign for cross_entropy on line 86 and we mask out the loss for the correct label on line 90.
2. It is possible to use a direct generation approach and match them against the true answer. Rank classification increases the accuracy of the model since the model only has to choose the correct choice from the list of choice, rather than having to generate the correct choice from the space of all possible outputs.