Logs27: Mutual Information

Steps

• Wait ... we'll have to use conversations.db finally? because we need p_seq2seq(a| pi, qi)
• Fully understand MI
  • Read the original paper
  • Understand where pi, qi comes from in the training
    • pi let's eat curry
    • qi How about kokoichi
    • pi+1 sounds good
• Start always with small model.
• Have backward seq2seq training in place.
• Find old implementation of mutual information.