Confused weight / input with gluon.SymbolBlock visualization?

[yifeim]

Hi,

I was using mxboard to visualize a symbolblock. It seems that instead of exposing inputs, the final graph exposed the weights and hid the inputs. The following codes generate the attached plots.

import mxnet as mx
import mxnet.symbol as S
from mxnet import gluon
from mxboard import SummaryWriter

def _signed_embed(vocab_size, num_embed, weight_initializer=mx.init.Uniform(0.1)):
    inputs = S.var('inputs')
    label  = S.var('label')
    embed_weight = S.var('embed_weight', shape=(vocab_size, num_embed),
                           init=weight_initializer)
    embed_outputs = S.broadcast_mul(
        embed_weight.take(inputs), # batch * ... * num_embed
        label.expand_dims(-1), # batch * ... * 1
    )
    return gluon.SymbolBlock(embed_outputs, [inputs, label])
with SummaryWriter(logdir='./logs') as sw:
    sw.add_graph(_signed_embed(100,12,)(S.var('inputs'), S.var('label')))

The output file (ignore the binaries)

^Lembed_weight^R^Dnull*T
^Eparam^RK^RI{ __init__ :  [\ uniform\ , {\ scale\ : 0.1}] ,  __shape__ :  (100, 12) }
^T
^Ltake8/inputs^R^Dnull
/
^Ktake8/take8^R^Dtake^Z^Lembed_weight^Z^Ltake8/inputs
^Z
^Rexpand_dims8/label^R^Dnull
W
^Yexpand_dims8/expand_dims8^R^Kexpand_dims^Z^Rexpand_dims8/label*^Y
^Eparam^R^P^R^N{ axis :  -1 }
V
^]broadcast_mul8/broadcast_mul8^R^Mbroadcast_mul^Z^Ktake8/take8^Z^Yexpand_dims8/expand_dims8"^B^Hd<95>vº.

Please let me know if this is indeed a bug or there are some other issues with my codes. My installed versions are mxboard==0.1.0 with mxnet-cu90==1.3.0b20180601.

[reminisce]

Can you double click 'take7to see ifdata` is included there? No matter what, this is not desirable. We fix it later.

[yifeim]

Appreciated! Yes, the data is included in the sub-block. I think the reason is that the hierarchy was somehow overwritten; hence I provided the binary file.

Another minor issues: gluon.block.name_scope currently uses '_' as separator, but tensorboard expects '/'. It would also be nice to allow custom prefix in gluon.block.name_scope to separate sub-block components. I did not remember having luck with the Prefix argument in the built-in blocks (could try again later). My current solution is mx.name.Prefix with my own '/'-separated hierarchy.

One last minor thing, when self.drop is reused, it appears as a self-loop in mxboard visualization. However, self.drop contains no trainable parameters and is reused in different parts. Instead of smart detection of reused blocks (which can be more challenging), it would be nice to have an option to simply hide a block from visualization.