Issue:
Sae_vis returns activation texts in "groups" according to what quantile they're in, or if they're in the top activating group. The problem is sometimes it will return an activation text in the wrong group. I'm able to reproduce an issue where it puts an activation text with max act of 2.88 into a group that is supposed to be range 0.000 to 0.578 (this is testing res-jb).
Code Details:
SaeVisData returns FeatureDatas. Each FeatureData has sequence_data (a SequenceMultiGroupData), which itself has seq_group_data (an array of SequenceGroupData). The SequenceGroupData has a title, which contains the Activation Group information (eg "INTERVAL min_interval to max_interval CONTAINS percent%"). In this case, the top activating token of an activation text is outside of the "INTERVAL min_interval to max_interval".
Reproduction:
poetry shell
poetry neuronpedia.py generate
source set id: [enter any source set name]
SAE path: [enter path to the 0-res-jb SAE]
sparsity threshold: -5
features per batch: 20
batches to sample from: 4096
prompts to select from: 24576
resume from: 1
Example incorrect output:
The attached example.json is 0-res-jb, feature index 18, and I've removed all activations except the bugged activation. Line 308 and 309 show a "binMin" (group min_interval) of 0 and "binMax" of 0.578, but line 485 and 570 show that this text has a max activating token of 2.88. I also manually tested the text to ensure that the max activating value of 2.88 is indeed correct.
example.json
Issue:
Sae_vis returns activation texts in "groups" according to what quantile they're in, or if they're in the top activating group. The problem is sometimes it will return an activation text in the wrong group. I'm able to reproduce an issue where it puts an activation text with max act of 2.88 into a group that is supposed to be range 0.000 to 0.578 (this is testing
res-jb).Code Details:
SaeVisDatareturnsFeatureDatas. EachFeatureDatahassequence_data(aSequenceMultiGroupData), which itself hasseq_group_data(an array ofSequenceGroupData). TheSequenceGroupDatahas atitle, which contains the Activation Group information (eg "INTERVAL min_interval to max_interval CONTAINS percent%"). In this case, the top activating token of an activation text is outside of the "INTERVAL min_interval to max_interval".Reproduction:
Example incorrect output:
The attached
example.jsonis 0-res-jb, feature index 18, and I've removed all activations except the bugged activation. Line 308 and 309 show a "binMin" (group min_interval) of 0 and "binMax" of 0.578, but line 485 and 570 show that this text has a max activating token of 2.88. I also manually tested the text to ensure that the max activating value of 2.88 is indeed correct.example.json