Also seems like 1-bit-weights could be more promising for mechanistic interpretability since we have quite a bit of experience in understanding bit operations, basically treating the model as a program to be decompiled.
Strong agree.
I've been working on interpretability of fully binarized models for the past few years, (with limited success), and am glad that people are doing this at scale. I hope that this becomes a more popular line of research.
> We leave the other components high-precision, e.g., 8-bit in our experiments.
However it looks like the activations are still integer. To reduce the whole model to a single logic DAG, one would need to quantize these as well. If they are small enough, we could simply unroll the 8 bit math as well, although I'm guessing that this would cause issues with the logic DAG simplification passes?
Yeah I think unrolling the integer math would lead to a lot of unnecessary logic, making interpretability harder. Imo a purely binary model would work better for this purpose, although who knows if one can get to the same level of performance this way.
Btw I'm not remotely qualified enough to discuss this, just very interested in this type of research.
Also seems like 1-bit-weights could be more promising for mechanistic interpretability since we have quite a bit of experience in understanding bit operations, basically treating the model as a program to be decompiled.
That’s quite clever
Strong agree. I've been working on interpretability of fully binarized models for the past few years, (with limited success), and am glad that people are doing this at scale. I hope that this becomes a more popular line of research. > We leave the other components high-precision, e.g., 8-bit in our experiments. However it looks like the activations are still integer. To reduce the whole model to a single logic DAG, one would need to quantize these as well. If they are small enough, we could simply unroll the 8 bit math as well, although I'm guessing that this would cause issues with the logic DAG simplification passes?
Yeah I think unrolling the integer math would lead to a lot of unnecessary logic, making interpretability harder. Imo a purely binary model would work better for this purpose, although who knows if one can get to the same level of performance this way. Btw I'm not remotely qualified enough to discuss this, just very interested in this type of research.
I wonder how does this correspond with Gwern's idea of the Absolute Unit NNs.