Toward Compositional Behavior

These are the survey questions (although not shown in the original survey presentation) from Toward Compositional Behavior in Neural Models: A Survey of Current Views. Take it yourself and see which cluster aligns with your views!

N.B. This survey is for reference purposes only, we do not store your responses or any other data.

Defining CB

(CB) When a model receives an input I that humans conceive as composed of component parts, if the model produces correct outputs for those parts (in isolation or in other combinations), then it will also produce a correct output for I.

S0. (CB) is a satisfactory working definition of compositional behavior, an important aspect of compositional generalization.

Evaluating CB

S1. Current methods for analyzing the behavior of neural models are sufficient to assess whether a model is capable of compositional behavior (CB). For example, consider methods used to assess performance on datasets designed to probe specific aspects of compositional generalization, such as SCAN, COGs, CFQ, PCFG, Colors, etc.

S2. Current methods for analyzing the representations within neural models are sufficient: if a model is capable of compositional behavior (CB), these analysis methods can identify the model-internal mechanisms supporting this behavior. For example, consider diagnostic probing, visualization, learning interpretable approximations of the representation space, etc.

S3. Current methods for analyzing the processing within neural models are sufficient: if a model is capable of compositional behavior (CB), these analysis methods can identify the model-internal mechanisms supporting this behavior. For example, consider analysis of circuits / induction heads, causal interventions such as ablation, etc.

S4 Interpretable representations are necessary: we cannot evaluate whether a model is capable of compositional behavior (CB) unless we can identify human-interpretable parts within its representational structure.

S5. Interpretable processing is necessary: we cannot evaluate whether a model is capable of compositional behavior (CB) unless we can identify human-interpretable parts within its representational structure, and establish that the model uses these parts as expected during processing. That is to say, if we observe in compositional behavior that certain parts stand in particular relations to one another, we can confirm that those parts interact in similar — ideally isomorphic — ways during the procedure carried out by the model, at some level of description. For example, consider the conceptual roles discussed by Piantadosi and Hill (2022).

S6. External grounding is necessary: we cannot evaluate whether a model is capable of compositional behavior (CB) unless we can identify human-interpretable parts within its representational structure, and establish that these parts are grounded with respect to some model-external structure in the world.

Achieving CB

S7. Current neural models show a sufficient degree of compositional behavior (CB); we don’t need to assign high priority to further research on this topic.

S8. Current neural models do not show a sufficient degree of compositional behavior (CB), but this issue will likely be resolved as a byproduct of increasing model capacity (i.e.~larger models and/or larger datasets). In other words, scale will solve this problem, and we don't need additional interventions to improve compositional behavior.

S9. Current neural models do not show a sufficient degree of compositional behavior (CB), and some intervention is required, but model-external interventions — as opposed to the model-internal interventions considered in the next claim — are likely to satisfactorily resolve this problem. Examples of model-external interventions include prompt engineering; strategic manipulation or augmentation of training data; and auxiliary tasks during training, pre-training, or fine-tuning.

S10. Current neural models do not show a sufficient degree of compositional behavior (CB), and model-external interventions are unlikely to resolve this issue. Model-internal interventions or novel architectures, focused on model representations/processing/learning, will be necessary to solve the problem.

S11. Current neural models do not show a sufficient degree of compositional behavior (CB), and model-internal interventions or novel architectures that incorporate explicit discrete symbolic computation (e.g., program synthesis) will be necessary to solve the problem.

Logical Geography of CB

Projected locations of survey respondents and clusters, calculated using principal component analysis
YOU