Our aim is to take inspiration from (and further develop) research into the cognitive science of conscious processing, to deliver greatly enhanced AI, with abilities observed in humans thanks to high-level reasoning leading among other things to greater abilities to face unusual or novel situations by reasoning, compositionally reusing existing knowledge and being able to communicate about that.

At the same time, new AI models could drive new insights into the neural mechanisms underlying conscious processing, instantiating a virtuous circle. Machine learning procedures have the advantage that they can be tested for their effective learning abilities, and in our case in terms of out-of-distribution abilities or in the context of causal environments changing due to interventions.

Imagine that you are driving a car from your office, back home. You do not need to pay a lot of attention to the road and you can talk to the passenger. Now imagine encountering a road block due to construction: you have to pay more attention, you have to be on lookout for new information, if the passengers starts talking to you, then you may have to tell the person, "please let me drive".

It is interesting to consider that when humans are confronted with a new situation, very commonly they require their conscious attention. In the driving example, when there is a road block you need to pay attention in order to think through what to do next, and you probably don't want to be disturbed, because your conscious attention can only focus on one thing at a time.

There is something in the way humans process information which seems to be different – both functionally and in terms of the neural signature in the brain – when we deal with conscious processing and novel situations (changes in the distribution) which require our conscious attention, compared to our habitual routines.

In those novel situations, we generally have to think, focus and attend to specific elements of our perception, actions or memories and sometimes inhibit our reactions based on context (e.g., facing new traffic rules or a road block). Why would humans have evolved to deal with such an ability with changes in distribution? Maybe simply because life experience is highly non-stationary.

System 1 and System 2. Cognitive scientists distinguish habitual versus controlled processing, where the former correspond to default behaviors, whereas the latter require attention and mental effort. Daniel Kahneman introduced the framework of fast and slow thinking, and describes the system 1 and system 2 styles of processing in our brain.

Some tasks can be achieved using only system 1 abilities whereas others also require system 2 and conscious processing. There are also notions of explicit (verbalizable) knowledge and explicit processing (which roughly correspond to system 2) and implicit (intuitive) knowledge and corresponding system 1 neural computations.

The default (or unconscious) processing of system 1 can take place very rapidly (as fast as about 100ms) and mobilize many areas of the brain in parallel. On the other hand, controlled (or conscious) processing involves a sequence of thoughts, usually verbalizable, typically requiring seconds to achieve.

Whereas we can act in fast and precise habitual ways without having to think consciously, the reverse is not true: controlled processing (i.e., system 2 cognition) generally requires unconscious processing to perform much of its work. It is as if the conscious part of the computation was just the top-level program and the tip of the iceberg.

Yet, it seems to be a very powerful one, which makes it possible for us to solve new problems creatively by recombining old pieces of knowledge, to reason, to imagine explanations and future outcomes, to plan and to apply or discover causal dependencies. It is also at that level that we interface with other humans through natural language.

And when a word refers to a complex concept for which we do not have a clear verbalizable and precise explanation (like how we manage to drive our bike), we can still name it and reason about how it relates with other pieces of knowledge, etc.

Even imagination and planning (which are hallmarks of system 2 abilities) require system 1 computations to sample candidate solutions to a problem (from a possibly astronomical number, which we never have to explicitly examine).

Our brain seems to thus harbour two very different types of knowledge: the kind we can explicitly reason about and communicate verbally (system 2 knowledge) and the kind that is intuitive and implicit (system 1 knowledge). When we learn something new, it typically starts being represented explicitly, and then as we practice it more, it may migrate to a different, implicit form.

When you learn the grammar of a new language, you may be given some set of rules, which you try to apply on the fly, but that requires a lot of effort and is done painfully slowly. As you practice this skill, it can gradually migrate to a habitual form, you make less mistakes (for the common cases), you can read / translate / write more fluently, and you may even eventually forget the original rules.

When a new rule is introduced, you may have to move back some of that processing to system 2 computation to avoid inconsistencies. It looks as if one of the key roles of conscious processing is to integrate different sources of knowledge (from perception and memory) in a coherent way.

The above division of labour is at the heart of the cognitive neuroscience Global Workspace Theory (or GWT) from Baars and its extension, the Global Neuronal Workspace model. The GWT suggests an architecture allowing specialist components to interact.

The key claim of the GWT is the existence of a shared representation—sometimes called a blackboard, sometimes a workspace—that can be modified by any selected specialist and whose content is broadcast to all specialists. That selection is based on a form of attention and can correspond to dynamically selecting (based on the input) a module or a few modules in a modular neural net that are most appropriate for a particular context and task.

The basic idea of deep learning frameworks inspired by the GWT is to explore a similar communication and coordination scheme for a neural net comprising of distinct modules. The GWT theory posits that conscious processing revolves around a communication bottleneck between selected parts of the brain which are called upon when addressing a current task.

There is a threshold of relevance beyond which information which was previously handled unconsciously gains access to this bottleneck, instantiated in a working memory. When that happens, that information is broadcast to the whole brain, allowing the different relevant parts of it to synchronize, forcing each module to learn to exchange with other modules in a way that allows swapping one module for another as source or destination of communicated content, i.e., with a shared "language".

These shared representations can be interpreted by many other modules. This gives rise to semantic representations that are not tied to a particular modality but can be triggered by any of the sensory channels. As we argue throughout this paper, this makes it possible to flexibly obtain new combinations of pieces of knowledge, enabling a compositional advantage aligned with the needs of systematic generalization out-of-distribution.

The GWT suggests a fleeting memory capacity in which only one consistent content can be dominant at any given moment, which suggests a sharper form of attention than the soft attention currently dominant in deep learning and described below. Attention is about sequentially selecting what computation to perform on what quantities.

Let us consider a machine translation task from English to French. To obtain a good translation generating the next French word, we normally focus especially on the "right" few words in the source English sentence that may be relevant to do the translation. This is the motivation that stimulated our work on content-based soft self-attention but may also be at the heart of conscious processing in humans as well as in future deep learning systems with both system 1 and system 2 abilities.

Content-Based Soft Attention. Soft attention forms a soft selection of one element (or multiple elements) from a set of elements at the previous level of computations, i.e we are taking a convex combination of the values of the elements at the previous level. These convex weights are coming from a softmax that is conditioned on how each of the elements' key vector matches some query vector.

In a way, attention is parallel, because computing these attention weights considers all the possible elements in some set, yielding a score for each of them, to decide which of them are going to receive the most attention. With stochastic hard-attention one samples from a distribution over elements to choose the attended content, whereas with soft attention one mixes these contents with different positive convex weights.

Content-based attention also introduces a non-local inductive bias into neural network processing, allowing it to infer long-range dependencies that might be difficult to discern if computations are biased by local proximity. Attention is at the heart of the current state-of-the-art NLP systems and is the standard tool for memory-augmented neural networks.

Attention and memory can also help address the problem of credit assignment through long-term dependencies by creating dynamic skip connections through time (i.e., a memory access) which unlock the problems of vanishing gradients and learning long-term dependencies.

Attention also transforms neural networks from machines that are processing vectors (e.g., each layer of a deep net), to machines that are processing sets, more particularly sets of key/value pairs, as with Transformers.

Soft attention uses the product of a query (or read key) represented as a matrix Q of dimensionality Nᵣ×d, with d the dimension of each key, with a set of Nₒ objects each associated with a key (or write-key) as a row in matrix Kᵀ (Nₒ×d), and after normalization with a softmax yields outputs in the convex hull of the values (or write-values) Vᵢ (row i of matrix V). The result is Attention(Q,K,V) = softmax(QKᵀ/√d)V.

With soft attention, one obtains a convex combination of the values in the rows of V, whereas stochastic hard attention would sample one of the value vectors with probability equal to that weight. If the soft attention is focused on one element for a particular row (i.e., the softmax is saturated), we get deterministic hard attention: only one of the objects is selected and its value copied to row j of the result.

Note that the d dimensions in the key can be split into heads which then have their attention matrix and write values computed separately. Note that hard attention is more biologically plausible (we only see one interpretation of the Necker cube at once, and have one thought at a time) but soft attention enables end-to-end training and has been the most commonly used in deep learning architectures up to now, e.g., with transformers.

However, there is recent evidence that if the communication bottleneck is discretized, better OOD generalization is observed, may be because the resulting simpler lingua franca would make it easier to swap one module for another in the attention-controlled communication between modules.

Attention as dynamic connections. We can think of attention as a way to create a dynamic connection between different blocks of computation, whereas in the traditional neural net setting, connections are fixed. On the receiving end (downstream module) of an attention-selected input, it is difficult to tell from the selected value vector from where it comes (among the selected upstream modules which competed for attention).

To resolve this, it would make sense that the information being propagated along with the selected value includes a notion of key or type or name, i.e., of where the information comes from, hence creating a form of indirection (a reference to where the information came from, which can be passed to downstream computations).

Attention implements variable binding. When the inputs and outputs of each of the modules are a set of objects or entities (each associated with a key and value vector), we have a generic object-processing machine which can operate on "variables" in a sense analogous to variables in a programming language: as interchangeable arguments of functions.

Because each object has a key embedding (which one can understand both as a name and as a type), the same computation can be applied to any variable which fits an expected "distributed type" (specified by a query vector). Each attention head then corresponds to a typed argument of the function computed by the factor.

When the key of an object matches the query of head k, it can be used as the k-th input vector argument for the desired computation. Whereas in regular neural networks (without attention) neurons operate on fixed input variables (the neurons which are feeding them from the previous layer), the key-value attention mechanisms make it possible to select on the fly which variable instance (i.e. which entity or object) is going to be used as input for each of the arguments of some computation, with a different set of query embeddings for each argument head.

The computations performed on the selected inputs can be seen as functions with typed arguments, and attention is used to bind their formal argument to the selected input, albeit in a soft differentiable way (that mixes multiple possibilities) in the case of soft attention. Type constraints have already been found useful in identification for causal discovery.

Current attention-based neural network already implement key-value-query soft attention mechanism (as above). What is missing is an ability to handle discrete types, hard (but possibly stochastic) choices of arguments, and more powerful inference machinery that uses not just type matching but is also able to reason about which modules and variables should be composed in a given context.

From a computational perspective, one hypothesis about the dynamics of communication between different modules is that different modules generally act in parallel and receive inputs from other modules. However, when they do need to communicate information with another arbitrary module, the information has to go through a routing bottleneck (the global workspace) controlled by an attention mechanism.

Because so few elements can be put in coherence at each step of the GWT selection, the inference process generally requires several such steps, leading to the highly sequential nature of system 2 computation (compared with the highly parallel nature of system 1 computation).

The contents which have thus been selected are essentially the only ones which can be committed to memory, starting with short-term memory. Working memory refers to the ability of the brain to operate on a few recently accessed elements (i.e., those in short-term memory).

These elements can be remembered and have a heavy influence on the next thought, action or perception, as well as on what learning focuses on, possibly playing a role similar to desired outputs, goals or targets in supervised learning for system 1 computations.

Partial State. From an RL perspective, it is interesting to note that if the GWT holds an important part of the state (including imagined future states, when planning), it does not describe all the aspects of the environment, only a handful of them, as already explored in the RL literature. This is different from standard RL approaches where the input (or the sequence of past inputs) is mapped to a fixed-size (estimated and latent) state vector.

The GWT suggests instead that, besides long-term memory content (which mostly does not change), the rapidly changing state should be seen as a very small set of entities (e.g., objects or particular attributes of objects, and their relation), with an information content similar to that of a single sentence.

This suggests neural net architectures in which very few modules and specific (variable, value) pairs are selected at every inference step, based on those that were recently selected, the current sensory input and the current contents of memory (which can also compete for write-access to the workspace).

Only the selected modules would be under pressure to adapt when the result of the combination needs to be tuned, leading to selective adaptation similar to that explored by (Bengio et al., 2019) (see Section 4.3 above) where just a few relevant modules need to adapt to a change in distribution.

System 2 to System 1 Consolidation. As an agent, a human being is facing frequent changes because of their actions or the actions of other agents in the environment. Most of the time, humans follow their habitual policy, but tend to use system 2 cognition when having to deal with unfamiliar settings. It allows humans to generalize out-of-distribution in surprisingly powerful ways, and understanding this style of processing would help us build these abilities in AI as well.

This is illustrated with our early example of driving in an area with unfamiliar traffic regulations, which requires full conscious attention (Section 3.1). This observation suggests that system 2 cognition is crucial in order to achieve the kind of flexibility and robustness to changes in distribution required in the natural world.

It looks like current deep learning systems are fairly good at perception and system 1 tasks. They can rapidly produce an answer (if you have parallel computing like that of GPUs) through a complex calculation which is difficult (or impossible) to dissect into the application of a few simple verbalizable operations. They require a lot of practice to learn and can become razor sharp good at the kinds of data they are trained on.

On the other hand, humans enjoy system 2 abilities which permit fast learning (I can tell you a new rule in one sentence and you do not have to practice it in order to be able to apply it, albeit awkwardly and slowly at first) and systematic generalization, both of which should be important characteristics of the next generation of deep learning systems.

Between-Modules Interlingua and Communication Topology. If the brain is composed of different modules, it is interesting to think about what code or lingua franca is used to communicate between them, such that it can lead to interchangeable pieces of knowledge being dynamically selected and combined to solve a new problem.

The GWT bottleneck may thus also play a role in forcing the emergence of such a lingua franca: the same information received by module A (e.g. "there is a fire") can come from any other module (say B, which detected a fire by smell, or C which detected a fire by sight). Hence B and C need to use a compatible representation which is broadcast via the GWT bottleneck for A's use.

Again, we see the crucial importance of attention mechanisms to force the emergence of shared representations and indirect references exchanged between the modules via the conscious bottleneck. However, the GWT bottleneck is by far not the only way for modules to communicate with each other.

Regarding the topology of the communication channels between modules, it is known that modules in the brain satisfy some spatial topology such that the computation is not all-to-all between all the modules. It is plausible that the brain uses both fixed local or spatially nearby connections as well as the global broadcasting system with top-down influence.

We also know that there are hierarchical communication routes in the visual cortex (on the path from pixels to object recognition), and we know how successful that has been in computer vision with convnets. Combining these different kinds of inter-module communication modalities in deep network thus seems well advised as well:

(1) Modules which are near each other in the brain layout can probably communicate directly without the need to clog the global broadcast channel (and this would not be reportable consciously). (2) Modules which are arbitrarily far from each other in the spatial layout of the brain can exchange information via the global workspace, following the theatre analogy of Baars' GWT.

The other advantage of this communication route is of course the exchangeability of the sources of information being broadcast, which we hypothesize leads to better systematic generalization.

The role of working memory in the GWT is not just as a communication buffer. It also serves as a blackboard (or analogously the "registers" in CPUs) where operations can be done locally to improve coherence. This enables a coherence-seeking mechanism: the different modules (especially the active ones) should adopt a configuration of their internal variables (and especially the more abstract entities they communicate to other modules) which is consistent with what other active modules "believe".

It is possible, that a large part of the functional role of conscious processing is for that purpose, which is consistent with the view of the working memory as a central element of the inference machinery seeking to obtain coherent configurations of the variables interacting according to some piece of knowledge (such as a factor of the factor graph, a causal dependency).