This page will be heavily extended in the near future. For instance, I will document and contrast the listed systems very soon.
AI Platform Architectures
Airbnb Bighead
Airbnb presented Bighead both at Strata and at Data Council (where they go into a bit more detail into architectures of components, e.g. see Deep Thought at 22:04 and Zipline at 28:01). Here is my summary diagram of the system:

Flyte
Lyft Flyte’s architecture is described here and its component architecture has a dedicated page. Another illustration can be found here. Below is my attempt to sketch out the overall architecture:

IBM Fabric for Deep Learning (FfDL)
The Fabric for Deep Learning is documented both in its paper (figure 1) and on its Github page. Its architecture looks approximately like this:

This section will be extended over the next weeks. Please refer to the landscape section about AI Platforms in the meantime.
Neural Network Architectures
- DeepMind’s Alpha Systems: AlphaGo, AlphaGo Zero, Alpha Zero, Alpha Star, AlphaFold
- OpenAI Codex, CLIP (Github), DALL·E, Five
- AllenAI Aristo, Mosaic and Prior
- Facebook BlenderBot (uses poly-encoder transformer architecture – might explain bi-encoder → cross-encoder → poly-encoder)
- BERT, GPT, ELMo, see Google blog post
- Distributed RL – A3C, GORILA, APE-X, IMPALA, APPO
- MANNs – memory nets → end-to-end memory nets → NTMs → DNC
- Modular neural nets, routing networks, capsule networks, neuro-symbolic concept learner
- Embeddings word2vec → sentence2vec → doc2vec and graph embeddings like node2vec and specialized embeddings like hotel2vec
Cognitive Architectures
See Comparison of Cognitive Architectures (Wikipedia) and Comparative Table of Cognitive Architectures
- ACT-R, Wikipedia
- BDI
- Chunk Hierarchy and REtrieval STructures (CHREST), Wikipedia
- CLARION, Wikipedia
- DUAL, Wikipedia
- Global Workspace Theory, Wikipedia
- ICARUS
- Learning Intelligent Distribution Agent (LIDA), Wikipedia
- Sigma
- Soar, Wikipedia
- Subsumption Architecture, Wikipedia
Intelligent Agent Design
- Also see Agent-Oriented Programming (AOP)
- Needs to be merged with Cognitive Architectures.
- Additional Architectures
- Neats and Scruffies distinction
See agent architectures and TouringMachines distinction into
- Deliberative Architectures
- IRMA (Bratman et al.)
- AUTODRIVE (Wood)
- Behaviour Hierarchies (Durfee and Montgomery)
- Agent-Oriented Programming (Shoham)
- Homer (Vere and Bickmore)
- Non-deliberative Architectures
- Subsumption Architecture (Brooks)
- Situated Automata (Rosenschein and Kaelbling)
- Pengi (Agre and Chapman)
- Reactive Action Packages (Firby)
- Universal Plans (Schoppers)
- Dynamic Action Selection
- Hybrid Architectures
- 3T
- AuRA
- Brahms (Agent-Oriented Language, BDI architecture)
- GAIuS
- GRL
- InteRRaP
- TinyCog
- TouringMachines
Other Architectures
Biological Inspiration
Components from Brain Computation as Hierarchical Abstraction by Dana H. Ballard:
- Cortex: Long-Term Memory
- Basal Ganglia: Program Sequencing
- Thalamus: I/O
- Hippocampus: Program Modifications
- Amygdala: Focus
Additional Considerations
- Different neurotransmitters beyond dopamine
- Four forms of neuroplasticity
- Spiking neural nets (so far haven’t had advantages; see zero-divergence inference learning (Oxford) for biologically plausible alternative to backprop)
- Reinforcement Learning: See Sutton & Barto Chapter 15
- CNNs seem biologically plausible
- Brain microcircuits and connectome
- It seems we need 5-8 layer DNN to approximate cortical neuron as published in “Single cortical neurons as deep artificial neural networks” (Beniaguev et al., Neuron Volume 109 Issue 17, 1 September 2021).
- Metacognition
- Sense of uncertainty (potentially unconscious) to be able to doubt oneself
- Monitoring of actions to recognize when they go off course