The following is a very preliminary sketch of the landscape of AI technologies (i.e. frameworks, libraries, platforms and product offerings). Please feel free to send enhancement suggestions via the contact form. Thank you in advance.
The Linux Foundation’s AI and Data Landscape is an interesting resource as well.
Deep Learning
Frameworks
- Caffe (BVLC) and Caffe 2.0
- Chainer
- CNTK
- CoreML
- DL4J
- DyNet, Github, Docs
- fast.ai, Docs, Course
- Keras, source
- Apache MXNet
- PaddlePaddle (PArallel Distributed Deep Learning)
- PyTorch
- Tensorflow
Compilers
- Glow Compiler by Facebook
- Google XLA
- Intel nGraph
- Latte
- Nvidia TensorRT
- OctoML (on top of Apache TVM)
- Open Neural Network Compiler (ONNC)
- PyTorch Glow
- PlaidML
- TACO – The Tensor Algebra Compiler
- Tensorflow Multi-Level Intermediate Representation, was originally here
- Tensor Comprehensions paper
- Tiramisu, paper
- TVM, NNVM
- Libraries like NNPACK, cuDNN, hipDNN/MIOpen etc.
Toolboxes
- Aequitas – Bias and Fairness Audit Toolkit
- IBM Adversarial Robustness Toolbox, Blog, Paper for adversarial hardening
- IBM AI Fairness 360 (AIF360), Demo, Paper, Docs for bias detection
- IBM AIX360, Demo – for explainability
- IBM Uncertainty Quantification Toolkit (UQ 360) – uncertainty quantification
- Intel Neural Network Distiller – for neural network compression
- Qualcomm AI Model Efficiency Toolkit
- XAI – An eXplainability toolbox for machine learning
Neural Network Modeler
- IBM Deep Leaqrning IDE / DarViz
- Microsoft NeuronBlocks
- Sony Neural Network Console
- SPSS Neural Networks
- Watson Studio Neural Network Modeler
Also see visualization solutions like Facebook ActiVis.
Embeddings
Traditional ML & Data Science
Machine Learning Frameworks
Reinforcement Learning
Please see the dedicated overview article. Still debating with myself whether I should add all the references to a map.
Natural Language Processing
NLP Architectures
- Ellogon
- General Architecture for Text Engineering (GATE)
- Heart of Gold (HoG)
- TIPSTER
- Apache UIMA, also OASIS standard
General
Translation
- fairseq (earlier Lua version was Facebook fairseq) – sequence modeling like translation or summarization
- OpenNMT – Neural Machine Translation
- PyTorch Translate
- StanfordNLP Phrasal
Speech Synthesis
Automatic Speech Recognition (ASR)
- Mozilla Project DeepSpeech (STT)
- RWTH ASR (RASR)
- CMUSphinx ASR: Kaldi and Vosk
NLU & Conversational AI
- Rasa – RL-driven NLU for dialog systems and virtual assistants
Other NLP
Computer Vision
- BoofCV
- Caffe2 (see deep learning frameworks)
- MATLAB, in particular due to its Toolboxes
- OpenCV, Github, Wikipedia
- OpenVINO, also useful for other tasks like NLP or recommendation
- OpenVX, open standard by Khronos Group
- torchvision (there are also things like PyTorchCV, but much less popular)
- scikit-image
- SimpleCV
Other
Embodied Agents
Recommender Systems
More
Symbolic Approaches
Reasoners
Planners
Optimizers & Solvers
These are only a few solvers – more extensive lists can be found on NEOS and YALMIP.
- Gurobi
- IBM CPLEX (linear, mixed-integer, quadratic programming), IBM ILOG CP Optimizer
- Lindo
- LocalSolver
- MOSEK
Linear
- Coin-or branch and cut (Cbc) – mixed integer linear programming (MILP)
- GNU Linear Programming Kit (GLPK)
- lpsolve – MILP
Non-Linear
- Interior Point OPTimizer (Ipopt)
- Basic Open-source Nonlinear Mixed INteger programming (Bonmin)
- Convex Over and Under ENvelopes for Nonlinear Estimation (Couenne)
Constraint Programming
Satisfiability (SAT)
Theorem Provers & Proof Assistants
Many more can be found in “A Survey on Theorem Provers in Formal Methods” which provides a great overview.
Logic Programming
- Clojure core.logic
- Datalog
- Julog, logic programming in Julia
- MiniKanren
- Prolog (e.g. SWI Prolog and GNU Prolog)
- pylo2 – Python wrapper around many logic programming engines, unfortunately not too active
Knowledge Bases & Ontologies
Also see Linked Open Data Cloud.
Knowledge Graph Technologies
AI Service Orchestration
Model and Data Catalogs
Model Zoos & Catalogs
- Acumos
- Caffe Model Zoo, Github
- IBM BotAssetExchange
- IBM Model Catalog, (on Docker Hub)
- IBM Model Asset eXchange (MAX) and Machine Learning eXchange (MLX)
- IBM OpenAIHub
- Caffe2 Models
- CNTK’s Pretrained Model List
- DL4J’s Zoo Models
- Gluon Model Zoo
- Microsoft Azure Gallery
- Microsoft Model Gallery
- MIT ModelDB
- modelzoo.co
- Model Zoo for AI Model Efficiency Toolkit
- mxnet’s Model Zoo
- Neon Model Zoo
- ONNX Models
- PyTorch Models
- TensorFlow Hub, tfhub.dev
- Tensorflow Models
- Torchvision Models
Primarily Commercial Model Marketplaces
- Algorithmia
- BigML Gallery
- Google AI Hub, alternative, docs
- ModelDepot
- Nvidia GPU Compute Cloud Catalog
- Wolfram Research’s Neural Net Repository
Data Catalogs
- Awesome Public Datasets – see esp. PublicDomains and SearchEngines
- IBM Watson Knowledge Catalog
- Data.gov
- DL4J Open Datasets
- EPA Dataset Gateway
- Google Dataset Search
- US Department of Commerce
- World Bank Data Catalog
Data Formats
Microsoft MMdnn can be used to convert models between frameworks. [There are additional related projects like onnxconverter. Apple has Core ML Tools, but it only converts to CoreML format.]
- Neural Network Exchange Format (NNEF)
- ONNX, IBM Research contributed Tensorflow Backend for ONNX
- Planning Domain Definition Language (PDDL)
- Predictive Model Markup Language (PMML)
- Portable Format for Analytics (PFA)
Cloud AI Services
This still needs to be extended and reorganized quite heavily. Relevant Cloud AI Service Portfolios to look at include:
- AWS Machine Learning
- Baidu AI Cloud
- Google Cloud AI & Machine Learning Services
- IBM Watson
- Microsoft Azure Cognitive Service
- Salesforce Einstein & Platform
Automatic Speech Recognition (ASR)
- Amazon Transcribe
- Baidu Speech Technology
- IBM Speech-to-Text
- Microsoft Speech-to-Text and Microsoft Speaker Recognition
Natural Language Processing, Understanding and Conversational AI
- AWS Comprehend
- Amazon Lex – conversational AI
- Azure Bot Service – conversational AI
- Einstein Bots and Einstein Language
- Google Dialogflow
Speech Synthesis
- AWS Polly
- IBM Text-to-Speech
Translation
- Amazon Translate
- Google Translate
- IBM Language Translator
- Microsoft Translator / Translator API and Speech Translation
Vision
- Amazon Rekognition (e.g. camera AWS DeepLens integrates with it)
- Amazon Textract – semantic OCR
- Baidu Image Recognition and Baidu Image Search
- Einstein Vision
- Microsoft Computer Vision
- Rossum – semantic OCR, automatic document handling
- Tencent Optical Character Recognition, Tencent Face Recognition, Tencent FaceID and Tencent Analysis Platform for Pneumonia CT Images
Other
- Amazon Quicksight – business intelligence
- Einstein Discovery, Einstein Prediction Builder
- Amazon Forecast – time-series forecasting
- Amazon Personalize – real-time recommendation
Workflow & Service Orchestration
- Apache Airflow
- Argo
- AWS Step Functions
- Azkaban
- Brigade
- Cadence
- Camel
- Camel K
- Captain
- CloudSlang
- Copper
- DigDag
- Fission Workflows
- Flor
- Imixs-Workflow
- Kiba
- Luigi
- Mistral
- MuleESB
- Netflix Conductor
- NodeRED
- Oozie
- Piper
- Pinball
- Spring Integration, Github
- RunDeck
- Titanoboa
- Viewflow, source
- Wexflow
- Workflow Core
- Workflow Engine
- Zeebe
Data Ingestion, Lakes & Warehouses
- Apache PDFBox (PDF manipulation), Apache Tika (content analysis)
- Alteryx
- databricks
- Dataiku
- Google BigQuery, Datalab (Github)
- Microsoft Azure HDInsight – to provision clusters, e.g. Hadoop, Spark or Storm
- Microsoft Data Lake Analytics
- OpenRefine (was Google Refine)
- TIBCO Clarity – data cleansing
- Trifacta, Wikipedia, also see Dataprep by Trifacta on Google Cloud
Systems & Platforms
AI Platforms
- Acumos AI
- AI Layer
- Airbnb BigHead & data management platform Zipline & metric platform Minerva
- Algorithmia MLOps Platform
- Alibaba Machine Learning Platform for AI (PAI 2.0) and Alibaba Cloud Intelligence Brain
- Apple Overton
- Arya AI
- AWS SageMaker (incl. SageMaker Ground Truth SageMaker Neo SageMaker RL, also this)
- Azure ML Studio
- Baidu Brain
- Bonsai
- Data Robot
- Deep Cognition
- DoorDash ML Platform
- eBay Krylov
- Facebook FBLearner
- Flipkart Hunch
- FloydHub
- Gojek’s ML Platform
- Google Vertex AI, also see Google Kubeflow & TFX & Google AutoML
- Groupon Flux
- H2O
- IBM Watson Studio and Watson Machine Learning
- Iguazio (also see this)
- Inuit ML Platform (based on Sagemaker, Argo Workflows, GitOps)
- KNIME
- Kubeflow
- LinkedIn Pro-ML (Productive Machine Learning)
- Lyft Flyte
- Meta AI Looper
- Microsoft OpenPAI, also see Microsoft DL Workspace (DLTS) and Azure ML
- mlflow (Github)
- Netflix Metaflow (and model lifecycle management platform Runway; also see this)
- NVIDIA TAO
- OpenAI Rapid
- OpenDataHub (has ties to RHOCP)
- Oracle AI
- Pachyderm
- Pinterest ML Platform
- Polyaxon
- Prowler.io
- RapidMiner
- Emerging Ray-Based Platforms (see Operator First Ray demo and CodeFlare)
- Stripe Railyard – Platform for Model Training
- Apache Submarine
- Swiftstack
- Tensorflow Extended (TFX) (also see this and Spotify’s TFX-Based ML Platform)
- TransmogrifAI
- Twitter DeepBird
- Uber Michelangelo (also see this and this)
- Valohai (also see this)
- Wix Machine Learning Platform
- Yelp ML Platform
There are many more platforms and also several specialized niche ones like Facebook ELF for (strategy) game research or AllenAct for Embodied AI. It is sometimes difficult to classify whether something is primarily a platform or framework.
Cloud AI
AI Development Experience & IDEs
- Cloudera Data Science Workbench
- IBM Watson Studio, IBM ILOG CPLEX Optimization Studio
- Jetbrains DataSpell (also Datalore, DataGrip, PyCharm for Data Science, IntelliJ for Data Engineers)
- Microsoft Azure ML Studio
- Weights and Biases
Model Training and Lifecycle Management
- Amazon Sagemaker (incl. its wide ecosystem such as SagemakerRL)
- IBM OpenScale, Watson Machine Learning and Watson AIOps
Acceleration & Distribution
- AdaptDL – resource-adaptive training & scheduling framework
- Apache Beam
- Bodo
- Dask
- Amazon Elastic Inference – low cost inference
- Apache Flink
- Apache Hadoop
- AWS Deep Learning AMIs – images
- Horovod
- Intel MKL, Wikipedia
- OpenMPI
- Ray
- Apache Spark
- Apache Storm
- Tencent Angel – parameter server for large-scale machine learning
- TensorRT
Hyperparameter Optimization (HPO)
- Bayesian Optimization
- DL4J Arbiter
- GPyOpt (stale!)
- HPOlib2 (stale!)
- hyperopt
- katib
- Optuna, paper
- Polyaxon Optimization Engine
- Ray Tune
- scikit-optimize / skopt – library to minimize expensive black-box functions
- SHERPA
- SigOpt
- Spearmint (stale!)
- Talos
Automated AI (AutoAI) and Neural Architecture Search (NAS)
The University of Freiburg maintains the great page about the underlying techniques at automl.org
Notebooks & Dashboards
- BeakerX
- Holoviz Panel
- Jupyter, JupyterLab, Docs
- nteract, Github
- Plotly Dash
- Streamlit
- Tableau – visual analytics
- Tensorboard, Tensorboard-Pytorch, TensorboardX (docs), Crayon (language agnostic interface to Tensorboard) – DL/DRL training dashboard
- Voilà
- Zeppelin
There are many related product offerings like Google Colab, binder, ContainDS. PixieDust is a tool to encapsulate business logic from notebooks.
Hardware
Neuromorphic Hardware
See WikiChip for more neural processors. And AI-Chip.
- Alibaba Ali-NPU
- Baidu Kunlun
- AWS Inferentia
- Cerebras CS-1
- Graphcore Intelligent Processing Unit (IPU)
- Intel Loihi – Intel Pohoiki Beach system has multiple Nahuku boards with Loihi chips, also see Habana Gaudi and Habana Goya (part of Intel)
- MIT Eyeriss
- Mythic
- Nvidia Deep Learning Accelerator (NVDLA)
- SpiNNaker
- Google TPU
- IBM TrueNorth, NorthPole
Quantum
- Amazon Braket – Quantum computing service on AWS
- D-Wave
- IBM Q
- Microsoft Quantum, docs, Github, Q#
- Paddle Quantum, Github
Benchmarks
While deciding what to measure is a topic on its own and e.g. discussed in SUPERB: Speech Processing Universal PERformance Benchmark, in Chollet’s excellent On the Measure of Intelligence as well as in Adams’ I-athlon: Towards A Multidimensional Turing Test, here are a few software projects that help with it:
- AI Matrix
- AIIA DNN Benchmark
- Fathom – reference workloads, there are also things like AI Benchmark which benchmarks smartphones on AI
- General Language Understanding Evaluation (GLUE) benchmark, SuperGLUE
- MLModelScope
- MLPerf – driven by ML Commons
- Stanford DAWNBench
- ParlAI (framework for training and evaluating AI models) Github
Probabilistic Programming
- AIC-PRAiSE
- Alchemy
- Anglican
- BayesDB & Venture (MIT)
- Birch
- BLOG
- BUGS – OpenBUGS, WinBUGS (related: JAGS and Biips)
- Church (deprecated?)
- Dimple
- Edward
- Factorie
- Figaro
- ForneyLab.jl (Julia Toolbox for Factor Graph-Based Probabilistic Programming)
- Gen
- Hakaru
- Infer.NET
- Kakuritu (Hakaru10, Hansei)
- LibBi
- Markov the Beast
- monad-bayes
- Nimble
- Omega.jl
- ProbTorch
- Probabilistic C
- Probabilistic C#
- Probabilistic Soft Logic
- ProbCog (now includes PyMLNs)
- ProbLog
- PSI Solver
- PyMC3
- Pyro
- R2 (Microsoft)
- Rainier (Bayesian Inference in Scala)
- RankPL
- Stan
- TensorFlow Probability
- Tuffy
- Turing.jl
- WebPPL, tutorial
Cloud
Oftentimes AI relies on cloud technology for its training and serving infrastructure. Hence, it makes sense to also list a few cloud technologies. This section is under construction and will be extended soon. Until then the CNCF Cloud Landscape is a good reference.
General
Service Meshes
- A10 Secure Service Mesh
- AspenMesh
- Cilium
- ConsulConnect
- Grey Matter
- Istio
- Kong
- Linkerd / Conduit
- Maesh
- Mesher
- Rotor
- SOFAMesh
- Yggdrasil
- Zuul
Debugging & Observability
Security
Other
- cloudevents.io – Specification for event data