ML Visuals
By dair.ai
https://github.com/dair-ai/ml-visuals
IMPORTANT NOTE: Please don’t request editing permission if you do not plan to add anything to the slides. If you want to edit the slides for your own purposes, just make a copy of the slides.
ML Visuals is a new collaborative effort to help the machine learning community in improving science communication by using more professional, compelling and adequate visuals and figures. You are free to use the visuals in your presentations or blog posts. You don’t need to ask permission to use any of the visuals but it will be nice if you can provide credit to the designer/author (author information found in the slide notes).
This is a project made by the dair.ai community and maintained in this GitHub repo. Our community members will continue to add more common figures and basic elements in upcoming versions. Think of this as free and open artefacts and templates which you can freely and easily download, copy, distribute, reuse and customize to your own needs. I am maintaining this set of slides on a bi-weekly basis where I will regularly organize and keep the slides clean from spam or unfinished content.
Contributing: To add your own custom figures, simply add a new slide and reuse any of the basic visual components (remember to request edit permissions). We encourage authors/designers to add their visuals here and allow others to reuse them. Make sure to include your author information (in the notes section of the slide) so that others can provide the proper credit if they use the visuals elsewhere (e.g. blog/presentations). Add your name and email just in case someone has any questions related to the figures you added. Also, provide a short description of your visual to help the user understand what it is about and how they can use it. If you need "Edit" permission, just click on the "request edit access" option under the "view only" toolbar above or send me an email at [email protected]. If you have editing rights, please make sure not to delete any of the work that other authors have added. You can try to improve it by creating a new slide and adding that improved version (that’s actually encouraged).
Downloading a figure from any of the slides is easy. Just click on File→Download→(choose your format).
If you need help with customizing a figure or have an idea of something that could be valuable to others, we can help. Just open an issue here and we will do our best to come up with the visual. Thanks.
Basic ML Visuals
Note:
Basic components go here.
Note:
This is a simple round rectangle that can represent some process, operation, or transformation
Author: Elvis Saravia ([email protected])
Note:
This can be used to represent a vector. If you want to edit the distance between each element, you can “ungroup” the shapes and then modify the distance between the individual lines.
Author: Elvis Saravia ([email protected])
Symbolizing an embedding
Note:
Can represent a neuron or some arbitrary operation
Author: Elvis Saravia ([email protected])
Note:
These visuals can represent a multi-directional array (3D) input, tensor, etc.
Author: Elvis Saravia ([email protected])
Softmax
Convolve
Sharpen
Note:
These visuals could represent transformations (left) or operations (right)
The visuals on the right use the Math Equations Add on for Google Slides.
If you click on the equation you are able to modify it by:
Selecting Add Ons → Math Equations → Menu
While the equation is selected, click “Connect to Equation” on the “Math Equations UI”
Then change the properties like color, size, etc.
Author: Elvis Saravia ([email protected])
Softmax
Convolve
Sharpen
Note:
These visuals could represent transformations (left) or operations (right)
The visuals on the right use the Math Equations Add on for Google Slides.
If you click on the equation you are able to modify it by:
Selecting Add Ons → Math Equations → Menu
While the equation is selected, click “Connect to Equation” on the “Math Equations UI”
Then change the properties like color, size, etc.
Author: Elvis Saravia ([email protected])
Log:
Added a dark theme version of the previous slide
Positional
Encoding
Masked
Multi-Head
Attention
Add & Norm
Output
Embedding
Multi-Head
Attention
Add & Norm
Outputs(shifted right)
Positional
Encoding
Multi-Head
Attention
Add & Norm
Input
Embedding
Feed
Forward
Add & Norm
Inputs
Feed
Forward
Add & Norm
Linear
Softmax
Note:
Using the same visual components from before and a few new custom ones, I was able to come up with the Transformer architecture
This is a reproduction of the Transformer architecture figure presented in the work of Vaswani et al. 2017
You can further customize it however you want
Author: Elvis Saravia ([email protected])
Multi-Head
Attention
Add & Norm
Input
Embedding
Output
Embedding
Feed
Forward
Add & Norm
Masked
Multi-Head
Attention
Add & Norm
Multi-Head
Attention
Add & Norm
Feed
Forward
Add & Norm
Linear
Softmax
Inputs
Outputs (shifted right)
Positional
Encoding
Positional
Encoding
Note:
Using the same visual components from before and a few new custom ones, I was able to come up with the Transformer architecture
This is a reproduction of the Transformer architecture figure presented in the work of Vaswani et al. 2017
You can further customize it however you want
Author: Elvis Saravia ([email protected])
Tokenize
I
love
coding
and
writing
“I love coding and writing”
Note:
This figure aims to demonstrate the process of tokenization
If you want to change the gradient colors of this shape, you can simply click on the shape and then select “Fill color” (the bucket looking icon at the top).
This is a gradient color so you can customize however you want
Author: Elvis Saravia ([email protected])
Input Layer
Hidden Layers
Output Layer
X = A[0]
a[4]
A[1]
A[3]
X
Ŷ
a[1]n
a[2]1
a[2]2
a[2]3
a[2]n
a[3]1
a[3]2
a[3]3
a[3]n
A[2]
A[4]
Created By @srvmshr
Input Layer
Hidden Layers
Output Layer
X = A[0]
a[4]
A[1]
A[3]
X
Ŷ
a[1]1
a[1]2
a[1]3
a[1]n
a[2]1
a[2]2
a[2]3
a[2]n
a[3]1
a[3]2
a[3]3
a[3]n
A[2]
A[4]
Original by @srvmshr
Dark themed version by (Elvis Saravia - [email protected])
Input Layer
Hidden Layers
Output Layer
X = A[0]
a[4]
A[1]
A[3]
X
Ŷ
[1a]1
a[1]2
a[1]3
a[1]n
a[2]1
a[2]2
a[2]3
a[2]n
a[3]1
a[3]2
a[3]3
a[3]n
A[2]
A[4]
Created By @srvmshr
NxNx3
+b1
+b2
MxM
MxM
+b1
+b2
ReLU
ReLU
a[l]
MxMX2
a[l-1]
CONV operation
Added by @srvmshr
Sourav @ yahoo.com
NxNx3
+b1
+b2
MxM
MxM
+b1
+b2
ReLU
ReLU
MxMX2
CONV operation
Added by @srvmshr
Sourav @ yahoo.com
NxNx3
+b1
+b2
MxM
MxM
+b1
+b2
ReLU
ReLU
MxMX2
CONV operation
Added by @srvmshr
Sourav @ yahoo.com
Abstract backgrounds
Note:
You can add abstract backgrounds here
Note:
You can change the gradient of each component
Author: Elvis Saravia
DAIR.AI
Note:
This is the header I use for the dair.ai NLP Newsletter
You can change the gradient of each component
You can customize the background as well by right clicking and changing to another gradient or static color
Author: Elvis Saravia
Note:
This is the header I use for the dair.ai NLP Newsletter
You can change the gradient of each component
Author: Elvis Saravia
Note:
This is the header I use for the dair.ai NLP Newsletter
You can change the gradient of each component
Author: Elvis Saravia
Gradient Backgrounds
Note:
Gradient background shows some inspirations and ideas for color themes that could be used for backgrounds
The examples used in this section are inspired by https://uigradients.com
Community Contributions
Note:
You can add abstract backgrounds here
S=1
S=2
Striding in CONV
Created by @srvmshr
Sourav @ yahoo.com
Created by @srvmshr
Sourav @ yahoo.com
NxNx192
NxNx64
NxNx32
NxNx128
NxNx192
1x1 Same
3x3 Same
5x5 Same
MaxPool Same s=1
Inception Module
Created by @srvmshr
Sourav @ yahoo.com
Retraining w/o expansion
t-1
t
Note:
1 hidden layer neural network
Author: Elvis Saravia ([email protected])
No-Retraining w/ expansion
Partial Retraining w/ expansion
Note:
1 hidden layer neural network
Author: Elvis Saravia ([email protected])
No-Retraining w/ expansion
Partial Retraining w/ expansion
t-1
t
t
t-1
Note:
1 hidden layer neural network
Author: Elvis Saravia ([email protected])
No-Retraining expansion
t
Partial Retraining expansion
t
t-1
Retraining expansion
t-1
t
t-1
Note:
1 hidden layer neural network
Author: Elvis Saravia ([email protected])
Size
#bed
ZIP
Wealth
Family?
Walk?
School
PRICE ŷ
X
Y
X
Ŷ = 0
Ŷ = 1
How does NN work (Insprired from Coursera)
Logistic Regression
Basic Neuron Model
By @srvmshr
Size
$
Size
$
Linear regression
ReLU(x)
By @srvmshr
I
V
128*128*1
128*128*1
CONV
1
CONV
2
CONV
4
CONV
3
CONV
5
CONV
6
CONV
7
I1
128*128*1
ENcoder
Decoder
V1
Encoder
Decoder
128*128*1
Training
By @srvmshr
Large NN
Med NN
Small NN
SVM,LR etc
η
Amount of Data
Why does Deep learning work?
By @srvmshr
a[1]1
a[1]2
a[1]3
Input
Hidden
Output
X = A[0]
a[1]4
a[2]
A[1]
A[2]
X
Ŷ
One hidden layer neural network
By @srvmshr
a[1]1
a[1]2
x[1]
a[2]
x[2]
x[2]
x[3]
x[1]
Neural network templates
By @srvmshr
a[1]1
a[1]2
x[1]
a[2]
x[2]
x[2]
x[3]
x[1]
Neural network templates
By @srvmshr
Train
Valid
Test
x1
x2
x1
x2
x1
x2
Train-Dev-Test vs. Model fitting
Underfitting
Good fit
Overfitting
By @srvmshr
x[2]
x[3]
x[1]
a[L]
x1
x2
r=1
x1
x2
DropOut
Normalization
By @srvmshr
w1
w1
w2
J
w1
w2
J
w1
w2
w2
Before Normalization
After Normalization
Early stopping
Dev
Train
Err
it.
By @srvmshr
x1
x2
w[1]
w[2]
w[L-2]
w[L-1]
w[L]
FN
TN
TP
FP
Deep neural networks
Understanding
Precision & Recall
By @srvmshr
w1
w2
SGD
BGD
w1
w2
SGD
Batch vs. Mini-batch Gradient Descent
Batch Gradient Descent vs. SGD
By @srvmshr
x[2]
x[3]
x[1]
p[1]
p[2]
Softmax Prediction with 2 outputs
By @srvmshr
sourav@ya