ML Visuals

By dair.ai

https://github.com/dair-ai/ml-visuals

IMPORTANT NOTE: Please don’t request editing permission if you do not plan to add anything to the slides. If you want to edit the slides for your own purposes, just make a copy of the slides.

ML Visuals is a new collaborative effort to help the machine learning community in improving science communication by using more professional, compelling and adequate visuals and figures. You are free to use the visuals in your presentations or blog posts. You don’t need to ask permission to use any of the visuals but it will be nice if you can provide credit to the designer/author (author information found in the slide notes).

This is a project made by the dair.ai community and maintained in this GitHub repo. Our community members will continue to add more common figures and basic elements in upcoming versions. Think of this as free and open artefacts and templates which you can freely and easily download, copy, distribute, reuse and customize to your own needs. I am maintaining this set of slides on a bi-weekly basis where I will regularly organize and keep the slides clean from spam or unfinished content.

Contributing: To add your own custom figures, simply add a new slide and reuse any of the basic visual components (remember to request edit permissions). We encourage authors/designers to add their visuals here and allow others to reuse them. Make sure to include your author information (in the notes section of the slide) so that others can provide the proper credit if they use the visuals elsewhere (e.g. blog/presentations). Add your name and email just in case someone has any questions related to the figures you added. Also, provide a short description of your visual to help the user understand what it is about and how they can use it. If you need "Edit" permission, just click on the "request edit access" option under the "view only" toolbar above or send me an email at [email protected] If you have editing rights, please make sure not to delete any of the work that other authors have added. You can try to improve it by creating a new slide and adding that improved version (that’s actually encouraged).

Downloading a figure from any of the slides is easy. Just click on File→Download→(choose your format).

If you need help with customizing a figure or have an idea of something that could be valuable to others, we can help. Just open an issue here and we will do our best to come up with the visual. Thanks.

Basic ML Visuals

Note:

Basic components go here.

Note:

This is a simple round rectangle that can represent some process, operation, or transformation

Author: Elvis Saravia ([email protected])

Note:

This can be used to represent a vector. If you want to edit the distance between each element, you can “ungroup” the shapes and then modify the distance between the individual lines.

Author: Elvis Saravia ([email protected])

Symbolizing an embedding

Note:

Can represent a neuron or some arbitrary operation

Author: Elvis Saravia ([email protected])

Note:

These visuals can represent a multi-directional array (3D) input, tensor, etc.

Author: Elvis Saravia ([email protected])

Softmax

Convolve

Sharpen

Note:

These visuals could represent transformations (left) or operations (right)

The visuals on the right use the Math Equations Add on for Google Slides.

If you click on the equation you are able to modify it by:

Selecting Add Ons → Math Equations → Menu

While the equation is selected, click “Connect to Equation” on the “Math Equations UI”

Then change the properties like color, size, etc.

Author: Elvis Saravia ([email protected])

Softmax

Convolve

Sharpen

Note:

These visuals could represent transformations (left) or operations (right)

The visuals on the right use the Math Equations Add on for Google Slides.

If you click on the equation you are able to modify it by:

Selecting Add Ons → Math Equations → Menu

While the equation is selected, click “Connect to Equation” on the “Math Equations UI”

Then change the properties like color, size, etc.

Author: Elvis Saravia ([email protected])

Log:

Added a dark theme version of the previous slide

Positional

Encoding

Masked

Multi-Head

Attention

Add & Norm

Output

Embedding

Multi-Head

Attention

Add & Norm

Outputs(shifted right)

Positional

Encoding

Multi-Head

Attention

Add & Norm

Input

Embedding

Feed

Forward

Add & Norm

Inputs

Feed

Forward

Add & Norm

Linear

Softmax

Note:

Using the same visual components from before and a few new custom ones, I was able to come up with the Transformer architecture

This is a reproduction of the Transformer architecture figure presented in the work of Vaswani et al. 2017

You can further customize it however you want

Author: Elvis Saravia ([email protected])

Multi-Head

Attention

Add & Norm

Input

Embedding

Output

Embedding

Feed

Forward

Add & Norm

Masked

Multi-Head

Attention

Add & Norm

Multi-Head

Attention

Add & Norm

Feed

Forward

Add & Norm

Linear

Softmax

Inputs

Outputs (shifted right)

Positional

Encoding

Positional

Encoding

Note:

Using the same visual components from before and a few new custom ones, I was able to come up with the Transformer architecture

This is a reproduction of the Transformer architecture figure presented in the work of Vaswani et al. 2017

You can further customize it however you want

Author: Elvis Saravia ([email protected])

Tokenize

I

love

coding

and

writing

“I love coding and writing”

Note:

This figure aims to demonstrate the process of tokenization

If you want to change the gradient colors of this shape, you can simply click on the shape and then select “Fill color” (the bucket looking icon at the top).

This is a gradient color so you can customize however you want

Author: Elvis Saravia ([email protected])

Input Layer

Hidden Layers

Output Layer

X = A[0]

a[4]

A[1]

A[3]

X

Ŷ

a[1]n

a[2]1

a[2]2

a[2]3

a[2]n

a[3]1

a[3]2

a[3]3

a[3]n

A[2]

A[4]

Created By @srvmshr

Input Layer

Hidden Layers

Output Layer

X = A[0]

a[4]

A[1]

A[3]

X

Ŷ

a[1]1

a[1]2

a[1]3

a[1]n

a[2]1

a[2]2

a[2]3

a[2]n

a[3]1

a[3]2

a[3]3

a[3]n

A[2]

A[4]

Original by @srvmshr

Dark themed version by (Elvis Saravia - [email protected])

Input Layer

Hidden Layers

Output Layer

X = A[0]

a[4]

A[1]

A[3]

X

Ŷ

[1a]1

a[1]2

a[1]3

a[1]n

a[2]1

a[2]2

a[2]3

a[2]n

a[3]1

a[3]2

a[3]3

a[3]n

A[2]

A[4]

Created By @srvmshr

NxNx3

+b1

+b2

MxM

MxM

+b1

+b2

ReLU

ReLU

a[l]

MxMX2

a[l-1]

CONV operation

Added by @srvmshr

Sourav @ yahoo.com

NxNx3

+b1

+b2

MxM

MxM

+b1

+b2

ReLU

ReLU

MxMX2

CONV operation

Added by @srvmshr

Sourav @ yahoo.com

NxNx3

+b1

+b2

MxM

MxM

+b1

+b2

ReLU

ReLU

MxMX2

CONV operation

Added by @srvmshr

Sourav @ yahoo.com

Abstract backgrounds

Note:

You can add abstract backgrounds here

Note:

You can change the gradient of each component

Author: Elvis Saravia

DAIR.AI

Note:

This is the header I use for the dair.ai NLP Newsletter

You can change the gradient of each component

You can customize the background as well by right clicking and changing to another gradient or static color

Author: Elvis Saravia

Note:

This is the header I use for the dair.ai NLP Newsletter

You can change the gradient of each component

Author: Elvis Saravia

Note:

This is the header I use for the dair.ai NLP Newsletter

You can change the gradient of each component

Author: Elvis Saravia

Gradient Backgrounds

Note:

Gradient background shows some inspirations and ideas for color themes that could be used for backgrounds

The examples used in this section are inspired by https://uigradients.com

Community Contributions

Note:

You can add abstract backgrounds here

S=1

S=2

Striding in CONV

Created by @srvmshr

Sourav @ yahoo.com

Created by @srvmshr

Sourav @ yahoo.com

NxNx192

NxNx64

NxNx32

NxNx128

NxNx192

1x1 Same

3x3 Same

5x5 Same

MaxPool Same s=1

Inception Module

Created by @srvmshr

Sourav @ yahoo.com

Retraining w/o expansion

t-1

t

Note:

1 hidden layer neural network

Author: Elvis Saravia ([email protected])

No-Retraining w/ expansion

Partial Retraining w/ expansion

Note:

1 hidden layer neural network

Author: Elvis Saravia ([email protected])

No-Retraining w/ expansion

Partial Retraining w/ expansion

t-1

t

t

t-1

Note:

1 hidden layer neural network

Author: Elvis Saravia ([email protected])

No-Retraining expansion

t

Partial Retraining expansion

t

t-1

Retraining expansion

t-1

t

t-1

Note:

1 hidden layer neural network

Author: Elvis Saravia ([email protected])

Size

#bed

ZIP

Wealth

Family?

Walk?

School

PRICE ŷ

X

Y

X

Ŷ = 0

Ŷ = 1

How does NN work (Insprired from Coursera)

Logistic Regression

Basic Neuron Model

By @srvmshr

Size

$

Size

$

Linear regression

ReLU(x)

By @srvmshr

I

V

128*128*1

128*128*1

CONV

1

CONV

2

CONV

4

CONV

3

CONV

5

CONV

6

CONV

7

I1

128*128*1

ENcoder

Decoder

V1

Encoder

Decoder

128*128*1

Training

By @srvmshr

Large NN

Med NN

Small NN

SVM,LR etc

η

Amount of Data

Why does Deep learning work?

By @srvmshr

a[1]1

a[1]2

a[1]3

Input

Hidden

Output

X = A[0]

a[1]4

a[2]

A[1]

A[2]

X

Ŷ

One hidden layer neural network

By @srvmshr

a[1]1

a[1]2

x[1]

a[2]

x[2]

x[2]

x[3]

x[1]

Neural network templates

By @srvmshr

a[1]1

a[1]2

x[1]

a[2]

x[2]

x[2]

x[3]

x[1]

Neural network templates

By @srvmshr

Train

Valid

Test

x1

x2

x1

x2

x1

x2

Train-Dev-Test vs. Model fitting

Underfitting

Good fit

Overfitting

By @srvmshr

x[2]

x[3]

x[1]

a[L]

x1

x2

r=1

x1

x2

DropOut

Normalization

By @srvmshr

w1

w1

w2

J

w1

w2

J

w1

w2

w2

Before Normalization

After Normalization

Early stopping

Dev

Train

Err

it.

By @srvmshr

x1

x2

w[1]

w[2]

w[L-2]

w[L-1]

w[L]

FN

TN

TP

FP

Deep neural networks

Understanding

Precision & Recall

By @srvmshr

w1

w2

SGD

BGD

w1

w2

SGD

Batch vs. Mini-batch Gradient Descent

Batch Gradient Descent vs. SGD

By @srvmshr

x[2]

x[3]

x[1]

p[1]

p[2]

Softmax Prediction with 2 outputs

By @srvmshr