ML Visuals
By dair.ai
https://github.com/dair-ai/ml-visuals
ML Visuals is a new collaborative effort to help the machine learning community in improving science communication by using more professional, compelling and adequate visuals and figures. You are free to use the visuals in your presentations or blog posts. You don’t need to ask permission to use any of the visuals but it will be nice if you can provide credit to the designer/author (author information found in the slide notes).
This is a project made by the dair.ai community and maintained in this GitHub repo. Our community members will continue to add more common figures and basic elements in upcoming versions. Think of this as free and open artefacts and templates which you can freely and easily download, copy, distribute, reuse and customize to your own needs. I am maintaining this set of slides on a bi-weekly basis where I will regularly organize and keep the slides clean from spam or unfinished content.
IMPORTANT NOTE: Please don’t request editing permission if you do not plan to add anything to the slides. If you want to edit the slides for your own purposes, just make a copy of the slides.
Contributing: To add your own custom figures, simply add a new slide and reuse any of the basic visual components (remember to request edit permissions). We encourage authors/designers to add their visuals here and allow others to reuse them. Make sure to include your author information (in the notes section of the slide) so that others can provide the proper credit if they use the visuals elsewhere (e.g. blog/presentations). Add your name and email just in case someone has any questions related to the figures you added. Also, provide a short description of your visual to help the user understand what it is about and how they can use it. If you need "Edit" permission, just click on the "request edit access" option under the "view only" toolbar above or send me an email at [email protected] If you have editing rights, please make sure not to delete any of the work that other authors have added. You can try to improve it by creating a new slide and adding that improved version (that’s actually encouraged).
Downloading a figure from any of the slides is easy. Just click on File→Download→(choose your format).
If you need help with customizing a figure or have an idea of something that could be valuable to others, we can help. Just open an issue here and we will do our best to come up with the visual. Thanks.
Feel free to ask us anything related to this project in our Slack group.
BCI的应用背景
应用挑战
高变异性
其他方法的不足（链状，单维度，手工设计特征）
提出方法解决问题（多维度，端到端，自动学习）
数据集及问题概述（Visual Stimulus)
���成EEG（时域、频域）地形图
DSNN的提出（多维度）
结果
与传统方法、最新深度学习方法比较（准确率）
分析CNN的工作原理（反卷积）
分析Attention value与ERP和ERD/ERS关系
讨论
DSNN的优越性（SS,TS)
Subject-dependent场景下的结果（Subject-Independent的困难以及必要性）
端到端框架的实现
对image-based方法的分析
sigma=0.01
sigma=0.1
sigma=0.5
sigma=1
Decision Hyperplane
Original Space
Feature Space
Zero Point
Lightweight Openpose
ST-GCNs
Fall
Dection
Input Video
ST-GCNs
Lightweight Openpose
Fall Detection
Video Stream
Video Stream
Skeleton Extraction
Fall
Detection
Alert
Abnormal
Normal
User
File System
Sender
SMTP
SMTP
Commands/Replies
Receiver
SMTP
File System
Video Stream
Skeleton Extraction
Fall
Detection
Alert
Abnormal
Normal
Conv3-128
Conv3-128
Conv3-128
Conv1-512
Conv1-512
Conv1
Number Pafs
Conv1
Number Keypoints
Conv1-128
Conv1-128
Conv3-128
dia=2
Skeleton
Image
Decision Hyperplane f(x) = 0
f(x) = 1 Unpurchased
f(x) = -1
Purchased
ΔF₁ו
Feature Space
Learning Spatial-Spectral-Temporal EEG Representations with Dual-Stream Neural Networks for Motor Imagery
WEIJIAN MAI
For
Representation
EEG-Based BCI的应用背景
运动康复
情感识别
机器人
脑控轮椅
基于EEG的脑机接口能够使用户通过大脑与外界进行交流，最初用��运动性残疾患者的运动康复，近年出现了对于普通用户的情感脑机接口
BCI的应用挑战
High Variation
Intra-Subject
Inter-Subject
Subject-
Independent
（挑战）分类的本质：找到大脑活动与EEG的对应关系，不变表示
BCI的应用挑战
（挑战）目前方法的不足：传统方法（依赖hand-craft features), SOTA方法（单维度），难以适用于Subject-Independent场景
0~1s
Rest
Rest
ERP
PSD
T0
T1/T2
Motor Imagery
Time
T0
T1/T2
1~3s
Visual
Stimulus
Visual
Stimulus
Rest
Motor Imagery
Rest
Time
4-sec
Visual
Stimulus
Visual
Stimulus
An EEG Trial
L
L
L
L
L
L
L
L
L
L
C
C
C
C
C
C
C
C
C
C
L
L
L
L
L
L
L
L
L
L
A
A
A
A
A
A
A
A
A
A
L
L
L
L
L
L
L
L
L
L
C
C
C
C
C
C
C
C
C
C
L
L
L
L
L
L
L
L
L
L
A
A
A
A
A
A
A
A
A
A
Spectral-Stream
Spatial-Stream
L
L
L
L
L
L
L
L
C
C
C
C
C
C
C
C
L
L
L
L
L
L
L
L
A
A
A
A
A
A
A
A
Spectral-Stream
Spatial-Stream
EEG of all channels in time domain
(0-1s, downsampling to 100Hz)
EEG of all channels in frequency domain
(0-50Hz, average of 0-4s (one trial))
Spatial Topographic Maps
(25 frames)
Spectral Topographic Maps
(25 frames)
(a) EEG Topographic Maps Generation
L
L
L
L
L
L
L
L
C
C
C
C
C
C
C
C
L
L
L
L
L
L
L
L
A
A
A
A
A
A
A
A
Spectral-Stream
Temporal-Stream
EEG of all channels in time domain
(0-1s, downsampling to 100Hz)
EEG of all channels in frequency domain
(0-50Hz, average of 0-4s (one trial))
Spatial Topographic Maps
(25 frames)
Spectral Topographic Maps
(25 frames)
EEG Topographic Maps Generation
(b) Spatial-Spectral-Temporal Feature Learning
BiLSTM
CNN
ANN
FC
Layer
Softmax
Layer
L
L
L
L
L
L
L
L
C
C
C
C
C
C
C
C
L
L
L
L
L
L
L
L
A
A
A
A
A
A
A
A
Spectral-Stream
EEG of all channels in time domain
(0-1s, downsampling to 100Hz)
EEG of all channels in frequency domain
(0-50Hz, average of 0-4s (one trial))
Spatial Topographic Maps
(25 frames)
Spectral Topographic Maps
(25 frames)
(a) EEG Topographic Maps Generation
(b) Spatial-Spectral-Temporal Feature Learning
Bi-LSTM
CNN
ANN
FC
Layer
Softmax
Layer
Temporal-Stream
0ms
40ms
80ms
960ms
0Hz
2Hz
4Hz
48Hz
L
L
L
L
L
L
L
L
C
C
C
C
C
C
C
C
L
L
L
L
L
L
L
L
A
A
A
A
A
A
A
A
Spectral-Stream
Amplitude of all channels in time domain
(0-1s, downsampling to 100Hz)
PSD of all channels in frequency domain
(0-50Hz, average of 0-4s (one trial))
Spatial Topographic Maps
(25 frames)
Spectral Topographic Maps
(25 frames)
(a) EEG Topographic Maps Generation
(b) Spatial-Spectral-Temporal Feature Learning
Bi-LSTM
CNN
ANN
FC
Layer
Softmax
Layer
Temporal-Stream
Bicubic
Amplitude Distribution
(Interval: 40ms)
20ms
60ms
100ms
940ms
980ms
Power Distribution
(Interval: 2Hz)
1Hz
3Hz
5Hz
47Hz
49Hz
Spectral EEG Topographic Maps
(25 frames)
Temporal EEG Topographic Maps
(25 frames)
EEG of all channels in time domain (0-1s, downsampling to 100Hz)
EEG of all channels in frequency domain (0-50Hz, average of 0-4s (one trial))
Bicubic
L
L
L
L
L
L
L
L
L
L
C
C
C
C
C
C
C
C
C
C
L
L
L
L
L
L
L
L
L
L
A
A
A
A
A
A
A
A
A
A
C
L
L
L
L
L
L
L
L
C
C
C
C
C
C
L
L
L
L
L
L
L
L
A
A
A
A
A
A
A
A
Spectral-Stream
Amplitude of all channels in time domain
(0-1s, downsampling to 100Hz)
PSD of all channels in frequency domain
(0-50Hz, average of 0-4s (one trial))
Spatial Topographic Maps
(25 frames)
Spectral Topographic Maps
(25 frames)
(a) EEG Topographic Maps Generation
(b) DSNN for Spatial-Spectral-Temporal Representations Learning
RNN
CNN
ANN
Softmax
Layer
Temporal-Stream
Bicubic
Bicubic
C
40ms
80ms
120ms
1000ms
2Hz
4Hz
6Hz
50Hz
(c) MI Classification
C
L
L
L
L
L
L
L
L
C
C
C
C
C
C
L
L
L
L
L
L
L
L
A
A
A
A
A
A
A
A
Spectral-Stream
Amplitude of all channels in time domain
(0-1s, downsampling to 100Hz)
PSD of all channels in frequency domain
(0-50Hz, average of 0-4s (one trial))
Spatial Topographic Maps
(50 frames)
Spectral Topographic Maps
(50 frames)
EEG Topographic Map Generation
DSNN for Spatial-Spectral-Temporal Representation Learning
RNN
CNN
ANN
Softmax
Layer
Temporal-Stream
Bicubic
Bicubic
C
20ms
40ms
60ms
1000ms
1Hz
2Hz
3Hz
50Hz
MI Classification
Frequency (Hz)
μV²/Hz (dB)
Time (s)
μV
EEG (64 channels)
EEG (64 channels)
C
L
L
L
L
L
L
L
L
C
C
C
C
C
C
L
L
L
L
L
L
L
L
A
A
A
A
A
A
A
A
Spectral-Stream
Amplitude of all channels in time domain
(0-1s, downsampling to 100Hz)
PSD of all channels in frequency domain
(0-50Hz, average of 0-4s (one trial))
Temporal Topographic Maps
(50 frames)
Spectral Topographic Maps
(50 frames)
(a) EEG Topographic Map Generation
(b) DSNN for Spatial-Spectral-Temporal Representation Learning
RNN
CNN
ANN
Softmax
Layer
Temporal-Stream
Bicubic
Bicubic
C
20ms
40ms
60ms
1000ms
1Hz
2Hz
3Hz
50Hz
(c) MI Classification
Frequency (Hz)
μV²/Hz (dB)
Time (s)
μV
EEG (64 channels)
EEG (64 channels)
head-like
topographic map
sensor positions
box-like
topographic map
head-like
topographic map
sensor positions
box-like
topographic map
(32 x 32)
Right Fist
Left Fist
Both Hands
Both
Feet
Right Fist
Left Fist
Both Hands
Both
Feet
Time
P7
P6
P5
P5
Spectral-Stream
Temporal-Stream
Temporal-Stream
Spectral-Stream
Temporal-Stream
(a)Spectral-Stream
(b)Temporal-Stream
Kernel #3
Kernel #7
Kernel #8
Kernel #15
Kernel #23
Kernel #35
Kernel #46
Kernel #63
Kernel #5
Kernel #13
Kernel #25
Kernel #35
Kernel #47
Kernel #51
Kernel #53
Kernel #64
C4
Cz
C3
Fz
CPz
Pz
Cz
(a)Spectral-Stream
(b)Temporal-Stream
Kernel #3
Kernel #7
Kernel #8
Kernel #15
Kernel #23
Kernel #35
Kernel #46
Kernel #63
Kernel #5
Kernel #13
Kernel #25
Kernel #35
Kernel #47
Kernel #51
Kernel #53
Kernel #64
C4
C4
C3
Cz
Subject #10
Subject #12
Subject #3
Subject #10
Subject #12
Subject #3
0.5
1
Kernel #35
Kernel #48
0
Spectral Power
No
Low
High
Intolerable
No
Low
High
Intolerable
1
0
0.5
1
Kernel #35
Kernel #48
0
Spectral Power
No
Low
High
Intolerable
No
Low
High
Intolerable
1
0
0.5
1
Kernel #35
0
Spectral Power
No
Low
High
Intolerable
1
0
Kernel #35
No
Low
High
Intolerable
1
0
1
0
1
0
Kernel #48
Kernel #61
Kernel #61
Kernel #35
No
Low
High
Intolerable
1
0
1
0
1
0
Kernel #48
Kernel #61
Kernel #61
1
0
1
0
1
No
Low
High
Intolerable
No
Low
High
Intolerable
Confusion Matrix
Predicted Class
True Class
No
Low
High
Intolerable
No
Low
High
Intolerable
Confusion Matrix
Predicted Class
True Class
Kernel #35
Kernel #48
0.5
0
1
Spectral Power
No
Low
High
Intolerable
No
Low
High
Intolerable
Attention Matrix
ERP
Attention Matrix
PSD
Attention Matrix
ERP
Attention Matrix
PSD
Test sample #12
Test sample #27
0
25
1
Test sample #12
Test sample #27
Test s