Machine Learning to Deep Learning_2

Presentation on theme: "Machine Learning to Deep Learning_2"— Presentation transcript:

1 Machine Learning to Deep Learning_2
Tutorial code: Jemin Lee Hompage: 역사 알고리즘 약간은 깊이 있는 이해 그것을 통한 실제 구현 과 응용 시스템 Area에서의 Deep-Learning

2 Table of Contents Fundamental Machine Learning (1일차)
Linear Regression: Gradient Descent Algorithm (optimization) Logistic Regression (Single Neuron=Perceptron): Sigmoid (Logistic function), Convexity, Cross Entropy, Decision Boundary Multiple Perceptron (Hidden Layer): Backpropagation algorithm Deep Neural Network Breakthrough (2-3일차) Rebirth of Neural Network, renamed DNN TensorFlow Basic DNN, ReLU, Pre-training, Dropout Convolutional Neural Network (CNN) How to apply DNN into real world problem (4일차) Use-case: smarttention 2016

3 Deep Learning Framework

4 왜 TensorFlow 인가?

5 Large-Scale Deep Learning With TensorFlow, JeffDean 2016

6 Large-Scale Deep Learning With TensorFlow, JeffDean 2016

7 Large-Scale Deep Learning With TensorFlow, JeffDean 2016

8 Architecture Large-Scale Deep Learning With TensorFlow, JeffDean 2016

9 Portable & Scalable Large-Scale Deep Learning With TensorFlow, JeffDean 2016

10 How is TensorFlow used at Google?
Recognizing Images with Inception Voice Recognition Smart reply in inbox by Gmail Large-Scale Deep Learning With TensorFlow, JeffDean 2016

11 실습: Basic TensorFlow 역사 알고리즘 약간은 깊이 있는 이해 그것을 통한 실제 구현 과 응용
시스템 Area에서의 Deep-Learning

12 Graph, Node, Edge 선언부와 실행부가 다름 코드 위치
“앞으로 어떻게 동작 할 것이다”라는 계획을 Graph로 표현한 것이 TensorFlow 이다. Operation: 동작을 정의 Node: Operation 정의 포함 Edge: node와 node를 연결함. 코드 위치 0.2.Basic/Basic Tutorial.ipynb

13 실습2: MNIST 역사 알고리즘 약간은 깊이 있는 이해 그것을 통한 실제 구현 과 응용
시스템 Area에서의 Deep-Learning

14 출처: 모두를 위한 머신러닝 시즌1, 김성훈 교수님

15 Coursera, Machine Learning, Andrew Ng

16 Diagnosing bias vs. variance
High Bias problem == underfitting High variance problem == overfitting Coursera, Machine Learning, Andrew Ng

17 Coursera, Machine Learning, Andrew Ng

18 Learning curves[1/3] Coursera, Machine Learning, Andrew Ng

19 Learning curves[2/3] Coursera, Machine Learning, Andrew Ng

20 Learning curves[3/3] Coursera, Machine Learning, Andrew Ng

21 Summarized bias vs. variance
Coursera, Machine Learning, Andrew Ng

22 Coursera, Machine Learning, Andrew Ng

23 코드 위치 al_DNN.ipynb

24 구현 내용 기본적인 MLP 구현 11-layer MLP with ReLU Pre-training and Dropout
One hidden layer Testing accuracy: ~92% 11-layer MLP with ReLU ReLU, Sigmoid, Softmax Testing accuracy: ~97% Pre-training and Dropout Xavier init. and new regularization Testing accuracy: ~99% Convolutional Neural Network Testing accuracy: ?%