回首頁

回第10期

什麼是深度學習
機器學習的卷土重來

全文閱覽

作        者　李宏毅

作者簡介　李宏毅為臺灣大學電機工程系暨資訊網路與多媒體研究所助理教授，主要研究領域：機器學習、深度學習、語意理解、語音辨識。

延伸閱讀 

• Nielsen, Michael A. "Neural Networks and Deep Learning" （2015） Determination Press。這是作者多次推薦的線上教科書：http://neuralnetworksanddeeplearning.com/
• 李宏毅 〈一天搞懂深度學習〉，作者2016 年9 月24 日在「2016 臺灣資料科學愛好者年會」給了一天四場的課程。底下是其演講幻燈片。對於本文有興趣的讀者可以參考：http://www.slideshare.net/tw_dsconf/ss-62245351
• Deep Learning CONCEPTS，YouTube 視頻的系列介紹課程。https://goo.gl/BD5djv

參考資料

[1] Michael A. Nielsen, "Neural Networks and Deep Learning", Determination Press, 2015, link: http://neuralnetworksanddeeplearning.com/
[2] Geoffrey E. Hinton, Simon Osindero, Yee-Whye The, “A fast learning algorithm for deep belief nets”, Neural computation, Vol. 18, No. 7, 2006
[3] Li Deng, Geoffrey E. Hinton, Brian Kingsbury, “New types of deep neural network learning for speech recognition and related applications: an overview,” ICASSP 2013
[4] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, “Deep Residual Learning for Image Recognition”, CVPR 2016
[5] Xavier Glorot, Antoine Bordes and Yoshua Bengio. “Deep sparse rectifier neural networks”. AISTATS, 2011
[6] Ian J. Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, Yoshua Bengio, “Maxout Networks.”, ICML, 2013
[7] Diederik Kingma, Jimmy Ba, “Adam: A Method for Stochastic Optimization”, ICLR, 2015
[8] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting” , Vol. 15, No. 1., JMLR, 2014
[9]
[10] Geoffrey Hinton, Li Deng, Dong Yu, George Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara Sainath, and Brian Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups." IEEE Signal Processing Magazine, vol. 29, no. 6, 2012
[11]  Sepp Hochreiter and Jürgen Schmidhuber,  "Long short-term memory", Neural Computation, Vol. 9, No. 8, 1997
[12] Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, Yoshua Bengio, “Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation, EMNLP, 2014
[13] Hasim Sak, Andrew Senior, Kanishka Rao, Francoise Beaufays, “Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition”, Interspeech, 2015
[14] Ilya Sutskever, Oriol Vinyals, Quoc V. Le, “Sequence to Sequence Learning with Neural Networks”, NIPS, 2014
[15] Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus. End-To-End Memory Networks.  NIPS, 2015
[16] Alex Graves, Greg Wayne, Ivo Danihelka. Neural Turing Machines. arXiv, 2014
[17] Ankit Kumar, Ozan Irsoy, Peter Ondruska, Mohit Iyyer, James Bradbury, Ishaan Gulrajani, Victor Zhong, Romain Paulus, Richard Socher. Ask Me Anything: Dynamic Memory Networks for Natural Language Processing. ICML 2016
[18] Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh, VQA: Visual Question Answering, ICCV, 2015
[19] Bo-Hsiang Tseng, Sheng-Syun Shen, Hung-Yi Lee, Lin-Shan Lee, Towards Machine Comprehension of Spoken Content: Initial TOEFL Listening Comprehension Test by Machine, Interspeech 2016
[20] Wei Fang, Juei-Yang Hsu, Hung-yi Lee, Lin-Shan Lee, Hierarchical Attention Model for Improved Machine Comprehension of Spoken Content, submitted to SLT 2016
[21] A. M. Rush, S. Chopra and J. Weston. A Neural Attention Model for Abstractive Sentence Summarization. EMNLP 2015.
[22] Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel,Yoshua Bengio. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. ICML, 2015
[23] William Chan, Navdeep Jaitly, Quoc V. Le, Oriol Vinyals, “Listen, Attend and Spell: A Neural Network for Large Vocabulary Conversational Speech Recognition”, ICASSP 2016
[24] D. Bahdanau, K. Cho, Y. Bengio. Neural Machine Translation by Jointly Learning to Align and Translate. ICLR, 2015
[25] Yu-An Chung, Chao-Chung Wu, Chia-Hao Shen, Hung-Yi Lee, Lin-Shan Lee, Audio Word2Vec: Unsupervised Learning of Audio Segment Representations using Sequence-to-sequence Autoencoder, Interspeech 2016
[26] Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean, “Efficient Estimation of Word Representations in Vector Space”,  ICLR, 2013
[27] Mnih, Volodymyr, Kavukcuoglu, Koray, Silver, David, Rusu, Andrei A, Veness, Joel, Bellemare, Marc G, Graves, Alex, Riedmiller, Martin, Fidjeland, Andreas K, Ostrovski, Georg, et al. Humanlevel control through deep reinforcement learning. Nature, 518(7540):529–533, 2015.
[28] Silver, David, Huang, Aja, Maddison, Chris J., Guez, Arthur, Sifre, Laurent, van den Driessche, George, Schrittwieser, Julian, Antonoglou, Ioannis, Panneershelvam, Veda, Lanctot, Marc, Dieleman, Sander, Grewe, Dominik, Nham, John, Kalchbrenner, Nal, Sutskever, Ilya, Lillicrap, Timothy, Leach, Madeleine, Kavukcuoglu, Koray, Graepel, Thore, and Hassabis, Demis. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484–489, 2016
[29] Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun, Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, ICCV, 2015