Author(s):
Zhang, Lei
Wang, Shuai
Liu, Bing
Abstract:
Deep learning has emerged as a powerful machine learning technique that learns multiple layers of representations or features of the data and produces state‐of‐the‐art prediction results. Along with the success of deep learning in many application domains, deep learning is also used in sentiment analysis in recent years. This paper gives an overview of deep learning and then provides a comprehensive survey of its current applications in sentiment analysis.
This article is categorized under:
- Fundamental Concepts of Data and Knowledge > Data Concepts
- Algorithmic Development > Text Mining
Document:
https://doi.org/10.1002/widm.1253
References:
[1] Liu B. Sentiment analysis: mining opinions, sentiments, and emotions. The Cambridge University Press, 2015.
[2] Liu B. Sentiment analysis and opinion mining (introduction and survey),Morgan & Claypool, May 2012.
[3] Pang B and Lee L. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2008. 2(1–2): pp. 1–135.
[4] Goodfellow I, Bengio Y, Courville A. Deep learning. The MIT Press. 2016.
[5] Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. In Proceedings of the International Conference on Artificial Intelligence and Statistics(AISTATS 2011), 2011.
[6] Rumelhart D.E, Hinton G.E, Williams R.J. Learning representations by back-propagating errors. Cognitive modelling,1988.
[7] Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, and Kuksa P. Natural languageprocessing (almost) from scratch. Journal of Machine Learning Research, 2011.[
8] Goldberg Y. A primer on neural network models for natural language processing. Journal of Artificial Intelligence Research, 2016.
[9] Bengio Y, Courville A, Vincent P. Representation learning: a review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013.
10] LeeH, GrosseR, RanganathR, andNgA.Y.Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the International Conference on Machine Learning(ICML 2009), 2009.
[11] Bengio Y, Ducharme R, Vincent P, and Jauvin C. A neural probabilistic language model. Journal of Machine Learning Research, 2003.
[12] Morin F,Bengio Y. Hierarchical probabilistic neural network language model. In Proceedings of the International Workshop on Artificial Intelligence and Statistics, 2005.
[13] Mikolov T, Chen K, Corrado G, and Dean J. Efficient estimation of word representations in vector space. In Proceedings of International Conference on Learning Representations(ICLR 2013), 2013.
[14] Mikolov T, Sutskever I, Chen K, Corrado G, and Dean J. Distributed representations of words and phrases and their compositionality. In Proceedings ofthe Annual Conference on Advances inNeural Information Processing Systems (NIPS 2013), 2013.
[15] Mnih A,Kavukcuoglu K. Learning word embeddings efficiently with noise-contrastive estimation. In Proceedings of the Annual Conference on Advances inNeuralInformation Processing Systems (NIPS 2013), 2013.
[16] Huang E.H, Socher R, Manning C.D. and Ng A.Y. Improving word representations via global context and multiple word prototypes. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2012), 2012.
[17] Pennington J, Socher R, Manning C.D. GloVe: global vectors for word representation. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP 2014), 2014.
[18] Bengio Y, Lamblin P, Popovici D, and Larochelle H. Greedy layer-wise training of deep networks. In Proceedings of the Annual Conference on Advances inNeural Information Processing Systems (NIPS 2006), 2006.
[19] Hinton G.E, Salakhutdinov R.R. Reducing the dimensionality of data with neural networks. Science, July 2006.
[20] Vincent P, Larochelle H, Bengio Y, and Manzagol P-A. Extracting and composing robust features with denoising autoencoders. In Proceedings of the International Conference on Machine Learning(ICML 2008), 2008. [
21] Sermanet P, LeCun Y. Traffic sign recognition with multi-scale convolutional networks. In Proceedings of the International Joint Conference on Neural Networks(IJCNN 2011), 2011.
[22] Elman J.L. Finding structure in time. Cognitive Science, 1990.
[23] Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 1994.
[24] Schuster M, Paliwal K.K. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing,1997.
[25] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 9(8): 1735-1780, 1997.
[26] Tai K.S, Socher R, Manning C. D. Improved semantic representations from tree-structured long short-term memory networks.In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL 2015), 2015.
[27] Cho K, Bahdanau D, Bougares F, Schwenk H and Bengio Y. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2014), 2014.
[28] Chung J, Gulcehre C, Cho K, Bengio Y. Empirical evaluation of gated recurrent neural networks on sequence modelling. arXiv preprint arXiv:1412.3555, 2014.
[29] Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.
[30] Weston J, Chopra S, Bordes A. Memory networks. arXiv preprint arXiv:1410.3916. 2014.
[31] Sukhbaatar S, Weston J, Fergus R. End-to-end memory networks. In Proceedings of the 29th Conference on Neural Information Processing Systems(NIPS 2015), 2015.
[32] Graves A,Wayne G, Danihelka I. Neural Turing Machines. preprint arXiv:1410.5401. 2014.
[33] Qian Q, Tian B, Huang M, Liu Y, Zhu X and Zhu X. Learning tag embeddings and tag-specific composition functions in the recursive neural network. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2015), 2015.
[34] Moraes R, Valiati J.F, Neto W.P. Document-level sentiment classification: an empirical comparison between SVM and ANN. Expert Systems with Applications. 2013.
[35] Le Q, Mikolov T. Distributed representations of sentences and documents. In Proceedings of the International Conference on Machine Learning(ICML 2014), 2014.
[36] Glorot X, Bordes A, Bengio Y. Domain adaption for large-scale sentiment classification: a deep learning approach. In Proceedings of the International Conference on Machine Learning(ICML 2011), 2011.
[37] Zhai S, Zhongfei (Mark) Zhang. Semisupervised autoencoder for sentiment analysis. In Proceedings of AAAI Conference on Artificial Intelligence(AAAI 2016), 2016.
[38] Johnson R, Zhang T. Effective use of word order for text categorization with convolutional neural networks. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies(NAACL-HLT 2015), 2015.
[39] Tang D, Qin B, Liu T. Document modelling with gated recurrent neural network for sentiment classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2015), 2015.
[40] Tang D, Qin B, Liu T. Learning semantic representations of users and products for document level sentiment classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL 2015), 2015.
[41] Chen H, Sun M, Tu C, Lin Y,andLiu Z. Neural sentiment classification with user and product attention. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2016), 2016.
[42] Dou ZY. Capturing user and product Information for document level sentiment analysis with deep memory network. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP 2017), 2017.
[43] Xu J, Chen D, Qiu X,andHuang X. Cached long short-term memory neural networks for document-level sentiment classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2016), 2016.
[44] Yang Z, Yang D, Dyer C, He X, Smola AJ,andHovy EH. Hierarchical attention networks for document classification. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2016), 2016.
[45] Yin Y, Song Y, Zhang M. Document-level multi-aspect sentiment classification as machine comprehension. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2017), 2017.
[46] Zhou X, Wan X, Xiao J.Attention-based LSTM network for cross-lingual sentiment classification.In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2016), 2016.
[47] Li Z, Zhang Y, Wei Y, Wu Y, and Yang Q. End-to-end adversarial memory network forcross-domain sentiment classification. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2017), 2017.
[48] Wiebe J, Bruce R, and O’Hara T. Development and use of a gold standard data set for subjectivity classifications. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL 1999), 1999.
[49] Socher R, Pennington J, Huang E.H, Ng A.Y, and Manning C.D. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2011), 2011.
[50] Socher R, Huval B, Manning C.D, and Ng A.Y. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP 2012), 2012.
[51] Socher R, Perelygin A, Wu J. Y, Chuang J, Manning C.D, Ng A. Y, and Potts C. Recursive deep models for semantic compositionality over asentiment treebank. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP 2013), 2013.
[52] Kalchbrenner N, Grefenstette E, Blunsom P. A convolutional neural network for modelling sentences. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2014), 2014.
[53] Kim Y. Convolutional neural networks for sentence classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2014), 2014.
[54] dos Santos, C. N., Gatti M. Deep convolutional neural networks for sentiment analysis for short texts. In Proceedings of the International Conference on Computational Linguistics(COLING 2014), 2014.
[55] Wang X, Liu Y, Sun C, Wang B, and Wang X. Predicting polarities of tweets by composing word embeddings with long short-term memory. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2015), 2015.
[56] Graves A,Schmidhuber J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks, 2005.
[57] Wang J, Yu L-C, Lai R.K., and Zhang X. Dimensional sentiment analysis using a regional CNN-LSTM model. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL 2016), 2016.
[58] Wang X, Jiang W, Luo Z. Combination of convolutional and recurrent neural network for sentiment analysis of short texts. In Proceedings of the International Conference on Computational Linguistics(COLING 2016), 2016.
[59] Guggilla C, Miller T, Gurevych I.CNN-and LSTM-based claim classification in online user comments. In Proceedings of the International Conference on Computational Linguistics(COLING 2016), 2016.
[60] Huang M, Qian Q, Zhu X. Encoding syntactic knowledge in neural networks for sentiment classification. ACM Transactions on Information Systems, 2017
[61] Akhtar MS, Kumar A, Ghosal D, Ekbal A, and Bhattacharyya P. A multilayer perceptron based ensemble technique for fine-grained financial sentiment analysis. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP 2017), 2017.
[62] Guan Z, Chen L, Zhao W, Zheng Y, Tan S, and Cai D. Weakly-supervised deep learning for customer review sentiment classification. In Proceedings of the International Joint Conference on Artificial Intelligence(IJCAI 2016), 2016.
[63] Teng Z, Vo D-T, and Zhang Y. Context-sensitive lexicon features for neural sentiment analysis. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2016), 2016.
[64] Yu J, Jiang J. Learning sentence embeddings with auxiliary tasks for cross-domain sentiment classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2016), 2016.
[65] Zhao Z, Lu H, Cai D, He X, Zhuang Y. Microblog sentiment classification via recurrent random walk network learning.In Proceedings of the Internal Joint Conference on Artificial Intelligence(IJCAI 2017), 2017.
[66] Mishra A, Dey K, Bhattacharyya P. Learning cognitive features from gaze data for sentiment andsarcasm classification using convolutional neural network. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2017), 2017.
[67] Qian Q, Huang M, Lei J, and Zhu X. Linguistically regularized LSTM for sentiment classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2017), 2017.
[68] Dong L, Wei F, Tan C, Tang D, Zhou M,andXu K. Adaptive recursive neural network for target-dependent Twitter sentiment classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2014), 2014.
[69] Vo D-T, Zhang Y. Target-dependent twitter sentiment classification with rich automatic features. In Proceedings of the Internal Joint Conference on Artificial Intelligence(IJCAI 2015), 2015.
[70] Tang D, Qin B, Feng X, and Liu T. Effective LSTMs for target-dependent sentiment classification. In Proceedings of the International Conference on Computational Linguistics(COLING 2016), 2016.
[71] RuderS, Ghaffari P, Breslin J.G. A hierarchical model of reviews for aspect-based sentiment analysis. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP 2016), 2016.
[72] Zhang M, Zhang Y,Vo D-T. Gated neural networks for targeted sentiment analysis. In Proceedings of AAAI Conference on Artificial Intelligence(AAAI 2016), 2016.
[73] Wang Y, Huang M, Zhu X, and Zhao L. Attention-based LSTM for aspect-level sentiment classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2016), 2016.
[74] Yang M, Tu W, Wang J, Xu F, and Chen X. Attention-based LSTM for target-dependent sentiment classification. In Proceedings of AAAI Conference on Artificial Intelligence(AAAI 2017), 2017.
[75] Liu J, Zhang Y. Attention modeling for targeted sentiment. In Proceedings of the Conference of the EuropeanChapter of the Association for Computational Linguistics(EACL 2017), 2017.
[76] Tang D, Qin B, and Liu T. Aspect-level sentiment classification with deep memory network. arXiv preprint arXiv:1605.08900, 2016.
[77] Lei T, Barzilay R, Jaakkola T. Rationalizing neural predictions. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP 2016), 2016.
[78] Li C, Guo X, Mei Q. Deep memory networks for attitude Identification. In Proceedings of the ACM International Conference on Web Search and Data Mining (WSDM2017), 2017.
[79] Ma D, Li S, Zhang X, Wang H. Interactive attention networks for aspect-Level sentiment classification.In Proceedings of the Internal Joint Conference on Artificial Intelligence(IJCAI 2017), 2017.
[80] Chen P, Sun Z, Bing L, and Yang W. Recurrent attention network on memory for aspect sentiment analysis. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP 2017), 2017.
[81] Tay Y, Tuan LA, Hui SC. Dyadic memory networks for aspect-based sentiment analysis. In Proceedings of the International Conference on Information and Knowledge Management(CIKM2017), 2017.
[82] Katiyar A, Cardie C. Investigating LSTMs for joint extraction of opinion entities and relations. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL 2016), 2016.
[83] Wang W, Pan SJ, Dahlmeier D,andXiao X. Recursive neural conditional random fields for aspect-based sentiment analysis. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2016), 2016.
[84] Wang W, Pan SJ, Dahlmeier D,andXiao X. Coupled multi-Layer attentions for co-extraction of aspect and opinion terms. In Proceedings ofAAAI Conference on Artificial Intelligence(AAAI 2017), 2017.
[85] Li X, Lam W. Deep multi-task learning for aspect term extraction with memory Interaction. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP 2017), 2017.
[86] He R, Lee WS, Ng HT,andDahlmeier D. An unsupervised neural attention model for aspect extraction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2017), 2017.
[87] Zhang M, Zhang Y, Vo D-T. Neural networks for open domain targeted sentiment. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2015), 2015.
[88] Zhou X, Wan X, XiaoJ. Representation learning for aspect category detection in online reviews. In Proceeding of AAAI Conference on Artificial Intelligence(AAAI 2015), 2015.
[89] Yin Y, Wei F, Dong L, Xu K, Zhang M, and Zhou M. Unsupervised word and dependency path embeddings for aspect term extraction. In Proceedings of the International Joint Conference on Artificial Intelligence(IJCAI 2016), 2016.
[90] Xiong S, Zhang Y, Ji D,andLou Y. Distance metric learning for aspect phrase grouping.In Proceedings of the International Conference on Computational Linguistics(COLING 2016), 2016.
[91] Poria S, Cambria E, Gelbukh A. Aspect extraction for opinion mining with a deep convolutional neural network. Journal of Knowledge-based Systems. 2016.
[92] Ying D, Yu J, Jiang J. Recurrent neural networks with auxiliary labels for cross-domain opinion target extraction. In Proceedings of AAAI Conference on Artificial Intelligence(AAAI 2017), 2017
[93] Irsoy O, Cardie C. Opinion mining with deep recurrentneural networks. In Proceedingsof the Conference on Empirical Methods on Natural Language Processing (EMNLP 2014), 2014.
[94] Yang B, Cardie C. Extracting opinion expressions with semi-markov conditional random fields. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2012), 2012.
[95] Liu P, Joty S, Meng H. Fine-grained opinion mining with recurrent neural networks and word embeddings. In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2015), 2015.
[96] Irsoy O, Cardie C. Deep recursive neural networks for compositionality in language. In Proceedings of the Annual Conference on Advances inNeural Information Processing Systems (NIPS 2014), 2014.
[97] Zhu X, Guo H, Sobhani P. Neural networks for integrating compositional and non-compositional sentiment in sentiment composition. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies(NAACL-HLT 2015), 2015.
[98] Yang B,Cardie C. Joint Inference for fine-grained opinion extraction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2013), 2013.
[99] Deng L, Wiebe J. Recognizing opinion sources based on a new categorization of opinion types. In Proceedings of the International Joint Conference on Artificial Intelligence(IJCAI 2016), 2016.
[100] Chen C, Wang Z, Lei Y, and Li W. Content-based influence modelling for opinion behaviour Prediction. In Proceedings of the International Conference on Computational Linguistics(COLING 2016), 2016.
[101] Rashkin H, Bell E, Choi Y,andVolkova S. Multilingual connotation frames: a case study on social media for targeted sentiment analysis and forecast. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2017), 2017.
[102] Mass A. L, Daly R. E, Pham P. T, Huang D, Ng A. Y. and Potts C. Learning word vectors for sentiment analysis. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2011), 2011.
[103] Bespalov D, Bai B, Qi Y, and Shokoufandeh A. Sentiment classification based on supervised latent n-gram analysis. In Proceedings of the International Conference on Information and Knowledge Management(CIKM 2011), 2011.
[104] Labutov I, Lipson H. Re-embedding words. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2013), 2013.
[105] Tang D, Wei F, Yang N, Zhou M, Liu T, and Qin B. Learning sentiment-specific word embedding for twitter sentiment classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL 2014), 2014.
[106] Tang D, Wei F, Qin B, Yang N, Liu T, and Zhoug M. Sentiment embeddings with applications to sentiment analysis. IEEE Transactions on Knowledge and Data Engineering, 2016.
[107] Wang L, Xia R. Sentiment Lexicon construction with representation Learning based on hierarchical sentiment Supervision. In Proceedings of the Conference on Empirical Methods on Natural Language Processing (EMNLP 2017), 2017.
[108] Yu LC, Wang J, Lai KR,andZhang X. Refining word embeddings for sentiment analysis. In Proceedings of the Conference onEmpirical Methods on Natural Language Processing (EMNLP 2017), 2017.
[109] Li J, Jurafsky D. Do multi-sense embeddings improve natural language understanding?In Proceedings of the Conference on Empirical Methods in Natural Language Processing(EMNLP 2015), 2015.
[110] Ren Y, Zhang Y, Zhang,M and Ji D. Improving Twitter sentiment classification using topic-enriched multi-prototype word embeddings. In Proceeding of AAAI Conference on Artificial Intelligence(AAAI 2016), 2016.
[111] Zhou H, Chen L, Shi F, Huang D. Learning bilingual sentiment word embeddings for cross-language sentiment classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics(ACL 2015), 2015.
[112]Barnes J,Lambert P, Badia T. Exploring distributional representations and machine translation for aspect-based cross-lingual sentiment classification.In Proceedings of the 27th International Conference on Computat