Multi-Objective Manifold Representation for Opinion Mining
Pshtiwan Rahman
Department of Computer Engineering, University of Kurdistan, Sanandaj, Kurdistan, Iran
Search for more papers by this authorCorresponding Author
Fatemeh Daneshfar
Department of Computer Engineering, University of Kurdistan, Sanandaj, Kurdistan, Iran
Correspondence:
Fatemeh Daneshfar ([email protected])
Search for more papers by this authorHashem Parvin
Department of Computer Engineering, University of Kurdistan, Sanandaj, Kurdistan, Iran
Search for more papers by this authorPshtiwan Rahman
Department of Computer Engineering, University of Kurdistan, Sanandaj, Kurdistan, Iran
Search for more papers by this authorCorresponding Author
Fatemeh Daneshfar
Department of Computer Engineering, University of Kurdistan, Sanandaj, Kurdistan, Iran
Correspondence:
Fatemeh Daneshfar ([email protected])
Search for more papers by this authorHashem Parvin
Department of Computer Engineering, University of Kurdistan, Sanandaj, Kurdistan, Iran
Search for more papers by this authorABSTRACT
Sentiment analysis plays a crucial role across various domains, requiring advanced methods for effective dimensionality reduction and feature extraction. This study introduces a novel framework, multi-objective manifold representation (MOMR) for opinion mining, which uniquely integrates deep global features with local manifold representations to capture comprehensive data patterns efficiently. Unlike existing methods, MOMR employs advanced dimensionality reduction techniques combined with a self-attention mechanism, enabling the model to focus on contextually relevant textual elements. This dual approach not only enhances interpretability but also improves the performance of sentiment analysis. The proposed method was rigorously evaluated against both classical techniques such as long short-term memory (LSTM), naive Bayes (NB) and support vector machines (SVMs), and modern state-of-the-art models including recurrent neural networks (RNN) and convolutional neural networks (CNN). Experiments on diverse datasets: IMDB, Fake News, Twitter and Yelp demonstrated the superior accuracy and robustness of MOMR. By outperforming competing methods in terms of generalizability and effectiveness, MOMR establishes itself as a significant advancement in sentiment analysis, with broad applicability in real-world opinion mining tasks (https://github.com/pshtirahman/Sentiment-Analysis.git).
Open Research
Data Availability Statement
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.
References
- Abdi, H. 2007. “Singular Value Decomposition (SVD) and Generalized Singular Value Decomposition.” Encyclopedia of Measurement and Statistics 907, no. 912: 44.
- Abdulrahman, A., and M. Baykara. 2020. “ Fake News Detection Using Machine Learning and Deep Learning Algorithms.” In 2020 International Conference on Advanced Science and Engineering (ICOASE), 18–23.
10.1109/ICOASE51841.2020.9436605 Google Scholar
- Akritidis, L., and P. Bozanis. 2022. “ How Dimensionality Reduction Affects Sentiment Analysis Nlp Tasks: An Experimental Study.” In IFIP International Conference on Artificial Intelligence Applications and Innovations, 301–312. Springer.
10.1007/978-3-031-08337-2_25 Google Scholar
- Alamoudi, E. S., and N. S. Alghamdi. 2021. “Sentiment Classification and Aspect-Based Sentiment Analysis on Yelp Reviews Using Deep Learning and Word Embeddings.” Journal of Decision Systems 30, no. 2–3: 259–281.
- Ali, N. M., M. M. Abd El Hamid, and A. Youssif. 2019. “Sentiment Analysis for Movies Reviews Dataset Using Deep Learning Models.” International Journal of Data Mining & Knowledge Management Process 9: 19–27.
10.5121/ijdkp.2019.9302 Google Scholar
- Angiani, G., L. Ferrari, T. Fontanini, et al. 2016. “ A Comparison Between Preprocessing Techniques for Sentiment Analysis in Twitter.” In International Workshop on Knowledge Discovery on the WEB, vol. 7, 37–56.
- Anusha, M., and R. Leelavathi. 2024. “Sentiment Analytics on Sarcasm Detection Using Bi-LSTM-1DCNN Model for Fake News Detection.” International Journal of Intelligent Systems and Applications in Engineering 12, no. 5s: 122–141.
- Artstein, R., and M. Poesio. 2008. “Inter-Coder Agreement for Computational Linguistics.” Computational Linguistics 34, no. 4: 555–596.
- Asghar, M. Z., A. Khan, S. Ahmad, and F. M. Kundi. 2014. “A Review of Feature Extraction in Sentiment Analysis.” Journal of Basic and Applied Scientific Research 4, no. 3: 181–186.
- Başarslan, M. S., and F. Kayaalp. 2021. “Sentiment Analysis on Social Media Reviews Datasets With Deep Learning Approach.” Sakarya University Journal of Computer and Information Sciences 4, no. 1: 35–49.
10.35377/saucis.04.01.833026 Google Scholar
- Belkin, M., and P. Niyogi. 2003. “Laplacian Eigenmaps for Dimensionality Reduction and Data Representation.” Neural Computation 15, no. 6: 1373–1396.
- Bharadwaj, P., and Z. Shao. 2019. “Fake News Detection With Semantic Features and Text Mining.” International Journal on Natural Language Computing 8: 35–55.
10.5121/ijnlc.2019.8302 Google Scholar
- Breiman, L. 2001. “Random forests.” Machine Learning 45: 5–32.
- Burke, R. 2002. “Hybrid Recommender Systems: Survey and Experiments.” User Modeling and User-Adapted Interaction 12: 331–370.
- Chawla, N. V. 2010. “ Data Mining for Imbalanced Datasets: An Overview.” In Data Mining and Knowledge Discovery Handbook, 875–886. Springer.
- Chawla, N. V., K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer. 2002. “Smote: Synthetic Minority Over-Sampling Technique.” Journal of Artificial Intelligence Research 16: 321–357.
- Chowdhary, K., and K. Chowdhary. 2020. “Natural Language Processing.” Fundamentals of Artificial Intelligence 20: 603–649.
- Conneau, A., D. Kiela, H. Schwenk, L. Barrault, and A. Bordes. 2017. “Supervised Learning of Universal Sentence Representations From Natural Language Inference Data.” Preprint, arXiv.
- Cunningham, J. P., and Z. Ghahramani. 2015. “Linear Dimensionality Reduction: Survey, Insights, and Generalizations.” Journal of Machine Learning Research 16, no. 1: 2859–2900.
- Daneshfar, F., and M. B. Jamshidi. 2022. “ A Pattern Recognition Framework for Signal Processing in Metaverse.” In 2022 8th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS), 1–5. IEEE.
10.1109/ICSPIS56952.2022.10044059 Google Scholar
- Daneshfar, F., and S. J. Kabudian. 2021a. “Speech Emotion Recognition System by Quaternion Nonlinear Echo State Network.” Preprint, arXiv.
- Daneshfar, F., and S. J. Kabudian. 2021b. “Speech Emotion Recognition Using Deep Sparse Auto-Encoder Extreme Learning Machine With a New Weighting Scheme and Spectro-Temporal Features Along With Classical Feature Selection and a New Quantum-Inspired Dimension Reduction Method.” Preprint, arXiv.
- Daneshfar, F., B. S. Saifee, S. Soleymanbaigi, and M. Aeini. 2025. “Elastic Deep Multi-View Autoencoder With Diversity Embedding.” Information Sciences 689: 121482.
- Dang, N. C., M. N. Moreno-García, and F. Prieta. 2020. “Sentiment Analysis Based on Deep Learning: A Comparative Study.” Electronics 9, no. 3: 483.
- Eggert, J., and E. Korner. 2004. “ Sparse Coding and Nmf.” In 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No. 04CH37541), vol. 4, 2529–2533. IEEE.
10.1109/IJCNN.2004.1381036 Google Scholar
- Fang, X., and J. Zhan. 2015. “Sentiment Analysis Using Product Review Data.” Journal of Big Data 2: 1–14.
- Feng, X., X. Feng, and B. Qin. 2021. “A Survey on Dialogue Summarization: Recent Advances and New Frontiers.” Preprint arXiv.
- Fisher, R. A. 1936. “The Use of Multiple Measurements in Taxonomic Problems.” Annals of Eugenics 7, no. 2: 179–188.
- Giachanou, A., and F. Crestani. 2016. “Like It or Not: A Survey of Twitter Sentiment Analysis Methods.” ACM Computing Surveys 49, no. 2: 1–41.
- Graves, A., S. Fernández, and J. Schmidhuber. 2005. “ Bidirectional LSTM Networks for Improved Phoneme Classification and Recognition.” In International Conference on Artificial Neural Networks, 799–804. Springer.
10.1007/11550907_126 Google Scholar
- Graves, A., and J. Schmidhuber. 2005. “Framewise Phoneme Classification With Bidirectional Lstm and Other Neural Network Architectures.” Neural Networks 18, no. 5–6: 602–610.
- Hochreiter, S., and J. Schmidhuber. 1997. “Long Short-Term Memory.” Neural Computation 9, no. 8: 1735–1780.
- Hotelling, H. 1933. “Analysis of a Complex of Statistical Variables Into Principal Components.” Journal of Educational Psychology 24, no. 6: 417–441.
10.1037/h0071325 Google Scholar
- Huang, G., Z. Liu, L. Van Der Maaten, and K. Q. Weinberger. 2017. “ Densely Connected Convolutional Networks.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4700–4708.
10.1109/CVPR.2017.243 Google Scholar
- IŞIK, M., and H. Dağ. 2020. “The Impact of Text Preprocessing on the Prediction of Review Ratings.” Turkish Journal of Electrical Engineering and Computer Sciences 28, no. 3: 1405–1421.
- Karimi, Z. 2023. “Manifold-Based Sparse Representation for Opinion Mining.” Scientific Reports 13, no. 1: 15904.
- Khan, M., and A. Srivastava. 2024. “Sentiment Analysis of Twitter Data Using Machine Learning Techniques.” International Journal of Engineering and Management Research 14, no. 1: 196–203.
- Khurana, D., A. Koli, K. Khatter, and S. Singh. 2023. “Natural Language Processing: State of the Art, Current Trends and Challenges.” Multimedia Tools and Applications 82, no. 3: 3713–3744.
- Klema, V., and A. Laub. 1980. “The Singular Value Decomposition: Its Computation and Some Applications.” IEEE Transactions on Automatic Control 25, no. 2: 164–176.
- Kumar, S., and A. Solanki. 2023. “An Abstractive Text Summarization Technique Using Transformer Model With Self-Attention Mechanism.” Neural Computing and Applications 35, no. 25: 18603–18622.
- Kumbhar, R., S. Mhamane, H. Patil, S. Patil, and S. Kale. 2020. “ Text Document Clustering Using k-Means Algorithm With Dimension Reduction Techniques.” In 2020 5th International Conference on Communication and Electronics Systems (ICCES), 1222–1228. IEEE.
- Lee, D. D., and H. S. Seung. 1999. “Learning the Parts of Objects by Non-Negative Matrix Factorization.” Nature 401, no. 6755: 788–791.
- Li, J., K. Cheng, S. Wang, et al. 2017. “Feature Selection: A Data Perspective.” ACM Computing Surveys 50, no. 6: 1–45.
- Liu, S. 2020. “Sentiment Analysis of Yelp Reviews: A Comparison of Techniques and Models.” Preprint arXiv.
- Lunga, D., S. Prasad, M. M. Crawford, and O. Ersoy. 2013. “Manifold-Learning-Based Feature Extraction for Classification of Hyperspectral Data: A Review of Advances in Manifold Learning.” IEEE Signal Processing Magazine 31, no. 1: 55–66.
10.1109/MSP.2013.2279894 Google Scholar
- Maas, A., R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts. 2011. “ Learning Word Vectors for Sentiment Analysis.” In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, 142–150. Association for Computational Linguistics.
- Maaten, L., and G. Hinton. 2008. “Visualizing Data Using t-SNE.” Journal of Machine Learning Research 9, no. 11: 11–26.
- Maćkiewicz, A., and W. Ratajczak. 1993. “Principal Components Analysis (PCA).” Computers & Geosciences 19, no. 3: 303–342.
- Martinez, A. M., and A. C. Kak. 2001. “PCA versus LDA.” IEEE Transactions on Pattern Analysis and Machine Intelligence 23, no. 2: 228–233.
- Medhat, W., A. Hassan, and H. Korashy. 2014. “Sentiment Analysis Algorithms and Applications: A Survey.” Ain Shams Engineering Journal 5, no. 4: 1093–1113.
10.1016/j.asej.2014.04.011 Google Scholar
- Mu, J., S. Bhat, and P. Viswanath. 2017. “All-But-the-Top: Simple and Effective Postprocessing for Word Representations.” Preprint arXiv.
- Nargiza, E. N. K. T. 2024. “Computational Linguistics Uses Statistical and Machine Learning Techniques to Extract Patterns.” Journal of Modern Educational Achievements 3, no. 1: 138–140.
- Nebu, C. M., and S. Joseph. 2016. “ A Hybrid Dimension Reduction Technique for Document Clustering.” In Innovations in Bio-Inspired Computing and Applications: Proceedings of the 6th International Conference on Innovations in Bio-Inspired Computing and Applications (IBICA 2015) Held in Kochi, India During December 16–18, 2015, 403–416. Springer.
10.1007/978-3-319-28031-8_35 Google Scholar
- Neri, F., C. Aliprandi, F. Capeci, and M. Cuadros. 2012. “ Sentiment Analysis on Social Media.” In 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, 919–926. IEEE.
10.1109/ASONAM.2012.164 Google Scholar
- Ojha, A. C., P. K. Shah, S. Gupta, and S. Sharma. 2024. “Classifying Twitter Sentiment on Multi-Levels Using a Hybrid Machine Learning Model.” International Journal of Intelligent Systems and Applications in Engineering 12, no. 3s: 328–333.
- Pang, B., and L. Lee. 2008. “Opinion Mining and Sentiment Analysis.” Foundations and Trends in Information Retrieval 2, no. 1–2: 1–135.
10.1561/1500000011 Google Scholar
- Parvin, H., P. Moradi, and S. Esmaeili. 2019. “Tcfaco: Trust-Aware Collaborative Filtering Method Based on Ant Colony Optimization.” Expert Systems with Applications 118: 152–168.
- Parvin, H., A. R. Naghsh-Nilchi, and H. M. Mohammadi. 2023a. “Image Captioning Using Transformer-Based Double Attention Network.” Engineering Applications of Artificial Intelligence 125: 106545.
- Parvin, H., A. R. Naghsh-Nilchi, and H. M. Mohammadi. 2023b. “Transformer-Based Local-Global Guidance for Image Captioning.” Expert Systems with Applications 223: 119774.
- Pearson, K. 1901. “Liii. On Lines and Planes of Closest Fit to Systems of Points in Space.” London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2, no. 11: 559–572.
- Pennington, J., R. Socher, and C. D. Manning. 2014. “ Glove: Global Vectors for Word Representation.” In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1532–1543. Association for Computational Linguistics.
10.3115/v1/D14-1162 Google Scholar
- Pramana, R., J. J. Subroto, A. A. S. Gunawan, et al. 2022. “ Systematic Literature Review of Stemming and Lemmatization Performance for Sentence Similarity.” In 2022 IEEE 7th International Conference on Information Technology and Digital Applications (ICITDA), 1–6. IEEE.
10.1109/ICITDA55840.2022.9971451 Google Scholar
- Qaisar, S. M. 2020. “ Sentiment Analysis of Imdb Movie Reviews Using Long Short-Term Memory.” In 2020 2nd International Conference on Computer and Information Sciences (ICCIS), 1–4. IEEE.
- Rakshit, P., and A. Sarkar. 2024. “A Supervised Deep Learning-Based Sentiment Analysis by the Implementation of WORD2VEC and GloVe Embedding Techniques.” Multimedia Tools and Applications 84: 979–1012.
10.1007/s11042-024-19045-7 Google Scholar
- Rizal, C., D. A. Kifta, R. H. Nasution, A. Rengganis, and R. Watrianthos. 2023. “ Opinion Classification for IMDb Review Based Using Naive Bayes Method.” In AIP Conference Proceedings, vol. 2913, 10–24. AIP Publishing.
- Roweis, S. T., and L. K. Saul. 2000. “Nonlinear Dimensionality Reduction by Locally Linear Embedding.” Science 290, no. 5500: 2323–2326.
- Sadikin, M., and A. Fauzan. 2023. “Evaluation of Machine Learning Approach for Sentiment Analysis Using Yelp Dataset.” European Journal of Electrical Engineering and Computer Science 7, no. 6: 58–64.
10.24018/ejece.2023.7.6.583 Google Scholar
- Salem, R. 2017. “ A Manifold Learning Framework for Reducing High-Dimensional Big Text Data.” In 2017 12th International Conference on Computer Engineering and Systems (ICCES), 347–352. IEEE.
10.1109/ICCES.2017.8275330 Google Scholar
- Sampedro, G. A. 2024. “ What's Next?: Exploring Machine Learning-Based Approaches to Content Suggestions Using Imdb Movie Reviews.” In 2024 International Conference on Electronics, Information, and Communication (ICEIC), 1–4. IEEE.
10.1109/ICEIC61013.2024.10457130 Google Scholar
- Shah, D., and S. S. Prasad. 2023. “ Investigating Challenges in Opinion Mining Using Logistic Regression, Knn and Decision Tree.” In Proceedings of the KILBY 100 7th International Conference on Computing Sciences 2023 (ICCS 2023), vol. 7.
- Shorten, C., T. M. Khoshgoftaar, and B. Furht. 2021. “Text Data Augmentation for Deep Learning.” Journal of Big Data 8, no. 1: 101.
- Springenberg, J. T., A. Dosovitskiy, T. Brox, and M. Riedmiller. 2014. “Striving for Simplicity: The All Convolutional Net.” Preprint arXiv.
- Srivastava, N., G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. 2014. “Dropout: A Simple Way to Prevent Neural Networks From Overfitting.” Journal of Machine Learning Research 15, no. 1: 1929–1958.
- Sun, S., C. Luo, and J. Chen. 2017. “A Review of Natural Language Processing Techniques for Opinion Mining Systems.” Information Fusion 36: 10–25.
- Tang, E. K., P. N. Suganthan, X. Yao, and A. K. Qin. 2005. “Linear Dimensionality Reduction Using Relevance Weighted Lda.” Pattern Recognition 38, no. 4: 485–493.
- Tenenbaum, J. B., V. de Silva, and J. C. Langford. 2000. “A Global Geometric Framework for Nonlinear Dimensionality Reduction.” Science 290, no. 5500: 2319–2323.
- Tiffani, I. E. 2020. “Optimization of naïve Bayes Classifier by Implemented Unigram, Bigram, Trigram for Sentiment Analysis of Hotel Review.” Journal of Soft Computing Exploration 1, no. 1: 1–7.
- Van Buuren, S., and C. G. Oudshoorn. 2000. Multivariate Imputation by Chained Equations. TNO.
- Van Der Maaten, L., E. Postma, J. Herik, et al. 2009. “Dimensionality Reduction: A Comparative.” Journal of Machine Learning Research 10: 66–71.
- Vaswani, A., N. Shazeer, N. Parmar, et al. 2017. “ Attention Is all You Need.” In 31st Conference on Neural Information Processing Systems (NIPS 2017), vol. 30.
- Vlachos, M., C. Domeniconi, D. Gunopulos, G. Kollios, and N. Koudas. 2002. “ Non-Linear Dimensionality Reduction Techniques for Classification and Visualization.” In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 645–651. Association for Computing Machinery.
10.1145/775047.775143 Google Scholar
- Von Luxburg, U. 2007. “A Tutorial on Spectral Clustering.” Statistics and Computing 17: 395–416.
- Voulodimos, A., N. Doulamis, A. Doulamis, and E. Protopapadakis. 2018. “Deep Learning for Computer Vision: A Brief Review.” Computational Intelligence and Neuroscience 2018: 7068349.
- Xia, T., D. Tao, T. Mei, and Y. Zhang. 2010. “Multiview Spectral Embedding.” IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 40, no. 6: 1438–1446.
- Xu, W., X. Liu, and Y. Gong. 2003. “ Document Clustering Based on Non-Negative Matrix Factorization.” In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval, 267–273. Association for Computing Machinery.
10.1145/860435.860485 Google Scholar
- Yang, Z., F. Yao, K. Fan, and J. Huang. 2017. “Text Dimensionality Reduction With Mutual Information Preserving Mapping.” Chinese Journal of Electronics 26, no. 5: 919–925.
- Zhang, L., S. Wang, and B. Liu. 2018. “Deep Learning for Sentiment Analysis: A Survey.” Wiley Interdisciplinary Reviews. Data Mining and Knowledge Discovery 8, no. 4: 1253.
- Zhang, M., J. Chai, J. Cao, J. Ji, and T. Yi. 2024. “Aspect-Level Sentiment Analysis Based on Deep Learning.” Computers, Materials & Continua 78, no. 3: 3743–3762.
- Zheng, J., and L. Zheng. 2019. “A Hybrid Bidirectional Recurrent Convolutional Neural Network Attention-Based Model for Text Classification.” IEEE Access 7: 106673–106685.
- Zhou, P., W. Shi, J. Tian, et al. 2016. “ Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification.” In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 207–212. Association for Computational Linguistics.
10.18653/v1/P16-2034 Google Scholar