An Empirical Study of Deep Learning Sentiment Detection Tools for Software Engineering in Cross-Platform Settings

内容摘要

Sentiment detection in software engineering (SE) has shown promise to support diverse development activities. However, given the diversity of SE platforms, SE-specific sentiment detection tools may suffer in performance in cross-platform settings. Recently deep learning (DL)-based SE-specific sentiment detection tools are found to offer superior performance than shallow machine learning (ML) based/rule-based tools. However, it is not known how the DL tools perform in cross-platform settings. In this paper, we study whether SE-specific DL sentiment detectors are more effective than shallow ML-based/rule-based sentiment detectors in cross-platform settings. In three datasets, we study three DL tools (SEntiMoji, BERT4SEntiSE, RNN4SentiSE) and compare those against three baselines: two shallow learning tools (Senti4SD, SentiCR) and one rule-based tool (SentistrengthSE). We find that (1) The deep learning SD tools for SE, BERT4SentiSE outperform other supervised tools in cross-platform settings in most cases, but then the tool is outperformed by the rule-based tool SentistrengthSE in most cases. (2) BERT4SentiSE outperforms SentistrengthSE by large margin in within-platform settings acr