نوع مقاله : مقاله پژوهشی
نویسندگان
1 کارشناسی ارشد رشته مدیریت منابع اطلاعاتی، دانشگاه الزهرا ، تهران، ایران
2 دانشیار گروه مدیریت، دانشکده علوم اجتماعی و اقتصاد، دانشگاه الزهرا، تهران، ایران نویسنده مسئول: a.khadivar@alzahra.ac.ir
3 استادیار گروه مدیریت صنعتی و فناوری اطلاعات ، دانشگاه شهید بهشتی، تهران، ایران
چکیده
بررسیها نشان داده است که بسیاری از سازمانها برای ایجاد دانش و بهبود تصمیمگیری از تحلیل نظرات و محتوای تولید شده توسط کاربران در شبکههای اجتماعی بهرهبرداری کردهاند. در این پژوهش، به تحلیل عملکرد مالی سازمانهای غیرانتفاعی با استفاده از تحلیل نظرات کاربران پرداخته شده است. مجموعه دادهی استفاده شده در این پژوهش شامل 26714 توییت کاربران از جمعا 23 حساب توییتری در سراسر جهان میباشد و ده سال دادهی مالی شامل سالهای 2010-2020 این سازمانها از 5 سازمان غیرانتفاعی منتخب جمعآوری شده است. نتایج حاصل از روشهای مدلسازی موضوع و تحلیل احساسات به دادههای پانلی تبدیل شدهاند. نتایج مدلسازی موضوعی و تحلیل احساسات به دادههای پنلی تبدیل و با استفاده از روشهای حداقل مربعات معمولی و حداقل مربعات معمولی پویا تحلیل شدهاند. نتایج نشان میدهند که محتوای تولید شده توسط کاربران و عملکرد مالی سازمانهای غیرانتفاعی رابطه معنی داری دارند. در حالی که احساسات مثبت تأثیر معنیداری بر عملکرد مالی این سازمانها ندارند، احساسات منفی و بازتوئیتها تأثیر منفی و علاقمندیها رابطه ی مثبتی با عملکرد مالی را نشان می دهند.
کلیدواژهها
موضوعات
عنوان مقاله [English]
Performance Investigation of Nonprofit Organizations by Sentiment Analysis on Social Networks
نویسندگان [English]
- Elnaz Valizadeh Hamzekolaei 1
- Ameneh khadivar 2
- Fatemeh Abbasi 3
1 Master of Management of Information Technology, Alzahra University, Tehran, Iran.
2 Associate Professor, Management Department, Faculty of Social Sciences and Economics, Alzahra University, Tehran, Iran Corresponding Author: a.khadivar@alzahra.ac.ir
3 Assistant Professor, Faculty of Management and Accounting, Shahid Beheshti University, Tehran, Iran
چکیده [English]
Abstract
Social networks have become vital for sharing opinions and feelings through user-generated content. Many organizations leverage analytics to enhance decision-making, yet most sentiment analysis studies focus on commercial businesses, neglecting non-profits despite their significant social media presence. This research investigates the impact of user-generated content on the financial performance of non-profit organizations using a dataset of 26,714 tweets from 23 accounts. Results indicate that while positive emotions do not affect financial performance, negative emotions and retweets harm it, whereas likes positively influence revenue. Future research should explore additional social networks and broader data collection methods.
Introduction
Free market societies comprise three main sectors: the public sector, the private business sector, and private non-profit organizations, collectively known as the "three-sector economy" (Weisbrod, 1975). Non-profit organizations are characterized by being organized, private, self-managed, non-profit distributing, and voluntary (Salamon et al., 1996). This research focuses on non-profits dedicated to animal welfare, which aims to prevent animal abuse and ensure proper care (Navigator Charity, n.d.). Recently, these organizations have gained prominence and significantly influenced societal modernization (Lee & Nowell, 2014). Evaluating their performance is crucial for enhancing efficiency amid increasing competition for funding. Given the challenges of measuring performance in non-profits, this study employs sentiment analysis of user-generated content on Twitter to assess organizational effectiveness.
Research Question(s)
How is the performance of non-profit organizations evaluated using user opinion analysis?
Literature Review
This literature review examines existing studies relevant to the research topic and identifies gaps that necessitate this investigation. Non-profit organizations generate revenue and publish annual financial statements (Rathi et al., 2016). They increasingly use social networks to engage with stakeholders (Lai et al., 2017), producing content that can yield valuable insights through user opinion analysis (Miller, 2011). Social networks enable these organizations to gather stakeholder feedback, enhancing decision-making (Waters & Lo, 2012). Non-profits typically focus on measuring performance through donor revenue and budget progress, emphasizing the importance of both financial and non-financial metrics (Epstein & McFarlan, 2011; Kaplan, 2001).
Research has explored the effects of user-generated content on the performance of both non-profit and for-profit organizations. For instance, studies have shown that Twitter content can predict sales for commercial enterprises (Liu et al., 2016) and that negative user messages elicit more responses, prompting businesses to adapt their communication strategies (Hewett et al., 2016). Additionally, analysis of user content has identified key factors influencing millennial engagement online (Saura et al., 2019). Another study linked customer feedback on Twitter to satisfaction and dissatisfaction factors in the hotel industry (Xu et al., 2017), while research demonstrated that emotions in text comments significantly affect product sales performance (Li, 2018).
In non-profit contexts, stakeholder-generated content can enhance participation strategies (Saxton & Waters, 2014), and increased social campaign popularity correlates with heightened discussion (Tayal & Yadav, 2016). However, most existing research focuses on emotional impacts on specific campaigns rather than overall organizational performance, revealing a significant gap. This study aims to fill this gap by evaluating non-profit performance through sentiment analysis of social media content, presenting an innovative model that incorporates econometric methods. Thus, this research represents a novel contribution to understanding non-profit performance evaluation.
Methodology
This study examines selected non-profit organizations evaluated through "Charity Navigator," a prominent charity evaluator in the United States. To refine our sample and focus on larger, more active organizations on social media, we began with a pool of 9,000 non-profits and used advanced search filters (see Table 1) to identify 60 organizations. From this group, five organizations were randomly chosen for further analysis.
Table 1. Title of filters for advanced search of non-profit organizations on charitynavigator website
The title of the characteristics of non-profit organizations
Characteristics of non-profit organizations
Social network
Twitter
Select category-type
Place
Income
Site ranking
Work area
Animals - rights, welfare and services to animals
The entire United States of America
No restrictions
No restrictions
international
Data collection involved manually extracting financial reports detailing total income for each organization from 2010 to 2020 via the "ProPublica" website. We also identified 23 English-language Twitter accounts related to these organizations, from which we gathered tweet data using a Python-based web crawler.
Data preprocessing included removing non-English texts, hashtags, mentions, URLs, punctuation, and stop words, as well as performing tokenization and lemmatization. This resulted in a dataset of 22,829 tweets from the five selected non-profits.
Data Visualization
Word Cloud: We generated a word cloud using the hyperwords package in Python, displaying the 50 most frequently used words, with their size reflecting usage frequency.
Topic Modeling: To explore underlying topics in the tweets, we applied Dirichlet’s hidden allocation algorithm, identifying five main themes:
Vegetarian education
Addressing cruelty and rescuing animals
Animal protection
Monitoring organizational actions
Supporting organizational activities
Results showed vegetarian education was the most discussed topic, while support for non-profits was the least frequent, indicating users prioritize other issues.
Sentiment Analysis
We conducted sentiment analysis using a vocabulary-based approach. Texts were standardized to lowercase, and stop words and punctuation were removed. A dataset of 1,000 tagged examples was used to evaluate sentiment accuracy.
Econometric Analysis
This research assesses how user-generated content on Twitter impacts the annual income of non-profits. After necessary tests and model estimation using Evioz 10 software, we evaluated the effects of independent variables on total revenue, summarized in Table 2.
Table 2. Variables used in the model
Mathematical symbol
English symbol
Mean
1
y
Total Revenue
Total Revenue
X1
X2
X3
X4
X5
Volume
Neg
Positive
Retweet
Favorite
Volume of tweets
Negative sentiments
Positive sentiments
Number of retweets
Number of favorits
The regression model is expressed as:
Total Revenue = αi + β1X1it + β2X2it + β3X3it + β4X4it + β5X5it + eit
We conducted Chow or Flimer tests to determine data structure and used Fisher’s test for the unit root test, confirming stationarity at a 95% confidence level.
Model Estimation
Using the OLS method, we found the model significant at the 95% confidence level, with a coefficient of determination of 0.512638, indicating that over half of the variability in total revenue is explained by the model. The Durbin-Watson statistic indicated no autocorrelation among residuals.
Results showed a significant relationship between Twitter content and financial performance. Negative sentiments and retweets inversely affected financial outcomes, while likes positively correlated with revenue, indicating active support from followers. No significant relationship was found between tweet volume or positive sentiments and financial performance.
Results
The results show that users' opinions have a significant impact on the financial performance of these organizations. Therefore, non-profit organizations should pay special attention to their users and donors to increase their satisfaction. Otherwise, the lack of proper management of the organization's actions can damage its credibility. Also, there is a need for further investigation to determine the cause of the negative relationship between the number of retweets and financial performance of organizations. Modeling of topics discussed by users shows that managers of non-profit organizations should focus more on building trust among their users, because the results of topic modeling show that support for the organization is the least frequent.
Keywords: Sentiment Analysis, Nonprofit Organizations, Topic Modeling, Social Network, Panel Data.
کلیدواژهها [English]
- sentiment analysis
- nonprofit organizations
- topic modeling
- social network
- panel data
- Alarifi, A., Alsaleh, M., & Al-Salman, A. (2016). Twitter turing test: Identifying social machines. Information Sciences, 372, 332–346. https://doi.org/10.1016/j.ins.2016.08.036
- Butcher, L. (2009). Nonprofit organizations outpace businesses in use of social media
- Canada boasts second largest nonprofit sector in the world [Internet]. CharityVillage. 2005 [cited 2023 Sep 24]. Available from: https://charityvillage.com/canada_boasts_second_largest_nonprofit_sector_in_the_world/#:~:text=Although%20the%20United%20States%20has
- Charity Navigator [Internet]. Charity Navigator. 2017. Available from: https://www.charitynavigator.org
- Chianese, A., & Piccialli, F. (2016b). A service oriented framework for analysing social network activities. Procedia Computer Science, 98, 509–514. https://doi.org/10.1016/j.procs.2016.09.087
- Epstein, M. J., & McFarlan, F. W. (2011). Measuring the efficiency and effectiveness of a nonprofit's performance. Strategic finance, 93(4), 27.
- Goh, K., Heng, C., & Lin, Z. (2013). Social Media Brand Community and Consumer Behavior: Quantifying the Relative impact of User- and Marketer-Generated content. Information Systems Research, 24(1), 88–107. https://doi.org/10.1287/isre.1120.0469
- Goldberg, K. L. (2020). Adopting Analytics: A Framework for Nonprofit Organizations(Doctoral dissertation, University of Maryland University College).
- González-Rodríguez, M. R., Díaz-Fernández, M. C., Bilgihan, A., Shi, F., & Okumus, F. (2021). UGC involvement, motivation and personality: Comparison between China and Spain. Journal of Destination Marketing & Management, 19, 100543. https://doi.org/10.1016/j.jdmm.2020.100543
- Hewett, K., Rand, W., Rust, R. T., & Van Heerde, H. J. (2016). Brand buzz in the echoverse. Journal of Marketing, 80(3), 1–24. https://doi.org/10.1509/jm.15.0033
- Kaplan, R. S. (2001). Strategic performance measurement and management in nonprofit organizations. Nonprofit Management and Leadership, 11(3), 353–370. https://doi.org/10.1002/nml.11308
- Lai, C., She, B., & Tao, C. (2017). Connecting the dots: A longitudinal observation of relief organizations’ representational networks on social media. Computers in Human Behavior, 74, 224–234. https://doi.org/10.1016/j.chb.2017.04.037
- Lee, C., & Nowell, B. (2014). A framework for assessing the performance of nonprofit organizations. American Journal of Evaluation, 36(3), 299–319. https://doi.org/10.1177/1098214014545828
- Li, S. (2018). Topic modeling and latent dirichlet allocation (lda) in python. Towards Data Science, 31.
- Liu, X. (2020). Analyzing the impact of user-generated content on B2B Firms’ stock performance: Big data analysis with machine learning methods. Industrial Marketing Management, 86, 30–39. https://doi.org/10.1016/j.indmarman.2019.02.021
- Liu, X., Singh, P. V., & Srinivasan, K. (2016). A structured analysis of unstructured big data by leveraging cloud computing. Marketing Science, 35(3), 363–388. https://doi.org/10.1287/mksc.2015.0972
- Miller, D. (2011). Miller (1983) revisited: A reflection on EO research and some suggestions for the future. Entrepreneurship Theory and Practice, 35(5), 873–894. https://doi.org/10.1111/j.1540-6520.2011.00457.x
- Palomino, M., Taylor, T., Göker, A., Isaacs, J., & Warber, S. (2016). The Online Dissemination of Nature–Health Concepts: Lessons from Sentiment Analysis of Social Media Relating to “Nature-Deficit Disorder.” International Journal of Environmental Research and Public Health, 13(1), 142. https://doi.org/10.3390/ijerph13010142
- Rathi, D., Given, L., Forcier, E., & Vela, S. (2016). Every Task its Tool, Every Tool its Task: Social Media Use in Canadian Non-Profit Organizations. Proceedings of the Annual Conference of CAIS / Actes Du Congrès Annuel De L ACSI. https://doi.org/10.29173/cais866
- Razi, M., Tarn, J. M., & Mumuni, A. G. (2019). Effectiveness of incentivized social media campaigns for a Fortune 500 company’s brand. American Journal of Business, 34(1), 37–56. https://doi.org/10.1108/ajb-07-2017-0019
- Rui, H., Liu, Y., & Whinston, A. (2013). Whose and what chatter matters? The effect of tweets on movie sales. Decision Support Systems, 55(4), 863–870. https://doi.org/10.1016/j.dss.2012.12.022
- Salamon, L. M., Anheier, H. K., Studies, J. H. I. F. P., & Project, J. H. C. N. S. (1996). The International Classification of Nonprofit Organizations - ICNPO Revision 1.0.
- Saura, J. R., Debasa, F., & Reyes-Menendez, A. (2019). Does User Generated Content Characterize Millennials’ Generation Behavior? Discussing the Relation between SNS and Open Innovation. Journal of Open Innovation Technology Market and Complexity, 5(4), 96. https://doi.org/10.3390/joitmc5040096
- Saxton, G. D., & Waters, R. D. (2014). What do StakeholdersLikeon Facebook? Examining Public Reactions to Nonprofit Organizations’ Informational, Promotional, and Community-Building Messages. Journal of Public Relations Research, 26(3), 280–299. https://doi.org/10.1080/1062726x.2014.908721
- Tayal, D. K., & Yadav, S. K. (2016). Sentiment analysis on social campaign “Swachh Bharat Abhiyan” using unigram method. AI & Society, 32(4), 633–645. https://doi.org/10.1007/s00146-016-0672-5
- Waters, R. D., & Lo, K. D. (2012). Exploring the impact of culture in the social media sphere: A content analysis of nonprofit organizations’ use of Facebook. Journal of Intercultural Communication Research, 41(3), 297–319. https://doi.org/10.1080/17475759.2012.728772
- Weisbrod, B. A. (1975). Research in Health Economics: A survey. International Journal of Health Services, 5(4), 643–661. https://doi.org/10.2190/kvhp-37n5-f6cd-rl6a
- Xu, X., Wang, X., Li, Y., & Haghighi, M. (2017). Business intelligence in online customer textual reviews: Understanding consumer perceptions and influential factors. International Journal of Information Management, 37(6), 673–683. https://doi.org/10.1016/j.ijinfomgt.2017.06.004