Mining Twitter to test how fan communities drive pop success

A communication studies graduate with a music industry minor wanted to test whether “stan” communities, the most active fans on Twitter, increasingly shape the success of pop artists. She had a detailed research proposal and needed help getting the data.

Julia Wood, a statistics major and the first Norman Powell data assistant at the Data Science Center, worked with her to narrow the project to four artists, five fan accounts for each, and five keywords for each, so that what they collected would be accurate and comparable. Julia then wrote Python to scrape follower counts, retweets, outbound link clicks, and keywords such as “buy” and “stream”.

The analysis in SPSS found a clear correlation, but not causation, between fan community activity and artist success. The project won a 2021 Undergraduate Research Week Dean’s Prize.

Leigh Phan, the Data Science Center’s facilitator, helped set up the software to collect the tweets. The project is a small version of a pattern the squad repeats: a student with a good question, a consultant who can get the data, and a result the student presents under their own name.