Social media as passive geo-participation in transportation planning – how effective are topic modeling & sentiment analysis in comparison with citizen surveys?

We live in an era of rapid urbanization as many cities are experiencing an unprecedented rate of population growth and congestion. Public transport is playing an increasingly important role in urban mobility with a need to move people and goods efficiently around the city. With such pressures on existing public transportation systems, this paper investigates the opportunities to use social media to more effectively engage with citizens and customers using such services. This research forms a case study of the use of passively collected forms of big data in cities – focusing on Sydney, Australia. Firstly, it examines social media data (Tweets) related to public transport performance. Secondly, it joins this to longitudinal big data – delay information continuously broadcast by the network over a year, thus forming hundreds of millions of data artifacts. Topics, tones, and sentiment are modeled using machine learning and Natural Language Processing (NLP) techniques. These resulting data, and models, are compared to opinions derived from a citizen survey among users. The validity of such data and models versus the intentions of users, in the context of systems that monitor and improve transport performance, are discussed. As such, key recommendations for developing Smart Cities were formed in an applied research context based on these data and techniques.