Fan, Z., Zhang, F., Loo, B. P., & Ratti, C. (2023). Urban visual intelligence: Uncovering hidden city profiles with street view images. Proceedings of the National Academy of Sciences, 120(27), e2220417120.

A longstanding line of research in urban studies explores how cities can be understood through their appearance. However, what remains unclear is to what extent urban dwellers’ everyday life can be explained by the visual clues of the urban environment. In this paper, we address this question by applying a computer vision model to 27 million street view images across 80 counties in the U.S. Then, we use the spatial distribution of notable urban features identified through the street view images, such as street furniture, sidewalks, building façades, and vegetation, to predict the socioeconomic profiles of their immediate neighborhood. Our results show that these urban features alone can account for up to 83% of the variance in people's travel behavior, 62% in poverty status, 64% in crime, and 68% in health behaviors. The results outperform models based on points of interest (POI), population, and other demographic data alone. Moreover, incorporating urban features captured from street view images can improve the explanatory power of these other methods by 5%-25%. We propose ``urban visual intelligence'' as a process to uncover hidden city profiles, infer, and synthesize urban information with computer vision and street view images. This study serves as a foundation for future urban research interested in this process and understanding the role of visual aspects of the city.

Paper available Here

Urban density is long considered to foster diverse exchanges of interpersonal knowledge and skills, which are intrinsic to sustainable human settlements. However, with current urban studies primarily devoted to city and district-level analysis, we cannot unveil the elemental connection between urban density and diversity. What makes a street more diverse? The Great Streets project uses high-resolution anonymous mobility data to quantify the income diversity of streets in urban areas. Our research shows that street diversity is not only linked to the concentration of visitors but rather to the kind of amenities, residential diversity, and income level around them. As a result, turning the corner can take you to a completely different diverse area.

Hong Kong, a well-known delicious city, is famous for the diversity of food. With social-media penetrating our lives, the food businesses are also adapting to the online-offline battle. A yummy shrimp dumpling right across the street, or the Japanese Kaiseki requires months ahead of booking, which one would you go?

I scraped data from OpenRice.com, which contains reviews since 1999, and plot the battle between Chinese food and Other cuisines in Hong Kong.

Fan, Z., & Loo, B. P. (2021). Street life and pedestrian activities in smart cities: opportunities and challenges for computational urban science. Computational urban science, 1(1), 1-17

Loo, B. P., Fan, Z., Lian, T., & Zhang, F. (2023). Using computer vision and machine learning to identify bus safety risk factors. Accident Analysis & Prevention, 185, 107017.

“Jaywalking“ is a term in the world of cars. Streets are designed for vehicles and people crossing the road at “uncomfortable” locations are breaking the rules. When would we switch this thinking paradigm and give streets back to pedestrians?

This work trained a MaskRCNN model to detect pedestrian jaywalking event observed through bus dashcam videos.

Z. Fan, F. Zhang and B. P. Y. Loo, "Rhythm of Transit Stations - Uncovering the Activity-Travel Dynamics of Transit-Oriented Development in the U.S.," in IEEE Transactions on Intelligent Transportation Systems, doi: 10.1109/TITS.2021.3115103.

Existing transit-oriented-development (TOD) classification studies primarily focus on the static characteristics around transit stations to measure the built environment's density, diversity, and design. As a community development model, time-variant variables, dynamic human activities throughout different times of the day and week matter in further unpacking the characteristics of TODs. Given that this aspect has been under-discussed in most previous TOD literature, this research provides an activity-based framework to classify commuter transit station areas by considering the degree of local vibrancy - the temporal visiting pattern of all points of interest (POIs) that fall within the station areas. We apply a two-step semi-unsupervised clustering algorithm to classify 4,290 station areas from 54 metropolitan areas across the U.S. This method produces 13 distinct station area types. Next, we further examine the connection between station area types and neighborhood travel behavior. A cross-sectional comparison reveals that stations with consistent active morning activities are associated with a higher ratio of commuting by walking and biking and lower automobile usage measured in vehicle miles traveled (VMT). Using stations opened after 2009, we show that active weekend activity patterns are associated with a more significant increase in commuting by public transit.

Zhuangyuan Fan, Maoran Sun, Tianyu Su, Sarah Williams

Working Paper

Interactive Platform

Retails, restaurants, and other urban amenities are critical indicators of street livelihood. The recent business restrictions coming with the COVID-19 pandemic have completely shifted how urban public spaces are being used. Many restaurants and retailers are either shut down or resorting to online services to maintain their business. The invisible cyber layer of cities is forced to take on further responsibilities in maintaining residents' basic daily demands. However, while cities are preparing for reopening, little is known if the past year's lockdown experience has shifted people’s habits and behaviors in cities. How do the online activities during the lockdown impact the offline activities during the intermediated reopening? How do we reimagine the urban public spaces given the further complicated social network effects introduced by a stronger cyber layer after the pandemic? These questions are still open to be explored.

Paper published at Computers, Environment and Urban Systems

Interactive Map Here

The experience of walking through a city is influenced by amenities and the visual qualities of its built environment. This paper uses thousands of pedestrian trajectories obtained from GPS signals to construct a desirability index for streets in Boston. We create the index by comparing the actual paths taken by pedestrians with the shortest path between any origin-destination pairs. The index captures pedestrians’ willingness to deviate from their shortest path and provides a measure of the scenic and experience value provided by different parts of the city. We then use computer vision techniques combined with georeferenced data to measure the built environment of streets. We show that desirable streets have better access to public amenities such as parks, sidewalks, and urban furniture. They are also sinuous, visually enclosed, have less complex facades, and have more diverse business establishments. These results further our understanding of the value that the built environment brings to pedestrians, enhancing our capacity to design more lively and functional environments.

Arianna Salazar Miranda, Zhuangyuan Fan, Fabio Duarte, Carlo Ratti, “Desirable Street: Using Deviations in Pedestrian

Trajectories to Measure the Value of the Built Environment”, Computers, Environment and Urban System ,https://doi.org/10.1016/j.compenvurbsys.2020.101563.

“Perception bias”: Deciphering a mismatch between urban crime and perception of safety

Paper Published on Landscape and Urban Planning Journal, https://authors.elsevier.com/a/1cFiPcUG5ISZs

Crime and perception of safety are two intertwined concepts affecting the quality of life and the economic development of a society. However, few studies have quantitatively examined the difference between the two due to the lack of granular data documenting public perceptions in a given geographic context. Here, by applying a pre-trained scene understanding algorithm, we infer the perception of safety score of streetscapes for census block groups in the city of Houston using a large number of Google Street View images. Then, using this inferred perception of safety, we create “perception bias” categories for each census block group. These categories capture the level of mismatch between people’s visually perceived safety and the actual crime rates. This measure provides scalable guidance in deciphering the relationship between the built environment and crime. Finally, we construct a series of models to examine the “perception bias” with static and dynamic urban factors, including socioeconomic features (e.g., unemployment rate and ethnic compositions), urban diversity (e.g., number and diversity of Points of Interest), and urban livelihood (i.e., hourly count of visitors). Analytical and numerical results suggest that the association between characteristics of urban space and “perception bias” over crime could be paradoxical. On the one hand, neighborhoods with a higher volume of day-time visitors appear more likely to be safer than it looks (low crime rate and low safety score). On the other hand, those with a higher volume of night-time visitors are likely to be more dangerous than it looks (high crime rate). The findings add further knowledge to the long-recognized relationship between built environment and crime as well as highlight the perception of safety in cities, which in turn enhances our capacity to design urban management strategies that prevent the emergence of extreme “perception bias”.

Yuan works on this brief research project visualizing New York City’s response to Covid-19 in the past couple of months.

The maps show the connection between income, infection rate, test rate across all zip codes in NYC from April to May. We are asking how could city better allocate resources to people who suffer the most during pandemics.

Read More about the project through Civic Data Design Lab’s blog at MIT

Project Prototype here

Yuan is the geo-spatial data scientist intern at Neighborly. She is currently working on optimizing the infrastructure distribution in the city of Stockton. More to come…

Computer Vision Project 2019, MIT

Tianyu Su, Zhuangyuan Fan

In computer vision, the photo geolocation problem has been usually approached at a global scale or regional scale. In this project, we derive knowledge from urban studies and present a classification based method looking into neighborhood scale photo geotagging. We subdivide Boston Chinatown into multiple geographic cells and train a deep network using 20K google street view images labeled with local restaurant density and street speed limit. We show that this multitask model could output a probability distribution over a couple of cells in the neighborhood.

Project Planning Phase Finished

image courtesy of Neighborly

Yuan was the data scientist performing spatial analysis, network analysis and visualization in this project.

" $30M dispersed among different communities across Maine will be impactful. We believe that Neighborly could be even more impactful by taking a quantitative and optimization-based approach, where we take a step back and think strategically at the global state level, and see how we can draw synergies and efficiencies of scale to build an overall network (composed of an array of smaller networks) that would deploy capital judiciously in a way to serve these unserved people of Maine. “

The plan was proposed to Maine ConnectMe Authority, and currently Neighborly is building the first network in South Portland.

2019 Sasaki Foundation Winner

Project in progress

Team: Helena Rong, Tianyu Su, Zhuangyuan Fan

Rentify Chinatown documents community narratives and generate data visualizations around gentrification in Boston Chinatown due to short-term rentals. Deliverables including documentation, online data platform, and pop-up exhibitions provide portals to engage communities and expose the issue to a wider audience.

Project in Progress

Team: Tianyu Su, Zhuangyuan Fan

Data Resource: Kauffman Data of Entrepreneurship 2008 - 2017

Check interactive Maps here.

“While immigration is often subject to contentious political debate, there is a little debate about the economic contributions of immigrant entrepreneurs. Immigrants are twice as likely to become entrepreneurs as native-born Americans. Immigrant entrepreneurs have begun and lead some of the world’s most successful and innovative companies. ”

Liu, C. Y., Painter, G., & Wang, Q. (2014). Lessons for U.S. Metro Areas: Characteristics and Clustering of High-Tech Immigrant Entrepreneurs. Kauffman Foundation.

Invited to present at 16th International Conference on Computers in Urban Planning and Urban Management

This research attempts to investigate the geography of entrepreneurship activities and identify the high priority urban spaces for entrepreneurs in San Francisco. We will approach this research by looking at the spatial pattern of these three types of innovation space: co-working, incubator, accelerator and figure out the key factor influence the developers to make the choices of the locations.

All Images @ Tianyu Su, Yonghui Chen and Zhuangyuan Fan

Student Work at University of Pennsylvania

Image Credit: Muhan Cui, Yi Li and Zhuangyuan Fan

In this project, our team designed a dynamic landscape that will reflect the relationship between water and wind through different times of the day and year. We conducted a simulation process with the help of flow analysis program, Eco-Tech and AquaSMS, and mapped the simulation with Grasshopper and Python, finally designed a mechanism that will reflect the wind and water interaction through pattern, space and lights.

Fayetteville , AR

UNDER CONSTRUCTION

In Collaboration with Leers Weinzapfel Associates

Yuan designed a parametric paving system inspired by the nature of water

All Images © OLIN

Suqian, China, 2017

Natian Cup International Design Competition Honorable Mention

TRANSPORTATION SYSTEM is one of most fundamental city infrastructures, through which we enjoy the almost ubiquitous connection but also pay a financial and environmental price. According to the report Calculation and Analysis of Transportation Energy Consumption Level in China, the transportation accounts for 60. l % of the total petroleum end-use consumption. The movement of freight and people from one place to another inevitably consumes large amount of energy. Our team wants to ask this question, how to convert this purely energy consuming activity to an energy generating activity?

The rising of a SHARED-BIKE INDUSTRY in China is offering a potential opportunity to it. Technically, it is possible to collect energy from cycling process, but it is economically inefficient to apply the technique to a personal bike: one complete energy storage system will cost more than a bike itself, and it will approximately require a person to bike for at least l 0 hours to compensate for his daily electricity usage. However, with the proliferating shared bike industry, it is calculated that each single bike will run for l Oto 15 hours a day, and rental fee and the economic value generated by the system will likely compensate for the equipment cost in the future.

Image Credit ©  Yadan Luo, You Wu, Xiaoye Xing and Zhuangyuan Fan

Zhengzhou, Henan

PROJECT IN PROGRESS

Bandirma, Turkey

COMPETITION ENTRY

In Collaboration with MIMOhaus

Image Credit: Hao Li, Zheming Cai, Zhuangyuan Fan and Xiaoye Xing