Current Research Projects

Research in IST cuts across traditional boundaries to drive interdisciplinary discovery and innovation. Our research is sponsored by a variety of national and international agencies, and we collaborate with diverse groups of scholars within and beyond Penn State. Explore our funded projects to see how IST's transformative research is addressing the world's most complex problems at the intersection of information, technology, and society.

|||||||||
Researcher:
Wang, James
Sponsoring Agency: Amazon Research Awards – Robotics Program
This project aims at developing emotion recognition technologies and algorithms for applications in social robotics. First, we will advance human pose estimation methods to generate more accurate human mesh representations. Second, due to limitations in dataset size and the inherent subjectivity of emotion annotations, we will build upon previous semi-supervised methods for noisy label training. Finally, we will explore the role social context plays in emotion understanding for the currently unexplored tasks of emotion localization and interaction classification. Learn more...
Research Areas: Data Sciences and Artificial Intelligence, Human-Computer Interaction
Term: -
Researcher:
Haynes, Steven
Sponsoring Agency: USDA National Institute of Food and Agriculture
The United States produces more than 938 million pounds of mushrooms annually valued at $1.2 billion, with Pennsylvania producing 66% of the total US mushroom crop (USDA NASS, 2020). Mushroom production is a very labor-intensive process. The Pennsylvania Farm Bureau and Sexsmith (2019) studies report a critical shortage of farm labor in Pennsylvania, and the US mushroom farms are short by 20% of the number of workers needed on their farms to complete required tasks, thereby limiting the crop yield and economic outcomes. This project will address the labor shortage in three ways: (1) develop automated fresh mushroom harvesting and packaging technologies through smart agriculture; (2) modify crop management techniques to optimize harvesting rates of manual laborers and mechanized systems; and (3) provide advanced technologies to industry stakeholders for labor efficient production and the economic well-being of rural communities. Learn more...
Research Areas: Human-Computer Interaction
Term: -
Researcher:
Huang, Ting Hao (Kenneth); Giles, C. Lee
Sponsoring Agency: Adobe Systems Incorporated
This project tackles the grand challenge of automating captioning for scientific figures in scholarly articles. We aim to (i) help readers better understand scientific figures and papers, (ii) support writers to compose better captions, and (iii) push the boundaries of AI's ability to automatically generate well-written captions. We will develop neural vision-to-language models that produce highquality captions for scientific figures at scale. These captioning models will be developed using large-scale real-world data such as captions and figures extracted from CiteSeerX and arXiv. Learn more...
Research Areas: Data Sciences and Artificial Intelligence, Human-Computer Interaction
Term: -
Researcher:
Sponsoring Agency: National Science Foundation
This project studies the ways that algorithmic management, using digital tools to automate and remotely manage workers, may negatively impact workers and their rights. The research will look specifically at ride-hailing platforms, which are rapidly replacing traditional taxi services. Researchers will develop an experimental ride-hailing platform that gives drivers and passengers control over parameters that impact algorithmic outcomes, as a means to understand and interact with the platform. Learn more...
Research Areas: Data Sciences and Artificial Intelligence, Human-Computer Interaction, Social and Organizational Informatics
Term: -
Researcher:
Huang, Ting Hao (Kenneth)
Sponsoring Agency: Meta Platforms, Inc.
This project proposes constructing LOGIFALLA, the first public benchmark dataset for common logical fallacies in online conversations. The core idea is to hire a group of crowd workers to simulate an online discussion thread about a given news article, where we secretly ask a subset of these workers to respond using specified types of logical fallacies (e.g., straw man, slippery slope, etc.). LOGIFALLA can move the AI community from spotting the problematic languages towards understanding the underlying problematic reasonings. We believe this will help create online discussion spaces that are more transparent, healthy, and safe. Learn more...
Research Areas: Human-Computer Interaction
Term: -
Researcher:
Kou, Yubo
Sponsoring Agency: College of IST
This project aims to design a social learning mechanism that supports users who are punished in online communities by encouraging the exchange of ideas between punished users and community members, so that the former can better evaluate their past behaviors against community norms and platform policies. Learn more...
Research Areas: Human-Computer Interaction
Term: -
Researcher:
Kou, Yubo
Sponsoring Agency: National Science Foundation
This is a study of human implications of online moderation systems that deal with disruptive online behaviors, such as offensive language and hate speech, by issuing penalties such as content removal or account suspension to users they determine to be disruptive. The study site is a high-population online community, where the research will document and describe human-punishment interaction in terms of how users experience punishment, what are users' post-penalty actions, and what support resources users use for a better understanding of community behavioral standards and behavioral improvement. Learn more...
Research Areas: Data Sciences and Artificial Intelligence, Human-Computer Interaction
Term: -
Researcher:
Bardzell, Jeffrey; Bardzell, Shaowen
Sponsoring Agency: National Science Foundation
This research will provide rich insights into a new line of regional experiments with computer-based economic development in the American Midwest, rooted in collaborations between government, industry, and universities to drive their traditions of manufacturing excellence into the next generation. The research will inform the development of sociotechnical interventions supporting bottom-up innovation procedures and emergent outcomes. Learn more...
Research Areas: Human-Computer Interaction
Term: -
Researcher:
Billah, Syed
Sponsoring Agency: National Institutes of Health
This project seeks to develop the next generation screen magnifier that will bridge the wide gap in smartphone user experience for people with low-vision impairments. It is rooted on three novel ideas. First, instead of indiscriminately magnifying the screen content as is done now, it will do object-aware magnification by identifying the objects in the graphical interface and compacting the space between the objects so as to keep contextually related objects close together in the magnified view. Second, by leveraging the untapped built-in sensors such as accelerometer, geometric field and barometric pressure sensors, it will expand the default surface gestures to include surfaceless natural gestures for magnification operations that can be done with only one hand, thus freeing the other hand for other tasks. Third, it will incorporate a novel keyboardless gesture-based text entry and editing technique to eliminate the difficulties that arise with virtual keyboards for text entry in magnification mode. Learn more...
Research Areas: Human-Computer Interaction
Term: -
Researcher:
Billah, Syed
Sponsoring Agency: College of IST
This project aims to design accessible segmentation algorithms – partition an image into meaningful regions, assign labels to each region, and are widely used in downstream computer vision tasks – to unlock their potential for people with disabilities. Learn more...
Research Areas: Data Sciences and Artificial Intelligence, Human-Computer Interaction
Term: -
Researcher:
Wang, Ting; Ma, Fenglong; Gui, Xinning
Sponsoring Agency: National Science Foundation
The goals of this project are to thoroughly investigate the potential security risks of automated machine learning and to develop rigorous yet easy-to-use mitigation to curb such risks without compromising the benefits of AutoML. Learn more...
Research Areas: Data Sciences and Artificial Intelligence, Human-Computer Interaction, Privacy and Security
Term: -
Researcher:
Sponsoring Agency: National Science Foundation
This project aims to uplift workers and improve the marketplace for online work so that digital work may help with the economic recovery of regions whose traditional industries have left. This research aims to develop sustainable methods for transitioning workers to high-skilled and creative digital jobs that are unlikely to be automated in the near to medium term future. Learn more...
Research Areas: Data Sciences and Artificial Intelligence, Human-Computer Interaction
Term: -