Research on questioning one's perspectives or frames about ambiguous situations, as a precursor to changing one's frame, through the lens of Klein's data-frame model of sensemaking, motivated reasoning, and counterfactual thinking. People tend to quickly form frames with limited information, then stick to those frames. Encouraging questioning a frame is relevant in everyday social situations, as well as more motivated domains wherein people are motivated to maintain their frames, regardless of how accurate they may be - such as climate change beliefs, workplace/academic gender bias, and misinformation. This project encompasses seven studies utilizing a range of methods.

Qualitatively, I investigated whether people naturally question their frames when instructed to generate multiple explanations across several ambiguous scenarios. Participants rarely showed evidence of questioning, operationalized through variance in explanation content and presence of counterfactual information. I created two potential conceptual models of the questioning process, a base model and a model inspired by Roese's counterfactual generation model. I compared the models using path analysis in R and found that the counterfactual model had better fit; availability of alternative frames mediated the effect of situational mutability on likelihood to question an initial frame. I developed and tested three strategies to prompt questioning a frame based in counterfactual thinking and tested their efficacy across three randomized, controlled experiments utilizing several ambiguous scenarios. A strategy to generate mutable factors within the situation did not increase participants' questioning compared to controls, but two other strategies were effective: reading a counterfactual "What if?" statement, and filling-in-the-blank on a counterfactual "What if?" statement. Simply reading the statement was most effective, but filling-in-the-blank (generating mutability in a structured way) still increased likelihood of questioning an initial frame compared to a control strategy. The effectiveness of the counterfactual-read strategy was replicated. 

I also applied the most effective strategy to the academic hiring context. Given that hiring decisions must be made quickly based on incomplete or ambiguous information, often without clear guidelines for judgments, biased decision making and heuristics are likely to happen. This plays out in the body of research finding that resumes are interpreted differently when they come from men vs. women, leading to differing judgments, hiring decisions, and pay. Strategies that are simple and can be utilized to mitigate bias in the moment are needed - strategies like considering a counterfactual statement. I tested this in a randomized, controlled trial using a hiring paradigm. No gender bias was found in hiring decisions or judgments of support. Participants who read a counterfactual "What if?" statement about a candidate they did not initially hire were more likely to question their initial support ratings and hiring decisions than were controls. This study was inspired by my work with the ADVANCE Initiative (below) as a possible strategy to introduce into hiring committee policies.

In preparation: Questioning Your Answers: Counterfactual Thinking Strategies for Questioning a Frame.
In preparation:Questioning a Frame in Ambiguous Social Situations.

SPSP 2024: "[268]Comparing Two Strategies for Frame Questioning: Providing Counterfactuals is More Effective than Self-Generation" (PDF)
JDM 2023: "Questioning in sensemaking: When counterfactual strategies are effective" (PDF)
CogSci 2022: "Changing Perspectives: Examining Factors Related To Counterfactual Thinking In Ambiguous Social Judgments" (PDF)
APS 2022: "X-45 - Easy Does It: Ease of Generating Explanations and Ambiguity Increase Questioning One’s Frame during Sensemaking" (PDF)

The Premortem is a technique used in the planning stage of design to mitigate hindsight bias and challenges with brainstorming and information sharing by adopting a failure frame from the beginning. When a plan is made, the Premortem begins by assuming that it has already failed catastrophically - the team then works both individually and together to generate reasons for failure and solutions to those reasons. This results in greater confidence in the plan, as solutions to specific, plausible failures can be incorporated into the design from the beginning. It also requires all team members to participate, leading to a diverse set of perspectives on failures and solutions.

Dr. Beth Veinott and I have explored applying the Premortem to video game development as well as compared human and generative AI responses to the method. For a game development team, incorporating the Premortem into their planning was both easy and effective, increasing their confidence in the plan in general and their ability to overcome challenges. When we compared humans' and generative AI's (ChatGPT-4) responses to the Premortem for university pandemic planning, we found they generated similar reasons for failure, and relatively similar solutions. However, ChatGPT-4 did not generate reasons for failure beyond the bounds of the given plan, whereas humans were more likely to foresee additional challenges. Using generative AI also foregoes the team buy-in for the plan and capturing of diverse team role perspectives that the Premortem technique facilitates. This suggests that generative AI could be used as an additional team member participating in the Premortem, but could not be used in place of the team.

(Accepted) HCI International 2024: "Enhancing Adaptive Instructional Approaches with New Artificial Intelligence Tools and Techniques"
HFES Proceedings 2023:  "Premortems in Game Development Teams: Impact and Potential." (PDF)

Graduate research assistant. Evaluating programs run through an NSF ADVANCE grant (#1760585) to improve knowledge, attitudes, and behavioral intentions regarding faculty gender diversity. For one article, analyzed attitudes held by participants of a mandatory, repeated implicit bias training program; found no correlation between number of trainings attended and cultural competence attitudes. For another article, analyzed attitudes held by participants before and after a workshop to promote advocacy for women and faculty of color, across five sessions (2x5 ANOVA). Attitudes reflected increased awareness of gender-based and racial inequity after the workshop, with no difference between the sessions. Additional analyses revealed no effects of participant gender or position at the university.

Colbert, K., Lehman, B., Goltz, S., & Minerick, A. (2024). Assessment of Advocates & Allies Workshops at a Predominantly White, STEM-Focused Institution. In progress.
Journal of Higher Education Policy and Management 2023: Effects of repeated implicit bias training in a North American university

A project to create a database of images to be used as stimuli in climate-related research. Participants rated 320 images on their relevance to climate change as well as their emotional valance and arousal; these images and their ratings are available for use in research at the database below. The images rated most relevant to climate change tended to be dramatic and negative, with subjects like polar bears on melting ice and outcomes of natural disasters. Subjects like windmills and solar panels were rated positively, but were rated less relevant. Images of people were most likely to be rated as the least relevant to climate change; this is in contrast with the types of images that news media often pairs with climate stories, indicating a mismatch between what the general public and the media believe represents climate change. Participants with greater pro-environmental dispositions were more likely to give higher climate-relevance ratings in general, and were later found to show more attention bias toward climate-relevant images in a dot-probe task.

Frontiers in Psychology 2019: Affective Images of Climate Change
Cognitive Processing 2019: Climate change images produce an attentional bias associated with pro-environmental disposition