Selected Projects 2025-2026

This project studies why LLMs struggle to reliably express their uncertainty through verbalized confidence, despite often knowing when their answers are correct. Through probing experiments, we show that pretrained LLMs encode a strong latent correctness signal in their hidden representations, which emerges during pretraining and remains largely unchanged by instruction tuning. However, this internal signal is not naturally accessible via zero-shot or in-context verbalized confidence, which we find to be dominated by instruction-following behavior rather than genuine uncertainty elicitation. We systematically demonstrate that zero-shot verbalized confidence is highly prompt-sensitive, weakly correlated with correctness, and poorly aligned with internal correctness representations. In contrast, lightweight supervised fine-tuning explicitly aligns verbalized confidence with the model’s latent correctness signal, enabling confidence outputs that closely match probe-level performance. Crucially, this alignment generalizes only to tasks where the underlying correctness representation itself generalizes.Shortly, we show that verbalized confidence is not an emergent capability of LLMs but an alignment problem: models possess internal correctness information yet require explicit supervision to expose it through language.

Expected Deliverables
Conference publication (COLM 2026 or EMNLP 2026)

Sungmin Kang, Yavuz Faruk Bakman, Duygu Nur Yaldiz, Baturalp Buyukates, Salman Avestimehr. “Uncertainty Quantification for Hallucination Detection in Large Language Models: Foundations, Methodology, and Future Directions”, 2025.

This project improves the reliability and interpretability of AI for financial decision-making by combining GraphRAG with Topological Data Analysis (TDA). Financial data are structured into a knowledge graph and embedded using graph neural networks. The mapper algorithm from TDA is then applied to these embeddings to identify clusters, gaps, and ambiguities in retrieved information. By analyzing the topology of relevant knowledge, the system can assess whether an answer is well-grounded or potentially unreliable. If results are ambiguous, the system automatically refines the query or flags hallucination risk. The outcome is a prototype system that reduces hallucinations, improves answer accuracy, and provides interpretable insights for more responsible AI in finance.

Expected Deliverables
Conference and journal publications, open-source software code

This project focuses on generative AI for financial decision-making in noisy and dynamic environments. We study problems governed by stochastic differential equations, where decisions must be made over time to optimize expected rewards under uncertainty. We investigate multimodal large language models that process time series and other structured inputs, with the goal of understanding and improving their ability to reason about stochastic dynamics and make reliable decisions. As an initial step, we benchmark the performance of existing MLLMs on carefully designed synthetic and real-world datasets that require reasoning over SDE-driven dynamics. A representative example is the optimal exercise problem for American options, where the model must identify the optimal stopping time given stochastic asset price trajectories. These benchmarks allow us to systematically assess current limitations in reasoning, temporal understanding, and decision-making under uncertainty. Building on these findings, we plan to improve model performance using reinforcement learning from human feedback (RLHF) and by developing novel algorithms tailored to reasoning over stochastic dynamics. Our approach emphasizes learning structured decision strategies rather than pattern matching, aiming to produce models that can reliably optimize decisions in stochastic environments. Finally, we will evaluate the extent to which the learned reasoning abilities generalize across domains. In particular, we will study whether models trained on financial decision-making tasks can transfer their understanding to other SDE-governed systems, such as problems in physics with different state representations and reward formulations. 

Expected Deliverables
A set of synthetic and real-world benchmarks for evaluating multimodal reasoning and decision-making in stochastic dynamic systems governed by stochastic differential equations (SDEs).

Improved multimodal models and training methods, incorporating reinforcement learning from human feedback (RLHF) and novel reasoning algorithms for decision-making under stochastic dynamics.

An assessment of cross-domain generalization, testing whether learned reasoning strategies transfer from financial tasks to other SDE-governed domains such as physics.

Research outputs, including peer-reviewed publications and, where appropriate, publicly released code and datasets.

Publications & Conferences 
Sepehri, M. S., Tinaz, B., Fabian, Z., & Soltanolkotabi, M. “Hyperphantasia: A Benchmark for Evaluating the Mental Visualization Capabilities of Multimodal LLMs”. Proceedings of the Thirty-Ninth

Annual Conference on Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2025. https://openreview.net/forum?id=NBU5IJGhCf

Project Resources: GitHub: https://github.com/AIF4S/Hyperphantasia

Hugging Face: https://huggingface.co/datasets/shahab7899/Hyperphantasia