Currently, I am interested in building mathematical foundations of large language models and mechanistic interpretability.

1. Mathematical Foundations of Large Language Models: In recent years, researchers have been trying to understand why modern neural architectures perform so well on various tasks. I try to understand what LLMs (mostly transformers) can or cannot do from an expressivity viewpoint, i.e. does there exist parameters/weights such that a transformer can perform certain tasks. Recently, I am working on applying theoretical ideas to speed up KV caching (in practice). 

2. Mechanistic interpretability: Mechanistic interpretability aims to understand the internal workings of neural networks. Recently, I am thinking about model steering and logical extrapolation.

See my papers below for details! (Papers with an asterisk* follow alphabetical order.)