Aashiq Muhamed

Ph.D. Student in Machine Learning, Carnegie Mellon University

aashiq.jpg

I am a Ph.D. student in the CMU Machine Learning Department, advised by Professors Mona Diab and Virginia Smith. I also closely collaborate with Professor Andrew Ilyas.

I am an Amazon PhD Fellow, a Cooperative AI PhD Fellow, a Siebel Scholar, and a two-time MATS Scholar (2024 and 2025). I was an AI Safety Research Fellow in the Anthropic Fellows Program and a Digital Minds Fellow at the University of Cambridge.

My research develops the technical foundations for building AI systems that remain safe and trustworthy after deployment. I organize this work around three problems:

  • Safeguards: I study how safety mechanisms in open-weight models can be bypassed or removed, and develop defenses against these attacks.
  • Poisoning: I investigate how backdoors and secret loyalties are introduced into models and persist through subsequent training. I develop white-box methods to detect hidden behavior that evades standard evaluation.
  • Oversight: I develop and study control protocols that use trusted monitors to limit harm from untrusted agents, including when model-level defenses fail or agents collude.

Across these directions, I use mechanistic interpretability to understand model behavior and inform the design of defenses and monitors.

My research has received grants and in-kind support from Coefficient Giving, the Center for AI Safety (CAIS), Goodfire, Snorkel AI, and Longview Philanthropy.

background

Before CMU, I spent nearly four years at Amazon building and deploying large-scale AI systems across AWS DeepComposer, Amazon Search M5, and AWS AI. This experience informs my focus on safety methods that remain effective in deployment.

I earned my B.Tech in Mechanical Engineering from the Indian Institute of Technology, Roorkee, where I received the President’s Gold Medal, and an MS in Mechanical Engineering from Stanford University. I completed my MS in Language Technologies at the CMU Language Technologies Institute.

I also mentor researchers through SPAR and MARS.

I’m always open to collaboration — feel free to reach out.

news

Oct 9, 2026 I presented Pando at COLM, where we ask whether interpretability methods remain effective when models do not faithfully explain themselves. I also presented Decoy Direction Optimization, our defense against attacks that remove model safeguards, at the Actionable Interpretability Workshop.
Oct 8, 2026 I presented CollusionBench at Snorkel AI’s Frontier Data Summit. We’re developing the benchmark to study collusion and undesirable coordination among LLM agents, with support from a Snorkel Open Benchmark Grant.
Aug 7, 2026 Participating in the Iliad Intensive starting August 7, 2026, back at Lighthaven — a three-week, full-time course on foundational AI alignment research, covering deep learning theory, agent foundations, interpretability, and safety guarantees.
May 11, 2026 Excited to be spending summer 2026 as a research intern with the Security group at Microsoft Research in Redmond, working on agentic security and monitoring.
Apr 25, 2026 I’ll be mentoring a project at MARS V on Behaviors That Survive Alignment — studying how hidden objectives persist through training and whether alignment erases or merely masks them. If you’re interested in mechanistic interpretability, emergent misalignment, or inductive backdoors, please consider applying!
Apr 14, 2026 Selected as one of 15 fellows (from 3,600+ applicants) for the Digital Minds Fellowship at the University of Cambridge, running August 3–9, 2026.
Feb 6, 2026 Honored to be selected as a Cooperative AI PhD Fellow for 2026, ranked among the top 5% of applicants!
Jan 9, 2026 Excited to be admitted to the 2026 cohort of the International Programme on AI Evaluation: Capabilities and Safety, selected among the top 2% of applicants globally! I’ll be joining a diverse group working on rigorous methods for evaluating AI capabilities and safety.
Nov 30, 2025 Excited to be selected as an Anthropic Fellow for AI safety research starting January 2025!
Oct 21, 2025 Delighted to be named an Amazon AI PhD Fellow! I’ll be working on monitoring and post-training for AI agents.
Aug 1, 2025 I’m excited to be a mentor for SPAR 2025 this Fall! I’ll be mentoring students on using SAEs for interpretable and tamper-resistant alignment. If you’re interested in advancing AI safety through hands-on interpretability research, please apply!
Jul 1, 2025 I am a visiting researcher at University of California, Berkeley this summer, hosted by Prof. Dawn Song and Xuandong Zhao.
Jun 1, 2025 Working as a FIG Fellow with Chi Nguyen, Caspar Oesterheld, and Emery Cooper on “Training AIs to Aid Decision Theory and Acausal Research” through the Future Impact Group’s Philosophy for Safe AI program.
May 2, 2025 I’m excited to present two papers at NAACL 2025: “CoRAG: Collaborative Retrieval-Augmented Generation” and “Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models”, with the latter also featuring an oral presentation at the TrustNLP Workshop.
Oct 16, 2024 Our work Inducing Elasticity in Foundation Models: Post-Training Techniques for Adaptable Inference was accepted at the The 4th Workshop on Efficient Natural Language and Speech Processing @ NeurIPS 2024. We study weight decomposition approaches to induce elasticity in pretrained LLMs.
Oct 15, 2024 I will be presenting my MATS project, “Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in LLMs,” at the Second NeurIPS Workshop on Attributing Model Behavior at Scale. Our results show that SSAEs pareto dominate pretrained SAEs within specific subdomains, with promising implications for broader applications in AI safety.
Sep 1, 2024 Delighted to have been named a Siebel Scholar 2025.
Aug 1, 2024 Spending the summer at Berkeley as an ML alignment and Theory Scholar working with Lucius Bushnaq and Jake Mendel from Apollo Research. Excited about pushing the frontiers of mechanistic interpretability.
Jul 20, 2024 We will present our work GRASS: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients at EMNLP 2024 and the Efficient Systems for Foundation Models Workshop @ICML 2024.
Mar 5, 2024 We will present our work “Fed Up with Complexity: Simplifying Many-Task Federated Learning with NTKFedAvg”, and “Cache Me If You Can: The Case For Retrieval Augmentation in Federated Learning” at the Privacy Regulation and Protection in Machine Learning Workshop @ ICLR 2024.
Mar 1, 2024 Our work “Less is Fed More: Sparsity Reduces Feature Distortion in Federated Learning” studying sparsity and feature distortion in federated learning was accepted at the Modular and Multilingual NLP Workshop at EACL 2024.
Feb 27, 2024 Our work “Adversarial Continuous Text to Image Generation” has been accepted to CVPR24!
Dec 20, 2023 We released An In-depth Look at Gemini’s Language Abilities, an impartial, in-depth, and reproducible study comparing Gemini, GPT, and Mixtral.
Dec 20, 2023 Our solution was the winning entry at the 1st Privacy Preserving Federated Learning Document VQA competition at NeurIPS 2023.
Sep 1, 2023 Started my MS/Ph.D at CMU!

selected publications

  1. Preprint
    MOLE: Detecting Insider Threats in AI Agents
    Aashiq Muhamed, and Virginia Smith
    arXiv preprint arXiv: 2609.06966, 2026
  2. PreprintCOLM Actionable InterpICML MechInterp
    Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration
    Aashiq Muhamed, Mona T. Diab, and Virginia Smith
    Preprint, 2026
  3. Preprint
    Pick Your Poison: Learning to Select Poison Sets for Stronger LLM Backdoor Attacks
    Aashiq Muhamed, Mona T. Diab, Virginia Smith, and 2 more authors
    Preprint, 2026
  4. COLM
    Pando: Do Interpretability Methods Work When Models Won’t Explain Themselves?
    Ziqian Zhong, Aashiq Muhamed, Mona T. Diab, and 2 more authors
    In Conference on Language Modeling (COLM), 2026