Aashiq Muhamed

Ph.D. Student in Computer Science, Carnegie Mellon University

aashiq.jpg

Hi, I’m Aashiq, a Ph.D. student at CMU Machine Learning Department, advised by Professors Mona Diab and Virginia Smith.

My research interests are broadly in the area of responsible AI. Topics I’m interested in include:

  • AI alignment, Safety, and Control
  • Compute, Data, and Communication Efficiency
  • Data-Centric AI and Science of AI

I hold a B.Tech in Mechanical Engineering from the Indian Institute of Technology, Roorkee, where I was the President’s Gold Medalist. I also hold an MS in Mechanical Engineering from Stanford University and an MS in Language Technologies from CMU Language Technologies Institute.

Before embarking on my Ph.D. journey, I accrued five years of industry experience, working as an Applied Scientist at AWS DeepComposer (2019-2021), Amazon Search M5 (2021-2022), and AWS AI (2022-2023).

Feel free to reach out if you’re interested in my research or contemplating a Ph.D. after industry experience.

news

Jun 1, 2025 Working as a FIG Fellow with Chi Nguyen, Caspar Oesterheld, and Emery Cooper on “Training AIs to Aid Decision Theory and Acausal Research” through the Future Impact Group’s Philosophy for Safe AI program.
May 2, 2025 I’m excited to present two papers at NAACL 2025: “CoRAG: Collaborative Retrieval-Augmented Generation” and “Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Models”, with the latter also featuring an oral presentation at the TrustNLP Workshop.
Oct 16, 2024 Our work Inducing Elasticity in Foundation Models: Post-Training Techniques for Adaptable Inference was accepted at the The 4th Workshop on Efficient Natural Language and Speech Processing @ NeurIPS 2024. We study weight decomposition approaches to induce elasticity in pretrained LLMs.
Oct 15, 2024 I will be presenting my MATS project, “Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in LLMs,” at the Second NeurIPS Workshop on Attributing Model Behavior at Scale. Our results show that SSAEs pareto dominate pretrained SAEs within specific subdomains, with promising implications for broader applications in AI safety.
Sep 1, 2024 Delighted to have been named a Siebel Scholar 2025.
Aug 1, 2024 Spending the summer at Berkeley as an ML alignment and Theory Scholar working with Lucius Bushnaq and Jake Mendel from Apollo Research. Excited about pushing the frontiers of mechanistic interpretability.
Jul 20, 2024 We will present our work GRASS: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients at EMNLP 2024 and the Efficient Systems for Foundation Models Workshop @ICML 2024.
Mar 5, 2024 We will present our work “Fed Up with Complexity: Simplifying Many-Task Federated Learning with NTKFedAvg”, and “Cache Me If You Can: The Case For Retrieval Augmentation in Federated Learning” at the Privacy Regulation and Protection in Machine Learning Workshop @ ICLR 2024.
Mar 1, 2024 Our work “Less is Fed More: Sparsity Reduces Feature Distortion in Federated Learning” studying sparsity and feature distortion in federated learning was accepted at the Modular and Multilingual NLP Workshop at EACL 2024.
Feb 27, 2024 Our work “Adversarial Continuous Text to Image Generation” has been accepted to CVPR24!

selected publications

  1. Preprint
    Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs
    Xiangchen Song, Aashiq Muhamed, Yujia Zheng, and 5 more authors
    arXiv preprint arXiv:2505.20254, 2025
  2. Preprint
    SAEs can improve unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMs
    Aashiq Muhamed, Jacopo Bonato, Mona Diab, and 1 more author
    arXiv preprint arXiv:2504.08192, 2025
  3. EMNLP
    Grass: Compute Efficient Low-Memory LLM Training with Structured Sparse Gradients
    Aashiq Muhamed, Oscar Li, David Woodruff, and 2 more authors
    In The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2024
  4. NeurIPS ENLSP
    CTR-BERT: Cost-effective knowledge distillation for billion-parameter teacher models
    Aashiq Muhamed, Iman Keivanloo, Sujan Perera, and 6 more authors
    In NeurIPS Workshop on Efficient Natural Language and Speech Processing (ENLSP) (Oral Spotlight), 2021