Projects

AdaptAI: A Personalized Solution to Sense Your Stress, Fix Your Mess, and Boost Productivity

AdaptAI is a multimodal AI system that enhances productivity and well-being by tailoring interventions to individual needs. Unlike generic tools, AdaptAI integrates egocentric vision, audio, physiological signals, and LLM-driven workflows to provide context-aware support. Its Tone-Adaptive Conversational Agent enhances productivity by adjusting responses to be more positive and supportive during stressful moments.

Code HCI Research

KhalasiIO: Detecting Situational Impairments With Reasoning-based Memory Bank

Detects and addresses temporary impairments caused by environmental factors like noise, lighting, and stress. Integrates wearable devices and contextual memory for personalized, real-time interventions, reducing cognitive load and frustration. Promising results highlight scalability and improved accessibility.

HCI Research

Strong Lensing Classification & Super-Resolution with Masked Autoencoders

Deep learning methods for astrophysics: explored ResNet-18 and Masked Autoencoder (MAE) models for classifying strong gravitational lensing images and enhancing resolution with ESRGAN. Included interpretability experiments with Grad-CAM to analyze model decisions and identify biases. Achieved improved super-resolution with physics-informed insights.

Code Astrophysics Deep Learning

Latent Space Collapse? Understanding the Effects of Narrow Fine-Tuning on LLMs

Explored how narrow fine-tuning reshapes GPT-2’s internal representations in a sentiment classification task. Using TransformerLens, we analyzed shifts in activation and embedding spaces, sentiment direction norms, and causal mediation experiments. Key finding: fine-tuning induces 'output squishing,' reducing diversity and potentially harming generalization.

Code LLMs Representation Learning

RAG Optimizations with Reinforcement Learning

A novel reinforcement learning-based optimization strategy to improve Retrieval-Augmented Generation (RAG) performance. Evaluates retriever performance across diverse conversational datasets using text analysis metrics, various retrieval techniques, and RL policies such as DQN, PPO, and Multi-Armed Bandits to reduce irrelevant content and improve response quality.

Deep Learning Reinforcement Learning

Quantum Machine Learning Classifiers

Implemented 24 combinations of Feature Maps, Ansatzs, and Optimizers to evaluate the efficacy of quantum kernel‐based classifiers. Our best combination — Z-Feature map, EfficientSU2 ansatz, and L_BFGS_B optimizer — outperformed classical SVM across all metrics.

Code Quantum Machine Learning Research

Visual Embeddings Solving Jigsaw Puzzles

A deep learning model using Graph Neural Networks and Autoencoders for representation learning via jigsaw puzzle paradigms. Introduced a Segmented Flow warp approach, achieving 60% validation accuracy, competing with SOTA methods. Original idea was to devise a efficient pre-training like solving jigsaw, to improve downstream performance in the wild for visual tasks.

Code Deep Learning Computer Vision

Soham Petkar