Divyam Goel

Divyam Goel

Machine Learning Engineer - L2

Adobe

Biography

I am a Machine Learning Engineer at Adobe, based in Bangalore. I earned my bachelor’s degree in E&CE from the Indian Institute of Technology, Roorkee. Concurrently, I work with Prof. Angel Xuan Chang, conducting research on conditional 3D scene generation at her lab in Simon Fraser University. Previously, I contributed to the groundbreaking research at Rephrase.ai (now part of Adobe) by leveraging generative AI for human avatar technologies.

My research at the GIST Computer Vision Lab, which also formed the basis of my undergraduate thesis, focused on developing embodied agents for vision and language navigation. During my time at the University of British Columbia (Summer 2021), I worked on developing efficient but accurate language models for software engineering tasks. My research interests are in computer vision and robotics, with a specialty in machine learning and (tractable) generative modeling.

At IIT Roorkee, I was a core member of a Deep Learning research centric reading group named the Vision and Language Group, and the IEEE Student Chapter, IIT Roorkee.

Interests

  • Computer Vision
  • Graph Learning
  • Probabilistic Generative Models
  • Robotics
  • Natural Language Processing

Education

  • B.Tech. in Electronics and Communications Engineering, 2022

    Indian Institute of Technology Roorkee

News

  • March 2024 Paper titled “Semi‑NMF Regularization‑Based Autoencoder Training for Hyperspectral Unmixing” accepted at NCC 2024.
  • November 2023 Rephrase AI gets acquired by Adobe! I will be joining Adobe - DVA as a Machine Learning Engineer following the acquisition.
  • June 2023 Joined Prof. Angel X. Chang’s lab at SFU as a Research Assistant.
  • April 2023 Promoted to Deep Learning Researcher at Rephrase AI!
  • June 2022 Joined Rephrase AI as a Deep Learning Junior Researcher!
  • June 2022 Paper titled “Language Guided Meta-Control for Embodied Instruction Following” accepted at the Embodied AI Workshop @ CVPR 2022! Preprint coming soon.
  • May 2022 Paper titled “Leveraging Dependency Grammar for Fine-Grained Offensive Language Detection using Graph Convolutional Networks” accepted at SocialNLP@NAACL 2022! Preprint coming soon.
  • March 2022: Paper “On the Cross-Modal Transfer from Natural Language to Code through Adapter Modules” accepted at ICPC 2022! Preprint coming soon.
  • Feb. 2022: Recipient of the IIT Roorkee Heritage Foundation Excellence Award for 2021!
  • July 2021: Joined Prof. Jonghyun Choi’s GIST Computer Vision Lab, South Korea (virtually) as a Research Assistant.
  • June 2021: Headed to Prof. Fatemeh H. Fard’s lab at UBC, Canada (virtually) to work as a Mitacs Globalink Research Intern.
  • Dec. 2020: Selected for the prestigious MITACS Globalink Research Internship 2021!!
  • July 2020: Will be working at PandoCorp as a research intern this fall in their machine learning research lab - Pando Labs.
  • March 2020: Recipient of the IIT Roorkee Heritage Foundation Excellence Award for 2019!
  • April 2019: Selected for the Quantum Computing summer training organized by QuLabs.

Experience

 
 
 
 
 

Machine Learning Engineer - L2

Adobe

Nov 2023 – Present Bangalore, India
  • Working on the research and development of Adobe’s family of GenAI models ‑ Firefly.
 
 
 
 
 

Research Assistant

3D Language Group, Simon Fraser University

Jun 2023 – Present Burnaby, Canada (virtually)
  • Working on the research and development of probabilistic generative models for the task of 3D scene generation.
 
 
 
 
 

Deep Learning Researcher

Rephrase AI

Jun 2022 – Nov 2023 Bangalore, India
  • My research focused on (tractable) generative models for speech AI and computer vision.
  • Enriched Rephrase.ai’s proprietary video generation pipeline, expanding its global outreach by enhancing avatar capabilities for multilingual applications.
  • Collaborated closely with DevOps engineers to efficiently deploy models on AWS, optimizing the generation pipeline to enable the creation of over 100K personalized videos daily.
  • Spearheaded a Text‑to‑Speech initiative, developing a voice cloning system from limited speech data. Also devised a speech restoration pipeline to extract clean speech audios of the source speaker from extremely low‑quality / noisy inputs.
 
 
 
 
 

Research Assistant

GIST Computer Vision Lab

Jul 2021 – Dec 2022 Gwangju, South Korea (virtually)
  • Published 1 research paper in EmbodiedAI at CVPR 2022 and submitted 1 paper at a top tier conference.
  • Released a work proposing a language‑guided meta‑controller to learn robust task‑agnostic representations, and an auxiliary reasoning loss to improve the overall cross‑modal grounding capabilities for embodied instruction following.
  • Submitted a work proposing a story‑visualization framework (SMART) using a multi‑stage multi‑modal transformer with in‑memory spatio‑temporal context, and a discretized variational autoencoder.
 
 
 
 
 

MITACS Globalink Research Intern

University of British Columbia

Jun 2021 – Sep 2021 Kelowna, Canada (virtually)
  • Published and presented a research paper at ICPC 2022.
  • Achieved 140× better parameter budget and ∼95% efficient storage in adapting foundational LLMs to source code.
  • Built the Super Code Clone Detection-88 (SCD-88) dataset to evaluate the proposed approach on python-specific code clone detection.
 
 
 
 
 

Research and Development Intern

Pando Labs - PandoCorp PLC

Aug 2020 – Nov 2020 Chennai, India
  • I worked on developing a practical Deep Reinforcement Learning solution to optimize shipping costs in the Supply Chain industry.
  • Achieved ~85% packing efficiency in 3D bin packing and ~50% cost efficiency in capacitative vehicle routing over existing solutions using off-policy agents in simulated environments.
  • Re-architected a heuristic algorithm to improve efficiency and allow parallelised operations on GPUs.
 
 
 
 
 

Undergraduate Student Researcher

Machine Vision Lab

Dec 2019 – Mar 2020 IIT Roorkee
  • Worked under the supervision of Prof. Balasubramanian Raman on a project titled - Medical Image Segmentation and Classification Problems.

Achievements

IIT Roorkee Heritage Foundation Excellence Award 2021

  • Awarded to 45 students from a pool of over 5000 undergraduates.
  • The Excellence Awards recognize students for outstanding academic, co-curricular and extra-curricular achievements.
See certificate

MITACS Globalink Research Internship

  • Selected for the prestigious MITACS Globalink Research Internship 2021 program.
  • The Mitacs-SICI partnership awards this research grant to top international undergraduates for a research internship under the supervision of Canadian university faculty members.
See certificate

IIT Roorkee Heritage Foundation Excellence Award 2019

  • Awarded to 45 students from a pool of over 5000 undergraduates.
  • The Excellence Awards recognize students for outstanding academic, co-curricular and extra-curricular achievements.
See certificate

Quantum Computing Summer Training

  • Among the 15 students selected for the Quantum Computing Summer Training - 2019.
  • The course comprised of intensive lectures on linear algebra, quantum mechanics and topics in quantum computation including entanglement, multi-qubit computation, quantum key distribution and quantum algorithms.

Featured Projects

*

SemiNMF-Autoencoders

PyTorch code for experiments with a semi-NMF based regularization objective over multiple autoencoder architectures for hyperspectral unmixing.
CODE

Super Code Clone Detection - 88

A dataset consisting of 11,400 python codes for a retrieval based code clone detection.
LINK

HyperspecAE

PyTorch reproduction of the paper titled ‘Hyperspectral Unmixing Using a Neural Network Autoencoder’ (Palsson et al. 2018).
CODE

Customer Churn Management

Led a collaborative effort to develop an agent that predicts consumer churn in the Indian Telecom Industry.
REPORT DEMO

Policy Gradient Methods

Implemented some Policy Gradient RL algorithms (using PyTorch) on the CartPole environment by OpenAI.
CODE

Indic-Language Translation

Trying to utilize a large pre-trained language model to build a English-Hindi machine translation system.
CODE

Sentiment Analysis with BERT

Fine-Tuning a BERT model for Sentiment Classification on the SMILE: Twitter Emotion Dataset.
CODE

NNVisualizer

Developed a Web Application using Flask and StreamLit for the visualization of the weights on neural network.
CODE

Self Driving Car

Designed an autonomous vehicle using a combination of image processing and IoT methods.
CODE DEMO