Hello!
I work on the machine learning infrastructure at Bloomberg for
ML model training.
My interests fall under the umbrella of understanding how large scale models (read, LLMs)
can be trained across GPU farms and eventually served in the most efficient way.
Previously, I was a grad student at Carnegie Mellon University, working
my way through systems and ML. During grad school, I interned at Nutanix where I got to play with Kubernetes container storage interface, volumes and AWS EKS.
I did my bachelor's in Electrical Engineering from Delhi Technological University.
Posts
-
Model Caching for AI Workloads: Open Source Summit 2025 Denver CO
June 24, 2025
Gave a talk on Model Caching for AI Training and Inference workloads to speed up model load time in a cloud native environment.
This talk discusses the challenges faced with hosting large models for inference and fine-tuning purposes, and how model caching can help mitigate some of these challenges by reducing load times during auto-scaling of services, improving resource utilization, and boosting data scientists’ productivity. The talk dives into how we integrated KServe’s Model Cache into its AI workloads and built an API to manage cache federation.
-
Quick thoughts on DSPy: automatic prompt optimization?
Sep 8, 2024
A new paradigm for giving structure to prompt engineering
-
NNLIBC: Neural Networks in C for WebAssembly
June 1, 2022
This is a Pytorch-like neural network library written in C.
Contact Me
Email: rtjsingh30@gmail.com