~/rishikesh portfolio
Research meets engineering

Ideas, implemented.

Voice AI.
Efficient models.
Intelligent agents.

I'm Rishikesh, an AI research engineer and technical founder working across model research and applied systems. I build multilingual speech synthesis, voice cloning and real-time conversational AI, alongside open-source implementations of language, audio and vision models.

I enjoy taking a paper apart, implementing its ideas in PyTorch, and exploring how to train and run models efficiently. My current interests include state-space and hybrid language models, GPU kernels, and coding agents with retrieval, persistent memory and tool use.

Different disciplines. Connected thinking.

Co-founder & CTO at Dubpro.ai. Previously consulting at MyClone.is. 10+ years in software, including 8+ years in AI.

1,977GitHub stars

308Repository forks

10+Years in software

Community counts across five research repositories ·

Jan 2019 — Present

Co-founder & CTO · Dubpro.ai / DeepSync Technologies

Leading multilingual dubbing, neural TTS, voice cloning, speech transformation and generative audio. Built agentic RAG dubbing workflows with human review, achieving approximately 2× faster dubbing. Developed multilingual foundation-model capabilities with RLHF-based post-training for translation quality and alignment.

Developed an in-house dubbing harness and tools, including an LLM-as-judge quality verifier, to automate dubbing and quality analysis, reduce reliance on human review, and save approximately 40% in time and 25% in costs.

—

AI Consultant / Developer · MyClone.is

Real-time voice AI, persona-aware retrieval, memory and conversational infrastructure. Remote, San Francisco-based company.

Oct 2018 — Jan 2019

Data Science / Software Development Engineer · Boutiqaat

Multimodal product matching, recommendations and catalog classification.

Dec 2017 — Oct 2018

Data Scientist · Humonics Global

Speech synthesis, question-answer generation, face recognition and liveness detection.

Jun 2015 — Dec 2017

Software Engineer · Nucleus Software

Banking analytics, loan-default prediction and enterprise financial software.

Education

Institute
National Institute of Technology, Silchar, India
Degree
Bachelor of Technology (B.Tech)
Branch
Electronics & Instrumentation Engineering
Years
–

Technical Skills

Models & training

PyTorch, Hugging Face, Transformers, SSMs, SFT, RLHF, RLVR, GRPO, distillation, diffusion and generative speech.

Inference & performance

Triton, CUDA, Flash Attention, vLLM, KV caching, quantization and GPU memory optimization.

Agents & voice

MCP, RAG, multi-agent orchestration, LlamaIndex, LangChain, LiveKit, Pipecat and Twilio.

Backend & infrastructure

Python, FastAPI, PostgreSQL, pgvector, Redis, Kafka, Docker, Serverless, AWS and GCP.

Selected research

Deep learning papers, implemented and explored.

Video understanding / PyTorch

ViViT

560 stars · 71 forks

A PyTorch implementation of the factorized video transformer encoder, exploring spatial and temporal representation learning.

Partial implementation · Work in progress

Semantic segmentation / PyTorch

ResUNet / ResUNet++

551 stars · 75 forks

Residual U-Net architectures with preprocessing and training workflows for road extraction and semantic segmentation.

Experimental · No longer maintained

Neural vocoders / PyTorch

VocGAN

321 stars · 59 forks

Modified neural vocoder with a MelGAN-based discriminator to address training overhead. Includes training, inference, checkpoints and audio samples.

Waveform synthesis / PyTorch

iSTFTNet

281 stars · 51 forks

Lightweight mel-to-waveform synthesis combining learned upsampling with inverse short-time Fourier transforms.

Experimental C8C8I implementation

Efficient architectures / PyTorch

FNet

264 stars · 36 forks

Fourier transforms in place of attention-based token mixing, implemented in PyTorch to explore efficient sequence architectures.

These are independent research implementations. Counts reflect community interest. Implementation details & context ↗

Currently exploring

Efficient architectures, GPU kernels and agent training.

State-space models

Mamba-3

Readable PyTorch reference with SISO/MIMO recurrence, trapezoidal discretization, rotary state updates and cached decoding.

Hybrid language models

LFM2.5 2.6B

Convolution and attention blocks, official checkpoint loading, KV caching and optional Triton inference kernels.

GPU programming

Triton Kernel Tutorials

31 tutorials covering tiled GEMM, normalization, attention, quantization and inference techniques.

Research infrastructure

Coding Agent RL Lab

Sandboxed code-repair environments, correctness rewards, model rollouts and optional TRL/GRPO integration.

Also exploring Qwen3.5, Gemma 3 and SoundStorm.

More speech, audio & vision projects

Speech & generative audio

Voxtral Codec · Neural codec
FastSpeech2 · Experimental TTS
NU-Wave · Diffusion audio upsampling

Vision architectures

CvT · Convolutional vision transformers
LocalViT · Locality in vision transformers
CoaT · Convolutional attention
SiT · Self-supervised vision transformer (2021)

Upstream contribution

Merged bug fixes to AcademiCodec / HiFi-Codec ↗.

Applied AI systems

Taking models into real conversations and workflows.

AI consultant · Completed six-month engagement

MyClone

Developed voice and backend features for real-time digital personas: streaming conversations, persona-aware retrieval, document ingestion and persistent memory.

LiveKit · LlamaIndex · FastAPI · PostgreSQL / pgvector

View project

Embedding migration from 1,536 to 512 dimensions supported approximately 3× lower vector storage and 15–20% lower voice-chat latency.

Nexus Code Agent

An experimental coding-agent runtime with MCP, sub-agents, tools, persistent sessions, context compaction and sandboxed execution.

View project

Voice AI Agent

A Pipecat-based demo connecting Twilio calls, browser WebRTC and text chat through a shared STT–LLM–TTS pipeline.

View project

07 / Start a conversation

Have something in mind?

I'm interested in research collaborations and engineering challenges in voice AI, efficient models and intelligent agents.

Let's talk