Rishikesh
AI Research Engineer · Voice AI, LLMs & Agentic Systems · Co-founder & CTO
New Delhi, India · rishikksh20@gmail.com · +91 96501 84220
Portfolio · GitHub · X / @ai_rishikesh · Printable CV · Original résumé
Professional summary
AI research engineer and technical founder with 10+ years in software development, including 8+ years in AI and generative systems. My work connects research implementations with practical systems across multilingual speech synthesis, voice cloning, real-time voice agents, language-model architectures, retrieval, and efficient inference.
As Co-founder and CTO of Dubpro.ai / DeepSync Technologies, I lead AI and engineering work on multilingual dubbing and generative audio. I also completed a six-month AI consulting engagement with MyClone.is, building real-time digital-persona capabilities, retrieval, persistent memory, and conversational infrastructure.
I implement research papers to understand their architecture and computational trade-offs, then explore training, decoding, GPU kernels, and application integration. My open-source portfolio spans established speech and vision implementations and newer work on selective state-space models, hybrid LLMs, Triton kernels, and coding agents.
Selected impact
- Open-source engagement: ViViT, ResUNet, VocGAN, iSTFTNet, and FNet together received 1,977 stars and 308 forks in the October 18, 2026 snapshot.
- Voice retrieval optimization: At MyClone, migrated embeddings from 1,536 to 512 dimensions, with improvements of approximately 3× lower vector storage, 2–2.5× faster retrieval, and 15–20% lower voice-chat latency.
- AI dubbing workflows: At Dubpro.ai, architected agentic RAG workflows with human review, achieving approximately 2× faster dubbing.
- Research-to-system range: Work spans neural vocoders, video transformers, segmentation, state-space models, real-time audio pipelines, persistent memory, and agent orchestration.
Engagement counts are a dated snapshot, not evidence of citations, production adoption, or benchmark superiority. Repository performance observations are identified separately below.
Professional experience
AI Consultant / Developer — MyClone.is, formerly Rappo
Completed six-month engagement · Remote · San Francisco-based company
- Developed backend features for a real-time Voice AI digital-persona platform that reproduces a user's voice, conversational style, knowledge, and communication patterns from text and voice data.
- Built low-latency conversational workflows with LiveKit, streaming STT/TTS, LLM orchestration, agent tools, and real-time session management.
- Implemented persona-aware retrieval and persistent memory using LlamaIndex, PostgreSQL/pgvector, hybrid RAG, document ingestion, and source-level access filters.
- Migrated embeddings from 1,536 to 512 dimensions; reported approximately 3× lower vector storage, 2–2.5× faster retrieval, and 15–20% lower voice-chat latency.
Stack: Python · FastAPI · LiveKit · LlamaIndex · PostgreSQL · pgvector · RAG · Streaming STT/TTS
Open-source project · MyClone.is
Co-founder & CTO — Dubpro.ai / DeepSync Technologies
January 2019–Present · Noida, India
- Co-founded and led development of AI dubbing, multilingual text-to-speech, voice cloning, speech transformation, and generative audio systems.
- Architected agentic RAG dubbing workflows with human-in-the-loop review, increasing dubbing speed by approximately 2×.
- Developed an in-house dubbing harness and tools, including an LLM-as-judge quality verifier, to automate dubbing and quality analysis, reduce reliance on human review, and save approximately 40% in time and 25% in costs.
- Developed multilingual foundation-model capabilities and applied RLHF-based post-training for translation-specific quality and alignment.
Focus: Multilingual speech · Neural TTS · Voice conversion · Translation · Agentic RAG · Human review · RLHF
Data Science / Software Development Engineer — Boutiqaat
October 2018–January 2019 · Gurugram, India
- Built text-and-image product matching, hybrid recommendation systems, and automated product categorization tools for commercial catalogs.
- Developed clustering, tagging, and product-labeling algorithms to support e-commerce data operations.
Focus: Multimodal matching · Recommendations · Classification · Catalog intelligence
Data Scientist — Humonics Global
December 2017–October 2018 · Gurugram, India
- Built automated question-answer generation systems for reading comprehension.
- Developed an LSTM/WaveNet-based TTS engine for conversational applications.
- Worked on face recognition and liveness detection solutions.
Focus: Speech synthesis · NLP · Conversational AI · Computer vision
Software Engineer — Nucleus Software
June 2015–December 2017 · India
- Developed banking transaction analytics and loan-default prediction systems.
- Designed and implemented financial supply-chain management software using MVC architecture.
Focus: Financial software · Predictive analytics · Enterprise application development
Selected open-source AI research
These implementations reflect work across video understanding, segmentation, speech synthesis, and efficient sequence modeling. The work implements and explores research by the original paper authors.
Community engagement snapshot
| Repository | Stars | Forks | Research area |
|---|---|---|---|
| ViViT-pytorch | 560 | 71 | Video transformers |
| ResUnet | 551 | 75 | Semantic segmentation |
| VocGAN | 321 | 59 | Neural vocoders |
| iSTFTNet-pytorch | 281 | 51 | Efficient audio generation |
| FNet-pytorch | 264 | 36 | Fourier token mixing |
| Total across these five repositories | 1,977 | 308 | Five research implementations |
Snapshot: October 18, 2026. The supplied review's counts matched the GitHub metadata checked for this portfolio update.
ViViT — Video Vision Transformer
Repository · 560 stars · 71 forks
- Implemented the factorized-encoder variant of ViViT for video classification in PyTorch.
- Explored how transformer architectures represent spatial and temporal information in video.
- Provides a research implementation reference for spatiotemporal representation learning and video understanding.
- Status: Partial, work-in-progress implementation. The README identifies Model 2 as implemented and Models 3/4 as unimplemented; this is not a complete reproduction.
Themes: Video understanding · Spatiotemporal modeling · Transformers · PyTorch
ResUNet / ResUNet++ — Residual image segmentation
Repository · 551 stars · 75 forks
- Implemented residual U-Net architectures for semantic image segmentation.
- Included crop preprocessing and training workflows for the Massachusetts Roads Dataset.
- Explored residual learning and encoder–decoder design for dense image prediction.
- Status: The README states that the repository is no longer maintained, experimental, and not production-ready.
Themes: Semantic segmentation · Residual networks · U-Net · Road extraction
VocGAN — Modified GAN neural vocoder
Repository · 321 stars · 59 forks
- Developed a modified VocGAN implementation with a MelGAN-based discriminator to reduce training overhead.
- Replaced the hierarchically nested JCU discriminator and modified the generator; the baseline branch preserves the original model.
- Included preprocessing, training, inference, pretrained models, and linked audio samples.
- Provided checkpoints for English LJSpeech, Korean KSS, and multispeaker VCTK datasets.
Themes: GANs · Neural vocoders · Speech synthesis · Training efficiency
iSTFTNet — Efficient neural waveform synthesis
Repository · 281 stars · 51 forks
- Implemented the C8C8I variant of iSTFTNet for lightweight mel-to-waveform synthesis using inverse short-time Fourier transforms.
- Explored how signal-processing structure can reduce neural audio synthesis cost.
- The README reports approximately 60% faster inference than HiFi-GAN and good audio quality with approximately 30% less training.
- Status and measurement scope: Experimental implementation for testing, without performance optimization. These observations are repository-reported and were not independently reproduced.
Themes: Inverse STFT · Neural audio · Efficient synthesis · Vocoders
FNet — Fourier token mixing
Repository · 264 stars · 36 forks
- Implemented FNet in PyTorch, exploring Fourier transforms as a replacement for attention-based token mixing.
- Studied an alternative sequence-modeling design through a compact research implementation.
- Connects earlier efficient token-mixing work with my continuing interest in hybrid attention and selective state-space models.
Themes: Fourier transforms · Sequence modeling · Attention alternatives · Efficient architectures
Mamba-3 — Selective state-space language models
Repository · 40 stars · 3 forks in the snapshot
- Implemented Mamba-3 in plain PyTorch, including SISO/MIMO recurrence, exponential-trapezoidal discretization, and per-head accumulated rotary state updates.
- Included residual blocks, a stacked language model, an inference cache, and single-token autoregressive decoding.
- Built a readable reference for investigating state updates and efficient sequence modeling.
- Status: Educational/research reference with serial scans; no custom CUDA/Triton kernels or production inference optimization.
Themes: Selective SSMs · MIMO recurrence · Rotary state updates · Cached decoding
Current architecture, inference & agent research
LFM2.5 2.6B — Hybrid convolution / attention LLM
Repository · 3 stars · 1 fork in the snapshot
- Implemented LiquidAI's LFM2.5-2.6B architecture in PyTorch with official checkpoint loading.
- Combines causal depthwise-convolution blocks and grouped-query attention, with KV/convolution caching, RoPE, RMSNorm, SwiGLU, and streaming sampling.
- Includes optional Triton RMSNorm, short-context GQA decoding, and sampling kernels, alongside portable fallbacks.
- The README reports 26.69→40.22 tokens/second on an RTX 4060 8 GB under its stated BF16 setup, approximately 50.7% higher throughput; this repository-reported result was not independently reproduced.
Qwen3.5 0.8B — Hybrid linear-attention exploration
- From-scratch PyTorch architecture implementation aligned with the official checkpoint.
- Explores hybrid GatedDeltaNet linear attention and full grouped-query attention, learned decay, partial RoPE, gated RMSNorm, checkpoint mapping, and sampling.
Gemma 3 270M — Small-model architecture playground
- PyTorch architecture exploration with sliding/full attention, grouped-query attention, gated feed-forward layers, RMSNorm, and RoPE.
- Focused on model internals and inference experimentation, without a claimed training or benchmark achievement.
Triton Kernel Tutorials — GPU programming experiments
Repository · 2 stars · 0 forks in the snapshot
- Created 31 self-contained tutorials with PyTorch references, correctness checks, and benchmarks.
- Covers tiled GEMM, reductions, normalization, FlashAttention forward/backward, quantization, RoPE, KV caches, decoding, SwiGLU, and mixture-of-experts operations.
- Scope: Educational kernels and fixed-configuration experiments; tutorial-specific results do not imply a general production kernel library.
Coding Agent RL Lab — Environment & training infrastructure
Repository · 1 star · 0 forks in the snapshot
- Stateful code-repair environment with immutable tasks, independent correctness grading, and Docker resource/network sandboxing.
- Includes CPU model rollouts, held-out evaluation, checkpoint auditing, and optional native TRL GRPO training integration.
- Provides tools for listing files, reading, patching, and running tests, with an MCP bridge for agent experimentation.
- Status: Experimental research infrastructure; successful environment/training integration does not establish improved policy learning or held-out model performance.
Voice AI & agent systems
MyClone — Real-time voice AI & digital personas
- Contributed backend engineering during a six-month consulting engagement, as detailed in the professional experience above.
- Combined real-time voice conversations with persona-aware retrieval, knowledge, persistent memory, and agent orchestration.
- Used LiveKit, LlamaIndex, FastAPI, PostgreSQL, and pgvector in voice and retrieval workflows.
- An organization project with multiple contributors; the role description reflects my supplied professional experience.
Nexus Code Agent — CLI-first coding assistant
- Experimental Python coding-agent runtime with MCP integration, sub-agents, skills/plugins, and tool orchestration.
- Explores persistent sessions, context compaction, repository memory, sandboxed commands, and resumable approvals.
- Status: Work in progress / active scaffold.
Voice AI Agent — Telephony, browser voice & chat
- Real-time assistant demo supporting inbound/outbound Twilio calls, browser WebRTC, and streamed text chat.
- Uses FastAPI and Pipecat-based STT–LLM–TTS pipelines with shared prompts and conversational logic.
- Basic showcase; extended RAG, long-term memory, and CRM workflows are potential extensions rather than claimed implemented features.
Additional speech & generative audio work
| Project | Research / engineering focus |
|---|---|
| FastSpeech2 | Experimental, work-in-progress FastSpeech 2 implementation based on ESPnet, with pitch/energy/duration modeling, a MelGAN vocoder, LJSpeech alignment and TorchScript export. 237 stars / 53 forks in the snapshot. |
| SoundStorm-pytorch | Work-in-progress semantic-to-acoustic codec-token generation with bidirectional attention and confidence-based parallel decoding. 131 stars / 13 forks in the snapshot. |
| Voxtral Codec | Experimental causal convolution-transformer audio codec with semantic/acoustic quantization; approximate distillation alignment, without a final-scale reproduction claim. |
| NU-Wave-pytorch | Diffusion-probabilistic neural audio upsampling with training and inference workflows. |
External open-source contribution — AcademiCodec / HiFi-Codec
Merged contribution: PR #8 · Upstream repository
- Contributed merged bug fixes to the AcademiCodec HiFi-Codec implementation.
- Addressed a missing MelSpectrogram import, input tensor shape handling during encoding, and required module files with upstream attribution.
- Listed as a contribution to an external research project, separate from repository or paper authorship.
Additional vision & architecture implementations
| Project | Research focus |
|---|---|
| CvT | Convolutional vision-transformer architecture implementation. |
| LocalViT | Locality-aware vision-transformer implementation. |
| CoaT | CoaT-Lite and parallel CoaT convolutional-attention image-transformer implementation. |
| SiT | The 2021 Self-supervised vIsion Transformer: rotation prediction, contrastive learning and image reconstruction. |
Technical skills & research interests
| Area | Technologies and methods |
|---|---|
| Model development | Python, PyTorch, Hugging Face, Transformers, hybrid/linear attention, state-space models, Mamba |
| Training & post-training | SFT, RLHF, RLVR, GRPO, distillation, alignment |
| Speech & generative audio | TTS, STT, voice cloning, voice conversion, vocoders, neural codecs, diffusion, flow matching |
| Efficient inference | Triton, CUDA, Flash Attention, vLLM, KV caching, quantization, GPU memory and latency optimization |
| Agents & retrieval | MCP, multi-agent orchestration, tool calling, skills, RAG, LlamaIndex, LangChain, vector search, persistent memory, sandboxing |
| Real-time voice | LiveKit, Pipecat, Twilio, streaming STT/TTS, WebRTC |
| Backend & data | FastAPI, PostgreSQL, pgvector, Redis, SQL |
| Infrastructure | Docker, AWS, GCP |
| Additional programming experience | C++, Rust, Java |
These areas combine professional experience, open-source implementations, and active research interests; individual projects above show where specific methods were applied.
Education & languages
Bachelor of Technology — Electronics and Instrumentation Engineering
National Institute of Technology Silchar · 2011–2015
Languages: Hindi — native · English — proficient
Collaboration interests
Open to research collaborations and engineering problems involving voice AI, efficient language models, GPU inference, generative audio, and intelligent agents.
Sources & scope
This profile combines the supplied résumé, brief biography, project repositories, and the user's GitHub review dated October 18, 2026. Exact calendar dates for the MyClone engagement were not provided. Model performance observations are repository-reported. Paper authorship, publication records, independent benchmark validation, and production adoption are not inferred from implementations or engagement counts.
See portfolio source notes for the collection record and claim boundaries.