Viraj Thakur — Software Engineer

🌟 Data Platform Engineer | BharatGen | Building AI for Bharat As a co-lead Data Platform Engineer at BharatGen, a semi-private, government-funded initiative, I contribute to India’s mission of building foundational AI models — including Large Language Models (LLMs), Vision-Language Models (VLMs), and speech systems — tailored to Indian languages and domains. I architect and manage end-to-end data pipelines that power generative AI training at scale. My focus spans multilingual data acquisition, curation, synthetic augmentation, tokenization, and infrastructure observability. Our data work spans 16 Indian languages and diverse domains such as healthcare, agriculture, law, and governance. 🚀 Key Responsibilities Data Curation & Augmentation: Lead scraping, filtering, and OCR-based digitization of Indian web content and documents. Designed persona-driven synthetic data pipelines for scalable instruction-tuning data generation — enabling trillion-scale datasets using LLMs and prompt engineering. Training Readiness: Built robust preprocessing pipelines for transforming raw data into training-ready, tokenized formats. Minimized manual overhead through automation and reproducibility-focused tooling. Compute & Infra Monitoring: Monitor large-scale GPU training clusters using Prometheus and Grafana. Developed automated storage tracking dashboards to optimize petabyte-scale distributed data systems. Benchmarking & Research: Contributed to SFT and checkpoint evaluations; collaborated on novel research for scaling synthetic data, with a publication underway. Mentorship & Enablement: Mentored interns on setting up OCR engines and metadata-based scraping. Contributed to internal best practices for scalable data workflows and tooling. ⚙️ Technical Stack Languages: Python, SQL, Bash Big Data: Spark, Hive, Hadoop, HDFS Pipelines: Airflow, Kafka Monitoring: Prometheus, Grafana Infra: DataHub, JuicyFS (in-house Data Lake) Cloud: AWS, GCP LLM Workflows: Tokenization, Synthetic Prompting, Evaluation 💡 What Drives Me I’m driven by the depth and diversity of work — from LLM data prep to high-performance compute systems — all aimed at shaping India's AI future. Working with deep learning researchers and large-scale systems has been both a dream and a growth journey. My next goal is to build a centralized observability platform offering real-time visibility into our data and compute stack, driving transparency, accountability, and performance optimization at scale.

Stackforce AI infers this person is a Data Engineering expert in AI and Finance sectors.

Location: Mumbai, Maharashtra, India

Experience: 3 yrs 9 mos