Rajat Agarwal

CTO

Noida, Uttar Pradesh, India18 yrs 6 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

17+ years in large scale distributed systems
Expert in building Big Data solutions from scratch
Proven track record in engineering leadership

Stackforce AI infers this person is a Big Data and Distributed Systems expert with strong leadership in SaaS and Telecom industries.

Contact

Skills

Core Skills

Distributed SystemsBig DataMicroservices

Other Skills

KafkaCloud native servicesData streamingData ScienceEngineering cultureHadoopSparkStormYARNSpark StreamingCJavaData StructuresUnixAlgorithms

About

Hands-on technology leader with experience building & growing engineering teams to deliver high business impact and drive innovation through data-driven decisions and agile execution ● 17+ years of hands-on experience in designing and developing Large Scale Distributed Systems. ● 12+ years in building Big Data solutions from scratch, handling 100Bn+ events, 500TB+ data / day. ● Co-authored products and partnered with stakeholders across functions to achieve amplified business growth and Impact. Specialties: Distributed computing, Micro services, Big Data Analytics, Hadoop/YARN, Spark, Spark Streaming, Storm, Kafka, Flume, Hive, PIG, Oozie, Falcon.

Experience

18 yrs 6 mos

Total Experience

3 yrs 1 mo

Average Tenure

4 yrs 1 mo

Current Experience

Confluent

Engineering leader

Apr 2022 – Present · 4 yrs 1 mo

Building cloud native services and Connectors to deliver highly reliable, highly scalable data streaming ecosystem with Kafka.
Looking for engineers & leaders who are passionate about building large scale, multi-tenant, realtime distributed data systems

KafkaCloud native servicesData streamingDistributed SystemsBig Data

Scaler

Career Coach

Apr 2021 – Present · 5 yrs 1 mo

Mentoring Professionals to uplevel their career growth

Microsoft

Principal Engineering Manager

Feb 2018 – Apr 2022 · 4 yrs 2 mos · Bengaluru, Karnataka, India · On-site

Built a compliant Big Data platform for Data Science experimentation and evolved it as a commercial offering handling data at petabyte scale and tens of billions of events per day. Built a strong team and engineering culture to take the product from 0-1 reaching $15M ARR in 3 years.

Big DataData ScienceEngineering cultureDistributed Systems

Inmobi

2 roles

Architect

Promoted

Jan 2016 – Jan 2018 · 2 yrs · Bangalore

Built scalable Big data solutions ground up using open source tech including Kafka, YARN, HIVE, Flume, Spark & Storm, handling 15TB+ data & 10Bn+ events / day with query latency of <5mins.
Partnership with Product and Business team to build customer centric features with the right engineering system for feature validations, monitoring and alerting. Introduced a strong culture of data driven approach and focus on writing generic and maintainable code. Drove engineering excellence for code reviews, deployment, feature validation and handling on-calls.

KafkaHadoopSparkStormBig DataDistributed Systems

Lead Engineer

Jan 2015 – Dec 2015 · 11 mos · Bangalore

Guavus

3 roles

Lead Technology

Promoted

Nov 2012 – Dec 2014 · 2 yrs 1 mo

Major contributions in adoption of big data tech stack in the company. Custom enhancements to open source Hadoop & YARN to build a centralised data platform in Guavus.
● Added Spark Streaming to Guavus core platform including Guavus specific custom enhancements and contribution to the Apache open source Spark community.
● Custom enhancements in YARN to add Guavus specific JVM reuse feature to YARN early release.
● Added support for skipping corrupt chunks of HDFS data, missing blocks, fixed Jobtracker memory leak, and many other enhancements in Guavus custom open source Hadoop distribution.

HadoopYARNSpark StreamingBig Data