Tarun Kumar

Engineering Manager

Gurugram, Haryana, India11 yrs 6 mos experience
Most Likely To SwitchHighly Stable

Key Highlights

  • Expert in Big Data technologies and analytics.
  • Designed scalable systems for massive data ingestion.
  • Proven track record in optimizing data query performance.
Stackforce AI infers this person is a Big Data Engineer with strong expertise in data ingestion and query optimization.

Contact

Skills

Core Skills

Big DataData Engineering

Other Skills

AlluxioAngularJSCC++CSSCascading Style Sheets (CSS)Custom S3 committerHTMLJavaJava Enterprise EditionJavaScriptMatlabNodejsOracle PL/SQL DevelopmentPresto Query Parser

About

Software Engineer with interest in Big Data technologies, Coding, Analytics. https://medium.com/@tarunmahajan1992

Experience

11 yrs 6 mos
Total Experience
2 yrs 10 mos
Average Tenure
7 yrs 4 mos
Current Experience

Airtel digital

3 roles

Engineering Manager - Big Data

Promoted

Dec 2022Present · 3 yrs 4 mos

Senior Lead Engineer Big Data

Dec 2020Nov 2022 · 1 yr 11 mos

Lead Engineer Big Data

Nov 2018Dec 2020 · 2 yrs 1 mo

  • 1. Designed and coded Query Engine to Perform Join and Query Data over 50 Peta byte Data Stored in Object Storage. We used best of all worlds. Used Presto Query Parser for Parsing and optimizing sql queries, spark for executing tasks and designed own query task scheduler and custom join logic. We are able to do joins on Peta byte scale data with very little memory footprint and without shuffle.
  • 2. Designed and coded federation layer, which serves OLAP queries with millisecond latency. Integrated latest ORC with Cloudera , enabled ZSTD compression and cached ORC files in Alluxio . Query done using presto and we are able to scan billion records per second over 4 node cluster.
  • 3. Currently writing data store for streaming key value based system, which will provide OLAP queries over streaming data with miilisecond latency. It is well known problem in industry and no single existing system works out of box to solve this.
  • 4. wrote system to ingest trillion records per day. System writes data over to object storage . Implemented custom S3 committer to support our purpose . It is horizontally scalable system and all data is sorted with custom logic before writing. System daily receive 25 Tera byte of data.
  • 5. Implementing Unified customer Master for all categories of Airtel.
Presto Query ParserSparkAlluxioZSTD compressionCustom S3 committerBig Data+1

Mobileum

2 roles

Senior Software Engineer - Big Data and Analytics

Promoted

Apr 2018Oct 2018 · 6 mos · Gurugram, Haryana, India

Software Engineer - Big Data and Analytics

Nov 2016Mar 2018 · 1 yr 4 mos · Gurugram, Haryana, India

Citi services india private limited

Assistant Manager

Nov 2015Nov 2016 · 1 yr · Chennai

Virtusa

Software Engineer

Jun 2014Oct 2015 · 1 yr 4 mos · Hyderabad, Telangana, India

Education

NIT Jalandhar

Bachelor's degree — Electronic and Communication engineering

Jan 2010Jan 2014

Stackforce found 100+ more professionals with Big Data & Data Engineering

Explore similar profiles based on matching skills and experience

Tarun Kumar - Engineering Manager | Stackforce