Balvant Yadav

Software Engineer

Palo Alto, California, United States11 yrs 6 mos experience

Highly Stable

Key Highlights

Over four years of experience in Big Data technologies.
Expertise in building data platform services and writing data pipelines.
Proven ability to debug complex issues in data systems.

Stackforce AI infers this person is a Big Data Engineer with expertise in stream processing and data pipeline development.

Contact

Skills

Core Skills

Big Data TechnologiesCloud ServicesStream ProcessingData Pipeline Development

Other Skills

.NETAerospikeAlgorithmsAmazon RedshiftApache KafkaApache SparkApache StormAzure DatabricksAzure SQL Data WarehouseAzure Stream AnalyticsBig Data AnalyticsCC#CGroupsCore Java

About

More than four years of experience in Big Data technologies like Hadoop, Storm, Kafka, Aerospike, Spark. Handful of experience in building data platform services, writing data pipelines and debugging complex issues.

Experience

Amazon

SDE-II

Jul 2022 – Present · 3 yrs 8 mos · Bengaluru, Karnataka, India

Infoworks.io

2 roles

Principal Software Engineer

Promoted

Jan 2020 – Jul 2022 · 2 yrs 6 mos

Senior Software Engineer

Dec 2018 – Jan 2020 · 1 yr 1 mo

Inmobi

3 roles

Tech Lead

Promoted

Jan 2018 – Dec 2018 · 11 mos · Bengaluru, Karnataka, India

CGroups and Multi tenancy in Storm : Explored resource allocation and control in storm using CGroups and did POC of storm multi-tenancy feature using resource aware scheduler and cgroups.
User Store Service: Designed RTA service and implemented user store service in spark.
Microsoft azure cloud services: Evaluated Azure SQL Data Warehouse and stream processing feature.
Data catalog: Built data catalog service which enables users to discover, understand and create value from existing datasets.

HadoopApache StormSparkAzure SQL Data WarehouseData CatalogBig Data Technologies+1

Senior Software Engineer

Promoted

Jan 2016 – Dec 2017 · 1 yr 11 mos · Bengaluru, Karnataka, India

CPI Billing and Strands reporting: Designed and implemented reporting pipeline for CPI billing and strands. This helped business team to isolate demand and supply side reporting and pay advertisers and publishers respectively on different events.
Egret: Egret is stream processing framework which provides lambda transformation and stream join solution. I was involved in designing and implementing this product from scratch. Currently it supports Kafka and Storm as messaging and stream processing framework respectively but it can be easily extended to support any messaging and stream processing framework.
Migration from batch pipeline to real time reporting: Evaluated resource requirement, designed, planned and implemented changes to move Map-Reduced based LocalDC batch processing to real time. We used egret lambda transformation with kafka and storm as messaging and processing framework respectively. This significantly reduced hardware requirements and reporting SLAs.
Storm security: POC of storm security using Kerberos. Planned and productionized this feature. This allows only authenticated and authorized users to access storm services.

KafkaStormMap ReduceSQL DWHStream ProcessingData Pipeline Development

Software Engineer

Jul 2014 – Dec 2015 · 1 yr 5 mos · Bengaluru, Karnataka, India

DCP Pipeline: Rewritten and automated E2E DCP data reporting pipeline using Map Reduce, SQL DWH, Falcon and Oozie.
LocalDC pipeline: Worked on data transformation, data enrichment and summarization of raw ad request response, render, click, download and billing events. We were processing 10 billions request per day.
HBase to Aerospike migration: Evaluated Aerospike performance and resource requirement. Planned, designed and implemented changes to move data enrichment in LocalDC from HBase to Aerospike. This signicantly reduced the operational and maintenance cost.

Map ReduceSQL DWHAerospikeData Pipeline Development