Shivank G.

Software Engineer

Bengaluru, Karnataka, India9 yrs 4 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Expert in developing high-performance software systems.
Proven track record in delivering complex low-latency systems.
Strong leadership in data engineering and team management.

Stackforce AI infers this person is a Data Engineering expert in AdTech and Fintech industries.

Contact

Skills

Core Skills

Data EngineeringBig DataAdtech

Other Skills

icebergEngineering ManagementSpring BootHiveAirflowDatadogHBaseCascadingJavaMavenBusiness Intelligence (BI)Data QualityData ModelsData ModelingGrafana

About

Results-Driven Professional with 6+ Years of Expertise in Developing High-Performance Software Systems across AdTech, Customer Profiling, Lending, and Postpaid Domains. Proven Track Record as a Technical Lead, Delivering Complex Low-Latency Distributed Systems, Transactional Solutions, and Ad- vanced Reporting Platforms. Thrives on taking ownership and crafting simple solutions for challenging business problems.

Experience

9 yrs 4 mos

Total Experience

2 yrs

Average Tenure

2 yrs 2 mos

Current Experience

Index exchange

Senior Software Data Engineer

Apr 2024 – Present · 2 yrs 2 mos · Bengaluru, Karnataka, India · Remote

Own and help operate a mission-critical shared data platform processing 500+ billion events daily, handling
400TB+ of data with peak throughput of up to 7 million events per second, while driving reliability, scalability,
and operational excellence.
Contribute to the architecture and evolution of a 1630+ node Hadoop/YARN platform running Trino, Spark,
Flink, and adjacent data services, enabling large-scale batch, streaming, and interactive analytics for multiple
teams and workloads.
Driving the design of a reusable Spark/Flink streaming platform to provide multiple teams with a common
foundation for building robust, high-throughput real-time data pipelines.
Prototyped and championed a platform approach for product reporting features that demonstrated potential
to reduce development cycle cost by 90%, and influenced product leadership to adopt it as a multi-quarter
roadmap initiative.
Led data lake modernization eﬀorts spanning metadata, table format strategy, and storage-performance trade-
oﬀs, including large-scale evaluation and migration planning around Iceberg-backed workloads.
Implemented Spark and storage optimizations to reduce small-file overhead and improve throughput, cost
eﬃciency, and operational stability across large HDFS-backed datasets.
Led Mimir migration work for a platform supporting 1+ billion active time series, strengthening scalability,
retention handling, and reliability of the observability stack.

icebergData EngineeringBig Data

Walmart

2 roles

Senior Software Engineer

Jul 2022 – Jan 2025 · 2 yrs 6 mos

Lead and manage a high-performing team, driving data engineering efforts for Sams sponsored Products.
Collaborate with business stakeholders and cross- functional teams to gather requirements and architect end-to-end solutions for the Search advertising platform UI, Ad Server, Campaign Management, and Data engineering components.
Implemented streaming jobs processing 300TB events per day using Apache Flink and Kafka
Process 100TB of data daily using the Krypton data engineering platform, ensuring seamless and scalable data processing.
Implement Kappa architecture pipelines to process Walmart and Sams data for mobile and desktop Ad attribution, maximizing return on ad spend (ROAS)/Sales for partners.
Successfully lead the implementation of Inclub Attribution, an industry-leading feature providing offline attribution for Sam’s Sponsored Products, resulting in significant ROAS/Sales increases for partners.

Engineering ManagementSpring BootData EngineeringAdTech

Data Engineer III

Oct 2020 – Jul 2022 · 1 yr 9 mos

Conducted POC comparing Kafka Streams and Spark Streaming, selecting Spark Streaming for real-time data processing.
Developed robust Spark Structured Streaming job(P13N) as a Kafka consumer for personalized recommendations and targeting.
Implemented custom offset management solution, ensuring fault tolerance and data consistency.
Integrated CCM into Spark jobs for efficient configuration management and dependency handling.

HiveAirflowData EngineeringBig Data

Paytm

Software Engineer

May 2019 – Oct 2020 · 1 yr 5 mos · Noida Area, India · On-site

Managedend-to-enddatapipelinesforthelendingand postpaid domains, processing large volumes of data and generating features for personalized user experiences.
Collaborated with ML engineers to scale and deploy models, improving customer engagement and conver- sion rates.
Orchestrated a flexible and config-driven workflow, en- abling rapid development and deployment of new fea- tures.

DatadogHBaseData Engineering

Caastle

Software Engineer

Jul 2017 – May 2019 · 1 yr 10 mos · Bengaluru Area, India

Owned, developed and maintained hadoop based ETL modules. Designed schedule based workflows to maximize parallelization in data loading.
Made major contributions in designing and migrating ETL to distributed computing based Java framework Cascading improving performance manyfold.
Worked at database level to incorporate new tenants data into a multi-tenant based DWH architecture. Developed re-usable and abstracted Java based ETL codebase integrating multiple systems, like mysql, impala, hive, s3, hbase, oozie, etc..
Worked in-sync with product managers to achieve aim with minimalistic technical changes to underlying systems. Designed and developed distributed architecture based solution for company’s internal development teams.

HBaseCascadingData Engineering

Sabre airline solutions

Intern

Jul 2016 – Dec 2016 · 5 mos

Automation of Deployment process using Continuous Integration Tool (Teamcity). It included automation of backend and frontend installations by writing XML
scripts in Maven pom and also automation of running of Tomcat and setting up SSH server for frontend installation in windows machine.

JavaMaven