Ankit Panwar

Data Engineer

Bengaluru, Karnataka, India5 yrs 10 mos experience

Key Highlights

Led migration of 10 PB data to S3 Lakehouse.
Optimized tax pipeline runtime from 7.5 hours to 1 hour.
Developed scalable ETL pipelines for major marketing campaigns.

Stackforce AI infers this person is a Data Engineering expert in Fintech and MarTech industries.

Contact

anshupanwar1801@gmail.com LinkedIn

Skills

Core Skills

AwsEtlPythonData ModelingSqlData PipelineSparkMicroservices

Other Skills

AWS Cloud MigrationAirflowAmazon RedshiftAnalytical SkillsAndroid DevelopmentApache IgniteApache SparkBig DataCC++Data AggregationData ProcessingData StructuresData WarehousingDatabases

About

5+ years of experience working as a Data Engineer with a major focus on data extraction, loading, transformation (ELT), Data curation using SQL, and creating optimized & scalable data pipelines. Domains:- Finance, Marketing, and Social media. Skilled in Python, Java, PySpark, ETL, AWS(Glue, S3, Redshift, IAM, EC2, Lambda, Cloud formation), Data Pipeline, Data warehousing. Apart from this, excellent communication skills and tend to pay attention to details that give me an edge while connecting with international customers. Also, is very good at time management & has a high degree of ownership. Thanks for reading, Stay Hungry! Stay Foolish!

Experience

Blackline

Senior Data Engineer

Oct 2024 – Present · 1 yr 5 mos · Bengaluru, Karnataka, India

Intuit

Data Engineer 2

Dec 2022 – Sep 2024 · 1 yr 9 mos · Bengaluru, Karnataka, India

Led the migration of 10 PB of data, including stored procedures, triggers, and ETLs from Oracle to an S3 Lakehouse architecture, which improved availability and significantly reduced operational costs by 40%.
Developed a customized batch pipeline to reconcile cash flow between BRM systems and the bank, saving 2.5 hours per day of manual effort.
Registrated 50+ tables as clean entities, enhancing data quality and boosting external team confidence in data integrity by 30%.
Optimized AWS costs by 10% by identifying unused tables, optimizing partitioning strategies, and implementing S3 bucket expiration policies.
Managed batch reconciliation processes accommodating over 70 pipelines, ingesting data into more than 30 tables, and processing 2 billion events to reconcile 30TB of data daily.
Optimized the tax pipeline by converting it from full load to incremental load and implementing Spark optimizations, reducing the runtime from an average of 7 hours and 50 minutes to just 1 hour.

AWSETLSQLData WarehousingData PipelineData Modeling+3

Sigmoid

Data Engineer

Sep 2021 – Nov 2022 · 1 yr 2 mos · Bangalore Urban, Karnataka, India

Deployed a comprehensive solution for a major US snack company, aggregating a 360‑degree view of marketing campaign impact throughout its lifecycle.
Built scalable ETL pipelines and architected serverless infrastructure to run Python and Spark jobs.
Implemented solutions for data scientists to deploy ML models in production.
Streamlined data aggregation from over 11 independent sources and automated the ETL process to efficiently roll out the solution.

PythonSparkETLData Aggregation

Contaque

2 roles

Software Developer

Oct 2020 – Aug 2021 · 10 mos · Noida, Uttar Pradesh, India

Worked with the Social media data team to develop and deliver effective BI solutions to get insight on feeds from different social media platforms.
Developed microservices for handling interactions across various social media platforms, including Facebook, Twitter, and WhatsApp.
Engineered scalable ETL pipelines for social media platforms using PySpark, ensuring efficient data processing and integration.

ETLMicroservicesData Processing