Vipul Bhardwaj

Data Engineer

Gurugram, Haryana, India10 yrs 1 mo experience

Most Likely To SwitchHighly Stable

Key Highlights

Expert in building data pipelines and frameworks.
Proven track record in GDPR compliance projects.
Strong background in anomaly detection and monitoring systems.

Stackforce AI infers this person is a Big Data Engineer with expertise in SaaS and Telecommunications.

Contact

Skills

Core Skills

Data EngineeringApache SparkBig Data

Other Skills

Anomaly DetectionBatch Data PipelinesCC++CI/CD PipelineChurn AnalysisData AnalysisData AnalyticsData Ingestion PipelinesData LakeData PipelinesData PublishingData Publishing FrameworkDruidExtract, Transform, Load (ETL)

About

Experienced Senior Big Data Developer with a demonstrated history of working in the software industry. Skilled in Apache Spark, Scala, Python, Data Engineering, and No SQL Databases. Strong engineering professional who graduated from Thapar University.

Experience

Mongodb

2 roles

Senior Data Engineer

Aug 2022 – Present · 3 yrs 7 mos · Gurugram, Haryana, India

Data Engineer III

Mar 2021 – Jul 2022 · 1 yr 4 mos · Gurugram, Haryana, India

PIIAnonymizer: Led the effort to make the data lake GDPR compliant. Designed and developed PII Anonymizer framework and made 700+ TB data lake GDPR compliant with minimum cost.
DataPublisher Framework: Created a config-driven data publishing framework that could cater to all types of data publishing needs of the team significantly decreasing the development time.
Built and revamped multiple data ingestion pipelines to help the company make data-driven decisions.
Decreased the build time of CI/CD pipeline by 30% by parallelizing the test cases.

Extract, Transform, Load (ETL)MongoDBData EngineeringData Publishing FrameworkData Ingestion PipelinesCI/CD Pipeline+1

Thales

Sr Big Data Developer

Jul 2019 – Mar 2021 · 1 yr 8 mos · Gurugram, Haryana, India

Anomaly Detection IoT Devices: Created an anomaly detection streaming framework using the Gaussian anomaly detection algorithm.
Tethering: Near real-time processing of data to create and update rules to find tethered devices based on OS signature which is used by the CISCO database to detect tethered devices. This helped bill the users accordingly.
Monitoring System: Set up a centralized monitoring system using Prometheus with Alerting mechanism. This proved to be a valuable tool providing insights into the running jobs. It helped identify long-running jobs which in turn reduced run time by more than 40%.

Anomaly DetectionStreaming FrameworkMonitoring SystemPrometheusBig DataData Engineering

Airtel

Sr Big Data Developer

May 2018 – Jul 2019 · 1 yr 2 mos · Gurgaon, Haryana, India

Live Work Play: Identify the Home and Work sites of users which is a key database for a lot of projects such as identifying target customers for any new service, sending updates about improved network coverage, and identifying problems with the network for any user, etc.
MyNetex: Built data pipelines to generate data for user-level KPIs such as usage pattern, cell usage, Time on Technology, Repeat Call Records, etc.
Churn Analysis: Segmented users in mobility classes(high, medium, and low) and identified, analyzed, and visualized patterns of churn users.
SetupthetechstackbeingoneofthefoundingmemberoftheData Engineering team. The effort resulted in a centralized data lake and enabled users to be self-sufficient in accessing the data via Superset powered by Druid on the backend which increased productivity by more than 75%.

Data PipelinesChurn AnalysisData LakeKPI GenerationData EngineeringBig Data

Blackrock

Analyst

Jan 2016 – May 2018 · 2 yrs 4 mos · Gurgaon, Haryana, India

AladdinBusinessPortal: Implemented batch data pipelines to generate data for multiple KPIs such as to analyze trades trends and monitor user ticket status, delays, hourly close rate, etc.

Batch Data PipelinesKPI AnalysisData Engineering