VIJAY GUPTA

Data Engineer

Delhi, India14 yrs 10 mos experience

Highly Stable

Key Highlights

Over 15 years of experience in diverse domains.
Expert in data engineering and ETL processes.
Strong background in data quality and automation.

Stackforce AI infers this person is a Data Engineer with expertise in ETL and data analytics across BFSI and E-Commerce sectors.

Contact

contactvijay777@gmail.com LinkedIn

Skills

Core Skills

Data EngineeringEtlNlpWeb ScrapingAutomationData MigrationTestingData AnalysisInfrastructure Management

Other Skills

API AutomationAgile MethodologiesAzure Data FactoryAzure DatabricksAzure DevOpsCommunicationData AnalyticsData Design ArchitectureData ModelingData QualityExtract, Transform, Load (ETL)GrafanaJavaJenkinsLinux

About

• 15+ Years of experience in BFSI(11+ years) , E-Commerce(3+) and Telecom(1+) domain.• 6+ Years in Data Handling, including ETL activities, cleansing, and transformation, loading , data analysis & validations. • Scrapping of website data, getting the data using various API responses in JSON/XML. Working with gigs of Data to be cleansed/transformed with knowledge in Python, PySpark, DataBricks, ADF etc.• Knowledge of Data Design Architecture and Data Modeling.• Have a good knowledge of production support and implementation of the latest code to various environments from Dev/QA to Staging to Production.• Have good Knowledge of database usage with extensive knowledge of complex SQL queries.• Tools Used:: DataBricks, ADF, Azure Pipeline, Eclipse, Jupyter Notebook and PyCharm, SOAPUI & Postman for API, Git, BitBucket, GitHub, Maven, Gradle.• Last but not least, never dying spirit of learning new technologies that come my way.

Experience

Animal friends

Data Engineer - Azure Databricks

Aug 2022 – Sep 2025 · 3 yrs 1 mo · United Kingdom · Remote

As part of the data engineering team, I focus on building and maintaining robust data solutions within the Azure ecosystem to support business intelligence and analytics.
Key Responsibilities:
Develop and validate scalable data pipelines using Azure Data Factory and Databricks.
Investigate and resolve daily ADF pipeline failures, addressing small change requests efficiently.
Write and enhance SQL code to accommodate evolving data requirements.
Monitor and maintain production pipelines to ensure consistent data availability and performance.
Uphold data quality through various validations and compliance with organizational data standards.
Collaborate with cross-functional teams to meet data requirements.

Azure Data FactoryAzure DatabricksTransact-SQL (T-SQL)PySparkPythonExtract, Transform, Load (ETL)+3

Bhavna corp

Technical Lead

Feb 2020 – Jun 2022 · 2 yrs 4 mos · India · On-site

Led the extraction and transformation of terabytes of data from third-party organizations using Azure Data Factory.
Utilized PySpark in Databricks for efficient data processing and transformation.
Successfully loaded data into Azure Synapse Analytics, enhancing data warehousing capabilities.
Collaborated with cross-functional teams to ensure seamless integration and data flow.

Azure Data FactoryAzure DevOpsNumPyExtract, Transform, Load (ETL)LinuxAzure Databricks+6

Global payments inc.

Senior Engineer

Apr 2017 – Jan 2020 · 2 yrs 9 mos

Using PySpark & ADF worked on a project for Data Cleansing of Financial data generated for billions of transaction every day for the credit cards.
Did POC of Natural Language Processing on comments of the Merchant & Customer against the Transaction for sentimental analysis.

NumPyExtract, Transform, Load (ETL)Data AnalyticsData EngineeringNatural Language Processing (NLP)API Automation+8

Makemytrip.com

Assistant Manager

Mar 2014 – Jan 2017 · 2 yrs 10 mos · Gurgaon, India · On-site

Worked on Web Application Scrapping & Automation using Python and Selenium.
Worked on automating various tasks on Linux server for Alerts & Monitoring.

GrafanaShell ScriptingSQLMySQLLinuxPython+6

Teoco

Analyst

Nov 2012 – Feb 2014 · 1 yr 3 mos · Gurgaon

Worked on
Data Migration from Oracle to Terradata for AT&T Client.
Parser testing , a tool used on various flavours of Linux and Windows machines to parse the streaming data.

LoadRunnerShell ScriptingSoapUISQLLinuxPython+4

Ntt data

Sr. Analyst

Apr 2012 – Oct 2012 · 6 mos · Bangalore · On-site

Client : Fidelity - UK
Worked on end to end testing of the data loading process along with front end validation of data.
Performance testing of web application using LoadRunner.

LoadRunnerData AnalysisMySQLCommunicationTesting

Oracle financial services software limited

Analyst

Mar 2010 – Apr 2012 · 2 yrs 1 mo · Bengaluru Area, India · On-site

Handling infrastructure setup on Linux machines, for the QA Team.
Automating Linux server manual tasks, using shell scripting.
Managed multiple App and DB servers. Did deployments ,creation of queues and tuning of Oracle Web logic, IBM WebSphere and Tomcat for proper functioning of Web Application.

TomcatWeblogic AdministrationLinuxWebSphere Application ServerCommunicationInfrastructure Management+1