Pulkit Gupta

Data Engineer

Bengaluru, Karnataka, India5 yrs 4 mos experience
Most Likely To SwitchHighly Stable

Key Highlights

  • Expert in architecting end-to-end data pipelines.
  • Proficient in real-time data processing and ETL automation.
  • Strong experience with AWS and big data technologies.
Stackforce AI infers this person is a Data Engineering expert with a focus on real-time data processing in SaaS environments.

Contact

Skills

Core Skills

Data EngineeringData ModelingBig Data AnalyticsEtl Automation

Other Skills

AWS DatabricksAWS GlueAmazon AthenaAmazon Elastic MapReduce (EMR)Amazon Web Services (AWS)Apache AirflowApache SparkBlockchainData ArchitectsData ScienceData StructuresDatabricksExtract, Transform, Load (ETL)HadoopHyperledger

About

I am your go-to data engineer, assisting you in the design and development of end-to-end data pipelines. Right now, I am responsible of architecting and modelling the DWH in Atlassian. Proficient in Data Engineering, Data Modelling, PySpark, Python, Hive, AWS EMR, MySQL, Databricks and Data Structures.

Experience

Atlassian

2 roles

Senior Data Engineer

Promoted

Oct 2024Present · 1 yr 5 mos · Bengaluru, Karnataka, India

  • 1. Responsible for modelling and architecting datasets.
  • 2. Developed the real-time streaming landscape for the team, using layered microservices architecture.
DatabricksPython (Programming Language)SQLData ModelingData EngineeringAmazon Web Services (AWS)+2

Data Engineer II

Dec 2022Oct 2024 · 1 yr 10 mos · Bengaluru, Karnataka, India

AWS DatabricksSQLMicroservicesExtract, Transform, Load (ETL)Data ModelingPySpark+2

Paytm

Data Engineer

May 2022Dec 2022 · 7 mos · Noida, Uttar Pradesh, India

Apache SparkHadoopSQLExtract, Transform, Load (ETL)Apache AirflowAmazon Web Services (AWS)+3

Teksystems

2 roles

Associate Software Engineer (Data Engineer)

Oct 2020Apr 2022 · 1 yr 6 mos · Bengaluru, Karnataka, India

  • Worked on the main product, which involved development of robust scripts in python on AWS Glue Console and writing AWS Athena queries to generate unique serial numbers for parts of mobile devices in real-time.
  • Responsible for development and management of Data Pipeline. ETL automation is done for files present in s3 locations using Python, PySpark as language and AWS glue as a platform. Data is then finally loaded to AWS Athena tables.
  • Developed ETL scripts and automated bigdata workloads in Amazon EMR (HADOOP Based) environment.
  • Responsible for post load processing of the data in final aggregate tables for further client usage.
  • Optimized the PySpark jobs by fine tuning the spark and sql configurations.
Apache SparkHadoopAmazon Elastic MapReduce (EMR)SQLAWS GlueAmazon Athena+3

Data Engineering Intern

Jan 2020Mar 2020 · 2 mos · Bangalore

  • Did my training on AWS platform.
  • Wrote a ETL script in python responsible for simultaneously updating MongoDB and SQL database from a single sql query.
  • Worked in DEV and QA environment for development of ETL pipelines.
SQLPython (Programming Language)

Auxesis group

Blockchain Intern

Jun 2019Jul 2019 · 1 mo · Mumbai Metropolitan Region

  • Worked on Docsafe. A Blockchain based system which allows user to store their important documents on Auxledger which is Auxesis group’s developed blockchain network.
  • Developed a prototype of Ride Sharing System on Hyperledger Fabric to make the whole taxi booking process tamper proof and this system will benefit the enterprise in keeping hackers and third parties away.
  • Handled one of Auxesis's client and provided him with technical support.
  • Learned basic development on Ethereum and Bitcoin platforms.
Data StructuresSQLPython (Programming Language)

Natural group

Research Intern

Jun 2018Jul 2018 · 1 mo · Jaipur Area, India

  • Researched on numerous cyber security attacks possible on banks and suggesting ways to reducing malware attacks
  • on banking apps and computers.
  • Performed Malware analysis on the samples and drew conclusions concerning origin raison d'être, techniques used.
  • Researched on Twitter Sentiment Analysis.
  • Concluded that with the extensive amount of social media data available online in different forms such as videos and images, the conventional text-based sentiment analysis with ample research and analysis can be evolved into more complex models of multimodal sentiment analysis thus, improving the accuracy of traditional systems.
Data Structures

Education

Manipal University Jaipur

Bachelor of Technology - BTech — Computer Science

Jan 2016Jan 2020

Stackforce found 100+ more professionals with Data Engineering & Data Modeling

Explore similar profiles based on matching skills and experience