Abhinav Dubey

DevOps Engineer

Gurgaon, Haryana, India12 yrs 6 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Expert in leading data engineering teams
Proficient in cloud technologies like AWS and GCP
Strong background in performance optimization and automation

Stackforce AI infers this person is a Data Engineering expert with a focus on SaaS and cloud solutions.

Contact

abhinavdubey129@gmail.com 8527006414 LinkedIn

Skills

Core Skills

Data EngineeringCloud ComputingMachine LearningAutomationData AnalysisSoftware Development

Other Skills

API DevelopmentAWSAgileAirflowAlgorithmsBashBig DataDatabricksDebuggingETLGCPGrafanaHadoopLinuxMicroservices

About

Lead data engineer with almost 10 years of experience, Seeking a challenging and rewarding opportunity across the industry that is in consonance with the professional experience to date and which effectively utilizes analytical and technical skills, currently working with Publicis Sapient as Lead Data Engineer. Skills: Python, Spark, PySpark, Hive, Hadoop, AWS, GCP, Databricks, Airflow, MySQL, MongoDB, Streamsets, Streamlit, Performance optimization, Automation Framework development, CI/CD, Git, Grafana, Agile

Experience

Publicis sapient

Lead Data Engineer

Apr 2021 – Present · 4 yrs 11 mos · Gurugram, Haryana, India · Hybrid

Leading and mentoring a high-performing Data Engineering team, fostering a collaborative environment and providing strategic guidance throughout the software development lifecycle.
Orchestrated close collaboration with multiple stakeholders to efficiently manage design and data requirements, ensuring streamlined and effective development processes across the organization.
Oversaw the design and development of comprehensive automation frameworks utilizing Python and PySpark, ensuring their robustness, reliability, and scalability for Big Data components.
Leading end-to-end performance analysis and performance optimizations of the legacy pipelines using Airflow and Pyspark, profiling performance bottlenecks, resource constraints, and driving targeted performance improvement initiatives across cross-functional teams.
Conducting thorough root cause analysis, and implementing preventive measures in collaboration with the architects to ensure continuous improvement.
Review application architecture and facilitated the implementation of spark memory optimization strategies, resulting in measurable enhancements in overall pipeline performance and efficiency.
Leading comprehensive code and design reviews for team members, ensuring alignment with best practices, standards, and architectural guidelines, thereby fostering a culture of continuous improvement and knowledge sharing within the team.

PythonPySparkAWSAirflowStreamSetsStreamlit+3

Dunnhumby

Lead Developer

Apr 2017 – Apr 2021 · 4 yrs · Gurugram, Haryana, India · On-site

Building ETL Ingress and egress pipelines
Implementing various science algorithms to generate product recommendations.
Implementing various science algorithms for customer acquisition, customer retention and evaluating customer DNA.
Features and Scoring pipelines.
Building Internal product for running complete ML based workflow.
Building REST-API's to create big data SAAS solutions.

ETLMachine LearningREST APIsBig DataData Engineering

Snapdeal

Software Development Engineer

Dec 2016 – Apr 2017 · 4 mos · Gurugram, Haryana, India

Domain : Automation Framework, Data Platform and CXE.
Projects : Monk, OCI and Reporting.
Technology : Python, Perl, Mysql, MongoDB, Bigdata, Bash and Linux.
Development of Performance Automation Framework.
Performance Test planning and execution of plan for API/Web Services.
Framework development for Monitoring and Deployment of various components.
Functional testing of recommendation and notification component for web and app.
Infra setup for various components including Big Data applications for Performance Testing.
Integration of Jmeter with Jenkins and Grafana for Performance Testing.
Writing python scripts to exhaust component resources respectively for Performance Testing.

PythonPerlMySQLMongoDBBig DataBash+3

Transporter.city(lugg cargo pvt. ltd.)

Python Developer

May 2016 – Dec 2016 · 7 mos · Gurugram, Haryana, India

1) Data Analysis
· Analyse large data sets to provide strategic direction to the company.
· Perform quantitative analysis of product usage trends to recommend pricing decisions. Created a customer based pricing model which generates a detailed bill as per their usage.
· Maintain uniformity of Incoming data from various sources for error free development environment.
· Scrutinise and track customer behaviour
· Assist in developing tools for internal data analysis.
2) Data Filtration :
· Analysing data and separating Noise from the useful data thereby making algorithms and codes based on previous data patterns much more efficient.
3) Algorithms :
· ETA model - This algorithm detects expected time of arrival of a fleet to the various points fed to the system.
· PI model - This algorithm helps various fleet owners to place their vehicles by letting them know a list of ranked vehicles at a given time.
· Automatic Order Creations : This algorithm creates automatic orders for the customers based on the fleet movement in a Real Time scenario.
4) API’s and Modules:
· Notifications Module : Provides customer specific alerts as per the phase of order life cycle. PubNub engine is used for publishing and subscribing.
· Reporting Module : Generates various types of reports with numerous categorisations.
· Bill Creation : Generates bill for the customers based on their usage depending upon the API usage and platform consumption.
· Touchdown API : Helps to track a particular vehicle movement at any given point.

Data AnalysisAPI DevelopmentAlgorithmsSoftware Development

Czentrix

Software Developer

Nov 2014 – May 2016 · 1 yr 6 mos · Gurugram, Haryana, India

Creating Project documents.
Collecting Requirement from client.
Debugging the existing code.
Development of the new code as required.
Unit Testing.
Test case/data preparation.
Execution and verification of the test results.
Customize the Project according to the need of Customers.
Guiding and Helping tech support team engineers for client specific issues and quickest ways to tackle them.
Modification to previously developed code so that It could be faster,better, more feasible and reliable.