Deepak Kumar

Backend Engineer

Pune, Maharashtra, India19 yrs 8 mos experience
Most Likely To SwitchHighly Stable

Key Highlights

  • Over 15 years of experience in software engineering and data architecture.
  • Expert in cloud platforms and big data solutions.
  • Proven track record in mentoring and training teams.
Stackforce AI infers this person is a seasoned Data Architect and Software Engineer specializing in Cloud and Big Data solutions.

Contact

Skills

Core Skills

Data EngineeringCloud Architecture

Other Skills

API DevelopmentAgile MethodologiesAmazon RedshiftAmazon Web Services (AWS)Apache KafkaApache PigApache SparkBig DataBusiness IntelligenceCore JavaDB2DBTData ModelingData WarehousingDatabase Design

About

Seasoned Lead software engineer and data architect with more than 15 years of experience. Committed to providing high-level solutions that drive business and provide outstanding cost efficiency on Big Data and Cloud platform. History of mentoring and training others in best practices. Skills :- Cloud ( AWS , Google Cloud) Programming ( Java , Ruby , Python , Shell Scripting , Rails, Scala , Spark, Apache Flink , SQL , PL/SQL) Hadoop Ecosystem ( HDFS , Hive, HBase, Apache Kylin , Yarn, Flume) Streaming Framework ( Kafka ) Data Ingestion ( Informatica , Kafka Connect , Nifi and MiNiFi) Database ( Oracle, MySQL, MSSQL, Hive, Impala, Presto) Visualisation (Birst , Qlik , Sisense , Looker) System , Data warehouse and Data lake designing SQL expert Application development Agile development methodologies Customer-oriented Troubleshooting and diagnostics API and Microservice designing OLAP and OLTP modeling Docker And Kubernetes

Experience

19 yrs 8 mos
Total Experience
2 yrs 8 mos
Average Tenure
3 yrs 11 mos
Current Experience

Apollo.io

Senior Data Engineer

Jul 2022Present · 3 yrs 11 mos · Pune, Maharashtra, India

  • Built and maintained a comprehensive Data Platform integrating self-hosted GKE solutions with Snowflake, DBT, and Metaplane
  • Implemented robust API webserver observability systems on GKE to enhance monitoring and performance tracking
  • Developed end-to-end ML pipelines for model training and inference, improving production deployment efficiency
  • Led cost optimization initiatives across multiple data platform components including Snowflake, GCP, and Databricks, resulting in significant operational savings
  • Standardized GenAI applications by architecting an enterprise-wide GenAI platform enabling seamless integration with various generative AI models
Databricks ProductsVector Databases (Pinecone, Vector Search)GenAIData EngineeringCloud Architecture

Coupa software

2 roles

Senior Lead Software Engineer

Feb 2022Jul 2022 · 5 mos

  • Setup self hosted Looker for VPC and FedRamp environment
  • Converted self hosted Looker as ECS service
  • Setup auto scaling and monitoring for self hosted Looker
  • Migrated Looker to GCM encrypted Looker
  • Wrote data pipeline supporting redshift along with snowflake

Lead Data Engineer / Software Engineer/ Data Architect

Oct 2017Feb 2022 · 4 yrs 4 mos

  • Orchestrated efficient large-scale software deployments using AWS stack , Ruby and Rails, Python , MPP data warehouse
  • Managed systems and infrastructures by defining hardware requirements and planning storage capacity for Redshift and Snowflake
  • Implemented BI solution framework for end-to-end business intelligence projects using Qlik , Sisense , Looker and Birst
  • Analyzed overall client performance and recommended reassessment of specific plan goals
  • Used cloud agnostic and cloud centric data warehouse technologies ( Redshift, Snowflake, Big Query) to research, gather, analyze and present data
  • Orchestrated smooth and effective flow of P2P data to support Analytics and support decision making process
  • Developed entire high performance analytics platform from scratch with all cutting edge technologies.
  • Created proof-of-concept prototype applications for demonstration and evaluation
  • Involved in team expansion and mentoring.
  • Rich experience in building OLAP and OLTP models for various transactional and analytics applications.
  • Experience in designing monolithic and microservice based application
  • Experience is docker and kubernetes.
  • Rich experience in architecting real time and bach ETL
  • Architected and built containerized solution using Docker and Kubernetes

Schlumberger

Senior Big Data Engineer

Jun 2016Sep 2017 · 1 yr 3 mos · Pune, Maharashtra, India

  • Administered, supported and monitored Hadoop clusters by proactively resolving cluster issues and maintaining servers
  • Coordinated with project management staff on Data Lake development
  • Part of team who built Enterprise Data Lake and framework for ingesting data from different source system into Data Lake
  • Created and implemented database designs and data models on Hive HBase and BigQuery on GCP
  • Collaborated with different departments and levels of management to gain consensus on Data lake and Data pond creation on in premise Hadoop cluster Developed framework for automated Data Pond Provisioning
  • Designed and developed PHM ( Prognostic Health Monitoring) system for drilling tools used in Oil rigs which is helpful in reducing operation cost and predicting failure much before the use
  • Implemented BI solution on top of Data Lake for business intelligence projects Developed generic parser for IoT logs for different IoT formats

Barclays

Senior Analyst

May 2013May 2016 · 3 yrs · Pune

  • Investigated and addressed in compliance system issues to enhance usability and improve functionality
  • Evaluated and adopted new technologies ( Hadoop and Cloud) to address changing needs in terms of user access and data volume
  • Enhanced and reconstructed compliance system datawarehouse on Oracle and later moving long running jobs to Hadoop cluster
  • Improved compliance system by adding new features and infrastructure, enhancing Supported and aligned efforts to meet business and customer needs
  • Presented business leaders with cost-benefit and return-on-investment estimations to drive project planning and execution
  • Reviewed performance benchmarks and established metrics for future tracking Configured entire Dev Ops pipeline for automated deployment

Amazon

Data Engineer

Jun 2012May 2013 · 11 mos · Hydrabad

  • Performed ad hoc deep-dive analysis for specific business problems.
  • Created and implemented database designs and data models.
  • Organised information by studying, analyzing, interpreting and classifying data. Enhancing the ETL process for loading such a huge data into Warehouse tables. Built Data Quality framework for data duplication issue

Oracle

Senior Application Engineer

Apr 2011Jun 2012 · 1 yr 2 mos · Banglore

  • Designing data model.
  • Enhancing the ETL process for loading.
  • Automation of analytics processes.
  • Fixing customer issue.
  • Coordinated with hardware and system engineering leads to gather and develop system requirements.
  • Created proofs of concept for innovative new solutions.
  • Consulted with engineering team members to determine system loads and develop improvement plans.
  • Improved system performance by making proactive adjustments and resolving bugs.
  • Orchestrated efficient large-scale software deployments, including testing features and correcting code

Cvs caremark corporation

2 roles

TCS SME

Jan 2008Jan 2010 · 2 yrs

  • Identifying gaps in financial reporting
  • Worked with customer for smooth UAT and ST
  • Worked with customer and offshore team on change requests
  • Walk through of features and change requests with customer
  • Built framework to identify issues in financial reporting
  • Tuned ETL for better reporting

TCS SME

Jan 2008Jan 2010 · 2 yrs

  • Techno functional SME

Tcs

IT Analyst

Aug 2006Apr 2011 · 4 yrs 8 mos

  • Participated with preparation of design documents for track work, including alignments, specifications, criteria details and estimates.
  • Used database to research, gather, analyze and present data.
  • Created and implemented database designs and data models.
  • Identified areas of weakness and recommended or implemented process improvements.
  • Built Data warehouse and ETL pipeline
  • Built business dashboards / reports
  • Developed reconciliation framework for to validate data processing and dashboards
  • Part of performance review team
  • Managing reporting team

Education

Great Lakes Institute of Management

Post Graduate Program — Artificial Intelligence and Machine Learning

Jan 2019Jan 2020

BIT Mesra Student-Industry Relations Cell

Bachelor of Engineering (B.E.) — computer science and engineering

Jan 2002Jan 2006

ST. Aloysius High School

high school

Jan 1998Jan 1999

Stackforce found 100+ more professionals with Data Engineering & Cloud Architecture

Explore similar profiles based on matching skills and experience