M

Mandeep Kumar

Data Engineer

Delhi, India10 yrs 11 mos experience
Most Likely To Switch

Key Highlights

  • Expert in building data pipelines and ETL processes.
  • Strong experience with AWS and data engineering tools.
  • Proven track record in developing scalable solutions.
Stackforce AI infers this person is a Data Engineer with expertise in building scalable data solutions across various industries.

Contact

Skills

Core Skills

PythonAwsSparkKafkaSpark Sql

Other Skills

AWS DMSAWS GlueAWS ecosystemAirflowApache KafkaApache SparkAthenaAuroraDjango rest-frameworkEMRElastic SearchHBaseHadoopHiveLinux

Experience

Angel one

Staff Data Engineer

May 2023Present · 2 yrs 10 mos · Bengaluru, Karnataka, India · Remote

Dehaat

Principal Software Engineer

Apr 2020Sep 2022 · 2 yrs 5 mos · Gurgaon, Haryana, India

  • Worked on advisory distribution to farmers and to provide corporate solutions to input partner companies.
  • Wrote apis for different features, optimised it and handled deployment of applications on AWS.
  • Worked on geo-spatial data(both raster and vector). Built pipelines to collect data from different applications and performed aggregations.
  • Wrote rule engines to generate dynamic advisories for farmers and handled disbursal of advisories
  • via mediums like IVRS, whatsapp, notification on app etc.
  • Merged and refactored the codebase of two different apps to use the shared libraries and common apis thereby reducing maintenance effort of the codebase.
  • Set up an ETL flow to store cdc data in S3 using AWS DMS and integrated Athena on top of processed data using AWS glue for running analytical queries
  • Software tools/Skills: Python, Django rest-framework, Postgresql, Postgis, RabbitMq, Airflow, AWS DMS, AWS Glue, Athena, Presto, Metabase, SQL queries, AWS ecosystem
PythonDjango rest-frameworkPostgresqlPostgisRabbitMqAirflow+8

Ht media ltd

Data Engineer II

Feb 2019Mar 2020 · 1 yr 1 mo · New Delhi Area, India

  • Worked on data-pipeline to collect and process clickstream data from various news-sites in batch and near real time to be used for analytics and wrote apis to show users read history and trending stories on the news-site.
  • Software tools/Skills: Spark, Kafka, Elastic Search, HBase, Snowplow, S3, Redshift,Scala etc
  • Wrote web crawlers to scrap and collect stories meta-data from various new-sites, google-news and google trends to find trending keywords and prepare metrics for improvements in publishing stories and minimizing the latency.
  • Software tools/Skills: Nodejs, Kafka, MongoDB etc
SparkKafkaElastic SearchHBaseSnowplowS3+4

Hdfc life

Data Engineer

May 2018Feb 2019 · 9 mos · Mumbai Area, India

  • Wrote spark sql jobs and nifi data flows to ingest EDW tables from inhouse DBs and put them in data lake and then processed these tables to extract information to be displayed on the cards in customer timeline.
  • Software tools/Skills: Spark SQL, Nifi, Elastic Search, Kafka, S3, Aurora, EMR, Scala etc
Spark SQLNifiElastic SearchKafkaS3Aurora+2

Lendingkart

Data Engineer II

Apr 2017Apr 2018 · 1 yr · Bengaluru Area, India

  • Designed and built pipeline to consume and track audit history changes with time to generate reports for tracking the delay in processing loans, status changes for lead, application, contract etc
  • Software tools/Skills: Kafka, S3, Redshift, mysql, python etc
  • Wrote apis and implemented a rule engine to generate the final loan disbursal TnC for customers to determine the user eligibility for pre-approved loans, final loans or no loans with proper terms and conditions.
  • Software tools/Skills: Scala, MySQL
KafkaS3RedshiftMySQLPythonScala

Persistent systems

Senior Software Engineer

Jul 2014Apr 2017 · 2 yrs 9 mos · Nagpur Area, India

  • Worked on media data to find statistics by combining viewership data with Ad and Promo data and plotting graphs to show these statistics on Audience Measurement Platform.
  • Software tools/Skills: ShareInsights(Spark and Hive), Mysql, Python
  • Worked with IBM BigInsights team to detect potential incompatibilities in BigInsights IOP packages and resolve the conflict to ensure that it runs seamlessly across different OS.
  • Software tools/Skills: Spark, hive, Kafka, Python, ambari, maven, ant etc
SparkHiveKafkaPythonMySQL

Mentor graphics

Internship

May 2013Jun 2013 · 1 mo · Noida Area, India

  • Wrote positive and negative test cases based on VHDL-AMS IEEE Reference Manual and compiled it on the tool vacom and vcom and prepared lists containing the test cases that showed error either in vacom or in vacom, error in both vcom and vacom and fatal error.
  • Software tools/Skills:
  • VHDL, VHDL-AMS, Verilog, Verilog-AMS, Vim, linux
VHDLVHDL-AMSVerilogVerilog-AMSVimLinux

Education

Indian Institute of Technology (Banaras Hindu University), Varanasi

Bachelor of Technology (B.Tech.) — Computer Science and Engineering

Jan 2010Jan 2014

Stackforce found 100+ more professionals with Python & Aws

Explore similar profiles based on matching skills and experience