Nadeesh Garg

Data Engineer

Bengaluru, Karnataka, India11 yrs 9 mos experience
Most Likely To SwitchAI ML Practitioner

Key Highlights

  • Expert in building scalable data pipelines.
  • Strong background in Quantum Computing and Data Engineering.
  • Proven leadership in modernizing data infrastructure.
Stackforce AI infers this person is a Data Engineering expert with a strong focus on SaaS and Healthcare industries.

Contact

Skills

Core Skills

Data EngineeringApache SparkResearch

Other Skills

Google BigQueryDbtApache AirflowPythonScalaQuantum InformationQuantum ComputingCompilersProphecyQuantum Field TheoryQuantum OpticsData VisualizationMachine LearningArtificial Intelligence (AI)Statistical Data Analysis

About

12 years of experience in Data Systems: From research in complex systems, network theory to working as a data scientist briefly in Finance and NLP, to conceptualizing and building full-stack web applications for web development, to designing, and building, with (managed) Kafka, Airflow and Spark, event-based data pipelines, to working as an ML Ops specialist Dabbling in Quantum Computing (Algorithms and Error Correction)

Experience

Prophecy

Data Engineer

Nov 2022Present · 3 yrs 4 mos · Bengaluru, Karnataka, India · Remote

  • 1. Developed custom Prophecy Gems for multiple clients and internal transpilation initiatives, creating visual components that dynamically generate native Spark and DBT code based on user interface configurations.
  • 2. Expanded the Expression Transpilation codebase to enable bidirectional code conversion across Scala, Python, and SQL, enhancing platform flexibility and developer productivity.
  • 3. Built documentation parsing libraries for Spark SQL, Python, and Scala to enable cross-language transpilation and power the Frontend Expression Builder, and extended support for BigQuery and DuckDB documentation to further enhance frontend capabilities.
  • 4. Used Scala Meta for a metaprogramming (code that writes code) related application for building backend support for cross language compilation.
  • 5. Led data infrastructure modernization for a large healthcare organization, migrating legacy Abinitio pipelines to Apache Spark.
  • Collaborated with a team of 5 engineers to successfully migrate 300+ pipeline configurations
  • Subsequently led the migration of 200+ additional pipelines, ensuring seamless transition and minimal downtime
  • 6. Developed and maintained the Lineage Extractor Project, a CLI extension that enhances Prophecy's data lineage capabilities and metadata tracking.
Google BigQueryDbtApache AirflowPythonApache SparkScala+1

Tekion corp

Senior Software Engineer

Jan 2021Nov 2022 · 1 yr 10 mos · India

  • Worked on productionizing various ML Models to put into production (recommendations, classification etc.)
  • Scaled various existing services to be able to handle upwards of 50 RPS by using ASGI and WSGI serving capabilites.
  • Designed and implemented Spark based ETL pipeline for doing massive computations for creation of feature sets (used by DS for creating models)
  • Designed and implemented workflow management for the same, to keep track of various pipelines.
  • Designed and implemented various ML models for Streaming data (Spark Streaming)

Cuemath

2 roles

Lead Data Engineer

Promoted

Dec 2019Jan 2021 · 1 yr 1 mo

  • Designed and developed, end to end, a schedulable, scalable, configurable ETL Pipeline system that makes it possible to use real-time events to construct clean, aggregated data using the Airflow and Spark DAG systems (plus Lambda, Kafka, S3, Redshift)
  • Enabled easier independent scale-up of the Product and Analytics suites through eventual decoupling of Analytics and Tech databases
  • Created the main ETL Scheduler, various Lambdas (to receive and parse events from clients and webhooks), the event emitter frontend library and the Flask service that interacts with our data warehouse (for custom admin/analytics dashboards + Alembic)

PM + SDE for Curriculum Analytics

Jun 2019Dec 2019 · 6 mos

  • Designed and developed an Analytics and Reporting System for teachers to view students data, from scratch using MongoDB, Flask and ReactJS.
  • Revamped the process of curriculum graph design by Designing and Developing a curriculum-graph frontend system to enhance the ease of use and speed, using a drag and drop interface
  • Mentored a team of 3 to create math animation videos, analytics dashboards (using python scripts and google sheets API) and an internal reporting tool powered by analytics
  • SPOC for product and curriculum specific data requests.

The institute of mathematical sciences, chennai

Research Assistant

May 2017Nov 2018 · 1 yr 6 mos · Greater Chennai Area

  • Worked with Prof. Sitabhra Sinha from the Complex Systems and Non Linear Dynamics team.
  • Analysed a globally coupled Ising Spin system with Hebbian Learning in a Multiplex Network; Direct application in Pattern Retrieval from Hopfield Networks (Undirected Neural Networks)
  • Presented a model on Iterated Prisoner’s Dilemma on a network with opponent awareness (Memory of opponents’ strategy)
  • Working on the Spotify Million Playlist Dataset using sparse graph representations for creating a playlist recommendation engine.
Quantum InformationQuantum ComputingResearch

I-pac (indian political action committee)

Senior Consultant

Aug 2016Jan 2017 · 5 mos

  • Designed strategy and execution workflow for 'Jan Shikayat Ki Panchayat' campaign targeting 7.5 million people nationwide
  • Headed Kisaan Door to Door Campaign for INC for upcoming Uttar Pradesh Elections in 3 districts: Shrawasti, Ghaziabad, Amethi; Total outreach of over 1000000 farmers
  • Managed Logistics, Execution for Rahul Gandhi Khat Sabha in Ghazipur, Ambedkar Nagar

Myolo

Back End Developer

Nov 2015Jul 2016 · 8 mos · Greater Delhi Area

  • Redesigned Spring, Hibernate based Backend to integrate Direct Messaging feature with GCM, in app-payments
  • Mentored 2 interns to create a model to extract user’s dating preferences from facebook likes, shared posts, friends’ preferences

Tripooler

Founder

Sep 2015Jul 2016 · 10 mos · Gurugram, Haryana, India

  • Worked on creation of a gamified social network that connects like-minded travelers
  • Responsible for product architecture, design, implementation and placement in market
  • Tested product concept and collected user preferences for product by accessing travel
  • communities as Head of Operations for two socio-travel startups: india42, wanderwagon

Nextgen techno ventures

Analyst: Strategy and Algorithms

Apr 2015Sep 2015 · 5 mos · Gurugram, Haryana, India

  • Designed price discovery models for derivative pricing in the Indian Stock Market
  • Implemented time series (ARIMA, GARCH) and portfolio theory based models (CAPM, MPT) and backtested various algorithms to check periodicity, basket arbitrage possibilities
  • Used various technical indicators and basic greek analysis to trade options and futures.
  • Built analytical tools using R to extract relevant information out of large datasets

Thilio

Data Scientist

Dec 2014Mar 2015 · 3 mos · Gurugram, Haryana, India

  • Designed management system that predicts person-task fit based on skills and past performance
  • Implemented supervised learning techniques (Regression, NLP) to predict time taken to project
  • completion using past performance data, skill tags, human learning curve

Ims health

Junior Consultant

Jul 2014Dec 2014 · 5 mos · Gurgaon

  • Currently work in the client services wing of SGPM group at IMS health. Have executed Social Media Analytics based Service requests as well as those related to MP reports.

India42

Head of Operations

Feb 2014Feb 2015 · 1 yr · Greater Delhi Area

  • Was responsible for event management on ground as well as off ground, heading core operations as well as marketing at india42 as a part time job.
  • india42 organizes experience based trips to offbeat locations across the country.

Make a difference

Placements Intern

Aug 2012May 2013 · 9 mos

Education

Indian Institute of Technology, Delhi

Bachelor of Technology (B.Tech.) — Engineering Physics

Jan 2010Jan 2014

Delhi Public School - R. K. Puram

Class 12 — Science with Computers

Jan 2008Jan 2010

Stackforce found 100+ more professionals with Data Engineering & Apache Spark

Explore similar profiles based on matching skills and experience