Sarthak Nagpal

Director of Engineering

Bengaluru, Karnataka, India7 yrs 3 mos experience
Highly Stable

Key Highlights

  • Expert in building robust big data pipelines.
  • Strong background in real-time distributed systems.
  • Proven leadership in data engineering projects.
Stackforce AI infers this person is a Data Engineer specializing in Fintech and Data Engineering solutions.

Contact

Skills

Core Skills

Big DataData EngineeringReal-time StreamingData WarehousingReal-time Distributed Systems

Other Skills

Apache AirflowApache SparkSnowflakeAzure DatabricksScalapythonJavaPower BIAzure Data LakeApache KafkaAzure Data FactorySQLCassandraxMattersKubernetes

About

Experienced Data Engineer with a strong background in building real-time distributed systems and enterprise web applications. Currently working on robust big data pipelines using Apache Airflow, Databricks, Snowflake, Python, Scala, and Java, with a strong focus on observability and monitoring. Skilled in working with Apache Kafka, Akka, and Play Framework, along with various Azure services such as Cosmos DB, ADLS and moderate experience in Cassandra, Docker, and Kubernetes. Proficient in data warehousing and integration tools like Data Vault 2.0, Azure Data Factory, Databricks, and Azure Synapse Analytics. Also experienced with monitoring and alerting tools like Grafana, xMatters, and Azure-native monitoring. Previously worked in a full-stack role using Java (Spring Boot), JavaScript, and React for building web applications. Quick to learn, highly adaptable, and effective at collaborating with both business stakeholders and technical teams. Known for strong communication skills and a proactive approach to solving problems and driving results.

Experience

7 yrs 3 mos
Total Experience
2 yrs 8 mos
Average Tenure
1 yr 11 mos
Current Experience

Morgan stanley

2 roles

Director

Promoted

Jan 2026Present · 4 mos

Manager

Jun 2024Jan 2026 · 1 yr 7 mos

  • Working as SDE-3 on Big Data Enterprise systems under Treasury Technology
Apache AirflowApache SparkSnowflakeAzure DatabricksScalapython+5

Hashedin by deloitte

2 roles

Software Engineer 2

Jan 2022Jun 2024 · 2 yrs 5 mos

  • Worked on making the real time streaming applications fault tolerant, scalable by identifying opportunities to improve performance whether through logic or making few performance improvements on the database front (Azure cosmos DB in particular).
  • Worked on adding alerting and montoring mechanisms in the system by adding kafka consumer lag alerts via xmatters and creating grafana dashboards.
  • Worked on creating a spring batch job to identify missing points in the system so that the issues can be resolved quickly.
  • Also, worked as a shadow lead and assisted the team members with their queries related to the tasks and provided solutions/alternatives.
  • Currently working as a team lead working on data curation and warehousing of batched data load in technologies such as ADF, Databricks for a renowned retail client and using synapse to expose the views as a DaaP to customers.
Apache KafkaAzure Data FactorySQLCassandraAzure DatabricksxMatters+11

Software Engineer

May 2021Dec 2021 · 7 mos

  • Currently working on a real time scalable and fault tolerant distributed system in Fintech domain integrating it with the existing data pipeline to consume certain transactions and transform them to a certain data model with some business rules applied to produce to a downstream application in the pipeline.
  • The transactions are consumed from Apache Kafka and then processed via Akka streams which helps to keep the app fault tolerant.
  • Scala is used to write business logic and produce to Kafka topic after transformation.
  • Application is containerized via Docker and deployed on Kubernetes.
  • Also working on maintenance and enhancement of certain features in another real time distributed application as and when required.
  • Also gained knowledge and experience on working on certain applications in the data pipeline using Apache Flink.
  • Data is consumed via Kafka using connector in Flink.
  • Data is processed and saved using checkpoints stored in Azure Blob storage
  • Data is transformed and produced to kafka topics and Azure cosmos DB for further usage.
Apache KafkaxMattersKubernetesAzure Cosmos DBScalaGrafana+3

Infosys

3 roles

Technology Analyst

Jan 2021Apr 2021 · 3 mos

Senior Systems Engineer

Promoted

Aug 2019Dec 2020 · 1 yr 4 mos

  • Maintaining applications for improved performance by writing efficient and testable code in Java(Core and Spring Boot) and javascript and identifying opportunities for improvement.
  • Conducting analysis,programming, testing and debugging on existing applications to polish their respective feature sets.
  • Support continuous improvement by suggesting alternatives and automation strategies to the Business clients.
  • Developed a web application in Spring Boot to validate and store the user data to a remote archive storage , also taking care of the faulty saves which the native application misses out thus, reducing dependency on Business clients and audit reporters for any missed data in the archive storage.
  • Created a service script using UNIX shell scripting to monitor certain services of an application as a value add thus reducing an overhead of numerous dollars every month for the Business clients.
  • Created an automation to check the health and status of the web application servers of a particular domain and provide alerts to the team on any failure so that the issue can be resolved there and then and the users don't face any downtime.

Systems Engineer

Jul 2018Jul 2019 · 1 yr

  • • Developed a data processing web application in Java where the user can upload and process their biological study data based on the conclusions of their respective experiments with objective being to provide the users a proper validation mechanism to process their sensitive data.

Slr infotech pvt. ltd

Engineering Trainee

May 2017Jul 2017 · 2 mos · Chandigarh, Chandigarh, India

  • Worked on a project named 'Library Management System' using concepts of advanced java such as collections, servlets, JSPs and Apache Tomcat server.

Netmax technologies pvt ltd

Trainee

May 2016Jul 2016 · 2 mos · Chandigarh, Chandigarh, India

  • Learnt the concept of core Java with new features from Java 8. Made an 'auto shutdown' mini program using concepts of network programming and AWT.

Niit technologies limited

Student Trainee

May 2015Jul 2015 · 2 mos · India

  • Made a mini hotel management system with C++ along with some basic OOP principles.

Education

Jaypee University of Information Technology

Bachelor’s Degree — CSE

Jan 2014Jan 2018

Sacred Heart Senior Secondary School, Dharamsala

Jan 2001Jan 2014

Stackforce found 100+ more professionals with Big Data & Data Engineering

Explore similar profiles based on matching skills and experience