Rachit Upadhyay

Data Engineer

Pune, Maharashtra, India14 yrs 3 mos experience
Most Likely To SwitchHighly Stable

Key Highlights

  • Expert in architecting data platforms for real-time processing.
  • Led development of a scalable Data Lake for enterprise solutions.
  • Proven track record in enhancing compliance systems in finance.
Stackforce AI infers this person is a Big Data Engineer with expertise in SaaS and Finance sectors.

Contact

Skills

Core Skills

Big DataData EngineeringComplianceSoftware EngineeringCloud Computing

Other Skills

Amazon Cloud WatchAmazon S3Amazon Web Services (AWS)Azure DatabricksBI SolutionsBigQueryBusiness AnalysisCloud IntegrationCommunicationCompliance SystemsData LakeData PlatformData ProcessingData WarehousingDatabases

Experience

Nvidia

Senior Data Engineer

Jan 2021Present · 5 yrs 2 mos

Boston consulting group (bcg)

Lead Data Engineer

Jan 2018Jan 2021 · 3 yrs · Gurgaon, Haryana, India

  • Architected and implemented data and analytics platform with capability of batch and real time data processing. Developed and scheduled spark streaming jobs using pyspark to perform near real time aggregation.
  • Develop framework to handle 10M+ Events daily, work on optimizing and tuning spark jobs.
  • Create and deliver powerful technical presentations and demos that explain our solution. Build reference architecture. Manage Technical aspect of RFP
  • Tailored shopping experiences, demand forecasting at scale, Immersive gaming experience some of use cases enabled.
  • 30+ use cases enabled with Data Platform As-A- Service built for data scientists, data Engineers and business analysts
  • Provide technical leadership in pre-sales and post sales capacity for customers to support successful understanding , evaluation and adaptation of technologies
HadoopSparkPySparkData ProcessingTechnical PresentationsData Platform+2

Schlumberger

Big Data Engineer

Apr 2016Sep 2017 · 1 yr 5 mos · Pune, Maharashtra, India

  • Coordinated with project management staff on Data Lake development
  • Part of team who built Enterprise Data Lake and framework for ingesting data from different source system into Data Lake
  • Created and implemented database designs and data models on Hive HBase and BigQuery on GCP
  • Collaborated with different departments and levels of management to gain consensus on Data lake and Data pond creation on in premise Hadoop cluster Developed framework for automated Data Pond Provisioning
  • Designed and developed PHM ( Prognostic Health Monitoring) system for drilling tools used in Oil rigs which is helpful in reducing operation cost and predicting failure much before the use
  • Implemented BI solution on top of Data Lake for business intelligence projects Developed generic parser for IoT logs for different IoT formats
Data LakeHiveHBaseBigQueryGCPBI Solutions+2

Bank of america merrill lynch

Lead Engineer

Mar 2014Mar 2016 · 2 yrs · Noida Area, India

  • Investigated and addressed in compliance system issues to enhance usability and improve functionality
  • Evaluated and adopted new technologies ( Hadoop and Cloud) to address changing needs in terms of user access and data volume
  • Enhanced and reconstructed compliance system datawarehouse on Oracle and later moving long running jobs to Hadoop cluster
  • Improved compliance system by adding new features and infrastructure, enhancing Supported and aligned efforts to meet business and customer needs
  • Presented business leaders with cost-benefit and return-on-investment estimations to drive project planning and execution
  • Reviewed performance benchmarks and established metrics for future tracking Configured entire Dev Ops pipeline for automated deployment
Compliance SystemsHadoopOracleDevOpsData EngineeringCompliance

Infosys

Senior Software Engineer

Jul 2011Mar 2014 · 2 yrs 8 mos · Pune Area, India

  • Worked on Infosys Cloud product Cloud Ecosystem Hub.
  • Integrated the product with Amazon Cloud Watch to get real time details of the CPU,Network,Data Traffic utilization
  • Worked on Spring MVC and implemented many different components of project involving Session Management
  • Integrated AmazonS3 with Local disk to automatically Sink new files on Cloud
Cloud IntegrationSpring MVCAmazon Cloud WatchAmazon S3Software EngineeringCloud Computing

Education

Symbiosis Centre For Management and Human Resource Development

Post Graduate diploma in Business Analytics — analytics

Jan 2017Jan 2018

Birla Vidya Niketan

Computer Science

Jan 2004Jan 2006

Stackforce found 100+ more professionals with Big Data & Data Engineering

Explore similar profiles based on matching skills and experience