Rohit Pandey

Engineering Manager

Mumbai, Maharashtra, India12 yrs 11 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

12+ years of experience in data engineering and architecture.
Led successful data migration projects improving performance by 3x.
Expert in developing scalable data pipelines for high concurrency.

Stackforce AI infers this person is a Data Engineering Leader with expertise in SaaS and Big Data solutions.

Contact

Skills

Core Skills

Data EngineeringData ProductsData ArchitectureBig Data Analytics

Other Skills

Data IntegrationSoftware Development Life Cycle (SDLC)SparkHiveFinancial ServicesCPG IndustryMicrosoft AzureGoogle Cloud Platform (GCP)DatabasesBusiness IntelligenceAnalysis ServicesMicrosoft Reporting ServiceMicrosoft SQL ServerSSISSSRS

About

Data Leader | 12+ Years of Industry Expertise --------------------------------------------- Data professional with 12+ years of experience in data engineering, architecture, and team leadership within startup and enterprise tech environments, including JioSaavn and Fractal. Proven ability to architect and implement scalable data pipelines and data products, with a strong background in batch processing. Currently leading Data Engineering teams at JioSaavn, responsible for developing and maintaining robust data pipelines and Data products that deliver analytical solutions to partners and labels across various platforms. Passionate about leveraging data to drive business outcomes and dedicated to mentoring and team development. I enjoy spending time with my family in my personal time.

Experience

12 yrs 11 mos

Total Experience

2 yrs 1 mo

Average Tenure

2 yrs 6 mos

Current Experience

Jiosaavn

Engineering Manager (Data - Infra & Analytics)

Dec 2023 – Present · 2 yrs 6 mos · Mumbai, Maharashtra, India · On-site

° Heading JioSaavn's data strategy, overseeing two engineering teams: Data Product Engineering and Data Platform Engineering.
° Addressed complex, large-scale data challenges presented by millions of daily active users and billions of content streams.
° Developed and executed multi-quarter roadmaps, architecting a modern data platform and deploying multiple data products leveraging these platform abstractions.
° Collaborated cross-functionally with Engineering, Product, Analytics, and Customer Support teams, contributing to JioSaavn's overall success.
° Managed the data infrastructure responsible for processing multi-billion event and asynchronous message volumes daily.
° Led the migration from a self-managed Hive system to a Spark-based architecture, resulting in a 3x+ performance improvement and significant year-over-year cost reductions.

Data ProductsData EngineeringBig Data AnalyticsData IntegrationSoftware Development Life Cycle (SDLC)

Fractal

2 roles

Data Architect

Jun 2022 – Dec 2023 · 1 yr 6 mos · Mumbai, Maharashtra, India

° Managing data side of projects end to end.
° Worked as a lead for all finance data projects for Major CPJ client.
° Set data culture with in team and lead data maturity initiative.

Financial ServicesCPG IndustryMicrosoft AzureBig Data AnalyticsGoogle Cloud Platform (GCP)Data Architecture

Senior Data Engineer - II

Nov 2020 – Feb 2023 · 2 yrs 3 mos · Mumbai, Maharashtra, India

Transforming data at scale via Pyspark on HDInsight Hadoop platform.
Understood data and business processes, performed data profiling and define dimensional data model for same for Major CPG company.
Understanding business requirement and created spark scripts to perform processing and load data in hive for publish purpose.
Loading high volume tables in hive with performance centric approach.
Cleaning data using pySpark scripts as per business standards and rule engine.
Writing Spark SQL scripts to load data in a dimensional model of hive.

Tata capital

Senior Data Engineer

Dec 2018 – Nov 2020 · 1 yr 11 mos · Mumbai Area, India

Working with all the stakeholders on data lake project from scratch
Requirement and KPI gathering from different business verticals and
evaluating tools and technologies to fulfill it.
Designing a data lake , ingestion, and data warehousing architecture.
Business as usual fulfillment.
worked on different architectural and implementation challenges .

Tiaa

Senior Big Data engineer

Oct 2016 – Dec 2018 · 2 yrs 2 mos · Mumbai Area, India

 Work with TIAA Business team to collect the requirements and then go through analysis and design phase to deliver the ETL solution using Hadoop and Datastage.
 Interact with Business partners and Business Analysts kick off sessions in translating business requirements into Functional Requirements Document.
 Develop Detailed Design Documents based on the Functional Requirements Document(FRD).
 Involve in the development, testing and migration of the DataStage jobs during offshore hours.
 Provide the support to the Testing team in the Test case preparation and execution during System Integration & User Acceptance Tests.
 Fetching data from source(RDBMS,Files) and loading to staging(Hive) and using staging as a source to populate Datawarehouse for reporting needs .
 Cosumes application specific delimited Files from the legacy source applications and design jobs to transorm it to target(Fusion) specific format.
 Writing T-SQL and PL-SQL queries, sub-queries and complex joins for generating resultset for ETL Source.
 Extensively used DataStage stages such as Lookup, Aggregate, Funnel,Transformer,Sequentialfile,Dataset,Oracle connector,ODBC Connector .
 Experience on designing parallel jobs,Sequencers,Error Handling in DataStage.
 Autosys for scheduling of ETL packages.
 Starteam for version control.
 Serena for deployment.

Citiustech

Big Data Engineer

Oct 2015 – Oct 2016 · 1 yr · Mumbai Area, India

 Daily use of ETL tools to develop and maintain Operational Data Store (ODS system).
 Created functional and technical design documents.
 Involved in Creation, Development and Deployment of ETL jobs in DataStage 11.5.
 Involved in designing and loading Datawarehouse in IBM Netezza and Datamart in Hbase and Hive.
 Writing T-SQL,PL-SQL Queries/Dynamic-queries using sub-queries and complex joins for generating resultset for ETL Source.
 Studied existing OLTP system and defined data in IBM Atomic Warehouse Model (AWM) in Hbase and Netezza.
 Studied IBM AWM model and moved data to facts, dimensions in IBM Dimensional Warehouse Model (DWM) in Netezza.
 Experience on designing parallel jobs, SCD, CDC, Sequencers,Error Handling in DataStage.
 Designed sequencer and master sequencer for scheduling of ETL packages.

Atos india

Software Engineer

Jul 2013 – Oct 2015 · 2 yrs 3 mos · Mumbai Area, India

 Worked end to end for ETL process from getting flat files from Ticketing tools to loading data from flat files to staging tables.
 Developed ETL process to transform staging table data and load it into the Data warehouse system for further processing.
 Extracting data from multiple sources, mainly Excel Spreadsheets, Text file, SQL server and CSV files to the data warehouse.
 Migrating all SSIS jobs to Datastage jobs.
 Responsible for the MSSQL 2012 DB for creating modifying and updating tables ,SP's according to the business requirements.
 Used Data flow components like Lookup, Derived Column, Merge join, Conditional Split to validate and modify data while loading to DWH using SSIS.
 Using control flow items Like Data Flow Task, Execute Package task, Send mail task.
 Creating and maintaining ETL source to target documentation and test cases and test logs
 Created basic types of reports like tables, matrix, subreports, drill through and drill down
 Configuring and troubleshooting DB mirroring log shipping.
 Creation of logins & DB users.
 Defining proper backup policy on databases are per business requirement and restoring the databases.
 Creating database maintenance plans and SQL Server jobs for backup, index rebuild, update stats etc
 Maintaining documentation for the site.(eg.Unit test case, TDD)