Jun 2024 – Dec 2024 · 6 mos · Bengaluru, Karnataka, India · Hybrid

• > Architect, develop, and deploy ETL/ELT pipelines using Azure Data Factory for a Fortune 500 company, integrating data from diverse sources. Leveraging Azure Databricks (PySpark/SparkSQL) and write Stored Procedures for advanced raw data transformations.
• > Write various stored procedures to automate & implement database refreshes, maintain these stored procedures & ADF pipelines/master pipelines and implemented data quality measures in ADF to improve data quality of various different sources, enhancing data integrity across multiple sources.
• > Design and implement data models that support flexible querying and data visualization. Expertise in writing SQL codes and in data-warehousing concepts such as star schemas, slowly changing dimensions, ELT/ETL, & MPP databases.
• > Maintenance and debugging of pipelines and dataflows, ensuring seamless refreshes of databases and Power BI reports.
• > Implement CI/CD pipeline methodologies to automate task using Azure DevOps and create artifacts, jobs and task to orchestrate data across DEV, UAT, and PROD environments through release pipelines utilizing ARM templates..
• ▪ Incremental Ingestion pipeline – Product Growth Data
• > Tech Stack – Shell Scripting, Azure Data Factory, Azure Data Lake gen2, Azure SQL DB, Power BI, PySpark
• Build generic, dynamic & optimized ingestion pipeline for highly critical & confidential Sales & Product Growth Data.
• Pipeline is designed in a way to handle GB’s of daily & weekly data together for different use cases like Audit, Promotions, Growth, Up-skilling etc.
• ▪ Pipeline Optimization & Enhancement
• > Tech Stack – Stored Procedures, Azure Data Factory, Azure Data Lake gen2, Azure SQL DB, Power BI, PySpark, Shell Scripting.
• Enhance & optimize multiple pipelines and stored procedures, built for different business and product units .
• Reduced execution time by 50% and improved alerting system for different edge cases.

Feb 2023 – Jun 2024 · 1 yr 4 mos · Hyderabad, Telangana, India · Hybrid

• Key skills :- MS SQL Server · SSIS · Informatica · Extract, Transform, Load (ETL) · Azure Data Factory · ADLS
• gen2 · Azure Synapse Analytics · Azure Databricks · Azure SQL DB · Azure Blob Storage · Apache Spark · PySpark · SparkSQL · Microsoft Power Bi · Python · Data Modelling · Data Warehousing · Data Analysis/Analytics · Statistical Analysis · Data Quality · Data Profiling · Data Engineering
• Responsibilities:
• > I extract, transform, and load (ETL) data via Azure Data Factory from various sources. Created ETL/ELT pipelines using Azure Data Factory to bring into Azure Databricks workspace for further transformations. Utilized Azure Databricks for raw data transformation (PySpark/SparkSQL).
• > Wrote various stored procedures to automate & implement data quality measures in SSIS and ADF to improve data quality of various different sources.
• > I have designed and implemented data models that support flexible querying and data visualization. Expertise in writing SQL codes and in data-warehousing concepts such as star schemas, slowly changing dimensions, ELT/ETL, and MPP databases.
• > Worked on: XML, CSV, JSON, Parquet, Delta & Binary file formats. Develop Gold layer tables in ADLS Gen 2, part of Synapse layer. Design & apply SSMS transformation logic when needed. Execute ADF master pipeline for data ingestion, creating external tables using pre-built Databricks scripts. Lastly storing output in Synapse Gold (Curated) layer for further automation.
• > Worked on various A/B testing & Regression Analysis projects. Wrote Python scripts using: NumPy, Pandas & SciPy for EDA & statistical analysis. Used Matplotlib & Seaborn for visualisation. Implemented business rules via IICS DG&C, Data Quality & Data Profiling to improve data quality from 64% to 98%.
• > I collaborate & work closely with multiple C-Suites stakeholders across the organization to provide support with regards to AVEVA's Products, product performance & improvement in terms of sales & revenue.

Jan 2017 – Dec 2022 · 5 yrs 11 mos · Guwahati, Assam, India · Remote

• Key skills - Data Analysis, Power BI, Tableau, Excel, SQL, Python Programming language, Azure Data Factory, Azure SQL DB, SparkSQL, Data Modelling, Data Warehousing
• > Analysed, Visualized, and Provided Analytics on data captured daily (> 1M records) using Azure SQL DB query editor, on procurement, sales & revenue inputs to build reporting solutions using Power BI & Tableau.
• > Worked with reporting team to design and develop Data pipelines and Dataflows for reporting needs using Azure Data Factory & reported to stakeholders on Commission process and analysed Sales data every month for improvement, monitored KPIs. Hence, resulting in 11% increase in revenue & peer performance.
• > Worked with Operations Team for effective & smart use of warehouse space, analysed inventory data for 3 months, resulted in significant decrease in warehouse cost by 18%.

Jun 2015 – Aug 2016 · 1 yr 2 mos · Bengaluru, Karnataka, India · On-site

• Key skills - Data Analysis, Power BI, Tableau, Excel, SQL.
• > Wrote SQL queries to extract data from Databases to identify and improve customer requirement process, resulting in a 15% increase in customer retention.
• > Worked in global and cross-functional virtual team's stakeholders to assist with data-related technical issues, analytics, and reporting needs and documented every process, thus saving time by 20%.
• > Designed, Developed, and Managed reports & dashboards using Tableau & Power Bi, reduced dashboard refresh time from 15 minutes to 4.5 minutes by optimizing functions, parameters & queries.