Jan 2022 – Jul 2022 · 6 mos · Greater Toronto Area, Canada

Aug 2019 – Oct 2021 · 2 yrs 2 mos · Toronto, Ontario, Canada

Oct 2021 – Jan 2022 · 3 mos · Greater Toronto Area, Canada

Dec 2018 – Aug 2019 · 8 mos · Toronto, Ontario, Canada

• Responsible for:
• 1. Designing and Implementing Azure Data Factory v2 pipeline to incrementally pull data from Salesforce
• 2. Leveraged Azure Databricks, PySpark to dynamically pull Salesforce object metadata to develop schema agnostic(schema drift) ADFv2 pipelines and additionally perform reads and writes to ADL Gen2
• 3. Implemented vertical partitioning in Azure Data Lake Gen2 for Salesforce data
• 4. Databricks Views over parquet files help catalog parquet files per entity
• 5. Using Databricks views Power BI pulls Salesforce data to perform analytics
• 6. In evaluation is Databricks Delta Lake to improve parquet read performance and still achieve efficient incremental pull
• The current reporting is performed over Azure SQL Database. The idea is to eliminate SQL foot print and instead perform reporting over Azure Databricks views/ ADL Gen2.
• In nutshell:
• Salesforce ->(ADFv2, Azure Databricks, PySpark) -> ADL Gen2-> Azure Databricks Views ->Power BI
• Challenges faced:
• 1. Evaluated various JDBC options for connecting Azure Databricks to Salesforce
• , none were performant or were expensive
• 2. Establishing Databricks connectivity with on-prem SQL boxes
• 3. Performing vertical partitioning in Databricks
• 4. Handling Delta Lake tables and performing SCD changes in Databricks
• 5. ADFv2 billing nuances and failed pipeline runs

Mar 2014 – Aug 2018 · 4 yrs 5 mos · Hyderabad Area, India

• Developed a large volume Data Lake leveraging Cosmos Scope scripts(predecessor of Azure Data Lake Store and U-SQL) for transforming raw input data into Dimensional Model data for analyzing patterns and trends
• Worked on prototype for migrating Cosmos data and scope scripts to Azure Data Lake Store and U-SQL scripts with minimal impact to customers
• Developed PowerBI reports to analyze data sourced from Data Lake

Aug 2012 – Feb 2014 · 1 yr 6 mos · Hyderabad Area, India

• Worked for Pramati's Imaginea division, later acquired by Accenture, India
• Twitter Sentiment Analysis, Leveraged Twitter API, HDInsight, SQL 2012 and SharePoint 2013. Implemented Dot Net application to push twitter data present in flat files to HDFS file system. Developed MapReduce program to perform sentiment analysis on data present in HDFS. Implemented Dot Net application to organize data in HDFS file system in order to help automate downstream SSIS loads.
• For an insurance client, developed dimension and fact load SSIS 2012 packages, SSRS 2012 and PPS reports. Installed and configured 3-tier and 2-tier SharePoint 2010 BI farm using Kerberos authentication leveraging Excel, PPS and SSRS service applications single-handedly.

Jan 2011 – Jul 2012 · 1 yr 6 mos · Hyderabad Area, India

• 1. Implemented prototype to fetch metadata from report definition files which includes details of the report properties, reportitems in the report etc using ReportingService2010 endpoint and proxy class generated from RDL XSD.
• 2. Involved in complete Data Quality Services 2012 evaluation with a practical scenario focusing on all the areas of DQS 2012
• 3. Involved in developing BI analytics solution using SQL Server 2008 R2 and SSRS 2008 R2 in SharePoint Integrated Mode
• 4. Developed MobileBI solution that was showcased on Avanade Showcase website

Oct 2008 – Jan 2011 · 2 yrs 3 mos · Bangalore

• Involved in developing reporting solution using MSBI suite.

Jul 2007 – Oct 2008 · 1 yr 3 mos · Bangalore

• Developed ETL and Reporting solution.