Rishi Anand — AI Researcher

I am a Data Architecture & Engineering leader and Databricks Delivery Solution Architect with 13+ years of experience designing and delivering modern, scalable data platforms for Fortune 500 organizations across Retail Banking, Healthcare, Retail, and Financial Services. Recognized as a Databricks Champion and trusted SME, I help customers solve complex Data problems and accelerate their journey on the Databricks Data Intelligence Platform. With deep expertise in Databricks Lakehouse Architecture, I have successfully architected and led large-scale implementations covering: Data ingestion, transformation, and governance frameworks using Delta Lake, Unity Catalog, and Delta Live Tables (DLT) Real-time streaming pipelines with Structured Streaming and Auto Loader Advanced ETL/ELT frameworks leveraging PySpark and Databricks SQL Performance optimization and cost governance in multi-cloud Databricks environments CI/CD pipelines with Databricks Asset Bundles (DAB), Azure DevOps, and GitHub Actions I partner closely with CDOs, Enterprise Architects, and technical teams to define data strategy, platform roadmaps, and migration blueprints, ensuring seamless onboarding to the Databricks platform while adhering to security, compliance, and governance best practices. My leadership style is customer-obsessed, transparent, and delivery-focused. I have scaled and managed multi-disciplinary teams of Data Engineers, BI Engineers, and Analysts, driving execution for end-to-end enterprise data programs under tight deadlines. Core Skills & Expertise Databricks Platform: Lakehouse Architecture, Delta Lake, Unity Catalog, Delta Live Tables, Structured Streaming, Auto Loader, MLflow, Databricks SQL, DBR tuning Cloud Platforms: Azure (ADF, Data Lake, Event Hub, Logic Apps), AWS (S3, Glue, Lambda – basic) Programming & Frameworks: PySpark, SQL, Python, T-SQL, PL/SQL, Unix scripting Data Architecture: Dimensional Modeling, Data Warehouse & Data Lake Design, Data Governance, Security & Privacy, Orchestration, DataOps DevOps & CI/CD: Azure DevOps, GitHub, Databricks Asset Bundle (DAB), Terraform (basic) Industry Expertise: Retail Banking, Healthcare, Retail, Financial Services,

Stackforce AI infers this person is a Data Engineering expert with a focus on cloud-based data solutions in Fintech and Healthcare.

Location: Bengaluru, Karnataka, India

Experience: 13 yrs 1 mo

Skills

Databricks Platform
Cloud Platforms
Data Architecture
Data Engineering
Data Analysis

Career Highlights

Led large-scale data platform implementations for Fortune 500 companies.
Recognized as a Databricks Champion and trusted SME.
Expert in optimizing data architecture and cloud costs.

Work Experience

Tiger Analytics

Senior Azure Data Architect (1 yr 9 mos)

Azure Data Architect (1 yr 3 mos)

Genpact Digital

Senior Data Engineer (2 yrs 4 mos)

KPMG India

Data Engineer (1 yr 3 mos)

Tata Consultancy Services

Spark Developer (2 yrs 5 mos)

Hadoop Developer (2 yrs 5 mos)

PL/SQL Developer (1 yr 8 mos)

Education

B.tech at National Institute of Technology Silchar

Rishi Anand

AI Researcher

Bengaluru, Karnataka, India13 yrs 1 mo experience

Key Highlights

Led large-scale data platform implementations for Fortune 500 companies.
Recognized as a Databricks Champion and trusted SME.
Expert in optimizing data architecture and cloud costs.

Stackforce AI infers this person is a Data Engineering expert with a focus on cloud-based data solutions in Fintech and Healthcare.

Contact

Skills

Core Skills

Databricks PlatformCloud PlatformsData ArchitectureData EngineeringData Analysis

Other Skills

Agile MethodologiesAnalytical SkillsApache ImpalaApache SparkApache SqoopAzure Data FactoryAzure Data LakeAzure DatabricksAzure Logic AppsAzure Synapse AnalyticsCI/CDClouderaCoachingCost ManagementCross-functional Team Leadership

About

Experience

13 yrs 1 mo

Total Experience

3 yrs 4 mos

Average Tenure

3 yrs

Current Experience

Tiger analytics

2 roles

Senior Azure Data Architect

Promoted

Jul 2024 – Present · 1 yr 9 mos · Bengaluru, Karnataka, India · Hybrid

Cut Cloud Costs by 20% by leading the end-to-end migration from Azure Synapse Analytics to Databricks, optimizing resource utilization and query performance.
Accelerated Data Processing by transitioning 50+ Synapse pipelines to Databricks Workflows and converting 250+ stored procedures from PL/SQL to Databricks Notebooks (SQL/PySpark), improving efficiency.
Delivered Project on Time under tight deadlines by leading a 6-member team, implementing CI/CD pipelines to automate deployment and reduce manual intervention.
Enhanced Data Quality & Reliability by designing a metadata-driven ingestion framework and integrating Great Expectations in Tiger Data Fabric, ensuring robust validation and monitoring.
Enabled Real-Time Data Processing by automating Databricks Workflow Generation and building CDC pipelines with Autoloader & Delta Live Tables, streamlining incremental data ingestion.
Optimized Cloud Spend & Visibility by developing a cost monitoring dashboard using Azure Cost Management API & Databricks, providing actionable insights into cloud expenditures.
Standardized & Scaled Data Ingestion by designing a reusable ADF pipeline, automating data ingestion from SQL, Snowflake, and files into Delta Lake (Silver & Gold layers) with built-in quality and validation checks.

Azure Synapse AnalyticsDatabricksCI/CDData QualityData IngestionDatabricks Platform+1

Azure Data Architect

Mar 2023 – Jun 2024 · 1 yr 3 mos · Bengaluru, Karnataka, India · Hybrid

Genpact digital

Senior Data Engineer

Nov 2020 – Mar 2023 · 2 yrs 4 mos · India · On-site

Better Customer Targeting: Helped PayPal group 1B+ customers and merchants based on their transaction behavior for smarter marketing.
Faster Campaign Launches: Automated data processing to quickly start marketing campaigns across platforms like Facebook, Instagram, Gmail, LinkedIn, and Twitter.
Higher Engagement & Conversions: Improved targeting accuracy, leading to better customer response and increased sales.
Efficient Big Data Processing: Used Spark to process large amounts of data faster and more efficiently.
Scalable & Reliable System: Built a strong ETL pipeline to handle billions of records smoothly without delays.

SparkETLData ProcessingData Engineering

Kpmg india

Data Engineer

Aug 2019 – Nov 2020 · 1 yr 3 mos · Greater Bengaluru Area · On-site

Improved Scalability & Performance: Migrated from Greenplum & DB2 to Spark & Hive, eliminating scalability bottlenecks and enabling high-performance, distributed data processing across large datasets.
Faster Data Processing & Analytics: Optimized Spark transformations and reduced data processing time by 20%, allowing analysts to derive insights much faster.
Optimized Storage & Cost Efficiency: Implemented efficient ORC & Parquet formats with Snappy compression, leading to reduced storage costs and faster data retrieval, improving query performance.
Automated ETL Workflows for Efficiency: Integrated Autosys & Oozie, reducing manual intervention in ETL processes and ensuring seamless data ingestion, transformation, and storage.
Seamless API Data Integration: Enabled real-time and batch data ingestion from REST APIs, improving data availability for analytics and reporting.
Reliable & Scalable Data Architecture: Designed a robust, scalable data pipeline capable of handling large-scale data ingestion and transformation, ensuring long-term system efficiency and growth.

SparkETLData ProcessingData Engineering

Tata consultancy services

3 roles

Spark Developer

Promoted

Feb 2017 – Jul 2019 · 2 yrs 5 mos · India · On-site

Improved Data Processing Speed: Migrating SAS jobs to PySpark on Databricks significantly reduced data processing time, enabling faster reporting and analysis.
Enhanced Scalability & Performance: Leveraged Azure Databricks & PolyBase to handle large-scale loan data efficiently, improving system performance.
Reduced Infrastructure Costs: Moving from the mainframe & SAS to Azure resulted in significant cost savings on licensing and hardware.
Automated Data Workflows: Streamlined job scheduling and execution using Azure Databricks, minimizing manual intervention and improving operational efficiency.
Seamless Cloud Transition: Successfully transitioned legacy SAS workflows to Azure Cloud, ensuring data accuracy, reliability, and compliance.

PySparkAzure DatabricksData WorkflowsData Engineering

Hadoop Developer

Aug 2014 – Jan 2017 · 2 yrs 5 mos · India · On-site

PL/SQL Developer

Nov 2012 – Jul 2014 · 1 yr 8 mos · India · On-site

Developed and optimized PL/SQL queries and stored procedures to improve data retrieval efficiency, reducing report generation time for business users.
Assisted in designing and maintaining database objects (tables, views, indexes) to support core business applications and workflows.
Collaborated with senior developers to debug and resolve data issues, ensuring the accuracy and reliability of business reports.
Automated routine data processing tasks using PL/SQL scripts, saving manual effort for the operations team.
Supported stakeholders by creating ad-hoc queries and reports, enabling faster decision-making.
Participated in database code reviews, learning best practices that improved application performance and maintainability.
Worked closely with business analysts to translate requirements into efficient database solutions.