Vamsi Krushna Lingala

Software Engineer

Bengaluru, Karnataka, India13 yrs 9 mos experience

Most Likely To Switch

Key Highlights

Expert in building scalable data systems for analytics.
Proficient in optimizing data ingestion and processing pipelines.
Experienced in integrating AI/ML workflows with data governance.

Stackforce AI infers this person is a Data Architect specializing in scalable data systems and event-driven architectures.

Contact

Skills

Core Skills

Data ArchitectureData EngineeringEvent Streaming

Other Skills

Agile MethodologiesAirflowAmazon Web Services (AWS)Apache KafkaApache PigApache SparkArchitectsBig DataData AnalysisData IngestionData IntegrationElasticsearchGitGoogle Cloud Platform (GCP)Graph Databases

About

I’m a Data Architect building scalable, observable, and privacy-safe data systems for analytics and AI/ML. My work focuses on modernizing data management by integrating a Data Lake with a robust pipeline platform for efficient processing and analytics. I’ve also designed a custom Spark Profiler to optimize performance and resource efficiency. I’ve worked extensively on large-scale clickstream platforms (~3B events/day)—building ingestion and transformation pipelines, data quality controls, and analytics-ready star schema models (facts/dimensions) including session metrics, CTA impression→click linkage, attribution, and platform/product/location segmentation. More recently, I’m worked on Agentic AI to automate data workflows—such as PII discovery/remediation, query optimization, data model recommendations, and self-serve analytics enablement—while keeping governance, reliability, and cost-efficiency at the core.

Experience

O9 solutions, inc.

Software Architect

Jul 2023 – Present · 2 yrs 8 mos

This project establishes a scalable data ecosystem that integrates the o9 Data Lake with an interactive drag-and-drop pipeline platform. Designed for efficient data ingestion, processing, and analytics, the platform enables users to seamlessly create and manage ETL workflows. A custom-built Spark Profiler enhances the ecosystem by providing detailed monitoring and optimization capabilities, ensuring high performance and resource efficiency for all Spark tasks. The architecture prioritizes scalability, data consistency, and governance compliance, supporting a robust environment for real-time data insights and operational excellence.
Responsibilities
● Defined and implemented architecture for integrating the o9 Data Lake and pipeline platform.
● Built and integrated a Spark Profiler for monitoring and optimizing Spark job performance.
● Architected the data lake for large-scale data ingestion, transformation, and storage with high
availability.
● Optimized data ingestion pipelines and workflows, focusing on resilience and efficiency.
● Identified bottlenecks and optimized resource allocation using the Spark Profiler.
● Provided technical guidance to teams on Spark and data lake best practices.
● Aligned solutions with business objectives, compliance standards, and governance needs.

icebergTypeScriptData ArchitectureData Engineering

Chingari®

Lead Data Engineer

Jan 2022 – Nov 2022 · 10 mos · Bengaluru, Karnataka, India · Hybrid

Built a high-performance, scalable, and durable event streaming platform capable of ingesting gigabytes of events per second
from multiple sources. The system processes data in near real-time, providing instant insights for event-driven applications. This platform
ensures that data is available in milliseconds, enabling businesses to react quickly to changing events.

icebergTypeScriptData EngineeringEvent Streaming