Vivek Kumar

Software Engineer

Bengaluru, Karnataka, India4 yrs experience

Key Highlights

Expert in cloud migration and data processing.
Proven track record in developing scalable APIs.
Strong background in real-time data engineering.

Stackforce AI infers this person is a Data Engineer with expertise in cloud-native solutions and real-time data processing.

Contact

Skills

Core Skills

Cloud MigrationData ProcessingApi DevelopmentData-as-a-serviceReal-time Data ProcessingEtl Workflows

Other Skills

AWSAWS EMRAirflowAlgorithmsAmazon Web Services (AWS)Android DevelopmentAndroid SDKAndroid StudioApache AirflowApache KafkaApache SparkBack-End Web DevelopmentBashC++CI/CD

About

Working on Designing and Implementing Software Products that enables Distributed Computing on Big Data through REST APIs Skills: Backend Development, Big-Data, Distributed Systems, Problem-Solving, Agile Development, Functional Programming Programming Languages : Java, Scala, Python, SQL, Bash Frameworks and Libraries : Spring Boot, MicroServices, Apache Spark, PySpark, Apache Kafka Cloud Technology : AWS ECS, EC2, EMR, KMS, S3, Athena, Redshift, DynamoDB, API Gateway, CloudWatch Databases : MySQL, DynamoDB, Redshift Workflow Management : Airflow Tools: Docker, Git, GitHub, GitLab CI/CD, Linux Coursework: Data Structures & Algorithms, Operating Systems, Object-Oriented Programming, DBMS

Experience

Oracle

Application Developer 2

Dec 2024 – Present · 1 yr 3 mos · Bengaluru, Karnataka, India · Hybrid

Cloud Migration of SQL-Based Pipelines to Apache Spark
Technologies: Apache Spark (3.x), SQL, AWS EMR, S3, Airflow, Python, CI/CD, Data Warehousing
Description:
Leading the transformation of legacy SQL-based ETL pipelines into scalable, cloud-native Apache Spark workflows to handle high-volume data processing in near real-time.
Key Contributions:
Migrated over 30+ complex SQL pipelines, each processing 500M+ records daily, to Spark, improving scalability and fault tolerance.
Reduced end-to-end pipeline runtime by 55%, bringing down processing time from ~48 hours to under 22 hours.

Apache SparkSQLAWS EMRS3AirflowPython+4

Mobilewalla

3 roles

Senior Software Engineer

Promoted

Aug 2024 – Nov 2024 · 3 mos · Hybrid

LendBetter API – Data-as-a-Service (DaaS) for Fintech Lending
Technologies: Spring Boot, Java, MySQL (AWS RDS), DynamoDB, Apache Spark, Apache Kafka, Redis
Description:
Designed and maintained a robust DaaS API to deliver real-time financial insights and credit scores, empowering fintech lenders to make faster, data-driven lending decisions.
Key Contributions:
Developed scalable API endpoints to manage and deliver real-time credit scoring and risk features.
Integrated with third-party data providers for batch ingestion and downstream processing using Spark and Kafka.
Improved API performance by 35% through intelligent caching (Redis) and SQL query optimization.

Spring BootJavaMySQLDynamoDBApache SparkApache Kafka+3

Software Engineer

Jun 2022 – Sep 2024 · 2 yrs 3 mos · Hybrid

Scalable Data Processing API for Real-Time ETL Requests
Technologies:
Apache Spark, Apache Kafka, Java, Spring Boot, AWS (EMR, Redshift), Airflow, Data Warehousing, Data Lake
Description:
Built and maintained a scalable API and Spark-based processing layer to handle real-time, high-volume ETL requests. The system dynamically triggered AWS EMR clusters based on request parameters, with robust execution monitoring and alerting mechanisms.
Key Contributions:
Engineered and optimized a multi-terabyte data processing pipeline to support dynamic ETL workflows.
Achieved 40% cost savings by implementing efficient data partitioning, leveraging AWS spot instances, and using EMR instance fleets for flexible compute resource allocation.
Integrated orchestration through Airflow to streamline task execution and improve system reliability.