Krishna Kumar

Backend Engineer

Canada6 yrs 11 mos experience

Highly Stable

Key Highlights

Reduced fraud rate from 20% to less than 2%
Achieved 45% reduction in platform costs
Pioneered real-time data solutions for critical systems

Stackforce AI infers this person is a Data Engineering expert specializing in scalable cloud solutions for Fintech and SaaS industries.

Contact

Skills

Core Skills

Data EngineeringAws Cloud ServicesCloud InfrastructureSoftware Engineering

Other Skills

.NET FrameworkASP.NETAWSAWS API GatewayAWS ECS FargateAWS RedshiftApache AirflowApache FlinkApache KafkaApache SparkAzure DatabricksBack-End Web DevelopmentBig DataC#CI/CD

About

Highly accomplished Senior Data Engineer with over 9 years of experience architecting, building, and optimizing large-scale, distributed data platforms within the Data Management Domain. I am dedicated to enabling data-driven decisions and empowering businesses through robust data solutions. My proven ability includes leading complex technical initiatives from conception to deployment, spearheading core data platform infrastructure development, integrating essential data assets and third-party systems, and delivering high-impact, real-time data solutions. I possess deep expertise in technologies such as Apache Flink, Spark, Kubernetes, Kafka, AWS cloud services, and modern data warehousing. Throughout my career, I have been instrumental in driving significant business impact and cost savings, evidenced by achievements like eliminating over $1M in vendor unit pricing and achieving approximately 45% reduction in platform costs. Equally important is my ability to cultivate strong stakeholder relationships and translate technical capabilities into valuable analytical insights that drive business functions. Passionate about bridging the gap between data and software engineering, I leverage my comprehensive expertise to deliver innovative, scalable data solutions that foster measurable business growth and success. I am seeking challenging opportunities to apply my deep technical skills and strategic platform vision. Let's connect to explore how we can collaborate on impactful projects in the realm of data-driven technologies.

Experience

6 yrs 11 mos

Total Experience

3 yrs 5 mos

Average Tenure

Current Experience

Mistplay

Senior Data Engineer II

May 2023 – Present · 3 yrs 1 mo · Barrie, Ontario, Canada · Remote

Pioneered the foundational real-time data pipelines for a mission-critical fraud detection system, cutting data latency from 24 hours to <5 minutes. This work directly enabled the ML models to reduce the company's fraud rate from 20% to less than 2%.
Redirected SageMaker inference logs to an async Kinesis stream, shrinking response payloads 70 % and slicing p99 latency from 500 ms to 200 ms.
Engineered and implemented advanced compression techniques in Redis, reducing critical feature store operational costs by 65% annually (saved $350k/yr).
Drove platform-wide cost optimization initiatives, reducing overall data platform costs by approximately 45% through strategic infrastructure tuning, optimizing CDC pipeline efficiency, and refining SQL warehouse usage patterns.
Architected the end-to-end feature streaming platform that feeds Mistplay's recommendation engine. The design for sub-minute latency data processing was critical for the regular re-training and performance of the core recommendation models.
Advised board‑level and CTO forums alongside principal engineers, steering a telemetry‑first backend redesign that’s on track to trim AWS + Databricks spend by 20 % and unlock org‑wide observability.

Apache FlinkSparkKubernetesKafkaAWS cloud servicesreal-time data solutions+7

Udacity

2 roles

Data Engineer III

Promoted

Aug 2021 – May 2023 · 1 yr 9 mos · Noida, Uttar Pradesh, India

Lead design and execution of a Unified Data Pipeline Infrastructure Platform on Kubernetes to unify hosting of fault tolerant data pipeline technologies and standardization of deployments. Reduced AWS platform costs by 30% as a result.
Lead design, implementation and review efforts to migrate legacy Apache Airflow cluster from version 1.x to 2.1 using AWS ECS Fargate for computing, ECR, S3, implemented application security architecture using security groups and ACL’s, networking (VPC, Gateways, Endpoints). IaaS code management using Terraform and robust CI/CD deployment pipelines using CircleCI.
Design and implementation effort to integrate analytics engineering tool (dbt) into the new Airflow cluster. The solution now enables citizen analysts to build Data Marts using Golden (data) Layer. Enabled analysts and data scientists to be more productive by allowing them to focus on writing business logic in SQL while adhering to Software Engineering best practices.
Lead the effort to design and implement a centralized User Profile message bus to enable downstream third-party vendors to get a 360-degree view of users from a single source of truth. Use of Apache Spark, protobuf as message format and Apache Kafka on AWS.
Architected and implemented a middleware proxy microservice in Golang to serve AWS Sagemaker ML data science models to backend microservices using GraphQL. Use of AWS API gateway to manage interaction between the GraphQL API and Sagemaker ML Endpoint.
ETL pipeline development and maintenance via Apache Airflow. Workflows involved processing millions of records stored centrally within Data Lake built on AWS S3 using Apache Spark on AWS EMR. Expertise in Spark performance tuning, resulting in significant improvement in processing time and resource utilization.
Automated builds and deployments using CircleCI (Optimized build run times down to 2 mins from 35+ minutes), Docker and Terraform. Built data products and services using Go and Python.

KubernetesApache AirflowAWS ECS FargateTerraformCircleCIApache Spark+6

Data Engineer II

May 2020 – Aug 2021 · 1 yr 3 mos · Noida, Uttar Pradesh, India

Data Engineer, contributed towards building the centralized data ecosystem to enable data driven decisions at Udacity. ETL pipelines developed and maintained via Apache Airflow. Experience with cloud warehouses such as AWS Redshift.
Workflows involved processing millions of records stored centrally within Data Lake built on AWS S3 using Apache Spark on AWS EMR. Building dashboards on Chartio (Bi Tool) using AWS Athena as the query layer.
Automated builds and deployments using CircleCI (Optimized build run times down to 2 mins from 35+ minutes), Docker and Terraform. Built data products and services using Go and Python.

Apache AirflowAWS RedshiftApache SparkCircleCIDockerTerraform+1

Indus valley partners

2 roles

Software Engineer

Promoted

Apr 2019 – May 2020 · 1 yr 1 mo

Software Engineer, implemented ETL backend systems on top of Apache Spark via AWS/Azure Databricks for Cloud Big Data Management, led re-architecture of the above system that now scales to serve 10 million records and above in production.
Responsibilities included mentoring of junior team members, implementing best practices such as Static Code Analysis, Integration testing, Automated CI/CD build flows and refactoring legacy code.

Apache SparkAWSAzure DatabricksStatic Code AnalysisCI/CDSoftware Engineering

Associate Engineer

Jun 2016 – Apr 2019 · 2 yrs 10 mos

Experience in Product consultancy and solutions to Alternate Asset Management institutions (Hedge Funds, Private Equity Firms) with projects involving custom implementations in the form of Business rules, pipeline management, interfaces to Geneva accounting system and DevOps.

Defence research and development organisation (drdo)

Intern

May 2015 – Jun 2015 · 1 mo · Chennai

Software Developer Intern at DRDO - Research and Innovation Centre, IIT Madras. I was a part of a team which was responsible in creating a GUI for a audio processing software. My main task was to create a elegant, robust and ready to use GUI. The IDE chosen was Netbeans 8.0.2 in Ubunutu 14.04 operating system.