Saket Harsh

Senior Software Engineer

Bengaluru, Karnataka, India6 yrs 6 mos experience

Highly Stable

Key Highlights

Expert in optimizing on-device AI execution.
Proven track record in building scalable distributed systems.
Strong foundation in algorithms and mathematics.

Stackforce AI infers this person is a highly skilled Systems Engineer specializing in AI and Cloud Computing.

Contact

Skills

Core Skills

On-device Machine LearningSystems EngineeringDistributed SystemsCloud ComputingSoftware Development

Other Skills

AlgorithmsAzure Service FabricBack-End Web DevelopmentCC++Cloud-Native ApplicationsComputer ScienceFunctional ProgrammingGNU DebuggerGitGithubGo (Programming Language)Iterative Closest Point AlgorithmsLinuxMicroservices

About

Driven by a passion for building high-performance systems, I specialize in optimizing on-device AI execution to bring machine learning closer to the edge. As a Systems Engineering Lead at NimbleEdge, I focus on designing and implementing efficient systems that enable machine learning models to run seamlessly on mobile devices. My work involves deep interaction with ML runtimes, ensuring faster inference times and optimal resource utilization. Previously, I was a Software Engineer 2 at Microsoft Azure, where I contributed to large-scale distributed systems, managing mission-critical production workloads. This experience strengthened my expertise in building resilient, scalable architectures capable of handling immense computational demands. My technical proficiency spans across C++, Golang, and C#, with a strong foundation in distributed systems and on-device machine learning. Additionally, I have a keen interest in algorithms and mathematics, which helps me approach problem-solving with a rigorous and analytical mindset. I thrive on solving complex engineering challenges at the intersection of systems architecture, AI acceleration, and performance optimization. Always eager to explore cutting-edge advancements in systems engineering and AI, I welcome opportunities for collaboration and discussion.

Experience

Coram ai

Senior Software Engineer

Apr 2025 – Present · 11 mos · Bengaluru, Karnataka, India

Nimbleedge

Systems Engineering Lead

Dec 2022 – Feb 2025 · 2 yrs 2 mos · Bengaluru, Karnataka, India

Bootstrapped the engineering team from scratch, built out platform for on-device machine learning.
Built custom light weight Python interpreter in C++ that can be executed on mobile devices.
Worked with ONNXRuntime and TFLite C/C++ APIs to provide seamless APIs on Android via JNI and NDK Toolchain.
Integrated ONNXRuntime for GenAI to provide support for structured streaming LLM responses.

ONNXRuntimeTFLite C/C++ APIsPythonC++On-device Machine LearningSystems Engineering

Microsoft

2 roles

Software Engineer II

Sep 2021 – Nov 2022 · 1 yr 2 mos

Enabled Azure Service Fabric clusters to be hosted on Spot instances ,leading to lower evictions rate, and 10x fewer interruptions on clusters for thousands of Azure customers.
Streamlined the placement of stateless microservices on Service Fabric clusters by creating specific Stateless only host node types, which perform 10x faster in scale-in/out and can scale up to 1000 instances.

Azure Service FabricSpot InstancesStateless MicroservicesDistributed SystemsCloud Computing

Software Engineer

Jun 2019 – Aug 2021 · 2 yrs 2 mos

Simplified OS upgrade process on large clusters, by rewritine a service which considered all production service health, traffic patterns, and timely maintenance window constraints. The service got instantly adopted by large scale teams like Office and Microsoft Teams.
Established service-to-service communication between Compute and Cluster resource services of Azure, during host nodes allocation, resulting in reduced Node up time from 5 mins to 2 mins.

Samsung r&d institute india - bangalore private limited

Software Development Intern

May 2018 – Jul 2018 · 2 mos · Bengaluru Area, India

Built a C++ tool to construct skeleton from a 3D Point Cloud frame using Principal Component Analysis and Iterative Closest Point Algorithms
Devised an O(n) clustering algorithm to classify points on the basis of their normal distance from the skeleton to identify body parts

C++Principal Component AnalysisIterative Closest Point AlgorithmsSoftware Development