Amit Kumar

Senior Software Engineer

Pune, Maharashtra, India18 yrs 11 mos experience
AI EnabledAI ML Practitioner

Key Highlights

  • 18 years of IT experience with a focus on Big Data.
  • Expert in architecting scalable data solutions on AWS.
  • Proven track record in data modernization and hybrid cloud migrations.
Stackforce AI infers this person is a Big Data Architect specializing in SaaS and enterprise data solutions.

Contact

Skills

Core Skills

Big DataAwsData MeshBusiness IntelligenceReal-time Processing

Other Skills

Data AnalyticsAmazon Web Services (AWS)Apache SparkKubernetesBig Data AnalyticsApache Spark StreamingMicroservicesArtificial Intelligence (AI)Apache AirflowKafkaDatabricksClickHousePower BIEMRSageMaker

About

Experienced Data Engineering Architect with 18 years in the IT industry, including over 12 years specializing in Big Data and Microservices across diverse domains like Retail, Banking, and Automotive. I am passionate about designing and implementing innovative, scalable data solutions that empower businesses to make data-driven decisions. I bring expertise in data mesh, lambda architecture, and data governance, along with hands-on experience in building robust data platforms that support both real-time and batch processing needs. Skilled in hybrid cloud solutions across AWS and Azure, I have a proven track record of architecting solutions that integrate seamlessly into complex enterprise environments. Driven by a commitment to continuous improvement, I thrive in Agile settings, leading cross-functional teams, and fostering CI/CD practices to deliver high-impact results. Currently focused on hybrid cloud migrations and data modernization projects, I’m always eager to explore emerging technologies and best practices to drive business transformation through data.

Experience

18 yrs 11 mos
Total Experience
3 yrs 10 mos
Average Tenure
1 yr
Current Experience

Opengov inc.

Senior Software Engineering Manager

May 2025Present · 1 yr · Pune District, Maharashtra, India · On-site

  • Leading the development of a modern data platform on AWS that powers both real-time streaming and batch analytics across multiple business domains. The platform serves as the foundation for data-driven decision-making and agentic applications that support automation, intelligence, and contextual workflows.
  • Architected scalable data infrastructure to unify and deliver trusted data for operational and strategic reporting.
  • Integrated diverse datasets across domains to ensure consistency, traceability, and compliance.
  • Designed hybrid data pipelines leveraging technologies like Kinesis, Glue, Lambda, Redshift, and Athena to meet low-latency and high-throughput needs.
  • Partnered with engineering, analytics, and product teams to define SLAs, governance, and self-serve data capabilities.
  • Laying the groundwork for AI-powered, agentic systems by enabling contextual and actionable data flows.
Big DataData AnalyticsAmazon Web Services (AWS)Apache SparkKubernetesBig Data Analytics+4

Thoughtworks

Lead Data Engineer

Nov 2020May 2025 · 4 yrs 6 mos · Pune Division, Maharashtra, India · On-site

  • Project: Retail Online and offline -Thailand
  • Objective: Develop a robust data solution based on data mesh principles to support both real-time and batch reporting for offline and online channels. The solution empowers business intelligence insights through Power BI, providing timely and actionable data.
  • Solution: Architected a medallion data structure, enabling the creation of reliable and accessible data products that improve decision-making processes and customer insights across segments. This architecture allows seamless integration of various data sources for comprehensive analytics.
  • Technology Stack:
  • Data Orchestration and Processing: Apache Airflow, Kafka, Databricks
  • Analytics and Reporting: ClickHouse, Power BI
  • Cloud and Machine Learning: EMR, SageMaker
Apache AirflowKafkaDatabricksClickHousePower BIEMR+3

Cdk global

4 roles

Senior Consultant

Promoted

Mar 2017Nov 2020 · 3 yrs 8 mos

  • Ø Real-Time and Big Data Framework – using HDFS, Hive, Scala Spark, Kafka, Spring Boot & Batch, Containers (Docker), Kubernetes.
  • Ø Expertise in converting big monolithic legacy Batch ETL processes into real-time processing with the help of Kafka Stream and Connect and having self-audit capacity using Grafana.
  • Ø Building a Big Data Platform for company-wide use of analytics and data processing.
  • Ø Realtime notifications engine processing millions of events per hour and applying complex algorithms by using the AKKA framework with Kafka.
  • Ø Data ingestion, processing and KPI reporting for multiple products of CDK
  • Responsibilities:
  • Design and Implementation of the CDK's Data Platform based on the Hadoop Ecosystem
  • Data Acquisition Interfaces - REST (microservice) and FTP (Apache Flume) accepting data in
  • various formats (Text, JSON,Avro and Parquet)
  • Data Curation Framework for Realtime Flows using Kafka, Spark-Streaming
  • Workflow management using Oozie
  • Distributing data out of the Lake to different sinks like MongoDB (Reporting Cache), File
  • Exports, etc
  • Designing platform changes to make it more efficient to run on the Cloud - like Separation of Data and Compute Grids, On-Demand Spark Cluster creation for handling Peak Batch Loads with a Smaller Always-On Cluster for AdHoc querying and Realtime flows
  • Development of RESTful Microservices for our reporting layer serving data from Mongo. • Led the Adoption of Spring Boot and Containerization (using Docker) for our 100+
  • Created resilience and elasticity application in AKKA which can handling millions of messages per minute for Realtime notification for the dealer with the help past period 3-month algorithm.
  • Converted 25yr old legacy system to stream processing frameworks via Kafka Connect, Kafka Streams. Used Scala for low level stream API to implement various business rule which was in form on store procedure.
HDFSHiveScala SparkKafkaSpring BootDocker+4

Consultant

Apr 2016Nov 2020 · 4 yrs 7 mos

Senior Member Technical

Oct 2014Jan 2020 · 5 yrs 3 mos

Senior Consultant

Jun 2013Nov 2020 · 7 yrs 5 mos

Adp dealer services

Senior Member Technical

Jun 2013Sep 2014 · 1 yr 3 mos

Infosys technologies ltd

Technical Analyst

Jan 2011May 2013 · 2 yrs 4 mos · Pune/Pimpri-Chinchwad Area

  • (Customer Care Desktop Application) is a web based application to provide Cox Customer Care representative (CSRs) a tool which integrates and consolidates information from multiple systems into a single application. This application is used by most of the CSRs. It displays customer details, network outages, and data from treatment engine and also provides ability to login into ICOMS. iNav contains a series of portlets which provides quick access to an important tool or functions in iNav
  • Responsibilities:
  • ● Interacting with onsite and client to discuss and clarify business requirements
  • ● Discussing and finalizing technical design for requirements.
  • ● Development and Unit testing of the requirement.
  • ● Mentoring and training newly joined on Functionality and Framework.

Persistent systems

Senior Software Engineer

May 2007Jan 2011 · 3 yrs 8 mos · Pune/Pimpri-Chinchwad Area

  • Raptor is a customer and agent experience management application used for call centers. It lets the business owners to design the end to end Experience via sophisticated design tools. The Raptor allows business owners to seamlessly manage the customer conversation from the point a message is received to the delivery of an answer by blending knowledge into workflow
  • Responsibilities:
  • Active participation in requirement gathering from client and coming up with the estimation.
  • Requirement gathering, design and tracking for onshore and offshore tasks.
  • Building POCs as per client requirement in short span of time.
  • Coded the Wavemaker framework changes.
  • Modifications & enhancements into dojo framework to create web 2.0 widgets.
  • Coded business tier for supporting web 2.0 widgets.

Education

Army Institute of Tecnology

Bachelor of Engineering (BE) — Information Technology

Jan 2002Jan 2006

Kendriya Vidyalaya

Jan 1989Jan 2001

Stackforce found 100+ more professionals with Big Data & Aws

Explore similar profiles based on matching skills and experience