Aman Soni

Data Engineer

Bengaluru, Karnataka, India5 yrs 6 mos experience

Key Highlights

  • Engineered big data frameworks for diverse clients.
  • Achieved significant operational efficiency improvements.
  • Led teams in agile-driven data engineering projects.
Stackforce AI infers this person is a Data Engineering expert in Healthcare and Retail sectors.

Contact

Skills

Core Skills

Data EngineeringBig Data AnalyticsProcess AutomationData ModelingCloud MigrationData AnalyticsEtl Development

Other Skills

Apache SparkPythonbashAmazon AthenaSparkSQLSQLContinuous Integration and Continuous Delivery (CI/CD)AWSAirflowSnowflakeAWS LambdaAutomationData ProcessingMicrosoft ExcelAnalytical Skills

About

As a seasoned Data Engineer with over 4 years of experience, I excel in crafting robust data pipelines and platforms to drive analytics and data-driven insights. My expertise spans across managing and processing vast datasets, optimizing system performance, and implementing cost-effective solutions. Throughout my career, I have successfully executed ETL processes handling over 200+ TB of data, significantly reducing latency, enhancing storage, and minimizing error rates. I have engineered a robust big data framework processing, enhancing data analysis capabilities and providing analysis-ready data from 30+ sources for diverse pharmaceutical clients. My initiatives in production deployments and optimizing runtime achieved a 50% reduction, while automation of data tasks using AWS Lambda led to a 60% improvement in operational efficiency, cutting data processing time by 50 hours per week. Have led a team in agile-driven enterprise implementations, I have streamlined analytics and reporting processes, boosting data accessibility by 35%. My expertise in dimensional and multi-dimensional modeling, coupled with automating DQ Checks and report generation, resulted in $500K savings and reduced operational overhead by 70%. I am passionate about leveraging data to uncover insights and drive strategic decisions. I thrive in dynamic environments, constantly seeking innovative solutions to complex data challenges. Let's connect to explore how I can contribute to your organization's success with my data engineering expertise. Skills: - Programming Languages: SQL, Python, PySpark, Spark, Spark SQL - Cloud Platforms: AWS, GCP, Azure Data Lake - Scripting Languages: Bash, C++, Python, HTML - Data Warehousing: Snowflake, Amazon Redshift, Presto, Elasticsearch - ETL/ELT Tools: Microsoft Excel, Jira, Confluence, Azure Ops, Git, Gitlab, Bitbucket, Airflow - Competencies: ETL/ELT Development, CI/CD, Process Automation, Data Modeling, Business Analytics, Data Warehousing, Data Analytics, Data Reconciliation, Data Normalization/De-normalization, Data Validation, Data Management, and Testing Feel free to connect with me or message me to discuss potential opportunities or collaborations. I look forward to expanding my network and contributing to data-driven initiatives.

Experience

5 yrs 6 mos
Total Experience
1 yr 5 mos
Average Tenure
1 yr 2 mos
Current Experience

Databricks

Senior Analytics Engineer

Mar 2025Present · 1 yr 2 mos · Bengaluru, Karnataka, India · Hybrid

Siteminder

Senior Data Engineer

Oct 2024Mar 2025 · 5 mos · Remote

Concertai

Data Engineer

Sep 2023Oct 2024 · 1 yr 1 mo · Bengaluru, Karnataka, India · Hybrid

  • Engineered a robust big data framework to process high volumes of data weekly and monthly, significantly enhancing data analysis capabilities for diverse clients.
  • Directed production deployments, data loads, and testing (SIT and UAT), achieving a 50% reduction in run-time.
  • Built enterprise big data ingestion and processing solutions using AWS (Glue), Airflow, Azure (Databricks with ADF), and Snowflake.
  • Pioneered automation of data tasks with AWS Lambda, improving operational efficiency by 60% and reducing data processing time by 50 hours per week.
  • Enhanced data quality and SQL performance by optimizing table structure for scalability, reducing runtime by 30%, and generating comprehensive reports for execution time and cost analysis.
  • Facilitated the automation of data cleansing and enrichment processes, improving data accuracy and usability for analytics purposes.
Data EngineeringApache SparkPythonbashBig Data AnalyticsAmazon Athena+3

Zs

2 roles

Associate Consultant - Data Engineering

Jan 2023Sep 2023 · 8 mos · Hybrid

  • Architected a large-scale commercial data platform for a pharmaceutical client, managing 200+ TB of data across 50+ sources via Snowflake, boosting data accessibility by 35%.
  • Led a 4-member team in agile-driven enterprise implementations for Healthcare/Pharma clients, overseeing the entire project lifecycle from requirement gathering to rollouts.
  • Demonstrated proficiency in dimensional and multi-dimensional modeling, building logical data models.
  • Streamlined DQ Checks automation and report generation within data pipelines, reducing operational overhead by 70%, resulting in $500K savings.
  • Migrated traditional on-premises ELT pipelines to AWS cloud using technologies like EMR, RDS, Airflow, Control-M, and S3/ADLS, and implemented ML-Ops with Dataiku.
  • Innovated a robust streaming data solution with AWS Kinesis Data Streams, Firehose, and Analytics, enhancing real-time data processing efficiency by 35%.
  • Enhanced system performance by 55% through effective utilization of BigQuery.
  • Collaborated with cross-functional teams to ensure seamless data ingestion, development, and testing for dashboards on Tableau and MicroStrategy.
Microsoft ExcelAnalytical SkillsProblem AnalysisGitProject ManagementAirflow+30

Business Technology Analyst - Data Engineering

Oct 2020Dec 2022 · 2 yrs 2 mos · Hybrid

  • Executed and orchestrated an automated ETL process/data pipeline, resulting in a 60% increase in customer retention and 75% sales growth.
  • Developed modules for insights and KPI calculations using AWS Services (Lambda, S3, EC2, DynamoDB, Redshift, SNS, step functions) and Python for retail Buying Engine.
  • Leveraged Spark applications with Amazon Kinesis to ingest patient-related data into repositories like Hive, Cassandra, and HBase.
  • Engineered a comprehensive DQM system and automated report framework, achieving a 20% reduction in data errors and a 90% decrease in operational costs.
  • Designed and implemented data analytics tools, uncovering vital insights and increasing campaign effectiveness by 45%.
  • Developed APIs using AWS API Gateway for real-time data extraction, improving data ingestion rate by 40%.
Microsoft ExcelAnalytical SkillsProblem AnalysisGitProject ManagementAirflow+28

Siemens healthineers

Software Development Intern

Jul 2019Jan 2020 · 6 mos · Greater Bengaluru Area · On-site

PythonCommunication

Persistent systems

Project Trainee

Jun 2019Jul 2019 · 1 mo · Greater Nagpur Area · On-site

PythonCommunication

Indian institute of technology, guwahati

Research Intern

May 2018Jul 2018 · 2 mos · Dispur, Assam, India · On-site

  • Conducted research on algorithms for 3D integrated circuits and Through-Silicon Vias (TSVs) assignment, contributing to advancements in the field of electrical design and packaging.
  • Collaborated with a team of researchers to develop a shortest path algorithm for 3D ICs, optimizing the TSV assignment process.
  • Analyzed and interpreted complex data sets to enhance the efficiency and performance of integrated circuits.
  • Led the publication of findings in the IEEE Electrical Design of Advanced Packaging and Systems (EDAPS) conference.
PythonCommunication

Education

Indian Institute of Information Technology Nagpur

Bachelor of Technology - BTech — Electronics and Communication Engineering

Jan 2016Jan 2020

Central Board of Secondary Education

Senior Secondary Certificate

Apr 2014May 2015

Stackforce found 100+ more professionals with Data Engineering & Big Data Analytics

Explore similar profiles based on matching skills and experience