K

Kunal A.

Data Engineer

Allentown, Pennsylvania, United States5 yrs 11 mos experience
Highly StableAI Enabled

Key Highlights

  • Expert in designing scalable data pipelines.
  • Proven track record in data quality assurance.
  • Strong collaboration with cross-functional teams.
Stackforce AI infers this person is a Data Engineering and Data Science expert in the Agritech and Fintech industries.

Contact

Skills

Core Skills

Data EngineeringEtlData Quality AssuranceData WarehousingReal-time Data ProcessingData ScienceMachine Learning

Other Skills

AWS S3AirflowAlgorithmsAmazon S3Amazon Web Services (AWS)Apache AirflowApache KafkaApache SparkBatch ProcessingBig Data AnalyticsBusiness RequirementsChange Data CaptureComputer ScienceData AnalysisData Lakes

About

Data Engineer/Architect with expertise in data warehousing, data lakes, data modeling, analytics, deploying scalable test-driven data pipelines, and collaborating cross-functionally with data scientists, software engineers, and business intelligence analysts. Tech stacks and Databases ranging across complex data architectures, business tools, and hybrid cloud infrastructure. During my tenure at Raven Industries and Life Byte Systems, I actively contributed to the development and deployment of their innovative data-driven solutions by working on several Big Data Technologies. I also collaborated closely with cross-functional teams to design and optimize data analytics, ensuring seamless data flow and integration across systems. These professional experiences have equipped me with a comprehensive skill set encompassing statistics, data visualization, ETL processes, data quality assurance, and automation. I have hands-on experience with tools and technologies such as Python, SQL, Apache Spark, Hadoop, Kafka, Airflow, MongoDB, Snowflake, Docker, MLFlow and managing complex cloud-based infrastructures. Beyond technical prowess, I bring an analytical mindset and a knack for problem-solving to the table. I am keen on staying updated with the latest industry trends and technologies, ensuring that I remain at the forefront of advancements in the field. Collaboration is an essential aspect of my work approach. I excel in interdisciplinary teams, leveraging my organization and strong communication skills to effectively collaborate with stakeholders, data engineers, data scientists, software engineers, and business analysts. QUALIFICATIONS Programming: Python, Java, SQL, Scala, C++, Bash Data: Hadoop, Hive, PostgreSQL, MySQL, MongoDB, S3, Redshift, Iceberg, Elasticsearch, Redis, DBT Distributed systems: Apache Spark, Kafka, Databricks, Kubernetes, Storm, Flink, Snowflake AWS/Azure cloud: S3, EC2, EMR, Lambda, Athena, Glue, Redshift, DynamoDB, Cloudwatch, Kinesis Other: Linux, Docker, Git, Agile, Kibana, Flask, Tensorflow, PyTorch, Salesforce, Tableau, Power BI, Jira, GeoPandas, Terraform, PyTest, MLFlow, Great Expectations, Rest/Fast API, Alteryx, Informatica

Experience

5 yrs 11 mos
Total Experience
1 yr 5 mos
Average Tenure
--
Current Experience

Guardian life

Senior Data Engineer

Jun 2024Present · 2 yrs · Bethlehem, Pennsylvania, United States · Hybrid

Raven industries

2 roles

Data Engineer

Aug 2023Jun 2024 · 10 mos

  • - Utilized AWS S3, Airflow, Spark, and Iceberg to design data pipelines to seamlessly transition an acquired company's manual weed identification process on farmlands into an internally managed, automated, distributed, and scalable environment.
AWS S3AirflowSparkIcebergData EngineeringETL

Data Engineer

May 2023Aug 2023 · 3 mos

  • Enhanced memory management for the Apache Spark framework on EMR clusters, benefiting the entire team.
  • Completed a project to derive, comprehensively validate, and quantify the complexity of agricultural tasks within various terrains, aiding farmers in generating more accurate cost and time estimates for their operations.
Apache SparkEMRData ValidationData EngineeringData Quality Assurance

Colorado state university

2 roles

Graduate Teaching Assistant

Jan 2023May 2023 · 4 mos · Fort Collins, Colorado, United States · On-site

  • CS 481 A5 : Data Mining at Scale

Graduate Teaching Assistant

Aug 2022Dec 2022 · 4 mos · Fort Collins, Colorado, United States · On-site

  • Teaching 3 courses:
  • CS 152 : Python programming
  • CS 163 : Java programming
  • CS 480 : Computer Science Education

Tmgm

Senior Data Engineer

Mar 2019Aug 2022 · 3 yrs 5 mos · Hyderabad, Telangana, India · On-site

  • Led a project to design and create an Amazon S3 Data Lake for TMGM's reporting system migration, importing data from various sources including PostgreSQL, Salesforce, MT4 server, and Amazon RDS.
  • Built ETL pipelines with Presto SQL in AWS Athena for near real-time product and business intelligence.
  • Enhanced data quality assurance through automated data reconciliation and query optimization for client satisfaction and cost control in a fast-paced environment.
  • Streamed Bitcoin’s real-time limit order book data from Coinbase Pro FIX API via Apache Kafka for algo-trading.
  • Analyzed and transformed extensive financial temporal data in a data warehouse, leveraging AWS S3, EMR, Athena, HDFS, Databricks, and Apache Spark.
  • Supported high-frequency market forecasting CNN-LSTM model development in line with DeepLOB technique, utilizing distributed GPU clusters.
Amazon S3PostgreSQLSalesforceETLApache KafkaApache Spark+2

Laalsa

Data Scientist Engineer

Mar 2018Mar 2019 · 1 yr · Hyderabad, Telangana, India · On-site

  • Extracted business insights from raw data (user and geospatial) and built interactive dashboards for customer journey analytics using Elasticsearch, Logstash, Kibana (ELK) stack, boosting user conversion and retention.
  • Developed an NLP text classifier to predict cuisine types on restaurant menus, creating personalized taste profiles and user recommendations, resulting in a 15% improvement in user engagement.
  • Managed on-premise data infrastructure including Apache Kafka, Apache Spark, Elasticsearch, Redis, and MongoDB, using Docker containers.
  • Aided software developers in designing schemas for collecting and storing data from applications to MongoDB.
ElasticsearchLogstashKibanaNLPData ScienceData Engineering

Education

Colorado State University

Master's degree — Computer Science

Aug 2022May 2024

Jawaharlal Nehru Technological University

Bachelor of Technology - BTech — Computer Science

Jan 2015Jan 2019

St. Mary's Junior College

Board of Intermediate — Science

Jan 2013Jan 2015

The Hyderabad Public School, Begumpet

Jan 2003Jan 2013

Stackforce found 100+ more professionals with Data Engineering & Etl

Explore similar profiles based on matching skills and experience