Benny Kiwin

Data Engineer

Chennai, Tamil Nadu, India6 yrs 11 mos experience

Key Highlights

Expert in building complex data pipelines.
Proficient in Snowflake and PySpark.
Strong background in data automation and reporting.

Stackforce AI infers this person is a Data Engineering expert with a strong focus on automation and data pipeline optimization.

Contact

Skills

Core Skills

SnowflakePythonDagsterAwsPysparkApache Kafka

Other Skills

SQLDBTSAP HanaTableauDockerHDFSC/C++MySQLKSQLksqlDBAmazon Web Services (AWS)AirflowC (Programming Language)Shell ScriptingMachine Learning

About

Data Engineer with 5+ years of experience in developing Dagster pipelines, Spark jobs, solving complex problems. Proficient in Snowflake, Pyspark, Kafka. Holds B.Tech degree from College of Engineering, Guindy. I am also an aspiring screenwriter!

Experience

6 yrs 11 mos

Total Experience

2 yrs 8 mos

Average Tenure

1 yr 7 mos

Current Experience

Lseg (london stock exchange group)

Data Engineer

Nov 2024 – Present · 1 yr 7 mos · Hybrid

Cisco

Data Engineer

Jun 2022 – Nov 2024 · 2 yrs 5 mos · Bengaluru, Karnataka, India · Hybrid

Worked on creating a complex pipeline for transporting around 515 GB data from SAP Hana to Snowflake, reducing the latency and manual intervention by 50%.
Created Stored Procedures in Snowflake for automating report generation, reducing effort by 2x.
Utilized Snowflake, Dagster/Airflow, Python3, SQL, DBT, SAP Hana, Gen AI Wrappers, AWS and Tableau.
Developed a DAG of Dagster jobs, schedules, and sensors for telemetry data analysis using Python and SQL, optimized queries for efficiency, with audits, automated alerts, and High-Availability deployment incl. switchover mechanisms.
Developed a module to generate PPT reports on a monthly basis using Dagster, Docker, SQL and Python.
Automated the flow in SQL Server dtsx packages using Dagster and Python to reduce manual effort by 75%.
PoC for PPT table parser, Tool‑calling LLM Wrapper, and data transformations using DBT.
PoC for data pipeline from S3 to Snowflake ‑ Used AWS services such as S3,SQS,Lambda and EC2.

SnowflakeDagsterPythonSQLDBTSAP Hana+3

Tata communications

2 roles

Senior Software Engineer

Promoted

Oct 2021 – Jun 2022 · 8 mos

Developed an optimized Ingestion module, increasing efficiency by 50% in DDOS Alert Detection module.
Improved DDOS alert detection using complex Spark queries, reducing alert trigger time by 30%.
Utilized Python3, HDFS, PySpark (incl. Streaming), Pandas, Apache Kafka, C/C++, MySQL, YARN and Cloud‑
era Data Platform.
POC for capturing live network traffic and applying probabilistic data structures to detect alerts from parsed packet details.
Deployed distributed collectors collecting Netflow data in parallel and processing them in sync to generate alerts, which enabled High Availability to the platform.
Added a Kafka consumer module in GoBGP open source code to process (add/remove) route entries.
POC for ingesting data (CDC data) from MySQL table to Kafka topic using Kafka Connect.
Added modules pertaining to data enrichment of raw netflow data in open source nprobe written in C and stream the data to Kafka.