Pratik Kothari

Data Engineer

New Delhi, Delhi, India2 yrs 9 mos experience

Key Highlights

Achieved significant cost savings through innovative data solutions.
Expert in optimizing distributed systems for enhanced performance.
Passionate about building efficient data infrastructure.

Stackforce AI infers this person is a Data Engineer specializing in scalable data solutions and cloud computing.

Contact

Skills

Core Skills

Data EngineeringCloud Computing

Other Skills

AirflowAlgorithmsAmazon DynamoDBAmazon EC2Amazon QuickSightAmazon S3Amazon Web Services (AWS)Apache KafkaApache SparkApache Spark StreamingAzure DatabricksC (Programming Language)C++Cascading Style Sheets (CSS)Communication

Experience

Delhivery

Data Engineer

Jun 2023 – Present · 2 yrs 9 mos · Noida

Designed and implemented an intermediary GAP layer for Trino with features like quota management (daily/monthly data scan limits) and intelligent query routing across clusters, achieving a cost reduction of ~$5,000/month through off-peak downscaling.
Successfully upgraded Trino from version 405 to 444, leading to enhanced query performance and a significant reduction in S3 GET costs, saving ~$1,000/day.
Automated cluster autoscaling based on workload (queue lengths, memory utilization) and developed mechanisms to detect and terminate inefficient queries, ensuring system stability and optimized resource utilization.
Built a comprehensive Penny Counter Dashboard with an underlying Spark job to consolidate and allocate costs of DWH components (e.g., Kafka, Kubernetes, TiDB, S3) at user and team levels based on several metrics

SQLApache SparkTrinoData PipelinesCloud ComputingAmazon Web Services (AWS)+3