Abhas Upadhayay

Software Engineer

Bengaluru, Karnataka, India7 yrs 11 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Pioneered innovative solutions in data engineering.
Secured significant deals through impactful project delivery.
Optimized complex data workflows for enhanced performance.

Stackforce AI infers this person is a Data Engineering expert with a strong focus on cloud-based solutions and performance optimization.

Contact

Skills

Core Skills

SparkGcpData EngineeringApi Development

Other Skills

Protocol BuffersDataprocAirflowNoSQLDockerPysparkFastAPISolrElasticSearchYarnhadoopScalaData AnalysisHiveUnix

About

Results-driven Software Engineer with a versatile skill set encompassing data engineering, ETL, and expertise in a broad range of technologies. Demonstrated success in architecting and implementing large-scale distributed systems, driving efficiencies, and delivering impactful solutions. Proven ability to tackle complex business requirements, optimize data processing workflows, and maintain a strong focus on performance excellence.

Experience

7 yrs 11 mos

Total Experience

2 yrs 7 mos

Average Tenure

3 yrs 6 mos

Current Experience

Walmart global tech india

Software Engineer 3

Dec 2022 – Present · 3 yrs 6 mos · Bengaluru, Karnataka, India

Pioneered a client-centric feature leveraging Spark streaming and dataproc in gcp empowering advertisers with greater control over product selection for campaigns, optimizing their advertising experience and increasing campaign performance by 20%
Revamped and stabilized legacy Spark and Airflow jobs, resolving persistent stability issues and saving valuable developer time. Implemented an innovative solution to address IP exhaustion in Dataproc jobs, introducing a dynamic framework to intelligently schedule jobs in GCP regions with available IPs, significantly reducing job failures.

Protocol BuffersDataprocSparkGCP

Spoonshot

Senior Software Engineer

Jul 2021 – Dec 2022 · 1 yr 5 mos

Engineered scalable data pipelines and API endpoints for the client-facing "Concept-Generator" feature, using Pyspark, FastAPI, Solr, and ElasticSearch, resulting in a $91k deal with Pepsi.
Resolved performance bottlenecks in Spark pipelines, leveraging Spark-UI metrics reducing batch processing time from 2 hours to an impressive 12 minutes.
Elevated throughput across 7 pipelines for 3 teams by addressing Kafka rebalancing and consumer timeout issues, ensuring optimal performance of the Kubernetes-managed Kafka cluster.

NoSQLDockerData EngineeringAPI Development

Target

Data Engineer

Jul 2018 – Jul 2021 · 3 yrs

Engineered scalable Spark (Scala) pipelines, unlocking analytics on financial assets data and facilitating the generation of 5 critical finance reports.
Leveraged data analysis skills and Spark (Scala) to automate store processes, saving developers 2 hours of effort and providing enhanced visibility into stock movement between stores.

YarnhadoopData EngineeringSpark