P

Piyush Daga

Senior Software Engineer

Bengaluru, Karnataka, India7 yrs 9 mos experience
Most Likely To SwitchAI Enabled

Key Highlights

  • Expert in Data Engineering and Machine Learning.
  • Proven track record in building scalable data pipelines.
  • Strong background in software development and API integration.
Stackforce AI infers this person is a Data Engineering and Machine Learning specialist in the SaaS industry.

Contact

Skills

Core Skills

Software DevelopmentBackend DevelopmentData EngineeringApi DevelopmentData Pipeline DevelopmentData ScienceMachine LearningEmbedded Systems

Other Skills

API ObservabilityAWS EKSAWS KinesisAmazon RedshiftAmazon Web Services (AWS)AnalyticsApache BeamApache SparkArtificial Intelligence (AI)Big DataC (Programming Language)C++Coding ExperienceComputer ScienceComputer Vision

About

Engineer by day. Writer also by day 😬. Reader at night ✓. Sometimes funny. Pouring my thoughts in articles and tweets. Find me: https://linktr.ee/piyush_daga

Experience

7 yrs 9 mos
Total Experience
2 yrs 7 mos
Average Tenure
6 yrs 2 mos
Current Experience

Postman

3 roles

Senior Software Engineer

Promoted

Jun 2022 – Present · 4 yrs

  • Building Live Collections Backend to automatically update collections from traffic data.
  • Building the Postman SDKs to capture, batch and send live traffic data from users' systems.
  • Skills: TypeScript · Kubernetes · OpenTelemetry · AWS Kinesis · Redis · MySQL
TypeScriptKubernetesOpenTelemetryAWS KinesisRedisMySQL+2

Data Engineer - II

Promoted

Dec 2020 – Jun 2022 · 1 yr 6 mos

  • Working on API Observability and Discovery.
  • Self-hosted Airflow on Kubernetes (AWS EKS) to schedule, monitor and debug DBT jobs.
  • Built an integrations platform that brings in 10 billion rows/month running 100+ in-house and public integrations with logging, monitoring, alerting and metrics support.
  • Created data-reliability as a platform to ensure sanity of data using data-parity, correlation between source and target, freshness checks (using DBT) and anomaly detection.
  • Introduced Infra-as-Code to the team in the form of Terraform and built out the initial platform to deploy integrations.
DBTKubernetesAWS EKSAPI ObservabilityData ReliabilityTerraform+2

Data Engineer - I

Mar 2020 – Nov 2020 · 8 mos

  • Designed and developed a Change Data Capture streaming pipeline
  • which syncs over 150GB data per day from MySQL to Redshift. This pipeline is deployed
  • over Kinesis Firehose and Elastic Beanstalk with lambda functions along the way.
  • Created multiple integrations using the singer-spec for syncing data between Marketo, Iterable and SFDC to Amazon Redshift.
Kinesis FirehoseElastic BeanstalkMySQLRedshiftData EngineeringData Pipeline Development

Zoomtail

Data Science Engineer

Jun 2019 – Mar 2020 · 9 mos · Bangalore

  • > Built a multi-class multi-label apparel classification model using Fastai and Pytorch, to automate the manual product onboarding process, by tagging over 55k products a month.
  • > Created a collaborative-filtering based recommendation system to identify similar retailers and products to deliver a truly personalized listing experience for 1100+ SMEs.
  • > Developed a Reverse image search service using object detection in videos to recommend similar products from our catalog.
  • > Designed a real-time stream processing pipeline using Apache Beam and Google Cloud, to optimize analysis and warehousing of orders.
FastaiPytorchApache BeamGoogle CloudData ScienceMachine Learning

Tally solutions pvt ltd

Software Developer

Jul 2018 – May 2019 · 10 mos · Bengaluru Area, India

  • Embedded a C++ lib in Python using PyBind11, creating a static library and enabling use of a custom HTTP protocol (HTTP-DIT). This allowed testing of internal services to be possible, along with building an e2e automation testing framework
C++PythonPyBind11Software DevelopmentEmbedded Systems

Self employed

Data Analyst intern

May 2017 – Jun 2017 · 1 mo · Bengaluru Area, India

  • The data science internship for AnalyticSimplified is conducted under the guidance of Mr. Soudhakar Elumalai and Mr. Jaikumar, during this period I worked on a project that involved social media mining, generating an engagement index for brands, recommending the probability of new users liking their products, while completing said tasks I learnt how to interact with APIs, include Natural Language Processing to segregate useful results and making conclusions that are mathematically accurate and statistically sound.

Education

Dayananda Sagar College of Engineering, BANGALORE

Bachelor of Engineering (BE)

Jan 2014 – Jan 2018

M.C. kejriwal vidyapeeth

ICSE — Computer Science

Jan 2000 – Jan 2012

Stackforce found 100+ more professionals with Software Development & Backend Development

Explore similar profiles based on matching skills and experience