Ketan Khurana

Software Engineer

Noida, Uttar Pradesh, India9 yrs 8 mos experience

Most Likely To SwitchAI ML Practitioner

Key Highlights

Reduced data processing time by 50% through ETL automation.
Engineered multiple data pipelines ensuring accurate data availability.
Innovated a framework for automating data pipeline creation.

Stackforce AI infers this person is a Data Engineering specialist in the EdTech and Data Analytics sectors.

Contact

Skills

Core Skills

Data EngineeringCloud Computing

Other Skills

AWSAWS LambdaAmazon Web Services (AWS)Apache AirflowArtificial Intelligence (AI)CSVData AnalysisData PipelinesData ScienceDatabase ManagementExcelExtract, Transform, Load (ETL)FastAPIGenAIGenai

About

* Implemented data pipelines to fetch data from S3 into Snowflake , utilizing Pipes Tasks and Streams. * Engineered numerous Lambda and Glue jobs for seamless ETL thereby reducing the data processing time by 50% * Orchestrated data pipelines using Airflow ensuring timely and 20+ accurate data availability for batch-oriented workflows. * Proficient in Pandas and pyspark scripting for data processing and manipulation. * Innovated a versatile framework automating the creation of data pipelines in Snowflake . * Collaborated with cross-functional teams to identify and fulfill the data requirements facilitating efficient reporting and analytics. * Handled diverse data sources such as CSV, Excel, Database API, Parquet and JSON . * Created event based and near real-time pipelines using SNS, SQS, eventbridge and Firehose * Set up Liquibase and Github for streamlined code deployment and version control respectively . * Implemented Unit test framework to improve the overall quality of the data platform leveraging pytest

Experience

Globallogic

Senior Data Engineer

Apr 2022 – Present · 3 yrs 11 mos · Noida, Uttar Pradesh, India · Remote

Content automation: Designed and implemented a modular GenAI-based architecture using FastAPI, AWS (SQS, DynamoDB), and OpenAI LLM to automate text book ingestion, test bank generation, and instructor manual creation with full observability via Langfuse.
Common Data Platform: Developed a data platform using snowflake and AWS for automatic ingestion into the raw layer before being transformed and loaded into the base layer. This provides data to the business users and data analysts for various analytical needs.
Acquisition: Developed another data platform wherein the input feeds are cleaned and mapped before being ingested into the datalake. The feeds are automatically ingested into the snowflake using snowpipe.

FastAPIAWSSnowflakeGenAIData EngineeringCloud Computing

Altran

Senior Software Engineer

Sep 2019 – Apr 2022 · 2 yrs 7 mos · Gurugram, Haryana, India

• PIERO: Developed Piero tool which extracted the csv data and processed the intermediate output as xml which was further processed on Linux servers to produce end result. With the help of the same, we automated the task extensively thereby completing the repetitive and time-consuming work in couple of minutes.

CSVXMLLinuxData Engineering

Tata consultancy services

System Engineer

Mar 2016 – May 2019 · 3 yrs 2 mos · Noida, Uttar Pradesh, India · On-site

• TSA: Developed Task Statistical Analysis tool which executed several commands on different databases parallelly, processed the output from several databases and finally presented the output in excel format.

Database ManagementExcelData Engineering