Jun 2023 – Sep 2024 · 1 yr 3 mos · Hybrid

• As an experienced Data Engineer, I thrive on building robust pipelines in AWS using an array of powerful tools: Lambda, Glue, Redshift, Sagemaker, Step Functions, S3, and RDS. Handling diverse client requirements is my forte, as I carefully understand their needs, provide valuable feedback, and efficiently implement the work.
• Monitoring data integrity and ensuring smooth operations, I take pride in creating efficient CI/CD pipelines with Jenkins, streamlining development and deployment processes.
• Additionally, I excel at implementing advanced ML models, such as Demand Forecasting, leveraging Sagemaker's capabilities. Integrating MLOps orchestration with Snowflake, I ensure seamless integration of machine learning solutions into the broader data ecosystem.
• With a passion for data-driven innovation, I'm committed to delivering excellence in every aspect of data engineering, enabling businesses to harness the true power of their data. 🚀 #DataEngineer #AWS #DataPipelines #MLOps #Sagemaker #Snowflake #Jenkins #DemandForecasting

Jun 2022 – Jun 2023 · 1 yr · Hybrid

• Developed the data pipeline architecture to link it with the UI, such that any file submitted from the UI directly loads to S3, is processed in Glue, is loaded to Redshift, and is analyzed in Quick sight.
• Collaborated with the Chief Architect in the ideation and design of a new backend data pipeline, providing key insights and justifications for the selection of resources, and assisted in the development of the pipeline from scratch.
• Implemented data pipeline automation using event-based triggers with no manual intervention.
• Communicated with Project managers and analysts about data pipelines that uplifted efficiency.
• Worked with supervisors to understand business requirements and translate those requirements into actionable reports.
• Developed and implemented a job scheduling solution that dynamically adjusts to user-provided parameters, resulting in significant cost savings by reducing unnecessary lambda and other resource usage, while optimizing compute power usage.
• Deployed AWS platform version control to effectively manage and track changes to the organization's cloud infrastructure, ensuring reliability and streamlined collaboration among team members.
• Developed and implemented efficient and scalable database schemas, including designing, and optimizing tables, to support data storage and processing requirements.
• Designed and developed an AWS-based backend system that automatically updates information and seamlessly integrates with Microsoft Form using SES, enhancing data accuracy, and streamlining communication processes.
• Primary: SQL, PySpark, AWS, S3, Hadoop, Hive, Redshift, Power BI, Python, Glue, Lambda, Quick sight, CloudWatch, events bridge, SES, Athena, Aws RDS, Snowflake, Talend, MySQL, Microsoft SQL Server, SQL Lite,
• Secondary: PostgreSQL, MongoDB, DynamoDB, Azure

Jan 2022 – Apr 2022 · 3 mos · Chicago, Illinois, United States · Remote

• Successfully collected, processed, and loaded data to AWS RDS, which was then utilized to deduce associations between source and destination columns.
• Implemented and analyzed several approaches, as well as exhibited knowledge of Python libraries and the Pypi package.
• Queried database to extract all metadata to achieve table to column relations for source and target in SQL scripts.
• Contributed to UI design and development, exhibiting teamwork and quick learning abilities, and understanding of technologies such as JavaScript, react, Pyvis, and NodeJS.
• Collaborated with the UI team to incorporate data lineage backend programming.
• Demonstrated data storytelling through extraction of data insights from different databases/tables.