Feb 2021 – Jan 2022 · 11 mos · Hyderabad, Telangana, India

• Collaborated with SSD Engineer , Data Science and Analyst team to provide data for a model which will predict the delay in Manufacturing process and improve it.
• Designed and built automated date load pipelines with monitoring and notification systems which reduced manual effort by 29%
• Created an automated python scripts and framework to load multiple tables from Big Query to MySQL in cloud VM.
• Optimized previously running schedule queries in Big Query $7 per hour to $1 per hour. Which led to monthly GCP project cost from $2400 to $600
• Ingested data from disparate data sources using a combination of SQL, File, Kafka ,Hdfs and Snowflake using Python , Nifi to create data views to be used by the DS team
• Load High Volume Data ranging from File in HDFS, SQL server and Kafka topics to Big Query using DataProc and Dataflow and from Big Query to Cloud SQL using PUB/SUB and Cloud Function
• Built several automated tools to create API, generate ddl scripts for creating, altering and dropping table, views, routines.
• Designed and developed CI/CD Pipeline in Azure Devops.

Aug 2019 – Feb 2021 · 1 yr 6 mos · Bangalore

• Led the migration to cloud-based data integration, and completed the product acquisition before deadline
• Oversaw a team of 5 data engineers, and collaborated with different stakeholders to convert complex business requirement to simple data solutions
• Successfully engineered overall components of a high-volume data warehouse by efficiently deploying, integrating and updating integration/deployment scripts to improve continuous integration practices best suitable for industry standard
• Built efficient data models and metadata for ad hoc and pre-built reporting by carrying out Interface with business customers, gathering requirements and delivering complete data, reporting solutions, owning the design, and maintenance of ongoing metrics, reports, analyses, dashboards, etc. to drive key business decisions
• Provided input and recommendations on technical issues to BI Engineers, Business & Data Analysts and Data Scientists and supported the development and validation required through the lifecycle of the DWH and Business
• Spearheaded Experience in using Open source technologies (Apache Spark, Hadoop, Hbase, Hive, Pig, Kafka and implementation of Big Data technologies including Hadoop, HDInsight, Data Bricks, ADF on Azure
• Hands-on Experience in Azure Data bricks, Azure Data Lake, Azure Data Factory, Azure Event Hub, SQL Server, Cosmos DB and automating SSIS Package generation using BIML and Python.

Oct 2015 – Jul 2019 · 3 yrs 9 mos · Bhubaneshwar Area, India

• Analyze data, develop transformation scripts, remediate quality issues, and work collaboratively with SMEs to
• implement and test new data products
• Coordinate with technology teams to design, construct, install, test and maintain data management systems
• Create reusable components and data tools for broader Research department use in accessing and analyzing
• Research datasets
• Help create and evolve data standards and product documentation
• Supervise design throughout implementation process.
• Interface with other technology teams to extract, transform, and load (ETL) data from a wide variety of data
• sources
• Reduced cycle time of routine data migration activities by 50%.
• Ensured end to end process of data extraction until reporting.
• Worked With NoSql Technologies

May 2013 – Aug 2013 · 3 mos · Jamshedpur, Jharkhand, India