Sep 2023 – Jun 2025 · 1 yr 9 mos · Bengaluru, Karnataka, India · Remote

• Design, develop, and implement data integration processes using Apache NiFi to facilitate the smooth flow of data between various systems and sources.
• Create efficient data pipelines to ingest, transform, and route data across different platforms, ensuring data quality, integrity, and security.
• Configure and fine-tune Apache NiFi components to meet specific project requirements and optimize performance.
• Manage and monitor NiFi processors, queues, and other components to ensure the efficient and reliable movement of data within the system.
• Developing and maintaining ETL (Extract, Transform, Load) processes to move and transform data from various sources into the Mongodb
• Implementing real-time data processing systems to handle streaming data.
• Utilizing programming languages such as Python to build and maintain data processing applications.
• Writing scripts to automate data workflows and tasks.
• Staying updated with the latest Big Data technologies and tools, such as Apache Nifi, Spark, Kafka,RabbitMQ and Mongodb.
• Implementing and optimizing the use of these technologies to meet the organization's requirements.

Sep 2023 – Jun 2025 · 1 yr 9 mos

• Design, develop, and implement data integration processes using Apache NiFi to facilitate the smooth flow of data between various systems and sources. Create efficient data pipelines to ingest, transform, and route data across different platforms, ensuring data quality, integrity, and security. Implemented Kafka stream processing using Scala for real-time data ingestion and transformation. Configure and fine-tune Apache NiFi components to meet specific project requirements and optimize performance.

Aug 2021 – Sep 2023 · 2 yrs 1 mo · Pune District, Maharashtra, India

• Developed and optimized data processing pipelines using Python and PySpark, handling large volumes of data efficiently.
• Collaborated with data engineers and data scientists to gather requirements and implement data-driven solutions.
• Designed and implemented real-time data streaming solutions using Apache Kafka, ensuring timely and accurate data delivery.
• Utilized Hadoop and Spark frameworks for distributed data processing and analysis, improving performance and scalability.
• Conducted performance tuning and optimization to enhance data processing efficiency and reduce processing time.
• Worked with NoSQL databases, such as MongoDB, for storing and retrieving large-scale data sets.
• Utilized cloud platforms to deploy and manage big data applications, ensuring high availability and scalability.
• Collaborated with stakeholders to understand business needs and provide insights through data analysis.
• Actively stayed updated with the latest trends and technologies in the big data field, implementing best practices in projects.

Aug 2021 – Sep 2023 · 2 yrs 1 mo

• Developed and optimized data processing pipelines using Python and PySpark, Scala handling large volumes of data efficiently. Collaborated with data engineers and data scientists to gather requirements and implement data-driven solutions. Designed and implemented real-time data streaming solutions using Apache Kafka, ensuring timely and accurate data delivery. Utilized Hadoop and Spark frameworks for distributed data processing and analysis, improving performance and scalability. Conducted performance tuning and optimization to enhance data processing efficiency and reduce processing time. Worked with NoSQL databases, such as MongoDB, for storing and retrieving large-scale data sets. Utilized cloud platforms to deploy and manage big data applications, ensuring high availability and scalability. Collaborated with stakeholders to understand business needs and provide insights through data analysis. Actively stayed updated with the latest trends and technologies in the big data field, implementing best practices in projects.

Jun 2021 – Aug 2021 · 2 mos · India · Remote

• Process Execution: Carry out day-to-day operational tasks related to specific processes, ensuring adherence to defined standards and procedures. Execute tasks efficiently and accurately, meeting quality and productivity targets.
• Quality Assurance: Ensure adherence to quality standards and compliance with applicable regulations. Perform quality checks and audits to identify and resolve errors or discrepancies, maintaining data integrity and accuracy.

Jun 2021 – Aug 2021 · 2 mos

• Process Execution: Carry out day-to-day operational tasks related to specific processes, ensuring adherence to defined standards and procedures. Execute tasks efficiently and accurately, meeting quality and productivity targets. Quality Assurance: Ensure adherence to quality standards and compliance with applicable regulations. Perform quality checks and audits to identify and resolve errors or discrepancies, maintaining data integrity and accuracy.

Jan 2020 – May 2021 · 1 yr 4 mos · India · On-site

• System Maintenance: Install, configure, and maintain computer systems, servers, operating systems, and software applications. Ensure that systems are updated, patched, and secured against vulnerabilities.
• User Management: Create and manage user accounts, permissions, and access controls for various systems and applications. Handle user provisioning and deprovisioning, password resets, and account maintenance.
• Network Management: Monitor and manage network infrastructure, including routers, switches, firewalls, and VPN connections. Troubleshoot network issues and ensure network availability and performance.
• Backup and Recovery: Implement and maintain backup and recovery strategies to protect critical data and systems. Regularly perform backups and test restoration procedures to ensure data integrity and availability.
• Security Management: Implement and enforce security policies, procedures, and best practices to protect systems and data from unauthorized access, malware, and other threats. Conduct security audits and vulnerability assessments, and address security incidents.
• System Monitoring and Performance: Monitor system performance, resource utilization, and availability. Identify and resolve performance bottlenecks, optimize system configurations, and recommend hardware or software upgrades as needed.
• Troubleshooting and Technical Support: Investigate and resolve hardware and software issues reported by users or detected through system monitoring. Provide technical support and guidance to users, addressing their questions and concerns.
• Documentation and Reporting: Maintain accurate and up-to-date documentation of system configurations, procedures, and troubleshooting steps. Generate reports on system performance, incidents, and maintenance activities.
• Disaster Recovery Planning: Contribute to the development and implementation of disaster recovery and business continuity plans.