N

Nipun Agarwal

Software Engineer

Bengaluru, Karnataka, India17 yrs 11 mos experience
Most Likely To SwitchHighly Stable

Key Highlights

  • Expert in building real-time and batch data platforms.
  • Proficient in big data technologies like Spark and Kafka.
  • Strong background in cloud computing with Azure and AWS.
Stackforce AI infers this person is a Big Data Engineer with expertise in real-time analytics and cloud solutions.

Contact

Skills

Core Skills

Big Data EngineeringCloud ComputingData EngineeringReal-time ProcessingBig Data DevelopmentData ProcessingReal-time AnalyticsSoftware DevelopmentDatabase ManagementSoftware TestingAutomationRemote Solutions

Other Skills

.NetASP.NETAerospikeAndroidApache SparkAzureCC#C++EclipseGrafanaHDFSHadoopHiveInfluxDB

About

Wherever we go, whatever we do, technology has engulfed us. We are constantly surrounded by applications, be it mobiles, television, computers, internet etc. Huge amounts of events and data are generated from these devices, which gives us a lot to analyze. I am a software developer and big data engineer who enjoys playing with data using various open source platforms like spark, cassandra, kafka etc. I have a lot of experience in building real time and batch data platforms.

Experience

Atlassian

Senior Engineer

Dec 2020Present · 5 yrs 3 mos · India

Tvs motor company

Principal data engineer

Apr 2019Dec 2020 · 1 yr 8 mos · Bangaon Area, India

  • Leading the telematics team to build, capture and analyse telematics platform for 2W and 3W using Azure as
  • the cloud platform and open source time series database, influxdb and grafana as visualization tool
  • Design and build data lake for TVSM and TVS group companies
AzureInfluxDBGrafanaBig Data EngineeringCloud Computing

Hotstar

Software Development Engineer

Nov 2017Mar 2019 · 1 yr 4 mos · Mumbai Area, India

  • Building and maintaining Hotstar Data Platform as a Senior Data Engineer. As part of it, implemented various real time and batch processing for behavioral and transactional data (change data capture), ETL(s) using pyspark and dimensional modelling (star schema) for data warehouse
  • Architect and designed ingestion pipelines using kafka, analytics platform for analysis for business users using hive and presto and analytical platform using inhouse hortonworks amabari cluster.
  • Spearheaded personalisation team to implement user level personalisation using microservices in Java. Technologies included aerospike, RDS and spring framework
  • Design and implemented item to item collaborative filtering using ALS for personalisation, which includes technologies like AWS EMR, Pyspark, data pipeline, terraform, ansible
PysparkKafkaHivePrestoJavaAerospike+2

Credit suisse

Big Data Developer

Dec 2016Nov 2017 · 11 mos · Mumbai Area, India

  • Design and implemented pricing engine to calculate offer rates for stocks for various markets
  • Used big data technologies to consume huge amounts of data and facilitate the engine and python libraries to function
  • Relevant Technologies:
  • Apache Spark, Pig, Hbase, Python, Scala, HDFS, Apache Oozie, Cloudera, Yarn, Jupyter, hue
PythonScalaHadoopHDFSApache SparkBig Data Development+1

Monotype solutions india

Big data Developer

Mar 2015Oct 2016 · 1 yr 7 mos · Delhi

  • Led and executed Real time analytics platform for consuming events in real time. Helped in optimizing resources and lowering money in AWS.
  • Programmed a lot of api(s) to be consumed by mobile as well as desktop apps in Nodejs and .Net. Keyboard apps integrated them to send events for analytics and accessing resources.
  • Built Android SDK for assets in Java and C++ (JNI). This was integrated in numerous keyboard apps which are on playstore
  • Researched on open source technologies and tools like Zookeeper, Solr, ElasticSearch, Jenkins as part of continuous development and growth.
  • Relevant Technologies:
  • .Net(MVC), SqlServer, Android, C++, Nodejs, Cassandra, Spark(streaming and batch), Kafka, Mongodb, Solr, Postgres, Python, Scala, Shell Scripting, AWS(EMR, Redshift, Kinesis, Lambda, Firehose, S3, ElasticSearch), Jenkins, Tensorflow
.NetNode.jsC++SparkKafkaBig Data Development+1

Vegayan systems

Technical Lead

Dec 2013Mar 2015 · 1 yr 3 mos · Mumbai Area, India

  • Developed a Cisco and Juniper configuration file parser to parse these files without any manual intervention. This greatly reduced manual effort, giving network engineers to focus on problem solving.
  • Programmed api(s) in jersey(java) to be integrated with the dashboards for Simplus.
  • Worked on databases like neo4j and mysql to integrate network topologies. Faster processing, querying and storage needs were met after transitioning to this NOSQL database.
  • Built highly multi-threaded application for processing router configuration files. Time it took to process them greatly reduced.
  • Architect solution in core Simplus library to be integrated with wide variety of clients. This helped in creating clients in any technology or language using services of the Simplus server
  • Relevant Technologies:
  • C, C++, Java, Hadoop, Neo4j, MySQL
JavaMySQLNeo4jSoftware DevelopmentDatabase Management

Microsoft

Software Design Engineer in Testing 2

Dec 2010Dec 2013 · 3 yrs · Hyderabad Area, India

  • Engineered test designs and test automation. Almost all the test cases were automated with no manual intervention
  • Created architecture of automation platforms for mobile(lync) and desktop application(lync and office) as per the requirements. Used by different teams to integrate their applications to plug into this platform.
  • Relevant Technologies:
  • C,C++, Java, C#, Javascript
C#JavaScriptC++Software TestingAutomation

Symantec corporation

Software Engineer

Jan 2008Nov 2010 · 2 yrs 10 mos · Pune

  • Engineered feature design and implementation for product pcAnywhere (Remote controlling Solution).
  • Worked on understanding and implementation of solution from scratch using Notification Server Framework. Part of core development team of four to achieve this solution
  • Developed Agent in COM which interacted with Symantec Notification Agent for communication between the server and the client.
  • Created executable Client Manager which handled installation, un-installation, upgrade of host package.
  • Led and tailored reports showing various session data in XML which became a trade secret.
  • Created light weight client called pcAquickConnect. This was bundled with the larger solution, help customers in quickly connecting to the remote machines.
  • Relevant Technologies:
  • C#, Asp.net, XML, Javascript, C++, C, Visual C++, MFC
C#ASP.NETXMLSoftware DevelopmentRemote Solutions

Education

BITS, pilani

M.E. — Software Systems

Jan 2006Jan 2008

Birla Institute of Technology and Science, Pilani

Master of Engineering (M.Eng.) — Software Systems

Jan 2006Jan 2008

Ambedkar DSEU Shakarpur Campus-I

Bachelor of Technology (B.Tech.) — Computer Science and Engineering

Jan 2002Jan 2006

Stackforce found 100+ more professionals with Big Data Engineering & Cloud Computing

Explore similar profiles based on matching skills and experience