Feb 2016 – May 2018 · 2 yrs 3 mos · Pune Area, India

• My role is end to end automation of collecting data from web, process data , and store data in Mongodb and index data in Elaticsearch.
• For data validation, we are using voluptuous, static schema design.
• Also extracted new updates/changes for data through rssfeeds

Jan 2014 – Feb 2016 · 2 yrs 1 mo · Pune Area, India

• I am working with large data sets, play around big data, and extract structured information from raw dataset. I am using different database designs for storage and retrieval purpose i.e. like Relational datbases, MongoDB and Redis. Uses amazon s3(simple storage service) for online storage.
• Worked on data apps for data visualization purpose. Experience with d3.js, javascript, ajax(Learning Phase), django.
• Have experience in extracting structured information from large raw data sets.
• Worked on Relational as well as Non relational databases also. Implemented the distributed framework for data collection and processing using Celery

Oct 2013 – Dec 2013 · 2 mos · Pune Area, India

• Data collection and processing :
• > Experience in Scraping, collected data from different sources like crunch base, linkedin, indeed etc.
• > Worked on Data Analysis.
• > Wrote Crawlers (scrapy) for collecting global spec data from various sources.