Sep 2022 – Sep 2023 · 1 yr · Ford Greenfield Labs, Palo Alto, CA

• Led a team of talented machine learning and computer vision researchers towards building vision-centric 3D perception solutions at Ford Autonomy. My work included building an end-to-end 3D Perception stack for Ford L2+ vehicles on the road, and the development of a flexible and scalable cloud-based machine-learning framework for all ML tasks within Ford.

Sep 2019 – Sep 2022 · 3 yrs · Ford Greenfield Labs, Palo Alto, CA

• My research included computer vision and advanced machine learning methods including convolutional neural networks, generative adversarial networks, variational autoencoders, deep reinforcement learning, and 3D perception; with major emphasis on object detection, semantics learning, 3D scene understanding, multi-view geometry, and visual odometry.
• In addition to researching and developing novel methodology, I built the complete cloud-based MLOps pipeline including intelligent data sourcing, auto-annotation, training, model optimization, and deployment (TensorRT C++) for putting our prediction engine in production.
• Two major projects for which I built end-to-end perception pipelines are (1) Ford Autonomous Shuttles (2) Ford Factory of the Future - Infrastructure-based autonomous vehicle marshaling through assembly plants.
• TOPICS OF RESEARCH:
• Monocular RGB camera and LiDAR-based 3D object detection, Classification, and Tracking in both indoor and outdoor environments.
• Unsupervised and Semi-Supervised Object 6-DoF Pose Estimation to reduce the cost of manual data annotation from millions of dollars down to zero.
• Multi-headed multi-task neural networks for scene understanding incorporating Sim2Real methods for zero-cost training of the networks.
• Generative Adversarial Networks for realistic image generation with semantic and cycle consistency from simulation data to fill the gap between both worlds.
• A combination of computer vision and traditional methods including non-linear optimization for object pose estimation.
• Automated global localization of multiple spatially distributed sensors within the infrastructure to a common coordinate frame using an autonomous robot.
• Perception system for localizing objects of various classes to within 10 centimeters and sub-degrees orientation accuracy within Ford’s Factory of the Future.

Mar 2017 – Sep 2019 · 2 yrs 6 mos · Farmington Hills, Michigan

• I worked as part of a very small team towards building the ADAS & Autonomous Driving perception reference platform “Perception Quick Start” which includes end-to-end solutions for camera and LiDAR based road feature and object detection.
• Developed complete lane detection pipeline from scratch. Pipeline includes lane pixel extraction using a combination of classical computer vision method and deep learning, lane detection, polyfit, noise suppression, lane tracking, lane smoothing, confidence computation, lane extrapolation, lane departure warning, lane offset, lane curvature, and lane types.
• Developed C based Computer Vision Library for basic image processing functions like image read/write, hough transforms, edge detection (canny, horizontal, vertical), colorspace conversions, image filtering (sharpen, gaussian smooth, sobel, emboss, edge). Created advanced math library for functions like least-squares polyfit and matrix operations.
• Stereo Camera calibration, rectification, disparity map generation, flat road free-space estimation, Object Detection using V-Disparity and 3D density-based clustering, 3D point cloud rendering along with 3D bounding-box, and depth perception.
• Developed dynamic image ROI stabilization module for correcting rotation and translation using angular pose/velocity data by computing and applying homography at run-time. Developed a general-purpose positioning driver for bringing in GNSS/IMU data into the perception stack.
• Optimized embedded implementation of algorithms for parallel computing on R-Car SoC HW Accelerators.
• Developed the complete V2V Solution from Scratch for the Renesas’ V2X Platform including CAN Framework, GPS/INS driver, GPS+IMU fusion for localization, Concise Path History computation, Path Prediction, CSV and KML logging module, 360-degree lane-level Target Classification, Basic Safety Applications, and an HMI using QT for displaying Warnings, Vehicle Tracking, Maps, and Debug Information.

Aug 2016 – Mar 2017 · 7 mos · Plymouth, Michigan

• Worked with DSRC On-Board Unit, MobileEye EyeQ3, dSPACE MicroAutoBox II, EB Assist ADTF and other modules to design, develop, and improve various models for the connected car technology (V2X) and to implement them on Changan's Connected Car (Using MATLAB/Simulink Platform, C in Linux Environment, and C++ in Windows Environment).
• Developed the algorithm and created plant model in Simulink for Target Classification that provides a 360–degree, relative classification of the locations of communicating Remote Vehicles relative to the Host Vehicle for Curvy and Straight Paths. Also, mapped the classification to various VSC-A safety applications.
• Developed the DSRC V2V applications Blind-Spot Warning (BSW), Lane Change Warning (LCW), Left Turn Assist (LTA) and implemented them over Cohda Wireless MK5 software stack.
• Defined the interface and created hooks between dSPACE MicroAutoboxII, EB Assist ADTF, and Cohda MK5-OBU.
• Developed ADTF Filters for all the VSC-A safety applications by creating a wrapper corresponding to their simulink models in C++ environment to be able to perform V2X functionalities on an embedded PC.
• Worked on the development of DSRC Rebroadcasting to enable a Non-DSRC vehicle to be seen by every other vehicle within the DSRC broadcast range. Developed the algorithm using MobileEye EyeQ3 raw CAN messages, GPS data, and the vehicle data to deduce the information required to create a BSM and transmit it over DSRC.
• Programmed the touch screen module "New Eagle Raptor VeeCAN 800" to control the power distribution to all of the subsystems in Changan's Connected Autonomous Car using "EATON Multiplexed Vehicle Electrical Center (mVEC)", monitor for failures, and display the diagnostic information on the screen. Command, Reply, and Status messages were SAE J1939 based.

May 2016 – Aug 2016 · 3 mos · Troy, Michigan

• Worked with application teams, forward systems algorithm group and controller design groups to define functionality/capability to be designed into new controllers.
• Developed algorithms and implemented it depending on the changes required by customers in the current functionalities of software package provided by Delphi in accordance with the V-Model Software Development Life Cycle.
• Designed, developed, and tested for EMS Software solution to meet project requirements

Jan 2016 – May 2016 · 4 mos · Novi, Michigan

• Developed the BSP (Board Support Package) for a custom hardware with i.MX6 Solo Processor and several peripherals intended to be used for V2X Technology by DENSO. Worked towards the low-level board bring up and provided the following supports:
• Provided support for RAM file system to manipulate files during runtime.
• Provided support for SPI NOR Flash and Parallel NOR Flash mounted as a filesystem at startup.
• Provided support for Removable storage (SD, microSD, USB Flash). Also, provided support for its auto detection and auto mounting at insertion.
• Provided support for USB OTG to be used for the Console Service, Mass-Storage Device and USB-to-Ethernet Adapter attachments.
• Added new feature to QNX for the BSP (Auto detection and switching between device stack to provide console service, host stack to provide auto-mounting of mass storage devices, and host stack to provide networking with ASIX USB-Ethernet Adapter based on the type of attachment.)

Aug 2015 – Dec 2015 · 4 mos · Dallas-Fort Worth Metroplex

• Designed and Developed the control GUI for a prosthetic system used to help rehabilitate post-stroke patients.
• Wrote the code for Arduino controller for adaptive adjustment of the air bubble pressure at the desired psi value for various points on the leg.
• Also designed the GUI on MATLAB, which allows the user to enter the desired psi values for each air bubble and simultaneously measure current bubble pressure and display it on the GUI.
• Made use of two Arduino UNO, one for sending the signals to 32 solenoids controlling airflow from Alicat Mass Flow Controller into the respective air bubbles, and one for receiving sense signal from 32 corresponding air pressure sensor. Signals were also sent to the Solenoids for deflating the air bubbles when required.
• Designed the communication protocol involving ACK and NACK and implemented them in MATLAB GUI for sending commands to one of the Arduino and receiving sense signals from another and hence provided the synchronization and feedback.
• In addition used 15 colors (Blue to Red) to map the pressure range and show the current pressure on each bubble visually in real-time.

Jan 2014 – May 2014 · 4 mos · Bangalore

• Designed and implemented a 2 Dimensional plotter (Smart XY Plotter) capable of plotting any 2D image using a pen, which was controlled by means of a Solenoid and two stepper motors (responsible for x, y and z directional movement).
• The control was governed by an ARM Processor (STM32F4 Discovery Board) to plot images which features were extracted using MATLAB.
• Also developed the GUI on MATLAB, which allows a user to either upload the image of their choice or select any other plot (Arbitrary interpolated curve, texts, shapes).
• Used two timers for controlling and synchronizing the parallel movement of X and Y motors to provide a curve of desired slope.
• The solenoid setup was brought back to its initial position after every plot. Used limit switches to detect its arrival at the desired reset position.