End-to-end machine learning and computer vision for a self-driving car

Screen Shot 2017-04-06 at 1.13.51 PM.png

Using a non-robotic approach, I developed a full autonomous system for a car to be able to drive and predict steering at curves, and to recognize traffic signs and other cars in the road.

The project have 3 main components using diverse techniques:

  1. Deep learning model using a convolutional neural network to clone human driving behavior, based on Nvidia CNN learnt the best way to steer at curves and recognize several different road situations, as different road textures and driving patterns.
  2. Traffic sign classifier model, based on LeNet to recognize main 43 German road traffic signs from a video stream.
  3. Computer vision model to detect lanes and obstacles, in order to predict steering at curves, or to perform safe lane changes.

For the deep learning models, I customized and trained two different convolutional neural networks. One based in NVIDIA CNN for the human driving cloning and other based on LeNet for the traffic sign classifier. I used thousands of augmented datasets for the training and a GPU instance in AWS for computing and optimizing the training batches.

preprocessed gallery.png
Real-time preprocessed traffic signs images ready to be used in the classifier training.

 

neural network.png
End-to-end convolutional neural network used to train the model to clone human behavior driving.

For the computer vision part, I process in real-time frames from the camera to detect several features and patterns, and predict potential steering angles based on object geometry.

 

Technology stack of the project:

  • Python3
  • TensorFlow
  • Keras
  • OpenCV
  • Raspberry Pi 3 and Raspicam
  • Unity

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s