License Plate Recognition App

In the field of video surveillance (CCTV), artificial intelligence has been used for years to automate tasks like detecting people, vehicles, fires, and license plates. I had been supporting the development of these systems—including LPR (License Plate Recognition)—for some time, but I was always curious to understand in depth how the algorithm works to detect and extract license plate characters.

When the end of the Data Science and Machine Learning Bootcamp I was attending approached, I decided to turn that curiosity into my final project: an Automatic Number Plate Recognition (ANPR) app. Although the course had only briefly covered Computer Vision, I wanted to go deeper and explore the topic on my own.

My goal was to develop a complete application that could take an image or video frame, detect a license plate, and then recognize its characters. To do this, I split the problem into three independent phases:

1. License plate detection in images (Object Detection)

2. Character recognition inside the plate (custom OCR)

3. Correct ordering of the detected characters

I selected tools and algorithms that offered a good balance between accuracy, performance, and ease of deployment, since I didn’t have access to high-end hardware to run heavy models.

Phase 1: License Plate Detection

I used YOLOv5 for plate detection, due to its good performance and low computational cost. I started by searching for public datasets, but the ones I found had poor-quality images. I tried to create my own dataset by taking photos in the street, but this approach was not realistic due to time and geographical limitations (most plates were Spanish).

After some in-depth searching, I discovered a website for license plate enthusiasts. I scraped it and collected thousands of images, which I then manually labeled using tools like Roboflow.

Model training was done iteratively. The first versions were decent, and I used those initial detections to speed up the labeling of the rest of the dataset.

Phase 2: Character Recognition

I initially tried generic OCR libraries (like Tesseract and EasyOCR), but they didn’t work well for plates and had poor accuracy. So I decided to train a second custom neural network, again using YOLOv5, this time to detect each character individually.

This part was even more time-consuming: I had to label every character, considering there were at least 35 different classes (letters and numbers).

Phase 3: Character Ordering

Once the model was trained, a new challenge appeared: correctly ordering the detected characters so the plate could be read like a human would. This turned out to be more complex than expected, because plates vary greatly between countries and vehicle types (cars, motorcycles, police vehicles, etc.).

User Interface

To present the results, I built a simple interface using Streamlit. While it’s limited in terms of customization, it allowed me to create a functional and easily deployable demo in a short amount of time.

Results or Impact

- Full end-to-end ANPR system developed using open-source tools

- High accuracy in both plate detection and character recognition

- Functional interface to test with images and display results

- Full automation of the process: from image input to text output

This project showed that it’s possible to build a working ANPR system without advanced hardware, and it gave me the opportunity to explore the entire Computer Vision and Machine Learning pipeline in a real-world use case.

Lessons Learned

Technical:

Custom neural network training with YOLOv5

Hands-on experience with PyTorch and OpenCV

Deep understanding of the importance of good dataset labeling in model performance

Professional:

Ability to manage a full project independently, from research to delivery

Making smart technical decisions under hardware and time limitations

Personal:

Intensive time management, especially under pressure (two months of focused work)

Validation that I can successfully handle complex, multi-step projects