Using Apple’s Machine Learning for License Plate Recognition

With the massive growth of machine learning, Apple is inspiring leading businesses to build more applications that can perform complex functions with ease. License Plate Recognition is one such application that can be built using this framework.

LPR technology for vehicles fits in any part where safety in tag acknowledgment is required. Thanks to our technology developed with advanced algorithms, our app enables reading and recognition of different license plate formats as per the country it belongs to.

The US market will see significant growth. It is projected to reach $3.57 billion by the year 2023. – ANPR Market Research

Businesses must own an independently automated LPR software solution. This tech-based solution allows an advanced data processing capability, optimizes routine tasks, and tools to increase productivity adding value to workflows.

Smartphones can help overcome the limitation of LPR devices. The built-in camera of your smartphone has the ability to capture the image of the license plate number and save it into a database for further processing.

Before we delve into the specifics of machine learning applications, let us investigate understanding the concept of License Plate Recognition.

What is License Plate Recognition?

License plate recognition has existed since the 1970s. Earlier, it was generally related to law enforcement. Cameras attached to police cars or street fixtures recognize the license plates of passing vehicles and contrast results with databases.

It is a process of recognizing number plates using Optical Character Recognition (or OCR) on images. Here, the OCR undertakes recognition of the printed text or images using a three-step process called DCR – Detection, Capture, and Recognition.

Machine Learning (ML) essentially implies a phenomenon where the machine has the capability to learn and deliver relevant yet better experiences. It is a part of Artificial Intelligence involving data analysis and automatic analytical model building.

Core ML Model

Core ML is a framework built by Apple. It supports features such as face tracking, face detection, landmarks, text detection, rectangle detection, barcode detection, object tracking, and image recognition.

Core ML supports a variety of ML models and this includes neural networks, tree ensembles, support vector machines, and generalized linear models. Core ML requires the Core ML model format (models with a. Ml model file extension). Apple and Google provide many Core ML models that can be downloaded easily and be used in the application.

Additionally, various research groups and universities publish their models and training data, which may not be in the Core ML model format. To use these models, you need to convert them into Core ML format (through Keras and Caffe).

Core ML is strongly integrated and supports the Vision (image analysis), Natural Language Processing (NLP), and GameplayKit frameworks.

Building the Application for LPR

Now that you are aware of the key concepts used for bringing about this application, let us now delve into how this all began:

• Apple started by looking at various frameworks that use neural networks for image recognition in iOS. The prominent ones we found were Tesseract and SwiftOCR.
• Apple used both the frameworks one by one. While the Tesseract framework consumed considerable time for the setup, the SwiftOCR was relatively faster; however, the latter lacked in terms of accuracy.
• By experimenting with both, Apple launched a Vision framework for face recognition, text, barcodes, QR codes, etc.

Also, Apple’s Core ML 3 helped iOS developers bring machine learning to smartphone apps. This framework can train multiple models with different data sets. Core ML 3 helps in detecting objects and identifying sounds to offer a personalized experience to the users.

About Vision Framework

It gives you easy access to Apple’s models for detecting faces, face landmarks, text, rectangles, barcodes, and objects.

Key Features of Vision Framework

• Detects face rectangle and face landmarks
• Find projected rectangular regions’ surface.
• Finds and recognizes barcodes.
• Find regions of visible text.
• Determines the horizon angle in an image.
• Detects transforms needed to align the content of two images.
• Processes images with the Core ML model.
• Track movements of a previously identified arbitrary object across multiple images or video frames.

Using Vision Framework

Apple performed two functions using the vision framework – finding regions of visible text to split the word image into characters and process images with the Core ML model. So, let us learn how to cut the images with vision “VNDetectTextRectanglesRequest”.

Here through vision, they detected the characters but could not recognize them. For recognition, they required the core ML trained model as shown below:

Creating the Core ML Model

Apple first looked for the Core ML model, which could recognize the license plate. However, they ended up with a model that could only recognize the text and not only the entire license plate. Then, the company decided to train the model through Keras, which is an open-source neural network library written in Python. It is designed to enable fast experimentation with deep neural networks.

Apple decided to build the model in Python and added the following frameworks:

• Keras to create the model from a dataset
• Image to fetch the pixels of the character image
• Numpy for a high-performance multidimensional array object.
• Core ML tools to convert into ml model
• Sklearn for splitting the dataset for training and testing

Workflow of Core ML Model

Now, Apple developers needed the data (images of characters 28*28) which had to be trained. They collected the data from the web and clicked 40-50 images of number plates. And, sliced the characters and then converted it to 28*28 resolution from one number plate image.

Hence, the decision was made to go ahead with an automated iOS app development and generated thousands of character images with the help of a vision framework.

Here is the workflow for developing an IOS app with the help of Core ML looks like:

• You must build a data model by Core ML frameworks like Keras, Caffe, Turi, and so on.
• To install the Python framework known as Core ML Tools, it translates the data model to the Core ML format. The consequence of the conversion is a file with an ML model extension.
• Thus, you have the model created by Core ML Tools and use it within your mobile app.

Key Features of License Plate Recognition

1. Check on the Authenticity of Abandoned Vehicles

This feature can be leveraged by the police forces for law enforcement purposes. Here, the application can enable them to quickly check the authenticity of number plates installed on abandoned vehicles via mobile devices. Hence, this can curb illegal acts/anti-social elements.

An additional camera can focus on the driver’s face and save the image for future security tests. Additionally, this technology does not need separate installations for each vehicle, unlike other technologies that install a transmitter for each vehicle.

2. Automation of Electronic Toll Collection

LPR can be used for automating the regulated entry of vehicles through toll barriers. It couples with an access control system to recognize number plates listed in the toll collection database. The barrier would open only for those vehicles that are recognized at the electronic toll collection point as shown in the below image:

The white rays read the license number plate and match it with the database of registered number plates.

3. LPR Devices that are Available as Hardware

There are different types of devices available in the market to recognize the number plate. For instance, CARMEN Parking is a cost-effective solution used for vehicle entry; CARMEN FreeFlow is a device used for recognition of fast-moving vehicles are effective LPR solutions. Some other examples include:

• Dongles for various SW versions – USB, PCIe, FXVD4 card, single or multi-license, the FXVD4mc_s video capture card.
• Connecting any analog camera to the system, the FXCAMd 102 digital ANPR camera with a motorized optic for easy installation and ideal image quality, the FXCAM IBW_2000 ANPR camera.
• A “plug-and-play” pre-set solution for the optimal image and the smart ANPR camera, the all-in-one unit, integrated ANPR in the camera.

On the downside, the above hardware devices come with a hefty price as they require a setup. Another limitation is the size and portability of the devices as the setup consumes a considerable amount of time.

Bottom Line

With the rise in the number of vehicles, it’s beyond human’s ability to complete this cumbersome task. Thus, detecting and recognizing the license plate of vehicles is important.

Machine Learning via License plate recognition brings a set of advantages that surpasses all its limitations. This solution is not only cost-effective but also quick and needs no installation. Since this application runs through your mobile phone, it comes in handy too.