AI algorithms and challenges for autonomous vehicles
Article By : Anton Hristozov
The day when we are going to see autonomous vehicles in everyday life for the regular consumer is quickly approaching. Many of the operations that vehicles have to perform are based on sensor information and some AI algorithms...
Autonomous vehicles (AVs) are starting to become a real possibility in some parts of industry. Agriculture, transportation and military are some of the examples. The day when we are going to see autonomous vehicles in everyday life for the regular consumer is quickly approaching. Many of the operations that vehicles have to perform are based on sensor information and some artificial intelligence (AI) algorithms. Vehicles need to collect data, plan their trajectory and execute the trajectory. These tasks, especially the last two require non-traditional programming approaches and rely on machine learning techniques, which are part of AI. This two-part series illustrates the AI applications that make autonomous vehicles a reality, presenting their challenges and accomplishments. Part 1 offered an analysis of AI and its applications in AVs. This article explores the algorithms and challenges of AI in AVs.
3. AI algorithms used in autonomous vehicles
3.1. Route Planning and Control Algorithms
Traditional algorithms from computer science that are heuristic in nature can be used for this task. These are algorithms like Bellman-Ford and Dijkstra’s algorithm (Bugala, 2018). For these algorithms to work we need to have localization of the vehicle during the whole time. Localization is accomplished through sensors such as GPS as well as simultaneous localization and mapping (SLAM) techniques.
SLAM is used when there is no GPS availability such as underground or enclosed spaces for example. SLAM generates a map of the environment and at the same time estimates the state of a vehicle (Cadena et al., 2016). The map is composed of landmarks or obstacles in order to represent the environment. SLAM is used in applications where the map is not available and needs to be created. It uses sensors and special algorithms that create models of the data in order to produce the map.
3.2. Object Detection Algorithms
Object detection is one of the most important tasks that AI has to handle in a moving vehicle. These algorithms are an area of active research and they rely on different sensors. Object detection can be based on cameras or lidars, radars and other type of sensors. The algorithms used are normally deep learning algorithms which use some type of a neural network to do the job (Redmon et al., 2016).
One requirement for this task is that it needs to be fast. The reason is because there is a succession of images that need to be processed as the vehicle moves.
Some of the latest techniques here are based on the use of convolutional neural networks (CNN). These are R-CNN, Fast R-CNN and You Only Look Once (Yolo) methods (Redmon et al., 2016). RCNN first finds regions that contain potential objects in an image and then tries to analyze each region. This makes R-CNN somewhat slow and that is why there have been fast R-CNN methods developed and the Yolo method. Yolo works simultaneously to find the regions and to classify the objects in the regions by using a single convolutional neural network.This makes Yolo very fast compared to the other methods. In addition Yolo is able to see the entire image and does not suffer from the issues in R-CNN like mistaking background images for objects.
3.3. Decision Making Algorithms
Decision making determines the actions of the vehicle based on information from sensors. A vehicle constantly makes decision, based on its policy and the environment. The algorithms used for decision making are the following (Bugala, 2018):
- Decision Trees
- Support Vector Machine (SVM) Regression
- Deep Reinforcement Learning
4. Challenges of AI in Autonomous Vehicles
Some of the challenges of using artificial intelligence algorithms for autonomous vehicles are the same challenges that are universal for many other AI applications. The autonomous vehicles domain introduces some additional, unique challenges. These are concepts like real time, safety and machine ethics. The following subsections describe these areas in detail.
4.1 Real time response
Real time systems are specific class of embedded systems which have the characteristics to produce outputs or reactions within certain time restrictions. They have to be deterministic and minimalist in their design, so that they can always meet the expected real time behavior. For this to happen they often use special real time operating systems (RTOS) or bare metal executives that interact directly with the hardware, avoiding interpreted languages and dynamic memory allocation. In some cases real time systems intentionally use a subset of a programming language in order to guarantee speed and determinism.
The artificial intelligence solutions are normally on the opposite side of the spectrum — using higher level programming languages, full operating system dependencies, techniques like dynamic memory allocation and garbage collection. This creates a challenge in making real time guarantees with these systems.
Another aspect of these systems is whether they are centralized or distributed as described in (Carmody, Thomas, 2019). Centralized systems are easier to architect, but rely heavily on internal communications and a powerful central processor. Distributed system use dedicated CPUs that can handle different subsystems and sensors thus alleviating the need for a complex and powerful central CPU. Distributed architecture allow for systems with lower power consumption. They are also more flexible and can be cheaper.
4.2 Computational complexity
Artificial intelligence algorithms, especially deep learning, require special hardware solutions due to the amount of data and the complexity of computations. Hardware, like graphic processing units (GPU) or tensor processing units (TPU) can be highly optimized for fast parallel computation. The speed comes at a price of higher energy consumption and higher cost. Even with using specialized hardware there is still no certainty that a particular algorithms will be able to reach a solution in real time constraints. Therefore the selection of algorithms with regards to their complexity and CPU demands is a serious factor in real time systems. Usage of CPU computation and memory are the measures that can help determine if an algorithm is suitable for use in a typical CPU used in modern vehicles.
4.3 Black box behavior
AI algorithms have been criticized for being harder to analyze than standard computer algorithms. This comes from the fact that AI algorithms have a higher level of complexity and rely on a lot of data. A complex neural network can perform several AI tasks without understanding the process it controls.
In fact a deep neural network is approximating a function and this is part of its nature and explains why it can be so successful. The deep neural networks can have tens of thousands of nodes that can be trained to reach certain state. In addition they can have many hidden layers and many inputs and outputs. All this leads to problems when a system crashes and there is the need to do forensics analysis and find out the precise cause for the crash. This is an active area of research and successes in this area will solve many legal and technical issues related to using AI.
One idea is to use hybrid solutions which combine AI with traditional control algorithms (Abduljabbar and Dia, 2019). This issue is particularly important for accident prediction and handling (Narayanan, 2019). The decision history of why one decision was made versus another has to be justified by the AI algorithm and this is hard or impossible if the algorithm can be only analyzed and viewed as a black box.
4.4 Accuracy and Reliability
The computer vision applications used in autonomous vehicles may not be ready for prime time. The reason is that they can work in pristine conditions, but can fail with some even small disturbances at the sensor inputs. The training of AI algorithms happens slowly with training data that has certain characteristics.
Changing the data can make the classification and prediction algorithms change their behavior dramatically with catastrophic results. For example a person who carries a large bag, may not be perceived as a person by the vehicle.
In addition it is hard to predict what can happen with arbitrary data that can enter the system. This issue has been exploited by attackers who were able to fool deep learning algorithms by data in images that is not seen by humans, but makes the neural network stop classifying properly. There are clearly a lot of improvements needed to improve the reliability and accuracy of the machine learning algorithms.
4.5 Safety
The complexity of AI can lead to problems with safety. A more complicated system is one that is more difficult to develop, test and deploy. This is only part of the problem. Another reality is that new safety standards are now emerging and not yet adopted by the transportation industry (Carmody, Thomas, 2019).
What makes things even more complicated is that analysis and verification of AI systems is something that is still in its early stages and may not be tractable, because of their complexity. In general, formal verification is hard, even for traditional software systems and it is even harder for AI based software. The problem with creating robust, verified and validated AI solutions is something that we have to solve in general, not only for autonomous vehicles, but for all AI applications.
4.6 Security and AI
AI systems are so central to the autonomous vehicle that their security is directly related to robustness of the whole system. Machine learning systems are susceptible to attacks and there have been many examples of such cases (Newman, 2019).
The biggest threats are when an adversary can manipulate data that comes from sensors to the vehicle thus making the system make incorrect decisions. This behavior can be improved by training systems with data that has some adversarial characteristics. An experiment with a deep learning model for recognizing road signs showed that ”adding just a few black and white stickers to a stop sign tricked the algorithm into thinking it was a 45mph speed limit sign.” (Newman, 2019,p.17).
A big threat for affecting how AI operates in an autonomous vehicle is through perturbing sensor operation, which leads to changes in the sensor data stream and thus can completely confuse the AI algorithms. Solutions like hybrid systems, fuzzy logic and traditional control systems based on models to complement AI are some ways that researchers are trying to explore. The stakes are high, since AI can be found in some vehicle subsystems already and will be in every vehicle in the near future.
4.7 Ethics and AI
At this point, the ethical aspects of AI implementations in autonomous and semi-autonomous vehicles are not mature and are not significantly developed. Ethical values are a human quality, which is hard to formalize and implement in machines. In other words deciding what is right or wrong is kind of nebulous to a machine (Narayanan, 2019). Full development of machine ethics may be decades away. This is an area of active research which is a cross discipline between human psychology, machine learning and social policy. Possible some regulations can help in this direction while the technology becomes more mature.
5. Conclusion
AI solutions are everywhere today. They are part of devices like Alexa and Google Home, part of grass cutting robots and part of autonomous vehicles we expect to drive soon. The challenges and achievements of AI are common for many industries, but in the transportation industry and especially with autonomous vehicles we have extra challenges related to security and safety. Part of it is because of the complexity of the task to create a self driving vehicle and part of it is because of the level of development of the field of AI and its suitability to solve complex problems like object detection, route planning and real time decision making. Many of these obstacles will be resolved as we move forward towards more and more penetration of autonomous solutions in certain sectors of the economy. Maybe the right question to ask for this industry is ”Are we going to get there safely?” and not just ”Are we there yet ?”.
— Anton Hristozov graduated from the University of Pittsburgh with a Masters in Telecommunications and Information Science. He is currently pursuing a doctoral degree in Technology at Purdue University.