An adaptive control system is a system which can cope with changes and uncertainty of the plant. The adaptive controllers are playing a key role in intelligent control systems, which are required for complex systems. This paper surveys different methods in controlling a plant with uncertainties in the plant's model. The survey starts with theoretical adaptive controller schemes and moves to more abstract methodology of machine learning techniques in control.

Keywords: adaptive control, machine learning, neural networks, reinforcement learning, fuzzy logic

LTI, linear time invariant; NN, neural networks; FLC, fuzzy logic control; MRAC, model reference adaptive control; MRC, model reference control; RLC, reinforcement learning control; ANFIS, adaptive network-based fuzzy inference system.

Research on establishing a stable system using control theory has been ongoing for nearly two hundred years. Various methods and techniques have been developed throughout the years in order to control systems in a robust and stable manner. Naturally, as the controlled system is more complex, it is required to design more advanced control schemes. The most basic group of systems, (from here-on noted as a "plant"), which is analyzed and manipulated by a control system, are known linear time invariant (LTI) plants. For these sorts of systems, classic control theory produces remarkable benchmarks and is reviewed in many books and papers.¹ Modern control theory, such as optimal attempt to optimize the control law by applying a cost functional on the state and the control variables.^2,3 However, these methods rely on knowing the model of the plant or at least have a very good approximation.

In case the model of the plant is unknown or is non-linear, it is essential to generate a control scheme that is adaptive, and can change according to the plant's output. There are various methods for applying adaptive control and it depends on the problem at hand.⁴ In case the model of the system is known, but, the parameters of the model are unknown constants, Model Predictive Controllers (MRC) has shown remarkable results.⁵ Machine learning techniques have also shown great promise in the field of adaptive control. Methods, such as Neural Networks,⁶ reinforcement learning,⁷ and fuzzy logic,⁸ basically approximate a nonlinear function and therefore can provide a good representation of the nonlinear unknown plant. Machine learning is mostly used as a model-free controller. The plant is considered as a black box, and the input and output data from the plant is collected and later trained upon. After the training stage, the machine learning system represents the plant’s model and is able to control the plant without any need of a mathematical model. This paper provides a brief review on several adaptive control approaches, including theoretic approaches and machine learning based approaches. For each approach which will be reviewed a use case reference is provided.

In case the plant's model parameters are unknown, there are several methods in adaptive control and system identification to predict these parameters online and generate a control law for the system. One method that works extremely well and is the most used is Model Reference Adaptive Control (MRAC).⁵ MRAC changes the control law in a manner that will cause the system to mimic a reference model. This process is known as indirect control. In MRAC, the main idea is to have a reference model $y_m$  that has the same order as the original system; for a first order systems with unknown parameters a and b,

$\dot{x}+ax=bu$ (1)

Hence, the reference model would be,

${\dot{x}}_m+a_{m}x=b_{m}r$ (2)

The design objective is to make the tracking error $e=x-x_m$ converge to 0. By subtracting equation (2) from (1), we get

$\dot{e}+a_{m}e=b\left(u-a_x-a_{r}r\right)$  (3)

Where, $a_x=\frac{a-a_m}{b}$ ,$a_r=\frac{b_m}{b}$ .

The control law is given by (4) as,

$u={\widehat{a}}_{r}r+{\widehat{a}}_{x}x$ (4)

Where, it can be shown using the right Lyapunov function that the change in estimation of ${\widehat{a}}_r$  and ${\widehat{a}}_x$ can be calculated using (5) and (6).

${\widehat{a}}_r=-sgn\left(b\right){\gamma }_{r}er$ (5)

${\widehat{a}}_x=-sgn\left(b\right){\gamma }_{x}ex$ (6)

Where, ${\gamma }_r$  and ${\gamma }_x$ are tuning parameters.

A toy example is shown in Figure 1, for the system,

$\dot{x}=ax+u$  (7)

Where, the parameter a is unknown. If we choose a reference model as shown in (8)

${\dot{x}}_m=-2x_m+r$ (8)

The resulting control law $u={\widehat{a}}_{x}x+r$ with a tuning parameter of 0.5 will produce Figure 1. The main challenges of working with MRAC are that it requires an understanding of how the system actually operates. Providing a false reference model, could react with a devastating and unstable control law. In addition, in practice one should stop the adaptive component of the controller when the system is close to convergence.

Figure 1 Model reference adaptive control example.

Reinforcement learning control (RLC), which was initially introduced by Sutton et al.,⁷ and later improved by many others^9,10 is a natural approach. With RLC, the control system starts with no information whatsoever and solves the Bellman equation based on trial‑ and‑ error experience. Sutton et al.⁷ have shown that RLC is direct adaptive optimal control, and that the system will converge to the optimal solution given infinite amount of trials. Hwangbo et al.¹¹ have shown an example of training a controller for stabilizing a quad rotor using RLC. The RLC directly mapped the state of the quad rotor to actuator command, making any predefined control structure obsolete. The trained policy showed outstanding performance, without making any use of the model. The main disadvantage of the RLC is the long convergence time. Since the policy space can be extremely large, it requires a large number of iterations until convergence is reached. In addition, if the plant is unstable, or safety is an issue, using RLC in practice could be challenging.

Neural networks (NN) provide model-free learning controllers for a class of nonlinear systems, in the sense that a structural or parameterized model of the system dynamics is not needed.⁶ Throughout, there are repeatable design algorithms and guarantees of system performance including both small tracking errors and bounded NN weights.¹² It has been shown that as uncertainty about the controlled system increases or as one desires to consider human user inputs at higher levels of abstraction, the NN controllers acquire more and more structure, eventually acquiring a hierarchical structure that resembles some of the elegant architectures proposed by computer science engineers using high-level design approaches based on cognitive linguistics, reinforcement learning, psychological theories, adaptive critics, or optimal dynamic programming techniques¹³ Neural networks have shown great promise in complex non-linear system control.^14–16 One of the earlier successes in NN control was backing up a full trailer into a dock.¹⁷ It was shown in simulation how the backup of the full trailer into the dock was managed successfully after training several thousand times, thus replacing the need for a controller based on nonlinear mathematical equations. The disadvantage of the neural network control is that it does require a large training data to work with. It is not always simple to generate this data, and many times the data is generated by using a working controller, and extracting the plant’s input‑ output data for the training of the network. However, if this working controller already exists, the necessity of the NN controller is questionable.

The term "fuzzy" refers to the fact that the logic involved can deal with concepts that cannot be expressed as "true" or "false" but rather as "partially true." Fuzzy logic control (FLC) has met tremendous interest in applications over the past decades, especially due to the nonlinearities of the control and the fact that it is designed based more on human experience than on a model.¹⁸ The human experience is represented by a series of if then rules that correlates well-defined linguistic variables in the input to well-defined linguistic variables in the output. However, the fact remains that the performance of the resulting controller is difficult to analyze due to its nonlinear effects. It is even difficult to establish general and non-conservative criteria for such an essential, qualitative property as closed loop stability. Despite the drawbacks, there are many systems that utilize the fuzzy logic tool in order to develop complex adaptive systems. One example is for air‑conditioning control where it was shown that using an FLC instead of previous controller designs the FLC reduced power consumption by 24%, and using neuro-fuzzy controllers provides even better energy consumption.¹⁹ Another example is the Adaptive Network-based Fuzzy Inference System (a.k.a. ANFIS).²⁰ ANFIS is a fuzzy inference system that can construct an input-output mapping based on both human knowledge and stipulated input-output data pairs. The ANFIS architecture is employed to model nonlinear functions identify nonlinear components in a control system online. The ANFIS architecture has shown impressive results in system identification and in the control of stationary systems.^21,22

In this paper we have reviewed several self learning adaptive control schemes and applications. The MRAC scheme provides a simple control law by estimating the model’s parameters; the controller is straightforward and easy for implementation. The main disadvantage of the MRAC approach is that it requires an understanding of the plant’s dynamics. In addition, in practice the adaptive component should be turned off after convergence, and it is not always clear how to do this. The machine-learning controllers perform well even if the system is nonlinear and unknown. The main advantage of the machine learning controllers is that there is no need for a working knowledge of the plant’s mathematical model; the controller acts as a black box, and approximates the model based on the data gathered. However, it is hard to guarantee performance, and time to convergence can be very high.

No financial interest exists.

Author declares that no conflict of interest exists.