John S. Baras - US grants

The Systems Research Center, an Engineering research Center at the University of Maryland and Harvard University is pursuing theoretical and experimental studies and educational programs in systems aspects of manufacturing, communications and signal processing, chemical process systems, intelligent servomechanisms, and expert systems and parallel computer architectures. New approaches to optimization-based design of engineering systems have been developed in the intelligent servomechanisms area, and these have been found useful in several of the other thrust areas. In the coming period, the Center will build on its significant accomplishments in each of these areas, and will continue the process of integration of project efforts across all of the research areas. This action is a five-year renewal.

9727805
Baras
The proposed work aims to extend results in reinforcement learning theory to dynamic game problems relevant to output feedback robust nonlinear control. There are two primary motivations for this:
(a).To develop schemes to overcome the prohibitive computational cost encountered while designing and implementing robust nonlinear controllers.
(b).Employ the dynamic game framework as a stepping stone leading to the development of an analytical machinery suitable for posing, and solving intelligent control problems.
The former is concerned primarily with off-line schemes for approximating the key equations, and development of techniques to efficiently compute and represent the control policy. The latter is concerned with on-line schemes, where one needs to integrate identification, control, and the ability to improve performance in finite amount of time1 with finite computational resources. The latter has less available information on system model and environment; thus learning is an essential component of the methodology.
With these objectives in mind, special emphasis needs to be placed on obtaining algorithms that exhibit good finite time performance, and do so with finite amount of resources (computational). Furthermore, in order to efficiently integrate the components of the resulting architectures, one needs to also develop (finite time) performance bounds for these algorithms. The approach calls for first studying the problem in the context of finite state automata, and then extending the results to discrete time dynamical system models. The proposed work intends to study:
(a).Extensions of reinforcement learning to obtain finite time performance bounds.
(b).Development of schemes to directly identify the information most relevant for control (information state), and to do so with specified accuracy in a finite amount of time. This calls for the development of measures of risk to tradeoff exploration and control for on-line implementation.
(c)Model structures in. (b) that lead to reduction in complexity, and lend themselves to efficient learning.
(d).Extension of the current analytical framework for studying reinforcement learning to account for the unpredictability associated with intelligence.
(e).Exploiting the relationship between risk-sensitive control and dynamic games to harness the structure offered by probability theory.
(f).Development of architectures, and software that efflciently implement the algorithms obtained.
Results obtained from this research project, coupled with the development of appropriate complexity metrics would result in a framework for posing, and analyzing a wide variety of intelligent control problems. Such an approach would lead to controllers that are inherently robust, yet capable of adapting their behaviour to perceived changes in the system/environment. The results would be applicable to computation and implementation of robust nonlinear control at one end, to truly autonomous control for large, complex systems at the other. Specific applicatlon domains include chemical process control, semiconductor manufacturing, and control of large communication networks.
***

Abstract
0205330

The design, planning, control and management of high performance networks require a much more integrated approach than the conventional layered approach, where each layer is designed and optimized independently from the others. In this proposal the researchers propose to exploit inter-layer dependencies in network protocols for improved network performance. In particular, the researchers will focus on ad-hoc wireless networks, in which these interdependencies are more pronounced and in which the network will benefit significantly by crosslayer designs.

The main focus is on the interaction between the physical layer, the MAC layer, and the routing/transport layers. The researchers take into account the nature of the wireless medium by detailed modeling of the transmission parameters and of the detector structure and consider both TDMA(scheduled) and CDMAmedia-accesscontrol mechanisms. The researchers couple these with the flow and route assignment problems and, furthermore, consider how the transport protocol interacts with route selection and bandwidth allocation.

In addition, the researchers address the role of network control and management in ad-hoc wireless networks and exploit its interaction with the aforementioned layers. Finally, the researchers consider the interaction of signal compression with rate and quality control and are mindful of the energy consumption repercussions of the joint protocol design.

Systematic methodologies for the design of distributed and implementable routing and scheduling algorithms that enable one to design, provision and manage mobile wireless networks with predictable and controllable performance are lacking. The research project provides a new framework for modular cross-layer design of scheduling and routing algorithms for ad-hoc networks.

Clique based methods are used for scheduling, where cliques are defined in the interference graph. Clique based policies are developed to achieve optimal throughput and as basis for distributed implementable algorithms for scheduling. Clique based scheduling is easier and more flexible and provides a pathway to extend Network Calculus results, to provide deterministic performance bounds for wireless networks. For the routing, a component based design model is used that divides the routing protocol into components with separate design concerns. Stability, agility and flexibility are better achieved through a component based architecture. These solutions are still cross-layer, but they have well defined interfaces for signaling, control and information exchange between components and layers. Performance models provide a systematic methodology to study and quantify the relationship and sensitivity of the network performance to its components parameters.

The research will yield new principles and fundamental methodologies for the design, performance evaluation, and control of multi-hop wireless networks. Research results will be incorporated in communication, optimization and design courses at the graduate level. The results will be disseminated to industry and Government Labs. Validation and testing will be accomplished via emulation and real life wireless network testbeds in collaboration with industry and Government Labs.

Cyber-physical systems of the near future will collaborate with humans. Such cognitive systems will need to understand what the humans are doing. They will need to interpret human action in real-time and predict the humans' immediate intention in complex, noisy and cluttered environments. This proposal puts forward a new architecture for cognitive cyber-physical systems that can understand complex human activities, and focuses specifically on manipulation activities. The proposed architecture, motivated by biological perception and control, consists of three layers. At the bottom layer are vision processes that detect, recognize and track humans, their body parts, objects, tools, and object geometry. The middle layer contains symbolic models of the human activity, and it assembles through a grammatical description the recognized signal components of the previous layer into a representation of the ongoing activity. Finally, at the top layer is the cognitive control, which decides which parts of the scene will be processed next and which algorithms will be applied where. It modulates the vision processes by fetching additional knowledge when needed, and directs the attention by controlling the active vision system to direct its sensors to specific places. Thus, the bottom layer is the perception, the middle layer is the cognition, and the top layer is the control. All layers have access to a knowledge base, built in offline processes, which contains the semantics about the actions.

The feasibility of the approach will be demonstrated through the development of a smart manufacturing system, called MONA LISA, which assists humans in assembly tasks. This system will monitor humans as they perform assembly task. It will recognize the assembly action and determine whether it is correct and will communicate to the human possible errors and suggest ways to proceed. The system will have advanced visual sensing and perception; action understanding grounded in robotics and human studies; semantic and procedural-like memory and reasoning, and a control module linking high-level reasoning and low-level perception for real time, reactive and proactive engagement with the human assembler.

The proposed work will bring new tools and methodology to the areas of sensor networks and robotics and is applicable, besides smart manufacturing, to a large variety of sectors and applications. Being able to analyze human behavior using vision sensors will have impact on many sectors, ranging from healthcare and advanced driver assistance to human robot collaboration. The project will also catalyze K-12 outreach, new courseware (undergraduate and graduate), publication and open-source software.

The Institute for Systems Research (ISR), at the University of Maryland (UMD), College Park, will organize an International Symposium on Networked Cyber-Physical Systems (CPS) together with the Institute for Advanced Study (IAS), Technical University of Munich (TUM) in Germany. This first symposium of its kind in the new and emerging area of Networked CPS is to be held at the Institute for Advanced Study (IAS) of the Technical University of Munich (TUM) tentatively on September 19-20, 2016. The plan is to have this Symposium on an annual basis, with the location alternating each year between TUM, Munich, and UMD, College Park. This symposium will aim at the establishment of collaboration in the new area of Networked CPS at an international level. The broader societal impact of the proposed effort is likely to be high due to its strong relevance to a new generation of industry, namely exploiting the Internet-of-Things (IoT). The training, education, and workforce development involving under-represented groups are going to play a critical role in this new development and will be discussed as well.

At this workshop, the critical research challenges will be identified by world renowned researchers from several countries (USA, Germany, England, Switzerland, Greece, and The Netherlands), who are experts in a variety of fields related to the interdisciplinary area of Networked CPS. The workshop will also aim to explore the intellectual underpinnings of a highly important and emerging area, potentially having great global and technological impact due to its strong relevance to a new generation of industry, namely involving the so called the Internet-of-Things (IoT). The results of the workshop will be made available to the public in the form of a report. Requested funds will be used primarily to cover the expenses of plenary and invited speakers and researchers from the USA.

This project addresses the problem of determining the quality or value of the information delivered at a given point in space and time in Networked Control Systems. Modern networking systems often have the capability of sensing and actuating on the environment in an intelligent way. This capability strongly relies on guarantees that the right information at the right place and
at the right time is available to the system for deciding the right action to take. Failure of these guarantees can lead to catastrophic consequences in safety critical applications such as industrial processes control, autonomous navigation, robotics, automatic drug delivery, and so on. For this reason, adopting a proper definition of value of information in a networked control system is of paramount importance for national welfare. Our study has implications beyond the field of control and and information, in terms of the introduction of new mathematical methods and design tools. In laying a theoretical foundation, we expect to draw novel, synergistic connections between control, information, and real systems. The proposed research will also impact the training of a new generation of students through undergraduate student involvement, graduate mentoring and curriculum development, outreach activities, plans for broadening participation in computing and retention of minority students, and broad dissemination activities.

The importance of delivering the right information at the right place and at the right time has been recognized in the literature of networked control systems and it sparked the introduction of several utility functions including the Age-of-Information and the closely related Value-of-Information and Quality of Information. In this project our objective is to show that universal
measures, such as Age of Information, Value of Information, and Quality of Information, are insufficient to characterize the quality of information in terms of its utility for control of dynamical systems, and we characterize a system-dependent and task-dependent utility metric and describe its applications in the framework of control. On the one side, we consider the information flow
occurring in feedback loops governed by event-triggering control policies. We argue that focusing on event-triggering control will bring key insights on how to provide a definition with an immediate operational significance, which takes into account resolution, timing, reliability constraints, and relates them to the parameters of the system under control. On the other side, we consider a
general, and somewhat more informal definition, that has been adopted in diverse fields such as information economics, risk management, and stochastic programming, and we will cast it in a rigorous control-theoretic framework. This approach is fully compatible with the insights we get from event-triggering control, and generally applicable to networked control systems.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.