Thomas Goldstein, Ph.D. - US grants

Discrete programming (DP) deals with optimization problems involving variables that range over a discrete (e.g., integer-valued) solution space. DP is an important tool in a variety of practical applications including digital communications, operations research, power grid optimization, and computer vision. While discrete programs are typically solved offline by sophisticated software using powerful computers, DP has recently emerged as an important tool in applications requiring real-time processing in embedded systems with stringent area, cost, and power constraints. Since existing DP solvers entail prohibitive complexity and power consumption when implemented on existing embedded hardware, novel algorithms and hardware architectures are necessary to unlock the potential of DP in real-time applications. This project fuses optimization theory, numerical methods, and circuit design to develop fast algorithms and suitable hardware architectures for real-time DP in embedded systems. Besides a thorough theoretical analysis of the proposed methods, the project includes extensive software and hardware benchmarking to reveal the efficacy of real-time DP in practice. To bridge the ever-growing gap between recent advances in numerical optimization and hardware design, the project also includes the development of undergraduate and graduate courses that build upon the vertically-integrated research approach of this project, in addition to offering summer research internships (REUs) to introduce young scientists to the field of discrete programming.

The project develops a set of computationally efficient and hardware-aware algorithms and corresponding dedicated very-large scale integration (VLSI) architectures that enable DP for real-time embedded systems. The proposed DP algorithms rely on a variety of algorithmic transformations, ranging from semidefinite and infinity-norm-based relaxations to exact variable-splitting methods and non-convex approximations. These disparate approaches offer a wide range of tradeoffs between solution quality and hardware implementation complexity. The project studies these fundamental tradeoffs, as well as the effects of finite-precision arithmetic in VLSI, from both a theoretical and practical perspective. To carry out this investigation, three dedicated VLSI architectures will be developed that exploit the inherent parallelism of the proposed algorithms. These architectures target (i) data detection in multi-antenna (MIMO) wireless systems that is the key bottleneck in next-generation communication systems, (ii) signal recovery problems in hyperspectral imaging, and (iii) phase retrieval problems from x-ray crystallography. By investigating the domain-specific performance and complexity of various numerical solvers in a variety of conditions and hardware configurations, the project will reveal the efficacy and limits of DP for a broad range of real-time applications beyond the ones studied in this project.

Recent advances in machine learning and AI, particularly those based on artificial neural networks, have enabled us to build systems that solve difficult information processing problems with human-like accuracy. For example, neural networks can recognize objects, predict how proteins fold, automate manufacturing processing, and use computer vision to navigate a vehicle or analyze satellite imagery. Unfortunately, these advanced AI systems come with their own unique problems. Like humans, neural networks can behave erratically, sometimes making strange and unexplainable decisions when asked to perform tasks that differ even a little from their training. For this reason, classifical image and signal processing methods are still the go-to solution when reliability, interpretability, and computational speed at needed. The goal of this research project is to mash up the performance and power of neural networks with the speed and reliability and classical algorithms. This research project also features an integrated teaching plan involving graduate students and undergraduate interns.

To achieve this goal, we consider three interrelated research thrusts. First, we consider ways that deep networks can help to automate and improve classical algorithms. For example, networks can be used to automate the selection of hyper-parameters, choose objective functions to minimize, identify noise types and levels that are present in data, and make other decisions that are needed to optimally tune the performance of classical imaging system. Second, we consider ways that neural networks can be 'plugged in' to classical variational imaging methods. For example, classical image priors (such as wavelet sparsity or total variation), can be replaced with more sophisticated priors defined by neural networks. Third, we consider efficient algorithms for solving minimization problems that arise when complex neural networks are used as components in classical optimization problems. Better algorithms will allow us to solve these complex problems efficiently, and without human oversight. This new suite of approaches has the potential to improve that state of the art for a range of important practical problems that have been studied by the PI. This includes enhancing deblurring problems of the type used for microscopy of new materials, boosting segmentation algorithms used to identify faults in semiconductor manufacturing, and solving complex resource allocation problems for medical applications.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Generalization, or the ability to transfer knowledge from one context to the next, is a hallmark of human intelligence. In artificial intelligence (AI), however, models trained in one setting often fail when tested in a new setting, even if the shift is minor or imperceptible. To build more generalizable AI, most modern methods employ some form of data augmentation (DA), which applies transformations to the data to create virtual samples that are then added to the dataset. The resulting synthesis of new examples appears to build helpful properties in AI such as invariance or resistance to change to certain natural transformations, and robustness to new tasks as well as noise in existing tasks. Despite the promise and performance of DA procedures, they are mostly applied in an ad-hoc manner and need to be designed and tested on a dataset by dataset basis. A set of fundamental principles and theory to understand DA and its impact on model training and testing is lacking. To address this outstanding challenge, the investigators will provide a precise understanding of the impact of DA on generalization, and leverage this understanding to design novel augmentations that can be used across multiple applications and domains.<br/><br/>In this project, the investigators propose a principled mathematical framework to 1) understand when DA helps and when DA could potentially hurt learning, 2) understand the structure induced by DA and characterize what makes high-quality augmentations, and 3) provide novel, systematic, and scalable design principles for augmenting data in new domains where we lack prior knowledge to guide us. These design principles will significantly broaden the applicability and promise of DA from computer vision to new domains (e.g., neural data, graphs and tabular data) where principled augmentations are still not known. Of special focus in this project will be applications of DA to neural activity, where augmentations have shown promise in building a more generalizable link between the brain and behavior. This research will also yield prescriptions for the role of DA in advancing fairness, accountability and transparency in modern machine learning.<br/><br/>This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.