We gratefully acknowledge support from
the Simons Foundation and member institutions.

Computer Science

New submissions

[ total of 452 entries: 1-452 ]
[ showing up to 2000 entries per page: fewer | more ]

New submissions for Wed, 18 May 22

[1]  arXiv:2205.07851 [pdf, other]
Title: ST-ExpertNet: A Deep Expert Framework for Traffic Prediction
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Recently, forecasting the crowd flows has become an important research topic, and plentiful technologies have achieved good performances. As we all know, the flow at a citywide level is in a mixed state with several basic patterns (e.g., commuting, working, and commercial) caused by the city area functional distributions (e.g., developed commercial areas, educational areas and parks). However, existing technologies have been criticized for their lack of considering the differences in the flow patterns among regions since they want to build only one comprehensive model to learn the mixed flow tensors. Recognizing this limitation, we present a new perspective on flow prediction and propose an explainable framework named ST-ExpertNet, which can adopt every spatial-temporal model and train a set of functional experts devoted to specific flow patterns. Technically, we train a bunch of experts based on the Mixture of Experts (MoE), which guides each expert to specialize in different kinds of flow patterns in sample spaces by using the gating network. We define several criteria, including comprehensiveness, sparsity, and preciseness, to construct the experts for better interpretability and performances. We conduct experiments on a wide range of real-world taxi and bike datasets in Beijing and NYC. The visualizations of the expert's intermediate results demonstrate that our ST-ExpertNet successfully disentangles the city's mixed flow tensors along with the city layout, e.g., the urban ring road structure. Different network architectures, such as ST-ResNet, ConvLSTM, and CNN, have been adopted into our ST-ExpertNet framework for experiments and the results demonstrates the superiority of our framework in both interpretability and performances.

[2]  arXiv:2205.07852 [pdf, other]
Title: REMuS-GNN: A Rotation-Equivariant Model for Simulating Continuum Dynamics
Comments: Accepted at the ICLR 2022 Workshop on Geometrical and Topological Representation Learning
Subjects: Machine Learning (cs.LG); Fluid Dynamics (physics.flu-dyn)

Numerical simulation is an essential tool in many areas of science and engineering, but its performance often limits application in practice or when used to explore large parameter spaces. On the other hand, surrogate deep learning models, while accelerating simulations, often exhibit poor accuracy and ability to generalise. In order to improve these two factors, we introduce REMuS-GNN, a rotation-equivariant multi-scale model for simulating continuum dynamical systems encompassing a range of length scales. REMuS-GNN is designed to predict an output vector field from an input vector field on a physical domain discretised into an unstructured set of nodes. Equivariance to rotations of the domain is a desirable inductive bias that allows the network to learn the underlying physics more efficiently, leading to improved accuracy and generalisation compared with similar architectures that lack such symmetry. We demonstrate and evaluate this method on the incompressible flow around elliptical cylinders.

[3]  arXiv:2205.07853 [pdf, other]
Title: Heterogeneous Domain Adaptation with Adversarial Neural Representation Learning: Experiments on E-Commerce and Cybersecurity
Comments: Forthcoming in IEEE Transactions on Pattern Recognition and Machine Intelligence (TPAMI)
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Learning predictive models in new domains with scarce training data is a growing challenge in modern supervised learning scenarios. This incentivizes developing domain adaptation methods that leverage the knowledge in known domains (source) and adapt to new domains (target) with a different probability distribution. This becomes more challenging when the source and target domains are in heterogeneous feature spaces, known as heterogeneous domain adaptation (HDA). While most HDA methods utilize mathematical optimization to map source and target data to a common space, they suffer from low transferability. Neural representations have proven to be more transferable; however, they are mainly designed for homogeneous environments. Drawing on the theory of domain adaptation, we propose a novel framework, Heterogeneous Adversarial Neural Domain Adaptation (HANDA), to effectively maximize the transferability in heterogeneous environments. HANDA conducts feature and distribution alignment in a unified neural network architecture and achieves domain invariance through adversarial kernel learning. Three experiments were conducted to evaluate the performance against the state-of-the-art HDA methods on major image and text e-commerce benchmarks. HANDA shows statistically significant improvement in predictive performance. The practical utility of HANDA was shown in real-world dark web online markets. HANDA is an important step towards successful domain adaptation in e-commerce applications.

[4]  arXiv:2205.07854 [pdf, other]
Title: Functional2Structural: Cross-Modality Brain Networks Representation Learning
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV); Neurons and Cognition (q-bio.NC)

MRI-based modeling of brain networks has been widely used to understand functional and structural interactions and connections among brain regions, and factors that affect them, such as brain development and disease. Graph mining on brain networks may facilitate the discovery of novel biomarkers for clinical phenotypes and neurodegenerative diseases. Since brain networks derived from functional and structural MRI describe the brain topology from different perspectives, exploring a representation that combines these cross-modality brain networks is non-trivial. Most current studies aim to extract a fused representation of the two types of brain network by projecting the structural network to the functional counterpart. Since the functional network is dynamic and the structural network is static, mapping a static object to a dynamic object is suboptimal. However, mapping in the opposite direction is not feasible due to the non-negativity requirement of current graph learning techniques. Here, we propose a novel graph learning framework, known as Deep Signed Brain Networks (DSBN), with a signed graph encoder that, from an opposite perspective, learns the cross-modality representations by projecting the functional network to the structural counterpart. We validate our framework on clinical phenotype and neurodegenerative disease prediction tasks using two independent, publicly available datasets (HCP and OASIS). The experimental results clearly demonstrate the advantages of our model compared to several state-of-the-art methods.

[5]  arXiv:2205.07855 [pdf, other]
Title: Decentral and Incentivized Federated Learning Frameworks: A Systematic Literature Review
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)

The advent of Federated Learning (FL) has ignited a new paradigm for parallel and confidential decentralized Machine Learning (ML) with the potential of utilizing the computational power of a vast number of IoT, mobile and edge devices without data leaving the respective device, ensuring privacy by design. Yet, in order to scale this new paradigm beyond small groups of already entrusted entities towards mass adoption, the Federated Learning Framework (FLF) has to become (i) truly decentralized and (ii) participants have to be incentivized. This is the first systematic literature review analyzing holistic FLFs in the domain of both, decentralized and incentivized federated learning. \rawresults publications were retrieved, by querying 12 major scientific databases. Finally, \finalresults articles remained after a systematic review and filtering process for in-depth examination. Although having massive potential to direct the future of a more distributed and secure AI, none of the analyzed FLF is production-ready. The approaches vary heavily in terms of use-cases, system design, solved issues and thoroughness. We are the first to provide a systematic approach to classify and quantify differences between FLF, exposing limitations of current works and derive future directions for research in this novel domain.

[6]  arXiv:2205.07856 [pdf, other]
Title: Impact of Learning Rate on Noise Resistant Property of Deep Learning Models
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

The interest in analog computation has grown tremendously in recent years due to its fast computation speed and excellent energy efficiency, which is very important for edge and IoT devices in the sub-watt power envelope for deep learning inferencing. However, significant performance degradation suffered by deep learning models due to the inherent noise present in the analog computation can limit their use in mission-critical applications. Hence, there is a need to understand the impact of critical model hyperparameters choice on the resulting model noise-resistant property. This need is critical as the insight obtained can be used to design deep learning models that are robust to analog noise. In this paper, the impact of the learning rate, a critical design choice, on the noise-resistant property is investigated. The study is achieved by first training deep learning models using different learning rates. Thereafter, the models are injected with analog noise and the noise-resistant property of the resulting models is examined by measuring the performance degradation due to the analog noise. The results showed there exists a sweet spot of learning rate values that achieves a good balance between model prediction performance and model noise-resistant property. Furthermore, the theoretical justification of the observed phenomenon is provided.

[7]  arXiv:2205.07857 [pdf, other]
Title: Neural Program Synthesis with Query
Comments: Accepted by ICLR 2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Aiming to find a program satisfying the user intent given input-output examples, program synthesis has attracted increasing interest in the area of machine learning. Despite the promising performance of existing methods, most of their success comes from the privileged information of well-designed input-output examples. However, providing such input-output examples is unrealistic because it requires the users to have the ability to describe the underlying program with a few input-output examples under the training distribution. In this work, we propose a query-based framework that trains a query neural network to generate informative input-output examples automatically and interactively from a large query space. The quality of the query depends on the amount of the mutual information between the query and the corresponding program, which can guide the optimization of the query framework. To estimate the mutual information more accurately, we introduce the functional space (F-space) which models the relevance between the input-output examples and the programs in a differentiable way. We evaluate the effectiveness and generalization of the proposed query-based framework on the Karel task and the list processing task. Experimental results show that the query-based framework can generate informative input-output examples which achieve and even outperform well-designed input-output examples.

[8]  arXiv:2205.07858 [pdf, other]
Title: Predicting tacrolimus exposure in kidney transplanted patients using machine learning
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Quantitative Methods (q-bio.QM)

Tacrolimus is one of the cornerstone immunosuppressive drugs in most transplantation centers worldwide following solid organ transplantation. Therapeutic drug monitoring of tacrolimus is necessary in order to avoid rejection of the transplanted organ or severe side effects. However, finding the right dose for a given patient is challenging, even for experienced clinicians. Consequently, a tool that can accurately estimate the drug exposure for individual dose adaptions would be of high clinical value. In this work, we propose a new technique using machine learning to estimate the tacrolimus exposure in kidney transplant recipients. Our models achieve predictive errors that are at the same level as an established population pharmacokinetic model, but are faster to develop and require less knowledge about the pharmacokinetic properties of the drug.

[9]  arXiv:2205.07859 [pdf, other]
Title: Btech thesis report on adversarial attack detection and purification of adverserially attacked images
Authors: Dvij Kalaria
Comments: Btech thesis report of Dvij Kalaria, Indian Institute of Technology Kharagpur. arXiv admin note: substantial text overlap with arXiv:2111.15518; substantial text overlap with arXiv:1911.05268 by other authors
Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR)

This is Btech thesis report on detection and purification of adverserially attacked images. A deep learning model is trained on certain training examples for various tasks such as classification, regression etc. By training, weights are adjusted such that the model performs the task well not only on training examples judged by a certain metric but has an excellent ability to generalize on other unseen examples as well which are typically called the test data. Despite the huge success of machine learning models on a wide range of tasks, security has received a lot less attention along the years. Robustness along various potential cyber attacks also should be a metric for the accuracy of the machine learning models. These cyber attacks can potentially lead to a variety of negative impacts in the real world sensitive applications for which machine learning is used such as medical and transportation systems. Hence, it is a necessity to secure the system from such attacks. Int this report, I focus on a class of these cyber attacks called the adversarial attacks in which the original input sample is modified by small perturbations such that they still look visually the same to human beings but the machine learning models are fooled by such inputs. In this report I discuss 2 novel ways to counter the adversarial attack using AutoEncoders, 1) by detecting the presence of adversaries and 2) purifying these adversaries to make target classification models robust against such attacks.

[10]  arXiv:2205.07860 [pdf, other]
Title: AdaCap: Adaptive Capacity control for Feed-Forward Neural Networks
Subjects: Machine Learning (cs.LG)

The capacity of a ML model refers to the range of functions this model can approximate. It impacts both the complexity of the patterns a model can learn but also memorization, the ability of a model to fit arbitrary labels. We propose Adaptive Capacity (AdaCap), a training scheme for Feed-Forward Neural Networks (FFNN). AdaCap optimizes the capacity of FFNN so it can capture the high-level abstract representations underlying the problem at hand without memorizing the training dataset. AdaCap is the combination of two novel ingredients, the Muddling labels for Regularization (MLR) loss and the Tikhonov operator training scheme. The MLR loss leverages randomly generated labels to quantify the propensity of a model to memorize. We prove that the MLR loss is an accurate in-sample estimator for out-of-sample generalization performance and that it can be used to perform Hyper-Parameter Optimization provided a Signal-to-Noise Ratio condition is met. The Tikhonov operator training scheme modulates the capacity of a FFNN in an adaptive, differentiable and data-dependent manner. We assess the effectiveness of AdaCap in a setting where DNN are typically prone to memorization, small tabular datasets, and benchmark its performance against popular machine learning methods.

[11]  arXiv:2205.07861 [pdf, other]
Title: Depression Diagnosis and Forecast based on Mobile Phone Sensor Data
Comments: Accepted by EMBC 2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Previous studies have shown the correlation between sensor data collected from mobile phones and human depression states. Compared to the traditional self-assessment questionnaires, the passive data collected from mobile phones is easier to access and less time-consuming. In particular, passive mobile phone data can be collected on a flexible time interval, thus detecting moment-by-moment psychological changes and helping achieve earlier interventions. Moreover, while previous studies mainly focused on depression diagnosis using mobile phone data, depression forecasting has not received sufficient attention. In this work, we extract four types of passive features from mobile phone data, including phone call, phone usage, user activity, and GPS features. We implement a long short-term memory (LSTM) network in a subject-independent 10-fold cross-validation setup to model both a diagnostic and a forecasting tasks. Experimental results show that the forecasting task achieves comparable results with the diagnostic task, which indicates the possibility of forecasting depression from mobile phone sensor data. Our model achieves an accuracy of 77.0 % for major depression forecasting (binary), an accuracy of 53.7 % for depression severity forecasting (5 classes), and a best RMSE score of 4.094 (PHQ-9, range from 0 to 27).

[12]  arXiv:2205.07862 [pdf, other]
Title: A Safety Assurable Human-Inspired Perception Architecture
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Robotics (cs.RO); Systems and Control (eess.SY)

Although artificial intelligence-based perception (AIP) using deep neural networks (DNN) has achieved near human level performance, its well-known limitations are obstacles to the safety assurance needed in autonomous applications. These include vulnerability to adversarial inputs, inability to handle novel inputs and non-interpretability. While research in addressing these limitations is active, in this paper, we argue that a fundamentally different approach is needed to address them. Inspired by dual process models of human cognition, where Type 1 thinking is fast and non-conscious while Type 2 thinking is slow and based on conscious reasoning, we propose a dual process architecture for safe AIP. We review research on how humans address the simplest non-trivial perception problem, image classification, and sketch a corresponding AIP architecture for this task. We argue that this architecture can provide a systematic way of addressing the limitations of AIP using DNNs and an approach to assurance of human-level performance and beyond. We conclude by discussing what components of the architecture may already be addressed by existing work and what remains future work.

[13]  arXiv:2205.07863 [pdf, other]
Title: Quality versus speed in energy demand prediction for district heating systems
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Systems and Control (eess.SY)

In this paper, we consider energy demand prediction in district heating systems. Effective energy demand prediction is essential in combined heat power systems when offering electrical energy in competitive electricity markets. To address this problem, we propose two sets of algorithms: (1) a novel extension to the algorithm proposed by E. Dotzauer and (2) an autoregressive predictor based on hour-of-week adjusted linear regression on moving averages of energy consumption. These two methods are compared against state-of-the-art artificial neural networks. Energy demand predictor algorithms have various computational costs and prediction quality. While prediction quality is a widely used measure of predictor superiority, computational costs are less frequently analyzed and their impact is not so extensively studied. When predictor algorithms are constantly updated using new data, some computationally expensive forecasting methods may become inapplicable. The computational costs can be split into training and execution parts. The execution part is the cost paid when the already trained algorithm is applied to predict something. In this paper, we evaluate the above methods with respect to the quality and computational costs, both in the training and in the execution. The comparison is conducted on a real-world dataset from a district heating system in the northwest part of Poland.

[14]  arXiv:2205.07864 [pdf, other]
Title: Privacy Enhancement for Cloud-Based Few-Shot Learning
Comments: 14 pages, 13 figures, 3 tables. Preprint. Accepted in IEEE WCCI 2022 International Joint Conference on Neural Networks (IJCNN)
Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)

Requiring less data for accurate models, few-shot learning has shown robustness and generality in many application domains. However, deploying few-shot models in untrusted environments may inflict privacy concerns, e.g., attacks or adversaries that may breach the privacy of user-supplied data. This paper studies the privacy enhancement for the few-shot learning in an untrusted environment, e.g., the cloud, by establishing a novel privacy-preserved embedding space that preserves the privacy of data and maintains the accuracy of the model. We examine the impact of various image privacy methods such as blurring, pixelization, Gaussian noise, and differentially private pixelization (DP-Pix) on few-shot image classification and propose a method that learns privacy-preserved representation through the joint loss. The empirical results show how privacy-performance trade-off can be negotiated for privacy-enhanced few-shot learning.

[15]  arXiv:2205.07865 [pdf, other]
Title: Simple Contrastive Graph Clustering
Comments: 9 pages, 6 figures
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Contrastive learning has recently attracted plenty of attention in deep graph clustering for its promising performance. However, complicated data augmentations and time-consuming graph convolutional operation undermine the efficiency of these methods. To solve this problem, we propose a Simple Contrastive Graph Clustering (SCGC) algorithm to improve the existing methods from the perspectives of network architecture, data augmentation, and objective function. As to the architecture, our network includes two main parts, i.e., pre-processing and network backbone. A simple low-pass denoising operation conducts neighbor information aggregation as an independent pre-processing, and only two multilayer perceptrons (MLPs) are included as the backbone. For data augmentation, instead of introducing complex operations over graphs, we construct two augmented views of the same vertex by designing parameter un-shared siamese encoders and corrupting the node embeddings directly. Finally, as to the objective function, to further improve the clustering performance, a novel cross-view structural consistency objective function is designed to enhance the discriminative capability of the learned network. Extensive experimental results on seven benchmark datasets validate our proposed algorithm's effectiveness and superiority. Significantly, our algorithm outperforms the recent contrastive deep clustering competitors with at least seven times speedup on average.

[16]  arXiv:2205.07867 [pdf, other]
Title: Feature and Instance Joint Selection: A Reinforcement Learning Perspective
Comments: Accepted by IJCAI-ECAI 2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Feature selection and instance selection are two important techniques of data processing. However, such selections have mostly been studied separately, while existing work towards the joint selection conducts feature/instance selection coarsely; thus neglecting the latent fine-grained interaction between feature space and instance space. To address this challenge, we propose a reinforcement learning solution to accomplish the joint selection task and simultaneously capture the interaction between the selection of each feature and each instance. In particular, a sequential-scanning mechanism is designed as action strategy of agents, and a collaborative-changing environment is used to enhance agent collaboration. In addition, an interactive paradigm introduces prior selection knowledge to help agents for more efficient exploration. Finally, extensive experiments on real-world datasets have demonstrated improved performances.

[17]  arXiv:2205.07868 [pdf, other]
Title: Minimal Neural Network Models for Permutation Invariant Agents
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)

Organisms in nature have evolved to exhibit flexibility in face of changes to the environment and/or to themselves. Artificial neural networks (ANNs) have proven useful for controlling of artificial agents acting in environments. However, most ANN models used for reinforcement learning-type tasks have a rigid structure that does not allow for varying input sizes. Further, they fail catastrophically if inputs are presented in an ordering unseen during optimization. We find that these two ANN inflexibilities can be mitigated and their solutions are simple and highly related. For permutation invariance, no optimized parameters can be tied to a specific index of the input elements. For size invariance, inputs must be projected onto a common space that does not grow with the number of projections. Based on these restrictions, we construct a conceptually simple model that exhibit flexibility most ANNs lack. We demonstrate the model's properties on multiple control problems, and show that it can cope with even very rapid permutations of input indices, as well as changes in input size. Ablation studies show that is possible to achieve these properties with simple feedforward structures, but that it is much easier to optimize recurrent structures.

[18]  arXiv:2205.07870 [pdf, other]
Title: Unsupervised Driving Behavior Analysis using Representation Learning and Exploiting Group-based Training
Comments: 7 figures, 8 pages , 7 tables, accepted and presented conference AAAI 2022 AI for Transportation Workshop (Prefinal version)
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Driving behavior monitoring plays a crucial role in managing road safety and decreasing the risk of traffic accidents. Driving behavior is affected by multiple factors like vehicle characteristics, types of roads, traffic, but, most importantly, the pattern of driving of individuals. Current work performs a robust driving pattern analysis by capturing variations in driving patterns. It forms consistent groups by learning compressed representation of time series (Auto Encoded Compact Sequence) using a multi-layer seq-2-seq autoencoder and exploiting hierarchical clustering along with recommending the choice of best distance measure. Consistent groups aid in identifying variations in driving patterns of individuals captured in the dataset. These groups are generated for both train and hidden test data. The consistent groups formed using train data, are exploited for training multiple instances of the classifier. Obtained choice of best distance measure is used to select the best train-test pair of consistent groups. We have experimented on the publicly available UAH-DriveSet dataset considering the signals captured from IMU sensors (accelerometer and gyroscope) for classifying driving behavior. We observe proposed method, significantly outperforms the benchmark performance.

[19]  arXiv:2205.07871 [pdf, other]
Title: Mondrian Forest for Data Stream Classification Under Memory Constraints
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Supervised learning algorithms generally assume the availability of enough memory to store their data model during the training and test phases. However, in the Internet of Things, this assumption is unrealistic when data comes in the form of infinite data streams, or when learning algorithms are deployed on devices with reduced amounts of memory. In this paper, we adapt the online Mondrian forest classification algorithm to work with memory constraints on data streams. In particular, we design five out-of-memory strategies to update Mondrian trees with new data points when the memory limit is reached. Moreover, we design trimming mechanisms to make Mondrian trees more robust to concept drifts under memory constraints. We evaluate our algorithms on a variety of real and simulated datasets, and we conclude with recommendations on their use in different situations: the Extend Node strategy appears as the best out-of-memory strategy in all configurations, whereas different trimming mechanisms should be adopted depending on whether a concept drift is expected. All our methods are implemented in the OrpailleCC open-source library and are ready to be used on embedded systems and connected objects.

[20]  arXiv:2205.07872 [pdf, other]
Title: ScAN: Suicide Attempt and Ideation Events Dataset
Comments: Paper accepted at NAACL 2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

Suicide is an important public health concern and one of the leading causes of death worldwide. Suicidal behaviors, including suicide attempts (SA) and suicide ideations (SI), are leading risk factors for death by suicide. Information related to patients' previous and current SA and SI are frequently documented in the electronic health record (EHR) notes. Accurate detection of such documentation may help improve surveillance and predictions of patients' suicidal behaviors and alert medical professionals for suicide prevention efforts. In this study, we first built Suicide Attempt and Ideation Events (ScAN) dataset, a subset of the publicly available MIMIC III dataset spanning over 12k+ EHR notes with 19k+ annotated SA and SI events information. The annotations also contain attributes such as method of suicide attempt. We also provide a strong baseline model ScANER (Suicide Attempt and Ideation Events Retriever), a multi-task RoBERTa-based model with a retrieval module to extract all the relevant suicidal behavioral evidences from EHR notes of an hospital-stay and, and a prediction module to identify the type of suicidal behavior (SA and SI) concluded during the patient's stay at the hospital. ScANER achieved a macro-weighted F1-score of 0.83 for identifying suicidal behavioral evidences and a macro F1-score of 0.78 and 0.60 for classification of SA and SI for the patient's hospital-stay, respectively. ScAN and ScANER are publicly available.

[21]  arXiv:2205.07873 [pdf, other]
Title: HDR Lighting Dilation for Dynamic Range Reduction on Virtual Production Stages
Subjects: Graphics (cs.GR)

We present a technique to reduce the dynamic range of an HDRI lighting environment map in an efficient, energy-preserving manner by spreading out the light of concentrated light sources. This allows us to display a reasonable approximation of the illumination of an HDRI map in a lighting reproduction system with limited dynamic range such as virtual production LED Stage. The technique identifies regions of the HDRI map above a given pixel threshold, dilates these regions until the average pixel value within each is below the threshold, and finally replaces each dilated region's pixels with the region's average pixel value. The new HDRI map contains the same energy as the original, spreads the light as little as possible, and avoids chromatic fringing.

[22]  arXiv:2205.07874 [pdf, other]
Title: Revisiting the Updates of a Pre-trained Model for Few-shot Learning
Comments: 18 pages, 11 figures, 8 tables
Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)

Most of the recent few-shot learning algorithms are based on transfer learning, where a model is pre-trained using a large amount of source data, and the pre-trained model is updated using a small amount of target data afterward. In transfer-based few-shot learning, sophisticated pre-training methods have been widely studied for universal and improved representation. However, there is little study on updating pre-trained models for few-shot learning. In this paper, we compare the two popular updating methods, fine-tuning (i.e., updating the entire network) and linear probing (i.e., updating only the linear classifier), considering the distribution shift between the source and target data. We find that fine-tuning is better than linear probing as the number of samples increases, regardless of distribution shift. Next, we investigate the effectiveness and ineffectiveness of data augmentation when pre-trained models are fine-tuned. Our fundamental analyses demonstrate that careful considerations of the details about updating pre-trained models are required for better few-shot performance.

[23]  arXiv:2205.07875 [pdf]
Title: The Hard Truth about Soft Skills in Game Development
Subjects: Graphics (cs.GR); Human-Computer Interaction (cs.HC)

This article explores the value and measurable effects of hard and soft skills in academia when teaching and developing abilities for the game industry. As we discuss, each individuals engagement with the subject directly impacts their performance; which is influenced by their 'soft' skill level. Students that succeed in mastering soft skills earlier on typically have a greater understanding and satisfaction of the subject (able to see the underlying heterogeneous nature of the material). As soft and hard skill don't just help individuals achieve their goals (qualifications), they also change their mindset. While it is important to master both hard and soft skills, often when we talk about the quality of education (for game development); the measure is more towards quantitative measures and assessments (which don't always sit well with soft skills). As it is easy to forget, in this digital age, that 'people' are at the heart of video game development. Not just about 'code' and 'technologies'. There exists a complex relationship between hard and soft skills and their dual importance is crucial if graduates are to succeed in the game industry.

[24]  arXiv:2205.07877 [pdf]
Title: A Comprehensive Survey on Model Quantization for Deep Neural Networks
Subjects: Machine Learning (cs.LG)

Recent advances in machine learning by deep neural networks are significant. But using these networks has been accompanied by a huge number of parameters for storage and computations that leads to an increase in the hardware cost and posing challenges. Therefore, compression approaches have been proposed to design efficient accelerators. One important approach for deep neural network compression is quantization that full-precision values are stored in low bit-width. In this way, in addition to memory saving, the operations will be replaced by simple ones with low cost. Many methods are suggested for DNNs Quantization in recent years, because of flexibility and influence in designing efficient hardware. Therefore, an integrated report is essential for better understanding, analysis, and comparison. In this paper, we provide a comprehensive survey. We describe the quantization concepts and categorize the methods from different perspectives. We discuss using the scale factor to match the quantization levels with the distribution of the full-precision values and describe the clustering-based methods. For the first time, we review the training of a quantized deep neural network and using Straight-Through Estimator comprehensively. Also, we describe the simplicity of operations in quantized deep convolutional neural networks and explain the sensitivity of the different layers in quantization. Finally, we discuss the evaluation of the quantization methods and compare the accuracy of previous methods with various bit-width for weights and activations on CIFAR-10 and the large-scale dataset, ImageNet.

[25]  arXiv:2205.07881 [pdf]
Title: Developing patient-driven artificial intelligence based on personal rankings of care decision making steps
Authors: Lauri Lahti (1) ((1) Department of Computer Science, Aalto University School of Science, Finland)
Comments: Corresponding author: Lauri Lahti (email: [email protected]). This research article manuscript version was completed on 11 May 2022 and it was self-archived on the open-access Arxiv repository (this https URL) on 11 May 2022. This research article (104 pages, 17 tables and 17 figures) is supplemented with seven supplementing documents (2781 pages)
Subjects: Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)

We propose and experimentally motivate a new methodology to support decision-making processes in healthcare with artificial intelligence based on personal rankings of care decision making steps that can be identified with our methodology, questionnaire data and its statistical patterns. Our longitudinal quantitative cross-sectional three-stage study gathered self-ratings for 437 expression statements concerning healthcare situations on Likert scales in respect to "the need for help", "the advancement of health", "the hopefulness", "the indication of compassion" and "the health condition", and 45 answers about the person's demographics, health and wellbeing, also the duration of giving answers. Online respondents between 1 June 2020 and 29 June 2021 were recruited from Finnish patient and disabled people's organizations, other health-related organizations and professionals, and educational institutions (n=1075). With Kruskal-Wallis test, Wilcoxon rank-sum test (i.e., Mann-Whitney U test), Wilcoxon rank-sum pairwise test, Welch's t test and one-way analysis of variance (ANOVA) between groups test we identified statistically significant differences of ratings and their durations for each expression statement in respect to respondent groupings based on the answer values of each background question. Frequencies of the later reordering of rating rankings showed dependencies with ratings given earlier in respect to various interpretation task entities, interpretation dimensions and respondent groupings. Our methodology, questionnaire data and its statistical patterns enable analyzing with self-rated expression statements the representations of decision making steps in healthcare situations and their chaining, agglomeration and branching in knowledge entities of personalized care paths. Our results support building artificial intelligence solutions to address the patient's needs concerning care.

[26]  arXiv:2205.07882 [pdf, other]
Title: Aligning Robot Representations with Humans
Subjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Robotics (cs.RO)

As robots are increasingly deployed in real-world scenarios, a key question is how to best transfer knowledge learned in one environment to another, where shifting constraints and human preferences render adaptation challenging. A central challenge remains that often, it is difficult (perhaps even impossible) to capture the full complexity of the deployment environment, and therefore the desired tasks, at training time. Consequently, the representation, or abstraction, of the tasks the human hopes for the robot to perform in one environment may be misaligned with the representation of the tasks that the robot has learned in another. We postulate that because humans will be the ultimate evaluator of system success in the world, they are best suited to communicating the aspects of the tasks that matter to the robot. Our key insight is that effective learning from human input requires first explicitly learning good intermediate representations and then using those representations for solving downstream tasks. We highlight three areas where we can use this approach to build interactive systems and offer future directions of work to better create advanced collaborative robots.

[27]  arXiv:2205.07883 [pdf, other]
Title: Learning Car Speed Using Inertial Sensors
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

A deep neural network (DNN) is trained to estimate the speed of a car driving in an urban area using as input a stream of measurements from a low-cost six-axis inertial measurement unit (IMU). Three hours of data was collected by driving through the city of Ashdod, Israel in a car equipped with a global navigation satellite system (GNSS) real time kinematic (RTK) positioning device and a synchronized IMU. Ground truth labels for the car speed were calculated using the position measurements obtained at the high rate of 50 [Hz]. A DNN architecture with long short-term memory layers is proposed to enable high-frequency speed estimation that accounts for previous inputs history and the nonlinear relation between speed, acceleration, and angular velocity. A simplified aided dead reckoning localization scheme is formulated to assess the trained model which provides the speed pseudo-measurement. The trained model is shown to substantially improve the position accuracy during a 4 minutes drive without the use of GNSS position updates.

[28]  arXiv:2205.07885 [pdf, other]
Title: Enforcing KL Regularization in General Tsallis Entropy Reinforcement Learning via Advantage Learning
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Maximum Tsallis entropy (MTE) framework in reinforcement learning has gained popularity recently by virtue of its flexible modeling choices including the widely used Shannon entropy and sparse entropy. However, non-Shannon entropies suffer from approximation error and subsequent underperformance either due to its sensitivity or the lack of closed-form policy expression. To improve the tradeoff between flexibility and empirical performance, we propose to strengthen their error-robustness by enforcing implicit Kullback-Leibler (KL) regularization in MTE motivated by Munchausen DQN (MDQN). We do so by drawing connection between MDQN and advantage learning, by which MDQN is shown to fail on generalizing to the MTE framework. The proposed method Tsallis Advantage Learning (TAL) is verified on extensive experiments to not only significantly improve upon Tsallis-DQN for various non-closed-form Tsallis entropies, but also exhibits comparable performance to state-of-the-art maximum Shannon entropy algorithms.

[29]  arXiv:2205.07886 [pdf, other]
Title: An Empirical Investigation of Representation Learning for Imitation
Comments: Accepted to NeurIPS2021 Datasets and Benchmarks Track
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Imitation learning often needs a large demonstration set in order to handle the full range of situations that an agent might find itself in during deployment. However, collecting expert demonstrations can be expensive. Recent work in vision, reinforcement learning, and NLP has shown that auxiliary representation learning objectives can reduce the need for large amounts of expensive, task-specific data. Our Empirical Investigation of Representation Learning for Imitation (EIRLI) investigates whether similar benefits apply to imitation learning. We propose a modular framework for constructing representation learning algorithms, then use our framework to evaluate the utility of representation learning for imitation across several environment suites. In the settings we evaluate, we find that existing algorithms for image-based representation learning provide limited value relative to a well-tuned baseline with image augmentations. To explain this result, we investigate differences between imitation learning and other settings where representation learning has provided significant benefit, such as image classification. Finally, we release a well-documented codebase which both replicates our findings and provides a modular framework for creating new representation learning algorithms out of reusable components.

[30]  arXiv:2205.07890 [pdf, other]
Title: On the Difficulty of Defending Self-Supervised Learning against Model Extraction
Comments: Accepted at ICML 2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR)

Self-Supervised Learning (SSL) is an increasingly popular ML paradigm that trains models to transform complex inputs into representations without relying on explicit labels. These representations encode similarity structures that enable efficient learning of multiple downstream tasks. Recently, ML-as-a-Service providers have commenced offering trained SSL models over inference APIs, which transform user inputs into useful representations for a fee. However, the high cost involved to train these models and their exposure over APIs both make black-box extraction a realistic security threat. We thus explore model stealing attacks against SSL. Unlike traditional model extraction on classifiers that output labels, the victim models here output representations; these representations are of significantly higher dimensionality compared to the low-dimensional prediction scores output by classifiers. We construct several novel attacks and find that approaches that train directly on a victim's stolen representations are query efficient and enable high accuracy for downstream models. We then show that existing defenses against model extraction are inadequate and not easily retrofitted to the specificities of SSL.

[31]  arXiv:2205.07920 [pdf, other]
Title: An Extension to Basis-Hypervectors for Learning from Circular Data in Hyperdimensional Computing
Subjects: Machine Learning (cs.LG); Information Theory (cs.IT)

Hyperdimensional Computing (HDC) is a computation framework based on properties of high-dimensional random spaces. It is particularly useful for machine learning in resource-constrained environments, such as embedded systems and IoT, as it achieves a good balance between accuracy, efficiency and robustness. The mapping of information to the hyperspace, named encoding, is the most important stage in HDC. At its heart are basis-hypervectors, responsible for representing the smallest units of meaningful information. In this work we present a detailed study on basis-hypervector sets, which leads to practical contributions to HDC in general: 1) we propose an improvement for level-hypervectors, used to encode real numbers; 2) we introduce a method to learn from circular data, an important type of information never before addressed in machine learning with HDC. Empirical results indicate that these contributions lead to considerably more accurate models for both classification and regression with circular data.

[32]  arXiv:2205.07932 [pdf, other]
Title: Distributed Feature Selection for High-dimensional Additive Models
Comments: 40 pages, 2 figures
Subjects: Machine Learning (cs.LG); Statistics Theory (math.ST); Applications (stat.AP); Computation (stat.CO); Machine Learning (stat.ML)

Distributed statistical learning is a common strategy for handling massive data where we divide the learning task into multiple local machines and aggregate the results afterward. However, most existing work considers the case where the samples are divided. In this work, we propose a new algorithm, DDAC-SpAM, that divides features under the high-dimensional sparse additive model. The new algorithm contains three steps: divide, decorrelate, and conquer. We show that after the decorrelation operation, every local estimator can recover the sparsity pattern for each additive component consistently without imposing strict constraints to the correlation structure among variables. Theoretical analysis of the aggregated estimator and empirical results on synthetic and real data illustrate that the DDAC-SpAM algorithm is effective and competitive in fitting sparse additive models.

[33]  arXiv:2205.07938 [pdf, ps, other]
Title: The Diversity of Argument-Making in the Wild: from Assumptions and Definitions to Causation and Anecdote in Reddit's "Change My View"
Comments: 7 pages, 5 tables. Accepted as paper with oral presentation to CogSci 2022, Toronto
Subjects: Computation and Language (cs.CL); Social and Information Networks (cs.SI); Neurons and Cognition (q-bio.NC)

What kinds of arguments do people make, and what effect do they have on others? Normative constraints on argument-making are as old as philosophy itself, but little is known about the diversity of arguments made in practice. We use NLP tools to extract patterns of argument-making from the Reddit site "Change My View" (r/CMV). This reveals six distinct argument patterns: not just the familiar deductive and inductive forms, but also arguments about definitions, relevance, possibility and cause, and personal experience. Data from r/CMV also reveal differences in efficacy: personal experience and, to a lesser extent, arguments about causation and examples, are most likely to shift a person's view, while arguments about relevance are the least. Finally, our methods reveal a gradient of argument-making preferences among users: a two-axis model, of "personal--impersonal" and "concrete--abstract", can account for nearly 80% of the strategy variance between individuals.

[34]  arXiv:2205.07939 [pdf]
Title: Two-Stage Coded Federated Edge Learning: A Dynamic Partial Gradient Coding Perspective
Comments: submitted to a conference
Subjects: Networking and Internet Architecture (cs.NI)

Federated edge learning (FEL) can training a global model from terminal nodes' local dataset, which can make full use of the computing resources of terminal nodes and performs more extensive and efficient machine learning on terminal nodes with protecting user information requirements. Performance of FEL will be suffered from long delay or fault decision as the master collects partial gradients from stragglers which cannot return correct results within a deadline. Inspired by this, in this paper, we propose a novel coded FEL to mitigate stragglers for synchronous gradient with a two-stage dynamic scheme, where we start with part of workers for a duration of before starting the second stage, and on completion of at the first stage, we start remaining workers in the second stage. In particular, the computation latency and transmission latency is essential and should be quantitatively analyzed. Then the dynamically coded coefficients scheme is proposed which is based on historical information including worker completion time. For performance optimization of FEL, a Lyapunov function is designed to maximize admission data balancing fairness and two stage dynamic coding scheme is designed to maximize arrival data among workers. Experimental evidence verifies the derived properties and demonstrates that our proposed solution achieves a better performance for practical network parameters and benchmark datasets in terms of accuracy and resource utilization in the FEL system.

[35]  arXiv:2205.07944 [pdf, other]
Title: Design and Implement an Enhanced Simulator for Autonomous Delivery Robot
Subjects: Robotics (cs.RO)

As autonomous driving technology is getting more and more mature today, autonomous delivery companies like Starship, Marble, and Nuro has been making progress in the tests of their autonomous delivery robots. While simulations and simulators are very important for the final product landing of the autonomous delivery robots since the autonomous delivery robots need to navigate on the sidewalk, campus, and other urban scenarios, where the simulations can avoid real damage to pedestrians and properties in the real world caused by any algorithm failures and programming errors and thus accelerate the whole developing procedure and cut down the cost. In this case, this study proposes an open-source simulator based on our autonomous delivery robot ZebraT to accelerate the research on autonomous delivery. The simulator developing procedure is illustrated step by step. What is more, the applications on the simulator that we are working on are also introduced, which includes autonomous navigation in the simulated urban environment, cooperation between an autonomous vehicle and an autonomous delivery robot, and reinforcement learning practice on the task training in the simulator. We have published the proposed simulator in Github.

[36]  arXiv:2205.07945 [pdf, other]
Title: Constructing Trajectory and Predicting Estimated Time of Arrival for Long Distance Travelling Vessels: A Probability Density-based Scanning Approach
Subjects: Systems and Control (eess.SY); Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE); Machine Learning (cs.LG)

In this study, a probability density-based approach for constructing trajectories is proposed and validated through an typical use-case application: Estimated Time of Arrival (ETA) prediction given origin-destination pairs. The ETA prediction is based on physics and mathematical laws given by the extracted information of probability density-based trajectories constructed. The overall ETA prediction errors are about 0.106 days (i.e. 2.544 hours) on average with 0.549 days (i.e. 13.176 hours) standard deviation, and the proposed approach has an accuracy of 92.08% with 0.959 R-Squared value for overall trajectories between Singapore and Australia ports selected.

[37]  arXiv:2205.07947 [pdf, other]
Title: Genomic Compression with Read Alignment at the Decoder
Subjects: Information Theory (cs.IT)

We propose a new compression scheme for genomic data given as sequence fragments called reads. The scheme uses a reference genome at the decoder side only, freeing the encoder from the burdens of storing references and performing computationally costly alignment operations. The main ingredient of the scheme is a multi-layer code construction, delivering to the decoder sufficient information to align the reads, correct their differences from the reference, validate their reconstruction, and correct reconstruction errors. The core of the method is the well-known concept of distributed source coding with decoder side information, fortified by a generalized-concatenation code construction enabling efficient embedding of all the information needed for reliable reconstruction. We first present the scheme for the case of substitution errors only between the reads and the reference, and then extend it to support reads with a single deletion and multiple substitutions. A central tool in this extension is a new distance metric that is shown analytically to improve alignment performance over existing distance metrics.

[38]  arXiv:2205.07949 [pdf, other]
Title: Power and Skew Reduction Using Resonant Energy Recycling in 14-nm FinFET Clocks
Subjects: Systems and Control (eess.SY); Hardware Architecture (cs.AR)

As the demand for high-performance microprocessors increases, the circuit complexity and the rate of data transfer increases resulting in higher power consumption. We propose a clocking architecture that uses a series LC resonance and inductor matching technique to address this bottleneck. By employing pulsed resonance, the switching power dissipated is recycled back. The inductor matching technique aids in reducing the skew, increasing the robustness of the clock network. This new resonant architecture saves over 43% power and 91% skew clocking a range of 1--5 GHz, compared to a conventional primary-secondary flip-flop-based CMOS architecture.

[39]  arXiv:2205.07953 [pdf, other]
Title: Application of multilayer perceptron with data augmentation in nuclear physics
Subjects: Machine Learning (cs.LG); Nuclear Theory (nucl-th)

Neural networks have become popular in many fields of science since they serve as reliable and powerful tools. Application of the neural networks to the nuclear physics studies has also become popular in recent years because of their success in the prediction of nuclear properties. In this work, we study the effect of the data augmentation on the predictive power of the neural network models. Even though there are various data augmentation techniques used for classification tasks in the literature, this area is still very limited for regression problems. As predicting the binding energies is statistically defined as a regression problem, in addition to using data augmentation for nuclear physics, this study contributes to this field for regression in general. Using the experimental uncertainties for data augmentation, the size of training data set is artificially boosted and the changes in the root-mean-square error between the model predictions on test set and the experimental data are investigated. As far as we know, this is the first time that data augmentation techniques have been implemented for nuclear physics research. Our results show that the data augmentation decreases the prediction errors, stabilizes the model and prevents overfitting. The extrapolation capabilities of the MLP models with different depths are also tested for newly measured nuclei in AME2020 mass table.

[40]  arXiv:2205.07959 [pdf]
Title: Deep Apprenticeship Learning for Playing Games
Authors: Dejan Markovikj
Comments: A dissertation submitted in partial fulfillment of the requirements for the degree of Master of Science in Computer Science at University of Oxford
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)

In the last decade, deep learning has achieved great success in machine learning tasks where the input data is represented with different levels of abstractions. Driven by the recent research in reinforcement learning using deep neural networks, we explore the feasibility of designing a learning model based on expert behaviour for complex, multidimensional tasks where reward function is not available. We propose a novel method for apprenticeship learning based on the previous research on supervised learning techniques in reinforcement learning. Our method is applied to video frames from Atari games in order to teach an artificial agent to play those games. Even though the reported results are not comparable with the state-of-the-art results in reinforcement learning, we demonstrate that such an approach has the potential to achieve strong performance in the future and is worthwhile for further research.

[41]  arXiv:2205.07960 [pdf, other]
Title: Meta AI at Arabic Hate Speech 2022: MultiTask Learning with Self-Correction for Hate Speech Classification
Comments: Accepted at the 5th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT5/LREC 2022)
Subjects: Computation and Language (cs.CL)

In this paper, we tackle the Arabic Fine-Grained Hate Speech Detection shared task and demonstrate significant improvements over reported baselines for its three subtasks. The tasks are to predict if a tweet contains (1) Offensive language; and whether it is considered (2) Hate Speech or not and if so, then predict the (3) Fine-Grained Hate Speech label from one of six categories. Our final solution is an ensemble of models that employs multitask learning and a self-consistency correction method yielding 82.7% on the hate speech subtask -- reflecting a 3.4% relative improvement compared to previous work.

[42]  arXiv:2205.07965 [pdf, other]
Title: Flexible and curtailable resource activation in three-phase unbalanced distribution networks
Subjects: Systems and Control (eess.SY); Optimization and Control (math.OC)

The need for flexibility and curtailable resources is crucial for ensuring the healthy operation of future distribution networks (DN). In this work, we propose a network-state driven framework that distribution system operators (DSOs) can utilize for activating flexible and curtailable resources for alleviating network voltage and thermal issues, while accounting for network voltage and current imbalances. This approach assumes the availability of dynamic network state information and uses nodal sensitivities for calculating a flexibility activation signal (FAS). The signal design is motivated by volt-Var and volt-watt inverter control, and thus bounded. The FAS also considers network voltage and current imbalances and incentivizes activation of active and reactive power flexibilities for reducing imbalance in addition to mitigating voltage and thermal imbalances in a three-phase unbalanced distribution network. The FAS design resembles optimal power flow duals, often used as locational marginal prices. The gains associated with the imbalance component of the objective function of three-phase unbalanced resource activation (TPU-RA) is performed using Pareto optimality. A numerical case study is presented showing the efficacy of the proposed framework in avoiding network issues while reducing voltage unbalance factor by more than 80\%. Further, DN's flexibility needs are quantified for location and time of day.

[43]  arXiv:2205.07970 [pdf, other]
Title: SciLander: Mapping the Scientific News Landscape
Subjects: Computers and Society (cs.CY); Social and Information Networks (cs.SI); Physics and Society (physics.soc-ph)

The COVID-19 pandemic has fueled the spread of misinformation on social media and the Web as a whole. The phenomenon dubbed `infodemic' has taken the challenges of information veracity and trust to new heights by massively introducing seemingly scientific and technical elements into misleading content. Despite the existing body of work on modeling and predicting misinformation, the coverage of very complex scientific topics with inherent uncertainty and an evolving set of findings, such as COVID-19, provides many new challenges that are not easily solved by existing tools. To address these issues, we introduce SciLander, a method for learning representations of news sources reporting on science-based topics. SciLander extracts four heterogeneous indicators for the news sources; two generic indicators that capture (1) the copying of news stories between sources, and (2) the use of the same terms to mean different things (i.e., the semantic shift of terms), and two scientific indicators that capture (1) the usage of jargon and (2) the stance towards specific citations. We use these indicators as signals of source agreement, sampling pairs of positive (similar) and negative (dissimilar) samples, and combine them in a unified framework to train unsupervised news source embeddings with a triplet margin loss objective. We evaluate our method on a novel COVID-19 dataset containing nearly 1M news articles from 500 sources spanning a period of 18 months since the beginning of the pandemic in 2020. Our results show that the features learned by our model outperform state-of-the-art baseline methods on the task of news veracity classification. Furthermore, a clustering analysis suggests that the learned representations encode information about the reliability, political leaning, and partisanship bias of these sources.

[44]  arXiv:2205.07972 [pdf, other]
Title: Sparse Visual Counterfactual Explanations in Image Space
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Visual counterfactual explanations (VCEs) in image space are an important tool to understand decisions of image classifiers as they show under which changes of the image the decision of the classifier would change. Their generation in image space is challenging and requires robust models due to the problem of adversarial examples. Existing techniques to generate VCEs in image space suffer from spurious changes in the background. Our novel perturbation model for VCEs together with its efficient optimization via our novel Auto-Frank-Wolfe scheme yields sparse VCEs which are significantly more object-centric. Moreover, we show that VCEs can be used to detect undesired behavior of ImageNet classifiers due to spurious features in the ImageNet dataset and discuss how estimates of the data-generating distribution can be used for VCEs.

[45]  arXiv:2205.07973 [pdf, other]
Title: Many Field Packet Classification with Decomposition and Reinforcement Learning
Comments: 13 pages, published in IET Netw. arXiv admin note: substantial text overlap with arXiv:1902.10319 by other authors
Journal-ref: IET Netw 2022 1-16
Subjects: Networking and Internet Architecture (cs.NI); Artificial Intelligence (cs.AI); Performance (cs.PF)

Scalable packet classification is a key requirement to support scalable network applications like firewalls, intrusion detection, and differentiated services. With ever increasing in the line-rate in core networks, it becomes a great challenge to design a scalable packet classification solution using hand-tuned heuristics approaches. In this paper, we present a scalable learning-based packet classification engine by building an efficient data structure for different ruleset with many fields. Our method consists of the decomposition of fields into subsets and building separate decision trees on those subsets using a deep reinforcement learning procedure. To decompose given fields of a ruleset, we consider different grouping metrics like standard deviation of individual fields and introduce a novel metric called diversity index (DI). We examine different decomposition schemes and construct decision trees for each scheme using deep reinforcement learning and compare the results. The results show that the SD decomposition metrics results in 11.5% faster than DI metrics, 25% faster than random 2 and 40% faster than random 1. Furthermore, our learning-based selection method can be applied to varying rulesets due to its ruleset independence.

[46]  arXiv:2205.07975 [pdf, other]
Title: Key-Value Stores on Flash Storage Devices: A Survey
Subjects: Hardware Architecture (cs.AR); Databases (cs.DB)

Key-value stores (KV) have become one of the main components of the modern storage and data processing system stack. With the increasing need for timely data analysis, performance becomes more and more critical. In the past, these stores were frequently optimised to run on HDD and DRAM devices. However, the last decade saw an increased interest in the use of flash devices because of their attractive properties. Flash is cheaper than DRAM and yet has a lower latency and higher throughput than HDDs. This literature survey aims to highlight the changes proposed in the last decade to optimise key-value stores for flash devices and predict what role these devices might play for key-value stores in the future.

[47]  arXiv:2205.07976 [pdf, other]
Title: Accelerating X-Ray Tracing for Exascale Systems using Kokkos
Authors: Felix Wittwer (1), Nicholas K. Sauter (2), Derek Mendez (2), Billy K. Poon (2), Aaron S. Brewster (2), James M. Holton (2), Michael E. Wall (3), William E. Hart (4), Deborah J. Bard (1), Johannes P. Blaschke (1) ((1) National Energy Research Scientific Computing Center, Lawrence Berkeley National Laboratory, USA, (2) Molecular Biophysics and Integrated Bioimaging Division, Lawrence Berkeley National Laboratory, USA, (3) Computer, Computational, and Statistical Sciences Division, Los Alamos National Laboratory, USA, (4) Sandia National Laboratories, USA)
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)

The upcoming exascale computing systems Frontier and Aurora will draw much of their computing power from GPU accelerators. The hardware for these systems will be provided by AMD and Intel, respectively, each supporting their own GPU programming model. The challenge for applications that harness one of these exascale systems will be to avoid lock-in and to preserve performance portability.
We report here on our results of using Kokkos to accelerate a real-world application on NERSC's Perlmutter Phase 1 (using NVIDIA A100 accelerators) and the testbed system for OLCF's Frontier (using AMD MI250X). By porting to Kokkos, we were able to successfully run the same X-ray tracing code on both systems and achieved speed-ups between 13% and 66% compared to the original CUDA code. These results are a highly encouraging demonstration of using Kokkos to accelerate production science code.

[48]  arXiv:2205.07979 [pdf, ps, other]
Title: The Budge programming language
Authors: Boro Sitnikovski
Subjects: Programming Languages (cs.PL); Computation and Language (cs.CL); Logic in Computer Science (cs.LO)

We present a simple, esoteric programming language based on G\"odel numbering and prime factorization, enhanced with explicit, scoped loops, allowing for easy program composition. We will show the syntax and semantics and then provide a few example programs and their evaluation. We will also provide a few interpreter implementations.

[49]  arXiv:2205.07982 [pdf, other]
Title: TOCH: Spatio-Temporal Object Correspondence to Hand for Motion Refinement
Subjects: Computer Vision and Pattern Recognition (cs.CV)

We present TOCH, a method for refining incorrect 3D hand-object interaction sequences using a data prior. Existing hand trackers, especially those that rely on very few cameras, often produce visually unrealistic results with hand-object intersection or missing contacts. Although correcting such errors requires reasoning about temporal aspects of interaction, most previous work focus on static grasps and contacts. The core of our method are TOCH fields, a novel spatio-temporal representation for modeling correspondences between hands and objects during interaction. The key component is a point-wise object-centric representation which encodes the hand position relative to the object. Leveraging this novel representation, we learn a latent manifold of plausible TOCH fields with a temporal denoising auto-encoder. Experiments demonstrate that TOCH outperforms state-of-the-art (SOTA) 3D hand-object interaction models, which are limited to static grasps and contacts. More importantly, our method produces smooth interactions even before and after contact. Using a single trained TOCH model, we quantitatively and qualitatively demonstrate its usefulness for 1) correcting erroneous reconstruction results from off-the-shelf RGB/RGB-D hand-object reconstruction methods, 2) de-noising, and 3) grasp transfer across objects. We will release our code and trained model on our project page at this http URL

[50]  arXiv:2205.07983 [pdf, other]
Title: Test-Time Adaptation with Shape Moments for Image Segmentation
Comments: Early Accept at International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2022
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Supervised learning is well-known to fail at generalization under distribution shifts. In typical clinical settings, the source data is inaccessible and the target distribution is represented with a handful of samples: adaptation can only happen at test time on a few or even a single subject(s). We investigate test-time single-subject adaptation for segmentation, and propose a Shape-guided Entropy Minimization objective for tackling this task. During inference for a single testing subject, our loss is minimized with respect to the batch normalization's scale and bias parameters. We show the potential of integrating various shape priors to guide adaptation to plausible solutions, and validate our method in two challenging scenarios: MRI-to-CT adaptation of cardiac segmentation and cross-site adaptation of prostate segmentation. Our approach exhibits substantially better performances than the existing test-time adaptation methods. Even more surprisingly, it fares better than state-of-the-art domain adaptation methods, although it forgoes training on additional target data during adaptation. Our results question the usefulness of training on target data in segmentation adaptation, and points to the substantial effect of shape priors on test-time inference. Our framework can be readily used for integrating various priors and for adapting any segmentation network, and our code is available.

[51]  arXiv:2205.07985 [pdf]
Title: Expert Systems with Logic#. A Novel Modeling Framework for Logic Programming in an Object-Oriented Context of C#
Comments: 23 pages, 4 figures, 4 tables, 7 appendices
Subjects: Artificial Intelligence (cs.AI); Programming Languages (cs.PL)

We present a novel approach how logic programming for expert systems can be declared directly in an object-oriented language.

[52]  arXiv:2205.07991 [pdf, other]
Title: TopSort: A High-Performance Two-Phase Sorting Accelerator Optimized on HBM-based FPGAs
Subjects: Hardware Architecture (cs.AR); Distributed, Parallel, and Cluster Computing (cs.DC)

The emergence of high-bandwidth memory (HBM) brings new opportunities to boost the performance of sorting acceleration on FPGAs, which was conventionally bounded by the available off-chip memory bandwidth. However, it is nontrivial for designers to fully utilize this immense bandwidth. First, the existing sorter designs cannot be directly scaled at the increasing rate of available off-chip bandwidth, as the required on-chip resource usage grows at a much faster rate and would bound the sorting performance in turn. Second, designers need an in-depth understanding of HBM characteristics to effectively utilize the HBM bandwidth. To tackle these challenges, we present TopSort, a novel two-phase sorting solution optimized for HBM-based FPGAs. In the first phase, 16 merge trees work in parallel to fully utilize 32 HBM channels. In the second phase, TopSort reuses the logic from phase one to form a wider merge tree to merge the partially sorted results from phase one. TopSort also adopts HBM-specific optimizations to reduce resource overhead and improve bandwidth utilization. TopSort can sort up to 4 GB data using all 32 HBM channels, with an overall sorting performance of 15.6 GB/s. TopSort is 6.7x and 2.2x faster than state-of-the-art CPU and FPGA sorters.

[53]  arXiv:2205.07993 [pdf, other]
Title: Generalizable Task Planning through Representation Pretraining
Subjects: Robotics (cs.RO)

The ability to plan for multi-step manipulation tasks in unseen situations is crucial for future home robots. But collecting sufficient experience data for end-to-end learning is often infeasible in the real world, as deploying robots in many environments can be prohibitively expensive. On the other hand, large-scale scene understanding datasets contain diverse and rich semantic and geometric information. But how to leverage such information for manipulation remains an open problem. In this paper, we propose a learning-to-plan method that can generalize to new object instances by leveraging object-level representations extracted from a synthetic scene understanding dataset. We evaluate our method with a suite of challenging multi-step manipulation tasks inspired by household activities and show that our model achieves measurably better success rate than state-of-the-art end-to-end approaches. Additional information can be found at https://sites.google.com/view/gentp

[54]  arXiv:2205.08001 [pdf, other]
Title: Towards Debiasing Translation Artifacts
Comments: Accepted to NAACL 2022, Main Conference
Subjects: Computation and Language (cs.CL)

Cross-lingual natural language processing relies on translation, either by humans or machines, at different levels, from translating training data to translating test sets. However, compared to original texts in the same language, translations possess distinct qualities referred to as translationese. Previous research has shown that these translation artifacts influence the performance of a variety of cross-lingual tasks. In this work, we propose a novel approach to reducing translationese by extending an established bias-removal technique. We use the Iterative Null-space Projection (INLP) algorithm, and show by measuring classification accuracy before and after debiasing, that translationese is reduced at both sentence and word level. We evaluate the utility of debiasing translationese on a natural language inference (NLI) task, and show that by reducing this bias, NLI accuracy improves. To the best of our knowledge, this is the first study to debias translationese as represented in latent embedding space.

[55]  arXiv:2205.08002 [pdf, other]
Title: Lost in Compression: the Impact of Lossy Image Compression on Variable Size Object Detection within Infrared Imagery
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Lossy image compression strategies allow for more efficient storage and transmission of data by encoding data to a reduced form. This is essential enable training with larger datasets on less storage-equipped environments. However, such compression can cause severe decline in performance of deep Convolution Neural Network (CNN) architectures even when mild compression is applied and the resulting compressed imagery is visually identical. In this work, we apply the lossy JPEG compression method with six discrete levels of increasing compression {95, 75, 50, 15, 10, 5} to infrared band (thermal) imagery. Our study quantitatively evaluates the affect that increasing levels of lossy compression has upon the performance of characteristically diverse object detection architectures (Cascade-RCNN, FSAF and Deformable DETR) with respect to varying sizes of objects present in the dataset. When training and evaluating on uncompressed data as a baseline, we achieve maximal mean Average Precision (mAP) of 0.823 with Cascade R-CNN across the FLIR dataset, outperforming prior work. The impact of the lossy compression is more extreme at higher compression levels (15, 10, 5) across all three CNN architectures. However, re-training models on lossy compressed imagery notably ameliorated performances for all three CNN models with an average increment of ~76% (at higher compression level 5). Additionally, we demonstrate the relative sensitivity of differing object areas {tiny, small, medium, large} with respect to the compression level. We show that tiny and small objects are more sensitive to compression than medium and large objects. Overall, Cascade R-CNN attains the maximal mAP across most of the object area categories.

[56]  arXiv:2205.08007 [pdf, other]
Title: Perceptual Evaluation on Audio-visual Dataset of 360 Content
Comments: 6 pages, 5 figures, International Conference on Multimedia and Expo 2022
Subjects: Multimedia (cs.MM); Sound (cs.SD); Audio and Speech Processing (eess.AS); Image and Video Processing (eess.IV)

To open up new possibilities to assess the multimodal perceptual quality of omnidirectional media formats, we proposed a novel open source 360 audiovisual (AV) quality dataset. The dataset consists of high-quality 360 video clips in equirectangular (ERP) format and higher-order ambisonic (4th order) along with the subjective scores. Three subjective quality experiments were conducted for audio, video, and AV with the procedures detailed in this paper. Using the data from subjective tests, we demonstrated that this dataset can be used to quantify perceived audio, video, and audiovisual quality. The diversity and discriminability of subjective scores were also analyzed. Finally, we investigated how our dataset correlates with various objective quality metrics of audio and video. Evidence from the results of this study implies that the proposed dataset can benefit future studies on multimodal quality evaluation of 360 content.

[57]  arXiv:2205.08012 [pdf, other]
Title: CascadER: Cross-Modal Cascading for Knowledge Graph Link Prediction
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Knowledge graph (KG) link prediction is a fundamental task in artificial intelligence, with applications in natural language processing, information retrieval, and biomedicine. Recently, promising results have been achieved by leveraging cross-modal information in KGs, using ensembles that combine knowledge graph embeddings (KGEs) and contextual language models (LMs). However, existing ensembles are either (1) not consistently effective in terms of ranking accuracy gains or (2) impractically inefficient on larger datasets due to the combinatorial explosion problem of pairwise ranking with deep language models. In this paper, we propose a novel tiered ranking architecture CascadER to maintain the ranking accuracy of full ensembling while improving efficiency considerably. CascadER uses LMs to rerank the outputs of more efficient base KGEs, relying on an adaptive subset selection scheme aimed at invoking the LMs minimally while maximizing accuracy gain over the KGE. Extensive experiments demonstrate that CascadER improves MRR by up to 9 points over KGE baselines, setting new state-of-the-art performance on four benchmarks while improving efficiency by one or more orders of magnitude over competitive cross-modal baselines. Our empirical analyses reveal that diversity of models across modalities and preservation of individual models' confidence signals help explain the effectiveness of CascadER, and suggest promising directions for cross-modal cascaded architectures. Code and pretrained models are available at https://github.com/tsafavi/cascader.

[58]  arXiv:2205.08013 [pdf, other]
Title: Continual learning on 3D point clouds with random compressed rehearsal
Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)

Contemporary deep neural networks offer state-of-the-art results when applied to visual reasoning, e.g., in the context of 3D point cloud data. Point clouds are important datatype for precise modeling of three-dimensional environments, but effective processing of this type of data proves to be challenging. In the world of large, heavily-parameterized network architectures and continuously-streamed data, there is an increasing need for machine learning models that can be trained on additional data. Unfortunately, currently available models cannot fully leverage training on additional data without losing their past knowledge. Combating this phenomenon, called catastrophic forgetting, is one of the main objectives of continual learning. Continual learning for deep neural networks has been an active field of research, primarily in 2D computer vision, natural language processing, reinforcement learning, and robotics. However, in 3D computer vision, there are hardly any continual learning solutions specifically designed to take advantage of point cloud structure. This work proposes a novel neural network architecture capable of continual learning on 3D point cloud data. We utilize point cloud structure properties for preserving a heavily compressed set of past data. By using rehearsal and reconstruction as regularization methods of the learning process, our approach achieves a significant decrease of catastrophic forgetting compared to the existing solutions on several most popular point cloud datasets considering two continual learning settings: when a task is known beforehand, and in the challenging scenario of when task information is unknown to the model.

[59]  arXiv:2205.08016 [pdf, other]
Title: Pulsar: A Superconducting Delay-Line Memory
Comments: 10 pages, 6 figures, 1 table, draft
Subjects: Emerging Technologies (cs.ET); Hardware Architecture (cs.AR)

Logic and fabrication advancements have renewed interest in superconductor electronics for energy-efficient computing and quantum control processors. One of the most challenging obstacles ahead is the lack of a scalable superconducting memory technology. Here, we present a superconducting delay line memory based on Passive Transmission Lines built with high kinetic inductors. The developed memory system is fully superconducting; operates at speeds ranging from 20~GHz to 100~GHz with $\pm$24\% and $\pm$13\% bias margins, respectively; and exhibits data densities in the 10s of Mbit/cm$^2$ with the MIT Lincoln Laboratory SC2 fabrication process. Moreover, its circulating nature allows the miniaturization of control circuitry, the elimination of data splitting and merging, and the inexpensive implementation of both sequential-access and content-addressable memories. Further advancements to high kinetic inductor fabrication processes indicate even greater data densities of 100s of Mbit/cm$^2$ and beyond.

[60]  arXiv:2205.08017 [pdf, other]
Title: $\mathscr{H}$-Consistency Estimation Error of Surrogate Loss Minimizers
Comments: ICML 2022 (long presentation)
Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

We present a detailed study of estimation errors in terms of surrogate loss estimation errors. We refer to such guarantees as $\mathscr{H}$-consistency estimation error bounds, since they account for the hypothesis set $\mathscr{H}$ adopted. These guarantees are significantly stronger than $\mathscr{H}$-calibration or $\mathscr{H}$-consistency. They are also more informative than similar excess error bounds derived in the literature, when $\mathscr{H}$ is the family of all measurable functions. We prove general theorems providing such guarantees, for both the distribution-dependent and distribution-independent settings. We show that our bounds are tight, modulo a convexity assumption. We also show that previous excess error bounds can be recovered as special cases of our general results.
We then present a series of explicit bounds in the case of the zero-one loss, with multiple choices of the surrogate loss and for both the family of linear functions and neural networks with one hidden-layer. We further prove more favorable distribution-dependent guarantees in that case. We also present a series of explicit bounds in the case of the adversarial loss, with surrogate losses based on the supremum of the $\rho$-margin, hinge or sigmoid loss and for the same two general hypothesis sets. Here too, we prove several enhancements of these guarantees under natural distributional assumptions. Finally, we report the results of simulations illustrating our bounds and their tightness.

[61]  arXiv:2205.08018 [pdf, other]
Title: A Survey on Semantics in Automated Data Science
Subjects: Artificial Intelligence (cs.AI)

Data Scientists leverage common sense reasoning and domain knowledge to understand and enrich data for building predictive models. In recent years, we have witnessed a surge in tools and techniques for {\em automated machine learning}. While data scientists can employ various such tools to help with model building, many other aspects such as {\em feature engineering} that require semantic understanding of concepts, remain manual to a large extent. In this paper we discuss important shortcomings of current automated data science solutions and machine learning. We discuss how leveraging basic semantic reasoning on data in combination with novel tools for data science automation can help with consistent and explainable data augmentation and transformation. Moreover, semantics can assist data scientists in a new manner by helping with challenges related to {\em trust}, {\em bias}, and {\em explainability}.

[62]  arXiv:2205.08020 [pdf, other]
Title: Partial Product Aware Machine Learning on DNA-Encoded Libraries
Comments: 8 pages, 5 figures; Published at the MLDD workshop, ICLR 2022
Subjects: Machine Learning (cs.LG); Quantitative Methods (q-bio.QM)

DNA encoded libraries (DELs) are used for rapid large-scale screening of small molecules against a protein target. These combinatorial libraries are built through several cycles of chemistry and DNA ligation, producing large sets of DNA-tagged molecules. Training machine learning models on DEL data has been shown to be effective at predicting molecules of interest dissimilar from those in the original DEL. Machine learning chemical property prediction approaches rely on the assumption that the property of interest is linked to a single chemical structure. In the context of DNA-encoded libraries, this is equivalent to assuming that every chemical reaction fully yields the desired product. However, in practice, multi-step chemical synthesis sometimes generates partial molecules. Each unique DNA tag in a DEL therefore corresponds to a set of possible molecules. Here, we leverage reaction yield data to enumerate the set of possible molecules corresponding to a given DNA tag. This paper demonstrates that training a custom GNN on this richer dataset improves accuracy and generalization performance.

[63]  arXiv:2205.08022 [pdf, ps, other]
Title: A faster algorithm for Vertex Cover parameterized by solution size
Subjects: Data Structures and Algorithms (cs.DS); Combinatorics (math.CO)

We describe a new algorithm for vertex cover with runtime $O^*(1.25400^k)$, where $k$ is the size of the desired solution and $O^*$ hides polynomial factors in the input size. This improves over previous runtime of $O^*(1.2738^k)$ due to Chen, Kanj, & Xia (2010) standing for more than a decade. The key to our algorithm is to use a potential function which simultaneously tracks $k$ as well as the optimal value $\lambda$ of the vertex cover LP relaxation. This approach also allows us to make use of prior algorithms for Maximum Independent Set in bounded-degree graphs and Above-Guarantee Vertex Cover.
The main step in the algorithm is to branch on high-degree vertices, while ensuring that both $k$ and $\mu = k - \lambda$ are decreased at each step. There can be local obstructions in the graph that prevent $\mu$ from decreasing in this process; we develop a number of novel branching steps to handle these situations.

[64]  arXiv:2205.08024 [pdf, ps, other]
Title: Harnessing Multilingual Resources to Question Answering in Arabic
Subjects: Computation and Language (cs.CL)

The goal of the paper is to predict answers to questions given a passage of Qur'an. The answers are always found in the passage, so the task of the model is to predict where an answer starts and where it ends. As the initial data set is rather small for training, we make use of multilingual BERT so that we can augment the training data by using data available for languages other than Arabic. Furthermore, we crawl a large Arabic corpus that is domain specific to religious discourse. Our approach consists of two steps, first we train a BERT model to predict a set of possible answers in a passage. Finally, we use another BERT based model to rank the candidate answers produced by the first BERT model.

[65]  arXiv:2205.08025 [pdf, other]
Title: The Hamiltonian Path Graph is Connected for Simple $s,t$ Paths in Rectangular Grid Graphs
Subjects: Discrete Mathematics (cs.DM)

A \emph{simple} $s,t$ path $P$ in a rectangular grid graph $\mathbb{G}$ is a Hamiltonian path from the top-left corner $s$ to the bottom-right corner $t$ such that each \emph{internal} subpath of $P$ with both endpoints $a$ and $b$ on the boundary of $\mathbb{G}$ has the minimum number of bends needed to travel from $a$ to $b$ (i.e., $0$, $1$, or $2$ bends, depending on whether $a$ and $b$ are on opposite, adjacent, or the same side of the bounding rectangle). Here, we show that $P$ can be reconfigured to any other simple $s,t$ path of $\mathbb{G}$ by \emph{switching $2\times 2$ squares}, where at most ${5}|\mathbb{G}|/{4}$ such operations are required. Furthermore, each \emph{square-switch} is done in $O(1)$ time and keeps the resulting path in the same family of simple $s,t$ paths. Our reconfiguration result proves that the \emph{Hamiltonian path graph} $\cal{G}$ for simple $s,t$ paths is connected and has diameter at most ${5}|\mathbb{G}|/{4}$ which is asymptotically tight.

[66]  arXiv:2205.08028 [pdf, other]
Title: Browser-based Hyperbolic Visualization of Graphs
Comments: To appear in IEEE PacificVis 2022
Subjects: Graphics (cs.GR); Social and Information Networks (cs.SI)

Hyperbolic geometry offers a natural focus + context for data visualization and has been shown to underlie real-world complex networks. However, current hyperbolic network visualization approaches are limited to special types of networks and do not scale to large datasets. With this in mind, we designed, implemented, and analyzed three methods for hyperbolic visualization of networks in the browser based on inverse projections, generalized force-directed algorithms, and hyperbolic multi-dimensional scaling (H-MDS). A comparison with Euclidean MDS shows that H-MDS produces embeddings with lower distortion for several types of networks. All three methods can handle node-link representations and are available in fully functional web-based systems.

[67]  arXiv:2205.08029 [pdf, other]
Title: Automatic Error Classification and Root Cause Determination while Replaying Recorded Workload Data at SAP HANA
Comments: 10 pages
Subjects: Software Engineering (cs.SE); Machine Learning (cs.LG)

Capturing customer workloads of database systems to replay these workloads during internal testing can be beneficial for software quality assurance. However, we experienced that such replays can produce a large amount of false positive alerts that make the results unreliable or time consuming to analyze. Therefore, we design a machine learning based approach that attributes root causes to the alerts. This provides several benefits for quality assurance and allows for example to classify whether an alert is true positive or false positive. Our approach considerably reduces manual effort and improves the overall quality assurance for the database system SAP HANA. We discuss the problem, the design and result of our approach, and we present practical limitations that may require further research.

[68]  arXiv:2205.08032 [pdf, ps, other]
Title: On Algebraic Constructions of Neural Networks with Small Weights
Subjects: Computational Complexity (cs.CC); Discrete Mathematics (cs.DM); Information Theory (cs.IT); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)

Neural gates compute functions based on weighted sums of the input variables. The expressive power of neural gates (number of distinct functions it can compute) depends on the weight sizes and, in general, large weights (exponential in the number of inputs) are required. Studying the trade-offs among the weight sizes, circuit size and depth is a well-studied topic both in circuit complexity theory and the practice of neural computation. We propose a new approach for studying these complexity trade-offs by considering a related algebraic framework. Specifically, given a single linear equation with arbitrary coefficients, we would like to express it using a system of linear equations with smaller (even constant) coefficients. The techniques we developed are based on Siegel's Lemma for the bounds, anti-concentration inequalities for the existential results and extensions of Sylvester-type Hadamard matrices for the constructions.
We explicitly construct a constant weight, optimal size matrix to compute the EQUALITY function (checking if two integers expressed in binary are equal). Computing EQUALITY with a single linear equation requires exponentially large weights. In addition, we prove the existence of the best-known weight size (linear) matrices to compute the COMPARISON function (comparing between two integers expressed in binary). In the context of the circuit complexity theory, our results improve the upper bounds on the weight sizes for the best-known circuit sizes for EQUALITY and COMPARISON.

[69]  arXiv:2205.08033 [pdf]
Title: Using Embeddings for Causal Estimation of Peer Influence in Social Networks
Comments: 17 pages, 1 figure, 4 tables
Subjects: Social and Information Networks (cs.SI); Machine Learning (cs.LG); Machine Learning (stat.ML)

We address the problem of using observational data to estimate peer contagion effects, the influence of treatments applied to individuals in a network on the outcomes of their neighbors. A main challenge to such estimation is that homophily - the tendency of connected units to share similar latent traits - acts as an unobserved confounder for contagion effects. Informally, it's hard to tell whether your friends have similar outcomes because they were influenced by your treatment, or whether it's due to some common trait that caused you to be friends in the first place. Because these common causes are not usually directly observed, they cannot be simply adjusted for. We describe an approach to perform the required adjustment using node embeddings learned from the network itself. The main aim is to perform this adjustment nonparametrically, without functional form assumptions on either the process that generated the network or the treatment assignment and outcome processes. The key contributions are to nonparametrically formalize the causal effect in a way that accounts for homophily, and to show how embedding methods can be used to identify and estimate this effect. Code is available at https://github.com/IrinaCristali/Peer-Contagion-on-Networks.

[70]  arXiv:2205.08034 [pdf, other]
Title: DeepSim: A Reinforcement Learning Environment Build Toolkit for ROS and Gazebo
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Robotics (cs.RO)

We propose DeepSim, a reinforcement learning environment build toolkit for ROS and Gazebo. It allows machine learning or reinforcement learning researchers to access the robotics domain and create complex and challenging custom tasks in ROS and Gazebo simulation environments. This toolkit provides building blocks of advanced features such as collision detection, behaviour control, domain randomization, spawner, and many more. DeepSim is designed to reduce the boundary between robotics and machine learning communities by providing Python interface. In this paper, we discuss the components and design decisions of DeepSim Toolkit.

[71]  arXiv:2205.08041 [pdf, other]
Title: Detection and Physical Interaction with Deformable Linear Objects
Comments: ICRA 2022 2nd Workshop on Representing and Manipulating Deformable Objects (this https URL)
Subjects: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)

Deformable linear objects (e.g., cables, ropes, and threads) commonly appear in our everyday lives. However, perception of these objects and the study of physical interaction with them is still a growing area. There have already been successful methods to model and track deformable linear objects. However, the number of methods that can automatically extract the initial conditions in non-trivial situations for these methods has been limited, and they have been introduced to the community only recently. On the other hand, while physical interaction with these objects has been done with ground manipulators, there have not been any studies on physical interaction and manipulation of the deformable linear object with aerial robots.
This workshop describes our recent work on detecting deformable linear objects, which uses the segmentation output of the existing methods to provide the initialization required by the tracking methods automatically. It works with crossings and can fill the gaps and occlusions in the segmentation and output the model desirable for physical interaction and simulation. Then we present our work on using the method for tasks such as routing and manipulation with the ground and aerial robots. We discuss our feasibility analysis on extending the physical interaction with these objects to aerial manipulation applications.

[72]  arXiv:2205.08043 [pdf, ps, other]
Title: Explainable and Optimally Configured Artificial Neural Networks for Attack Detection in Smart Homes
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

In recent years cybersecurity has become a major concern in adaptation of smart applications. Specially, in smart homes where a large number of IoT devices are used having a secure and trusted mechanisms can provide peace of mind for users. Accurate detection of cyber attacks is crucial, however precise identification of the type of attacks plays a huge role if devising the countermeasure for protecting the system. Artificial Neural Networks (ANN) have provided promising results for detecting any security attacks for smart applications. However, due to complex nature of the model used for this technique it is not easy for normal users to trust ANN based security solutions. Also, selection of right hyperparameters for ANN architecture plays a crucial role in the accurate detection of security attacks, especially when it come to identifying the subcategories of attacks. In this paper, we propose a model that considers both the issues of explainability of ANN model and the hyperparameter selection for this approach to be easily trusted and adapted by users of smart home applications. Also, our approach considers a subset of the dataset for optimal selection of hyperparamters to reduce the overhead of the process of ANN architecture design. Distinctively this paper focuses on configuration, performance and evaluation of ANN architecture for identification of five categorical attacks and nine subcategorical attacks. Using a very recent IoT dataset our approach showed high performance for intrusion detection with 99.9%, 99.7%, and 97.7% accuracy for Binary, Category, and Subcategory level classification of attacks.

[73]  arXiv:2205.08046 [pdf, other]
Title: Shape complexity in cluster analysis
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

In cluster analysis, a common first step is to scale the data aiming to better partition them into clusters. Even though many different techniques have throughout many years been introduced to this end, it is probably fair to say that the workhorse in this preprocessing phase has been to divide the data by the standard deviation along each dimension. Like the standard deviation, the great majority of scaling techniques can be said to have roots in some sort of statistical take on the data. Here we explore the use of multidimensional shapes of data, aiming to obtain scaling factors for use prior to clustering by some method, like k-means, that makes explicit use of distances between samples. We borrow from the field of cosmology and related areas the recently introduced notion of shape complexity, which in the variant we use is a relatively simple, data-dependent nonlinear function that we show can be used to help with the determination of appropriate scaling factors. Focusing on what might be called "midrange" distances, we formulate a constrained nonlinear programming problem and use it to produce candidate scaling-factor sets that can be sifted on the basis of further considerations of the data, say via expert knowledge. We give results on a few iconic data sets, highlighting the strengths and potential weaknesses of the new approach. These results are generally positive across all the data sets used.

[74]  arXiv:2205.08048 [pdf, other]
Title: A Short Introduction to the Koopman Representation of Dynamical Systems
Authors: Bassam Bamieh
Subjects: Systems and Control (eess.SY); Dynamical Systems (math.DS)

The Koopman representation is an infinite dimensional linear representation of linear or nonlinear dynamical systems. It represents the dynamics of output maps (aka observables), which are functions on the state space whose evaluation is interpreted as an output. Conceptually simple derivations and commentary on the Koopman representation are given. We emphasize an important duality between initial conditions and output maps of the original system, and those of the Koopman representation. This duality is an important consideration when this representation is used in data-driven applications such as the Dynamic Mode Decomposition (DMD) and its variants. The adjoint relation between the Koopman representation and the transfer operator of mass transport is also shown.

[75]  arXiv:2205.08056 [pdf, other]
Title: "What makes a question inquisitive?" A Study on Type-Controlled Inquisitive Question Generation
Comments: Accepted in the 11th Joint Conference on Lexical and Computational Semantics (*SEM) conference, NAACL 2022
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

We propose a type-controlled framework for inquisitive question generation. We annotate an inquisitive question dataset with question types, train question type classifiers, and finetune models for type-controlled question generation. Empirical results demonstrate that we can generate a variety of questions that adhere to specific types while drawing from the source texts. We also investigate strategies for selecting a single question from a generated set, considering both an informative vs.~inquisitive question classifier and a pairwise ranker trained from a small set of expert annotations. Question selection using the pairwise ranker yields strong results in automatic and manual evaluation. Our human evaluation assesses multiple aspects of the generated questions, finding that the ranker chooses questions with the best syntax (4.59), semantics (4.37), and inquisitiveness (3.92) on a scale of 1-5, even rivaling the performance of human-written questions.

[76]  arXiv:2205.08062 [pdf, ps, other]
Title: Strong Revenue (Non-)Monotonicity of Single-parameter Auctions
Subjects: Computer Science and Game Theory (cs.GT)

Consider Myerson's optimal auction with respect to an inaccurate prior, e.g., estimated from data, which is an underestimation of the true value distribution. Can the auctioneer expect getting at least the optimal revenue w.r.t. the inaccurate prior since the true value distribution is bigger? This so-called strong revenue monotonicity is known to be true for single-parameter auctions when the feasible allocations form a matroid. We find that strong revenue monotonicity fails to generalize beyond the matroid setting, and further show that auctions in the matroid setting are the only downward-closed auctions that satisfy strong revenue monotonicity. On the flip side, we recover an approximate version of strong revenue monotonicity that holds for all single-parameter auctions, even without downward-closeness. As applications, we improve the sample complexity upper bounds for various single-parameter auctions.

[77]  arXiv:2205.08067 [pdf]
Title: Robust Perception Architecture Design for Automotive Cyber-Physical Systems
Subjects: Machine Learning (cs.LG); Robotics (cs.RO); Systems and Control (eess.SY)

In emerging automotive cyber-physical systems (CPS), accurate environmental perception is critical to achieving safety and performance goals. Enabling robust perception for vehicles requires solving multiple complex problems related to sensor selection/ placement, object detection, and sensor fusion. Current methods address these problems in isolation, which leads to inefficient solutions. We present PASTA, a novel framework for global co-optimization of deep learning and sensing for dependable vehicle perception. Experimental results with the Audi-TT and BMW-Minicooper vehicles show how PASTA can find robust, vehicle-specific perception architecture solutions.

[78]  arXiv:2205.08071 [pdf, other]
Title: How Not to Handle Keys: Timing Attacks on FIDO Authenticator Privacy
Comments: to be published in the 22nd Privacy Enhancing Technologies Symposium (PETS 2022)
Subjects: Cryptography and Security (cs.CR)

This paper presents a timing attack on the FIDO2 (Fast IDentity Online) authentication protocol that allows attackers to link user accounts stored in vulnerable authenticators, a serious privacy concern. FIDO2 is a new standard specified by the FIDO industry alliance for secure token online authentication. It complements the W3C WebAuthn specification by providing means to use a USB token or other authenticator as a second factor during the authentication process. From a cryptographic perspective, the protocol is a simple challenge-response where the elliptic curve digital signature algorithm is used to sign challenges. To protect the privacy of the user the token uses unique key pairs per service. To accommodate for small memory, tokens use various techniques that make use of a special parameter called a key handle sent by the service to the token. We identify and analyse a vulnerability in the way the processing of key handles is implemented that allows attackers to remotely link user accounts on multiple services. We show that for vulnerable authenticators there is a difference between the time it takes to process a key handle for a different service but correct authenticator, and for a different authenticator but correct service. This difference can be used to perform a timing attack allowing an adversary to link user's accounts across services. We present several real world examples of adversaries that are in a position to execute our attack and can benefit from linking accounts. We found that two of the eight hardware authenticators we tested were vulnerable despite FIDO level 1 certification. This vulnerability cannot be easily mitigated on authenticators because, for security reasons, they usually do not allow firmware updates. In addition, we show that due to the way existing browsers implement the WebAuthn standard, the attack can be executed remotely.

[79]  arXiv:2205.08072 [pdf]
Title: A Survey on Machine Learning for Geo-Distributed Cloud Data Center Management
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG)

Cloud workloads today are typically managed in a distributed environment and processed across geographically distributed data centers. Cloud service providers have been distributing data centers globally to reduce operating costs while also improving quality of service by using intelligent workload and resource management strategies. Such large scale and complex orchestration of software workload and hardware resources remains a difficult problem to solve efficiently. Researchers and practitioners have been trying to address this problem by proposing a variety of cloud management techniques. Mathematical optimization techniques have historically been used to address cloud management issues. But these techniques are difficult to scale to geo-distributed problem sizes and have limited applicability in dynamic heterogeneous system environments, forcing cloud service providers to explore intelligent data-driven and Machine Learning (ML) based alternatives. The characterization, prediction, control, and optimization of complex, heterogeneous, and ever-changing distributed cloud resources and workloads employing ML methodologies have received much attention in recent years. In this article, we review the state-of-the-art ML techniques for the cloud data center management problem. We examine the challenges and the issues in current research focused on ML for cloud management and explore strategies for addressing these issues. We also discuss advantages and disadvantages of ML techniques presented in the recent literature and make recommendations for future research directions.

[80]  arXiv:2205.08075 [pdf]
Title: Collaborative Attention Memory Network for Video Object Segmentation
Comments: Youtube-VOS 2021
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Semi-supervised video object segmentation is a fundamental yet Challenging task in computer vision. Embedding matching based CFBI series networks have achieved promising results by foreground-background integration approach. Despite its superior performance, these works exhibit distinct shortcomings, especially the false predictions caused by little appearance instances in first frame, even they could easily be recognized by previous frame. Moreover, they suffer from object's occlusion and error drifts. In order to overcome the shortcomings , we propose Collaborative Attention Memory Network with an enhanced segmentation head. We introduce a object context scheme that explicitly enhances the object information, which aims at only gathering the pixels that belong to the same category as a given pixel as its context. Additionally, a segmentation head with Feature Pyramid Attention(FPA) module is adopted to perform spatial pyramid attention structure on high-level output. Furthermore, we propose an ensemble network to combine STM network with all these new refined CFBI network. Finally, we evaluated our approach on the 2021 Youtube-VOS challenge where we obtain 6th place with an overall score of 83.5\%.

[81]  arXiv:2205.08076 [pdf]
Title: A Labeling Task Design for Supporting Algorithmic Needs: Facilitating Worker Diversity and Reducing AI Bias
Comments: 45 pages, 4 figures
Subjects: Human-Computer Interaction (cs.HC)

Studies on supervised machine learning (ML) recommend involving workers from various backgrounds in training dataset labeling to reduce algorithmic bias. Moreover, sophisticated tasks for categorizing objects in images are necessary to improve ML performance, further complicating micro-tasks. This study aims to develop a task design incorporating the fair participation of people, regardless of their specific backgrounds or task's difficulty. By collaborating with 75 labelers from diverse backgrounds for 3 months, we analyzed workers' log-data and relevant narratives to identify the task's hurdles and helpers. The findings revealed that workers' decision-making tendencies varied depending on their backgrounds. We found that the community that positively helps workers and the machine's feedback perceived by workers could make people easily engaged in works. Hence, ML's bias could be expectedly mitigated. Based on these findings, we suggest an extended human-in-the-loop approach that connects labelers, machines, and communities rather than isolating individual workers.

[82]  arXiv:2205.08078 [pdf, other]
Title: Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers
Comments: 38 pages, 2 figures. To appear in ICML 2022
Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Optimization and Control (math.OC)

Vision transformers using self-attention or its proposed alternatives have demonstrated promising results in many image related tasks. However, the underpinning inductive bias of attention is not well understood. To address this issue, this paper analyzes attention through the lens of convex duality. For the non-linear dot-product self-attention, and alternative mechanisms such as MLP-mixer and Fourier Neural Operator (FNO), we derive equivalent finite-dimensional convex problems that are interpretable and solvable to global optimality. The convex programs lead to {\it block nuclear-norm regularization} that promotes low rank in the latent feature and token dimensions. In particular, we show how self-attention networks implicitly clusters the tokens, based on their latent similarity. We conduct experiments for transferring a pre-trained transformer backbone for CIFAR-100 classification by fine-tuning a variety of convex attention heads. The results indicate the merits of the bias induced by attention compared with the existing MLP or linear heads.

[83]  arXiv:2205.08083 [pdf, other]
Title: Region-Aware Metric Learning for Open World Semantic Segmentation via Meta-Channel Aggregation
Comments: Accepted at IJCAI 2022
Subjects: Computer Vision and Pattern Recognition (cs.CV)

As one of the most challenging and practical segmentation tasks, open-world semantic segmentation requires the model to segment the anomaly regions in the images and incrementally learn to segment out-of-distribution (OOD) objects, especially under a few-shot condition. The current state-of-the-art (SOTA) method, Deep Metric Learning Network (DMLNet), relies on pixel-level metric learning, with which the identification of similar regions having different semantics is difficult. Therefore, we propose a method called region-aware metric learning (RAML), which first separates the regions of the images and generates region-aware features for further metric learning. RAML improves the integrity of the segmented anomaly regions. Moreover, we propose a novel meta-channel aggregation (MCA) module to further separate anomaly regions, forming high-quality sub-region candidates and thereby improving the model performance for OOD objects. To evaluate the proposed RAML, we have conducted extensive experiments and ablation studies on Lost And Found and Road Anomaly datasets for anomaly segmentation and the CityScapes dataset for incremental few-shot learning. The results show that the proposed RAML achieves SOTA performance in both stages of open world segmentation. Our code and appendix are available at https://github.com/czifan/RAML.

[84]  arXiv:2205.08084 [pdf, other]
Title: M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems
Comments: 10 pages, 8 figures, the paper has been submited to KDD2022
Subjects: Information Retrieval (cs.IR)

Industrial recommender systems have been growing increasingly complex, may involve \emph{diverse domains} such as e-commerce products and user-generated contents, and can comprise \emph{a myriad of tasks} such as retrieval, ranking, explanation generation, and even AI-assisted content production. The mainstream approach so far is to develop individual algorithms for each domain and each task. In this paper, we explore the possibility of developing a unified foundation model to support \emph{open-ended domains and tasks} in an industrial recommender system, which may reduce the demand on downstream settings' data and can minimize the carbon footprint by avoiding training a separate model from scratch for every task. Deriving a unified foundation is challenging due to (i) the potentially unlimited set of downstream domains and tasks, and (ii) the real-world systems' emphasis on computational efficiency. We thus build our foundation upon M6, an existing large-scale industrial pretrained language model similar to GPT-3 and T5, and leverage M6's pretrained ability for sample-efficient downstream adaptation, by representing user behavior data as plain texts and converting the tasks to either language understanding or generation. To deal with a tight hardware budget, we propose an improved version of prompt tuning that outperforms fine-tuning with negligible 1\% task-specific parameters, and employ techniques such as late interaction, early exiting, parameter sharing, and pruning to further reduce the inference time and the model size. We demonstrate the foundation model's versatility on a wide range of tasks such as retrieval, ranking, zero-shot recommendation, explanation generation, personalized content creation, and conversational recommendation, and manage to deploy it on both cloud servers and mobile devices.

[85]  arXiv:2205.08086 [pdf, other]
Title: EvoRobogami: Co-designing with Humans in Evolutionary Robotics Experiments
Comments: To be published in GECCO 2022
Subjects: Robotics (cs.RO)

We study the effects of injecting human-generated designs into the initial population of an evolutionary robotics experiment, where subsequent population of robots are optimised via a Genetic Algorithm and MAP-Elites. First, human participants interact via a graphical front-end to explore a directly-parameterised legged robot design space and attempt to produce robots via a combination of intuition and trial-and-error that perform well in a range of environments. Environments are generated whose corresponding high-performance robot designs range from intuitive to complex and hard to grasp. Once the human designs have been collected, their impact on the evolutionary process is assessed by replacing a varying number of designs in the initial population with human designs and subsequently running the evolutionary algorithm. Our results suggest that a balance of random and hand-designed initial solutions provides the best performance for the problems considered, and that human designs are most valuable when the problem is intuitive. The influence of human design in an evolutionary algorithm is a highly understudied area, and the insights in this paper may be valuable to the area of AI-based design more generally.

[86]  arXiv:2205.08087 [pdf, other]
Title: An Empirical Study of Blockchain Repositories in GitHub
Comments: The International Conference on Evaluation and Assessment in Software Engineering 2022 (EASE 2022)
Subjects: Cryptography and Security (cs.CR); Software Engineering (cs.SE)

Blockchain is a distributed ledger technique that guarantees the traceability of transactions. Blockchain is adopted in multiple domains like finance (e.g., cryptocurrency), healthcare, security, and supply chain. In the open-source software (OSS) portal GitHub, we observe a growing adoption of Blockchain-based solutions. Given the rapid emergence of Blockchain-based solutions in our daily life and the evolving cryptocurrency market, it is important to know the status quo, how developers generally interact in those repos, and how much freedom they have in applying code changes. We report an empirical study of 3,664 Blockchain software repositories from GitHub. We divide the Blockchain repositories into two categories: Tool (e.g., SDKs) and Applications (e.g., service/solutions developed using SDKs). The Application category is further divided into two sub-categories: Crypto and Non-Crypto applications. In all Blockchain repository categories, the contribution interactions on commits are the most common interaction type. We found that more organizations contributing to the Blockchain repos than individual users. The median numbers of internal and external users in tools are higher than the application repos. We observed a higher degree of collaboration (e.g., for maintenance efforts) among users in Blockchain tools than those in the application repos. Among the artifacts, issues have a greater number of interactions than commits and pull requests. Related to autonomy we found that less than half of total project contributions are autonomous. Our findings offer implications to Blockchain stakeholders, like developers to stay aware of OSS practices around Blockchain software.

[87]  arXiv:2205.08089 [pdf]
Title: Efficient Stereo Depth Estimation for Pseudo LiDAR: A Self-Supervised Approach Based on Multi-Input ResNet Encoder
Comments: 9 pages, 5 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Perception and localization are essential for autonomous delivery vehicles, mostly estimated from 3D LiDAR sensors due to their precise distance measurement capability. This paper presents a strategy to obtain the real-time pseudo point cloud instead of the laser sensor from the image sensor. We propose an approach to use different depth estimators to obtain pseudo point clouds like LiDAR to obtain better performance. Moreover, the training and validating strategy of the depth estimator has adopted stereo imagery data to estimate more accurate depth estimation as well as point cloud results. Our approach to generating depth maps outperforms on KITTI benchmark while yielding point clouds significantly faster than other approaches.

[88]  arXiv:2205.08090 [pdf, other]
Title: A Linear Comb Filter for Event Flicker Removal
Comments: 10 pages, 7 figures, published in IEEE International Conference on Robotics and Automation (ICRA), 2022
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Event cameras are bio-inspired sensors that capture per-pixel asynchronous intensity change rather than the synchronous absolute intensity frames captured by a classical camera sensor. Such cameras are ideal for robotics applications since they have high temporal resolution, high dynamic range and low latency. However, due to their high temporal resolution, event cameras are particularly sensitive to flicker such as from fluorescent or LED lights. During every cycle from bright to dark, pixels that image a flickering light source generate many events that provide little or no useful information for a robot, swamping the useful data in the scene. In this paper, we propose a novel linear filter to preprocess event data to remove unwanted flicker events from an event stream. The proposed algorithm achieves over 4.6 times relative improvement in the signal-to-noise ratio when compared to the raw event stream due to the effective removal of flicker from fluorescent lighting. Thus, it is ideally suited to robotics applications that operate in indoor settings or scenes illuminated by flickering light sources.

[89]  arXiv:2205.08093 [pdf, ps, other]
Title: Narrowing the LOCAL$\unicode{x2013}$CONGEST Gaps in Sparse Networks via Expander Decompositions
Subjects: Data Structures and Algorithms (cs.DS); Distributed, Parallel, and Cluster Computing (cs.DC)

Many combinatorial optimization problems can be approximated within $(1 \pm \epsilon)$ factors in $\text{poly}(\log n, 1/\epsilon)$ rounds in the LOCAL model via network decompositions [Ghaffari, Kuhn, and Maus, STOC 2018]. These approaches require sending messages of unlimited size, so they do not extend to the CONGEST model, which restricts the message size to be $O(\log n)$ bits.
In this paper, we develop a generic framework for obtaining $\text{poly}(\log n, 1/\epsilon)$-round $(1\pm \epsilon)$-approximation algorithms for many combinatorial optimization problems, including maximum weighted matching, maximum independent set, and correlation clustering, in graphs excluding a fixed minor in the CONGEST model. This class of graphs covers many sparse network classes that have been studied in the literature, including planar graphs, bounded-genus graphs, and bounded-treewidth graphs.
Furthermore, we show that our framework can be applied to give an efficient distributed property testing algorithm for an arbitrary minor-closed graph property that is closed under taking disjoint union, significantly generalizing the previous distributed property testing algorithm for planarity in [Levi, Medina, and Ron, PODC 2018 & Distributed Computing 2021].
Our framework uses distributed expander decomposition algorithms [Chang and Saranurak, FOCS 2020] to decompose the graph into clusters of high conductance. We show that any graph excluding a fixed minor admits small edge separators. Using this result, we show the existence of a high-degree vertex in each cluster in an expander decomposition, which allows the entire graph topology of the cluster to be routed to a vertex. Similar to the use of network decompositions in the LOCAL model, the vertex will be able to perform any local computation on the subgraph induced by the cluster and broadcast the result over the cluster.

[90]  arXiv:2205.08094 [pdf, other]
Title: MATrIX -- Modality-Aware Transformer for Information eXtraction
Subjects: Computer Vision and Pattern Recognition (cs.CV)

We present MATrIX - a Modality-Aware Transformer for Information eXtraction in the Visual Document Understanding (VDU) domain. VDU covers information extraction from visually rich documents such as forms, invoices, receipts, tables, graphs, presentations, or advertisements. In these, text semantics and visual information supplement each other to provide a global understanding of the document. MATrIX is pre-trained in an unsupervised way with specifically designed tasks that require the use of multi-modal information (spatial, visual, or textual). We consider the spatial and text modalities all at once in a single token set. To make the attention more flexible, we use a learned modality-aware relative bias in the attention mechanism to modulate the attention between the tokens of different modalities. We evaluate MATrIX on 3 different datasets each with strong baselines.

[91]  arXiv:2205.08095 [pdf, other]
Title: Reasoning About Vectors using an SMT Theory of Sequences
Comments: 36 pages, 7 figures, accepted at IJCAR 2022
Subjects: Logic in Computer Science (cs.LO)

Dynamic arrays, also referred to as vectors, are fundamental data structures used in many programs. Modeling their semantics efficiently is crucial when reasoning about such programs. The theory of arrays is widely supported but is not ideal, because the number of elements is fixed (determined by its index sort) and cannot be adjusted, which is a problem, given that the length of vectors often plays an important role when reasoning about vector programs. In this paper, we propose reasoning about vectors using a theory of sequences. We introduce the theory, propose a basic calculus adapted from one for the theory of strings, and extend it to efficiently handle common vector operations. We prove that our calculus is sound and show how to construct a model when it terminates with a saturated configuration. Finally, we describe an implementation of the calculus in cvc5 and demonstrate its efficacy by evaluating it on verification conditions for smart contracts and benchmarks derived from existing array benchmarks.

[92]  arXiv:2205.08096 [pdf, other]
Title: Can Bad Teaching Induce Forgetting? Unlearning in Deep Networks using an Incompetent Teacher
Subjects: Machine Learning (cs.LG)

Machine unlearning has become an important field of research due to an increasing focus on addressing the evolving data privacy rules and regulations into the machine learning (ML) applications. It facilitates the request for removal of certain set or class of data from the already trained ML model without retraining from scratch. Recently, several efforts have been made to perform unlearning in an effective and efficient manner. We propose a novel machine unlearning method by exploring the utility of competent and incompetent teachers in a student-teacher framework to induce forgetfulness. The knowledge from the competent and incompetent teachers is selectively transferred to the student to obtain a model that doesn't contain any information about the forget data. We experimentally show that this method is well generalized, fast, and effective. Furthermore, we introduce a zero retrain forgetting (ZRF) metric to evaluate the unlearning method. Unlike the existing unlearning metrics, the ZRF score does not depend on the availability of the expensive retrained model. This makes it useful for analysis of the unlearned model after deployment as well. The experiments are conducted for random subset forgetting and class forgetting on various deep networks and across different application domains. A use case of forgetting information about the patients' medical records is also presented.

[93]  arXiv:2205.08098 [pdf, other]
Title: Can We Do Better Than Random Start? The Power of Data Outsourcing
Comments: 22 pages, 5 figures
Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)

Many organizations have access to abundant data but lack the computational power to process the data. While they can outsource the computational task to other facilities, there are various constraints on the amount of data that can be shared. It is natural to ask what can data outsourcing accomplish under such constraints. We address this question from a machine learning perspective. When training a model with optimization algorithms, the quality of the results often relies heavily on the points where the algorithms are initialized. Random start is one of the most popular methods to tackle this issue, but it can be computationally expensive and not feasible for organizations lacking computing resources. Based on three different scenarios, we propose simulation-based algorithms that can utilize a small amount of outsourced data to find good initial points accordingly. Under suitable regularity conditions, we provide theoretical guarantees showing the algorithms can find good initial points with high probability. We also conduct numerical experiments to demonstrate that our algorithms perform significantly better than the random start approach.

[94]  arXiv:2205.08099 [pdf, other]
Title: Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey
Comments: Survey for pruning and freezing methods applied before training starts
Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)

State-of-the-art deep learning models have a parameter count that reaches into the billions. Training, storing and transferring such models is energy and time consuming, thus costly. A big part of these costs is caused by training the network. Model compression lowers storage and transfer costs, and can further make training more efficient by decreasing the number of computations in the forward and/or backward pass. Thus, compressing networks also at training time while maintaining a high performance is an important research topic. This work is a survey on methods which reduce the number of trained weights in deep learning models throughout the training. Most of the introduced methods set network parameters to zero which is called pruning. The presented pruning approaches are categorized into pruning at initialization, lottery tickets and dynamic sparse training. Moreover, we discuss methods that freeze parts of a network at its random initialization. By freezing weights, the number of trainable parameters is shrunken which reduces gradient computations and the dimensionality of the model's optimization space. In this survey we first propose dimensionality reduced training as an underlying mathematical model that covers pruning and freezing during training. Afterwards, we present and discuss different dimensionality reduced training methods.

[95]  arXiv:2205.08103 [pdf, ps, other]
Title: Deterministic 3-Server on a Circle and the Limitation of Canonical Potentials
Subjects: Data Structures and Algorithms (cs.DS)

The deterministic $k$-server conjecture states that there is a $k$-competitive deterministic algorithm for the $k$-server problem for any metric space. We show that the work function algorithm is $3$-competitive for the $3$-server problem on circle metrics, a case left open by Coester and Koutsoupias (2021). Our analysis follows the existing framework but introduces a new potential function which may be viewed as a relaxation of the counterpart by Coester and Koutsoupias (2021). We further notice that the new potential function and many existing ones can be rewritten in a canonical form. Through a computer-aided verification, however, we find that no such canonical potential function can resolve the deterministic $3$-server conjecture for general metric spaces under the current analysis framework.

[96]  arXiv:2205.08104 [pdf, other]
Title: Sequential Elimination Contests with All-Pay Auctions
Subjects: Computer Science and Game Theory (cs.GT); General Economics (econ.GN)

By modeling contests as all-pay auctions, we study two-stage sequential elimination contests (SEC) under incomplete information, where only the players with top efforts in the first stage can proceed to the second and final stage to compete for prizes. Players have privately held type/ability information that impacts their costs of exerting efforts. We characterize players' Perfect Bayesian Equilibrium strategies and discover a somewhat surprising result: all players exert weakly lower efforts in the final stage of the SEC compared to those under a one-round contest, regardless of the number of players admitted to the final stage. This result holds under any multi-prize reward structure, any type distribution and cost function. As a consequence, in terms of the expected highest effort or total efforts of the final stage, the optimal SEC is equivalent to a one-round contest by letting all players proceed to the final stage.

[97]  arXiv:2205.08105 [pdf, ps, other]
Title: Discretization of inherent ODEs and the geometric integration of DAEs with symmetries
Subjects: Numerical Analysis (math.NA); Optimization and Control (math.OC)

Discretization methods for differential-algebraic equations (DAEs) are considered that are based on the integration of an associated inherent ordinary differential equation (ODE). This allows to make use of any discretization scheme suitable for the numerical integration of ODEs. For DAEs with symmetries it is shown that the inherent ODE can be constructed in such a way that it inherits the symmetry properties of the given DAE and geometric properties of its flow. This in particular allows the use of geometric integration schemes with a numerical flow that has analogous geometric properties.

[98]  arXiv:2205.08108 [pdf, other]
Title: Unbiased Math Word Problems Benchmark for Mitigating Solving Bias
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

In this paper, we revisit the solving bias when evaluating models on current Math Word Problem (MWP) benchmarks. However, current solvers exist solving bias which consists of data bias and learning bias due to biased dataset and improper training strategy. Our experiments verify MWP solvers are easy to be biased by the biased training datasets which do not cover diverse questions for each problem narrative of all MWPs, thus a solver can only learn shallow heuristics rather than deep semantics for understanding problems. Besides, an MWP can be naturally solved by multiple equivalent equations while current datasets take only one of the equivalent equations as ground truth, forcing the model to match the labeled ground truth and ignoring other equivalent equations. Here, we first introduce a novel MWP dataset named UnbiasedMWP which is constructed by varying the grounded expressions in our collected data and annotating them with corresponding multiple new questions manually. Then, to further mitigate learning bias, we propose a Dynamic Target Selection (DTS) Strategy to dynamically select more suitable target expressions according to the longest prefix match between the current model output and candidate equivalent equations which are obtained by applying commutative law during training. The results show that our UnbiasedMWP has significantly fewer biases than its original data and other datasets, posing a promising benchmark for fairly evaluating the solvers' reasoning skills rather than matching nearest neighbors. And the solvers trained with our DTS achieve higher accuracies on multiple MWP benchmarks. The source code is available at https://github.com/yangzhch6/UnbiasedMWP.

[99]  arXiv:2205.08109 [pdf]
Title: Forecasting Solar Power Generation on the basis of Predictive and Corrective Maintenance Activities
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Signal Processing (eess.SP)

Solar energy forecasting has seen tremendous growth in the last decade using historical time series collected from a weather station, such as weather variables wind speed and direction, solar radiance, and temperature. It helps in the overall management of solar power plants. However, the solar power plant regularly requires preventive and corrective maintenance activities that further impact energy production. This paper presents a novel work for forecasting solar power energy production based on maintenance activities, problems observed at a power plant, and weather data. The results accomplished on the datasets obtained from the 1MW solar power plant of PDEU (our university) that has generated data set with 13 columns as daily entries from 2012 to 2020. There are 12 structured columns and one unstructured column with manual text entries about different maintenance activities, problems observed, and weather conditions daily. The unstructured column is used to create a new feature column vector using Hash Map, flag words, and stop words. The final dataset comprises five important feature vector columns based on correlation and causality analysis.

[100]  arXiv:2205.08115 [pdf, other]
Title: Fast and Provably Convergent Algorithms for Gromov-Wasserstein in Graph Learning
Subjects: Machine Learning (cs.LG)

In this paper, we study the design and analysis of a class of efficient algorithms for computing the Gromov-Wasserstein (GW) distance tailored to large-scale graph learning tasks. Armed with the Luo-Tseng error bound condition~\cite{luo1992error}, two proposed algorithms, called Bregman Alternating Projected Gradient (BAPG) and hybrid Bregman Proximal Gradient (hBPG) are proven to be (linearly) convergent. Upon task-specific properties, our analysis further provides novel theoretical insights to guide how to select the best fit method. As a result, we are able to provide comprehensive experiments to validate the effectiveness of our methods on a host of tasks, including graph alignment, graph partition, and shape matching. In terms of both wall-clock time and modeling performance, the proposed methods achieve state-of-the-art results.

[101]  arXiv:2205.08116 [pdf, other]
Title: On the Use of Refactoring in Security Vulnerability Fixes: An Exploratory Study on Maven Libraries
Comments: Accepted as ERA paper to EASE2022
Subjects: Software Engineering (cs.SE)

Third-party library dependencies are commonplace in today's software development. With the growing threat of security vulnerabilities, applying security fixes in a timely manner is important to protect software systems. As such, the community developed a list of software and hardware weakness known as Common Weakness Enumeration (CWE) to assess vulnerabilities. Prior work has revealed that maintenance activities such as refactoring code potentially correlate with security-related aspects in the source code. In this work, we explore the relationship between refactoring and security by analyzing refactoring actions performed jointly with vulnerability fixes in practice. We conducted a case study to analyze 143 maven libraries in which 351 known vulnerabilities had been detected and fixed. Surprisingly, our exploratory results show that developers incorporate refactoring operations in their fixes, with 31.9% (112 out of 351) of the vulnerabilities paired with refactoring actions. We envision this short paper to open up potential new directions to motivate automated tool support, allowing developers to deliver fixes faster, while maintaining their code.

[102]  arXiv:2205.08119 [pdf, other]
Title: ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks
Comments: Accepted by ICML 2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Neural networks (NNs) with intensive multiplications (e.g., convolutions and transformers) are capable yet power hungry, impeding their more extensive deployment into resource-constrained devices. As such, multiplication-free networks, which follow a common practice in energy-efficient hardware implementation to parameterize NNs with more efficient operators (e.g., bitwise shifts and additions), have gained growing attention. However, multiplication-free networks usually under-perform their vanilla counterparts in terms of the achieved accuracy. To this end, this work advocates hybrid NNs that consist of both powerful yet costly multiplications and efficient yet less powerful operators for marrying the best of both worlds, and proposes ShiftAddNAS, which can automatically search for more accurate and more efficient NNs. Our ShiftAddNAS highlights two enablers. Specifically, it integrates (1) the first hybrid search space that incorporates both multiplication-based and multiplication-free operators for facilitating the development of both accurate and efficient hybrid NNs; and (2) a novel weight sharing strategy that enables effective weight sharing among different operators that follow heterogeneous distributions (e.g., Gaussian for convolutions vs. Laplacian for add operators) and simultaneously leads to a largely reduced supernet size and much better searched networks. Extensive experiments and ablation studies on various models, datasets, and tasks consistently validate the efficacy of ShiftAddNAS, e.g., achieving up to a +7.7% higher accuracy or a +4.9 better BLEU score compared to state-of-the-art NN, while leading to up to 93% or 69% energy and latency savings, respectively. Codes and pretrained models are available at https://github.com/RICE-EIC/ShiftAddNAS.

[103]  arXiv:2205.08121 [pdf, other]
Title: Design of Joint Source-Channel Codes Based on a Single Protograph
Comments: 20 pages, 15 figures, 5 tables
Subjects: Information Theory (cs.IT)

In this paper, we propose using a single protograph to design joint source-channel codes (JSCCs). We present a generalized algorithm, called protograph extrinsic information transfer for JSCC algorithm (PEXIT-JSCC algorithm), for analyzing the channel threshold of the proposed JSCC. We also propose a source single protograph EXIT (SSP-EXIT) algorithm, which is more generic than the generalized source protograph extrinsic information transfer (GSP-EXIT) algorithm, for evaluating the source threshold of a single protograph. Moreover, a collaborative optimization method based on the SSP-EXIT and PEXIT-JSCC algorithms is proposed to construct single-protograph JSCCs with good source and channel thresholds. Finally, we construct single-protograph JSCCs, analyze their decoding thresholds, and compare their theoretical and error performance with JSCC systems based on optimized double-protographs. Results show that our proposed codes can attain channel thresholds within 1 dB from the Shannon limit and outperform double-protograph-based JSCCs.

[104]  arXiv:2205.08124 [pdf, other]
Title: When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learning
Comments: ACL 2022
Subjects: Computation and Language (cs.CL)

Transfer learning (TL) in natural language processing (NLP) has seen a surge of interest in recent years, as pre-trained models have shown an impressive ability to transfer to novel tasks. Three main strategies have emerged for making use of multiple supervised datasets during fine-tuning: training on an intermediate task before training on the target task (STILTs), using multi-task learning (MTL) to train jointly on a supplementary task and the target task (pairwise MTL), or simply using MTL to train jointly on all available datasets (MTL-ALL). In this work, we compare all three TL methods in a comprehensive analysis on the GLUE dataset suite. We find that there is a simple heuristic for when to use one of these techniques over the other: pairwise MTL is better than STILTs when the target task has fewer instances than the supporting task and vice versa. We show that this holds true in more than 92% of applicable cases on the GLUE dataset and validate this hypothesis with experiments varying dataset size. The simplicity and effectiveness of this heuristic is surprising and warrants additional exploration by the TL community. Furthermore, we find that MTL-ALL is worse than the pairwise methods in almost every case. We hope this study will aid others as they choose between TL methods for NLP tasks.

[105]  arXiv:2205.08128 [pdf, ps, other]
Title: Local Completeness Logic on Kleene Algebra with Tests
Subjects: Logic in Computer Science (cs.LO); Programming Languages (cs.PL)

Local Completeness Logic (LCL) has been put forward as a program logic for proving both the correctness and incorrectness of program specifications. LCL is an abstract logic, parameterized by an abstract domain that allows combining over- and under-approximations of program behaviors. It turns out that LCL instantiated to the trivial singleton abstraction boils down to O'Hearn incorrectness logic, which allows us to prove the presence of program bugs. It has been recently proved that suitable extensions of Kleene algebra with tests (KAT) allow representing both O'Hearn incorrectness and Hoare correctness program logics within the same equational framework. In this work, we generalize this result by showing how KATs extended either with a modal diamond operator or with a top element are able to represent the local completeness logic LCL. This is achieved by studying how these extended KATs can be endowed with an abstract domain so as to define the validity of correctness/incorrectness LCL triples and to show that the LCL proof system is logically sound and, under some hypotheses, complete.

[106]  arXiv:2205.08129 [pdf, other]
Title: Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space
Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

General-purpose robots require diverse repertoires of behaviors to complete challenging tasks in real-world unstructured environments. To address this issue, goal-conditioned reinforcement learning aims to acquire policies that can reach configurable goals for a wide range of tasks on command. However, such goal-conditioned policies are notoriously difficult and time-consuming to train from scratch. In this paper, we propose Planning to Practice (PTP), a method that makes it practical to train goal-conditioned policies for long-horizon tasks that require multiple distinct types of interactions to solve. Our approach is based on two key ideas. First, we decompose the goal-reaching problem hierarchically, with a high-level planner that sets intermediate subgoals using conditional subgoal generators in the latent space for a low-level model-free policy. Second, we propose a hybrid approach which first pre-trains both the conditional subgoal generator and the policy on previously collected data through offline reinforcement learning, and then fine-tunes the policy via online exploration. This fine-tuning process is itself facilitated by the planned subgoals, which breaks down the original target task into short-horizon goal-reaching tasks that are significantly easier to learn. We conduct experiments in both the simulation and real world, in which the policy is pre-trained on demonstrations of short primitive behaviors and fine-tuned for temporally extended tasks that are unseen in the offline data. Our experimental results show that PTP can generate feasible sequences of subgoals that enable the policy to efficiently solve the target tasks.

[107]  arXiv:2205.08146 [pdf, other]
Title: Formal verification of an industrial UML-like model using mCRL2 (extended version)
Comments: pre-print of a paper that is submitted to FMICS 2022
Subjects: Systems and Control (eess.SY); Logic in Computer Science (cs.LO)

Low-code development platforms are gaining popularity. Essentially, such platforms allow to shift from coding to graphical modeling, helping to improve quality and reduce development time. The Cordis SUITE is a low-code development platform that adopts the Unified Modeling Language (UML) to design complex machine-control applications. In this paper we introduce Cordis models and their semantics. To enable formal verification, we define an automatic translation of Cordis models to the process algebraic specification language mCRL2. As a proof of concept, we describe requirements of the control software of an industrial cylinder model developed by Cordis, and show how these can be verified using model checking. We show that our verification approach is effective to uncover subtle issues in the industrial model and its implementation.

[108]  arXiv:2205.08147 [pdf, other]
Title: Pairwise Comparison Network for Remote Sensing Scene Classification
Comments: 6 pages, 4 figures, published to GRSL
Journal-ref: IEEE Geoscience and Remote Sensing Letters, vol. 19, pp. 1-5, 2022
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Remote sensing scene classification aims to assign a specific semantic label to a remote sensing image. Recently, convolutional neural networks have greatly improved the performance of remote sensing scene classification. However, some confused images may be easily recognized as the incorrect category, which generally degrade the performance. The differences between image pairs can be used to distinguish image categories. This paper proposed a pairwise comparison network, which contains two main steps: pairwise selection and pairwise representation. The proposed network first selects similar image pairs, and then represents the image pairs with pairwise representations. The self-representation is introduced to highlight the informative parts of each image itself, while the mutual-representation is proposed to capture the subtle differences between image pairs. Comprehensive experimental results on two challenging datasets (AID, NWPU-RESISC45) demonstrate the effectiveness of the proposed network. The code are provided in https://github.com/spectralpublic/PCNet.git.

[109]  arXiv:2205.08148 [pdf, other]
Title: Gender and Racial Bias in Visual Question Answering Datasets
Comments: ACM Conference on Fairness, Accountability, and Transparency (FAccT 2022)
Subjects: Computer Vision and Pattern Recognition (cs.CV); Computers and Society (cs.CY)

Vision-and-language tasks have increasingly drawn more attention as a means to evaluate human-like reasoning in machine learning models. A popular task in the field is visual question answering (VQA), which aims to answer questions about images. However, VQA models have been shown to exploit language bias by learning the statistical correlations between questions and answers without looking into the image content: e.g., questions about the color of a banana are answered with yellow, even if the banana in the image is green. If societal bias (e.g., sexism, racism, ableism, etc.) is present in the training data, this problem may be causing VQA models to learn harmful stereotypes. For this reason, we investigate gender and racial bias in five VQA datasets. In our analysis, we find that the distribution of answers is highly different between questions about women and men, as well as the existence of detrimental gender-stereotypical samples. Likewise, we identify that specific race-related attributes are underrepresented, whereas potentially discriminatory samples appear in the analyzed datasets. Our findings suggest that there are dangers associated to using VQA datasets without considering and dealing with the potentially harmful stereotypes. We conclude the paper by proposing solutions to alleviate the problem before, during, and after the dataset collection process.

[110]  arXiv:2205.08149 [pdf, ps, other]
Title: A Novel K-Repetition Design for SCMA
Comments: 6 pages, 6 figures
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

This work presents a novel K-Repetition based HARQ scheme for LDPC coded uplink SCMA by employing a network coding (NC) principle to encode different packets, where K-Repetition is an emerging technique (recommended in 3GPP Release 15) for enhanced reliability and reduced latency in future massive machine-type communication. Such a scheme is referred to as the NC aided K-repetition SCMA (NCK-SCMA). We introduce a joint iterative detection algorithm for improved detection of the data from the proposed LDPC coded NCKSCMA systems. Simulation results demonstrate the benefits of NCK-SCMA with higher throughput and improved reliability over the conventional K-Repetition SCMA.

[111]  arXiv:2205.08150 [pdf, ps, other]
Title: UnPWC-SVDLO: Multi-SVD on PointPWC for Unsupervised Lidar Odometry
Authors: Yiming Tu
Subjects: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

High-precision lidar odomety is an essential part of autonomous driving. In recent years, deep learning methods have been widely used in lidar odomety tasks, but most of the current methods only extract the global features of the point clouds. It is impossible to obtain more detailed point-level features in this way. In addition, only the fully connected layer is used to estimate the pose. The fully connected layer has achieved obvious results in the classification task, but the changes in pose are a continuous rather than discrete process, high-precision pose estimation can not be obtained only by using the fully connected layer. Our method avoids problems mentioned above. We use PointPWC as our backbone network. PointPWC is originally used for scene flow estimation. The scene flow estimation task has a strong correlation with lidar odomety. Traget point clouds can be obtained by adding the scene flow and source point clouds. We can achieve the pose directly through ICP algorithm solved by SVD, and the fully connected layer is no longer used. PointPWC extracts point-level features from point clouds with different sampling levels, which solves the problem of too rough feature extraction. We conduct experiments on KITTI, Ford Campus Vision and Lidar DataSe and Apollo-SouthBay Dataset. Our result is comparable with the state-of-the-art unsupervised deep learing method SelfVoxeLO.

[112]  arXiv:2205.08151 [pdf, other]
Title: Cluster on Wheels
Comments: 8 pages, 7 figures, 2022 International Conference for Advancement in Technology(ICONAT). It is about the work of the mapping robot cluster computer platform
Journal-ref: Y. Yang, D. Feng and S. Schwertfeger, "Cluster on Wheels," 2022 International Conference for Advancement in Technology (ICONAT), 2022, pp. 1-8
Subjects: Robotics (cs.RO)

This paper presents a very compact 16-node cluster that is the core of a future robot for collecting and storing massive amounts of sensor data for research on Simultaneous Localization and Mapping (SLAM). To the best of our knowledge, this is the first time that such a cluster is used in robotics. We first present the requirements and different options for computing of such a robot and then show the hardware and software of our solution in detail. The cluster consists of 16 nodes of AMD Ryzen 7 5700U CPUs with a total of 128 cores. As a system that is to be used on a Clearpath Husky robot, it is very small in size, can be operated from battery power and has all required power and networking components integrated. Stress tests on the completed cluster show that it performs well.

[113]  arXiv:2205.08152 [pdf, ps, other]
Title: Dual-mode robust MPC for the tracking control of non-holonomoic mobile robots
Authors: Huan Meng
Subjects: Robotics (cs.RO)

In this paper, a novel dual-mode robust model predictive control (MPC) approach is proposed for solving the tracking control problem of non-holonomoic mobile robots with additive bounded disturbance. To reduce the negative effect of disturbance and drive the state of real system closer to the one of nominal system , a robust reference signal is introduced into the cost function of MPC. In order to reduced the computation burden caused by online optimization of MPC and further improve the tracking accuracy, a dual-mode control strucuture consisting of the robust MPC and the local nonlinear robust control is developed, in which the local nonlinear robust control law is applied within a specified terminal region. Finally, simulation results on the non-holonomic mobile robot are presented to show the validity of the proposed control approach.

[114]  arXiv:2205.08155 [pdf, other]
Title: Communication-Free Shepherding Navigation with Multiple Steering Agents
Comments: 6 pages, 2 figures, comments welcome
Subjects: Systems and Control (eess.SY); Multiagent Systems (cs.MA)

Swarm guidance addresses a challenging problem considering the navigation and control of a group of passive agents. To solve this problem, shepherding offers a bio-inspired technique of navigating such group of agents by using external steering agents with appropriately designed movement law. Although most shepherding researches are mainly based on the availability of centralized instructions, these assumptions are not realistic enough to solve some emerging application problems. Therefore, this paper presents a decentralized shepherding method where each steering agent makes movements based on its own observation without any inter-agent communication. Our numerical simulations confirm the effectiveness of the proposed method by showing its high success rate and low costs in various placement patterns. These advantages particularly improve with the increase in the number of steering agents.

[115]  arXiv:2205.08157 [pdf, other]
Title: Uncertainty-based Network for Few-shot Image Classification
Comments: Few-shot learning, Uncertainty, Mutual information
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

The transductive inference is an effective technique in the few-shot learning task, where query sets update prototypes to improve themselves. However, these methods optimize the model by considering only the classification scores of the query instances as confidence while ignoring the uncertainty of these classification scores. In this paper, we propose a novel method called Uncertainty-Based Network, which models the uncertainty of classification results with the help of mutual information. Specifically, we first data augment and classify the query instance and calculate the mutual information of these classification scores. Then, mutual information is used as uncertainty to assign weights to classification scores, and the iterative update strategy based on classification scores and uncertainties assigns the optimal weights to query instances in prototype optimization. Extensive results on four benchmarks show that Uncertainty-Based Network achieves comparable performance in classification accuracy compared to state-of-the-art method.

[116]  arXiv:2205.08159 [pdf]
Title: SEMI-FND: Stacked Ensemble Based Multimodal Inference For Faster Fake News Detection
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Fake News Detection (FND) is an essential field in natural language processing that aims to identify and check the truthfulness of major claims in a news article to decide the news veracity. FND finds its uses in preventing social, political and national damage caused due to misrepresentation of facts which may harm a certain section of society. Further, with the explosive rise in fake news dissemination over social media, including images and text, it has become imperative to identify fake news faster and more accurately. To solve this problem, this work investigates a novel multimodal stacked ensemble-based approach (SEMIFND) to fake news detection. Focus is also kept on ensuring faster performance with fewer parameters. Moreover, to improve multimodal performance, a deep unimodal analysis is done on the image modality to identify NasNet Mobile as the most appropriate model for the task. For text, an ensemble of BERT and ELECTRA is used. The approach was evaluated on two datasets: Twitter MediaEval and Weibo Corpus. The suggested framework offered accuracies of 85.80% and 86.83% on the Twitter and Weibo datasets respectively. These reported metrics are found to be superior when compared to similar recent works. Further, we also report a reduction in the number of parameters used in training when compared to recent relevant works. SEMI-FND offers an overall parameter reduction of at least 20% with unimodal parametric reduction on text being 60%. Therefore, based on the investigations presented, it is concluded that the application of a stacked ensembling significantly improves FND over other approaches while also improving speed.

[117]  arXiv:2205.08166 [pdf, other]
Title: CellTypeGraph: A New Geometric Computer Vision Benchmark
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

Classifying all cells in an organ is a relevant and difficult problem from plant developmental biology. We here abstract the problem into a new benchmark for node classification in a geo-referenced graph. Solving it requires learning the spatial layout of the organ including symmetries. To allow the convenient testing of new geometrical learning methods, the benchmark of Arabidopsis thaliana ovules is made available as a PyTorch data loader, along with a large number of precomputed features. Finally, we benchmark eight recent graph neural network architectures, finding that DeeperGCN currently works best on this problem.

[118]  arXiv:2205.08168 [pdf, other]
Title: Global existence of classical solutions and numerical simulations of a cancer invasion model
Comments: 43 pages, 16 figures
Subjects: Numerical Analysis (math.NA); Analysis of PDEs (math.AP)

In this paper, we study a cancer invasion model both theoretically and numerically. The model is a nonstationary, nonlinear system of three coupled partial differential equations modeling the motion of cancer cells, degradation of the extracellular matrix, and certain enzymes. We first establish existence of global classical solutions in both two- and three-dimensional bounded domains, despite the lack of diffusion of the matrix-degrading enzymes and corresponding regularizing effects in the analytical treatment. Next, we give a weak formulation and apply finite differences in time and a Galerkin finite element scheme for spatial discretization. The overall algorithm is based on a fixed-point iteration scheme. In order to substantiate our theory and numerical framework, several numerical simulations are carried out in two and three spatial dimensions.

[119]  arXiv:2205.08175 [pdf, other]
Title: Probabilistic Automata of Bounded Ambiguity
Comments: Short version in CONCUR'17, Long version in Information and Computation (special issue on Weighted Automata)
Journal-ref: Information and Computation, Volume 282, January 2022, 104648
Subjects: Formal Languages and Automata Theory (cs.FL)

Probabilistic automata are an extension of nondeterministic finite automata in which transitions are annotated with probabilities. Despite its simplicity, this model is very expressive and many of the associated algorithmic questions are undecidable. In this work we focus on the emptiness problem (and its variant the value problem), which asks whether a given probabilistic automaton accepts some word with probability greater than a given threshold. We consider a natural and well-studied structural restriction on automata, namely the degree of ambiguity, which is defined as the maximum number of accepting runs over all words. The known undecidability proofs exploits infinite ambiguity and so we focus on the case of finitely ambiguous probabilistic automata. Our main contributions are to construct efficient algorithms for analysing finitely ambiguous probabilistic automata through a reduction to a multi-objective optimisation problem called the stochastic path problem. We obtain a polynomial time algorithm for approximating the value of probabilistic automata of fixed ambiguity and a quasi-polynomial time algorithm for the emptiness problem for 2-ambiguous probabilistic automata. We complement these positive results by an inapproximability result stating that the value of finitely ambiguous probabilistic automata cannot be approximated unless PTIME = NP.

[120]  arXiv:2205.08178 [pdf, other]
Title: Active learning of causal probability trees
Authors: Tue Herlau
Subjects: Machine Learning (cs.LG); Methodology (stat.ME)

The past two decades have seen a growing interest in combining causal information, commonly represented using causal graphs, with machine learning models. Probability trees provide a simple yet powerful alternative representation of causal information. They enable both computation of intervention and counterfactuals, and are strictly more general, since they allow context-dependent causal dependencies. Here we present a Bayesian method for learning probability trees from a combination of interventional and observational data. The method quantifies the expected information gain from an intervention, and selects the interventions with the largest gain. We demonstrate the efficiency of the method on simulated and real data. An effective method for learning probability trees on a limited interventional budget will greatly expand their applicability.

[121]  arXiv:2205.08180 [pdf, other]
Title: SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

We propose the SAMU-XLSR: Semantically-Aligned Multimodal Utterance-level Cross-Lingual Speech Representation learning framework. Unlike previous works on speech representation learning, which learns multilingual contextual speech embedding at the resolution of an acoustic frame (10-20ms), this work focuses on learning multimodal (speech-text) multilingual speech embedding at the resolution of a sentence (5-10s) such that the embedding vector space is semantically aligned across different languages. We combine state-of-the-art multilingual acoustic frame-level speech representation learning model XLS-R with the Language Agnostic BERT Sentence Embedding (LaBSE) model to create an utterance-level multimodal multilingual speech encoder SAMU-XLSR. Although we train SAMU-XLSR with only multilingual transcribed speech data, cross-lingual speech-text and speech-speech associations emerge in its learned representation space. To substantiate our claims, we use SAMU-XLSR speech encoder in combination with a pre-trained LaBSE text sentence encoder for cross-lingual speech-to-text translation retrieval, and SAMU-XLSR alone for cross-lingual speech-to-speech translation retrieval. We highlight these applications by performing several cross-lingual text and speech translation retrieval tasks across several datasets.

[122]  arXiv:2205.08184 [pdf, other]
Title: SKILL: Structured Knowledge Infusion for Large Language Models
Comments: NAACL 2022
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from text. In this work, we propose a method to infuse structured knowledge into LLMs, by directly training T5 models on factual triples of knowledge graphs (KGs). We show that models pre-trained on Wikidata KG with our method outperform the T5 baselines on FreebaseQA and WikiHop, as well as the Wikidata-answerable subset of TriviaQA and NaturalQuestions. The models pre-trained on factual triples compare competitively with the ones on natural language sentences that contain the same knowledge. Trained on a smaller size KG, WikiMovies, we saw 3x improvement of exact match score on MetaQA task compared to T5 baseline. The proposed method has an advantage that no alignment between the knowledge graph and text corpus is required in curating training data. This makes our method particularly useful when working with industry-scale knowledge graphs.

[123]  arXiv:2205.08185 [pdf, ps, other]
Title: Large-stepsize integrators with improved uniform accuracy and long time conservation for highly oscillatory systems with large initial data
Subjects: Numerical Analysis (math.NA)

In this paper, we are concerned with large-stepsize highly accurate integrators for highly oscillatory second-order differential equations with large initial data and a scaling parameter $0 <\varepsilon\ll 1$. The highly oscillatory property of this model problem corresponds to the parameter $\varepsilon$. We propose and analyze a novel class of highly accurate integrators which is based on some formulation approaches to the problem, Fourier pseudo-spectral method and exponential integrators. Two practical integrators up to order four are constructed by using the symmetric property and stiff order conditions of implicit exponential integrators. The convergence of the obtained integrators is rigorously studied, and it is shown that the accuracy is improved to be $\mathcal{O}(\varepsilon^2 h^r)$ in the absolute position error for the time stepsize $h$ and the order $r$ of the integrator. The near energy conservation over long times is established for the integrators with large time stepsizes. Numerical results show that the proposed integrators used with large stepsizes have improved uniformly high accuracy and excellent long time energy conservation.

[124]  arXiv:2205.08189 [pdf, other]
Title: Automatic Acquisition of a Repertoire of Diverse Grasping Trajectories through Behavior Shaping and Novelty Search
Comments: 7 pages, 9 figures, accepted at ICRA 2022. Annex video available at this https URL
Subjects: Robotics (cs.RO); Machine Learning (cs.LG)

Grasping a particular object may require a dedicated grasping movement that may also be specific to the robot end-effector. No generic and autonomous method does exist to generate these movements without making hypotheses on the robot or on the object. Learning methods could help to autonomously discover relevant grasping movements, but they face an important issue: grasping movements are so rare that a learning method based on exploration has little chance to ever observe an interesting movement, thus creating a bootstrap issue. We introduce an approach to generate diverse grasping movements in order to solve this problem. The movements are generated in simulation, for particular object positions. We test it on several simulated robots: Baxter, Pepper and a Kuka Iiwa arm. Although we show that generated movements actually work on a real Baxter robot, the aim is to use this method to create a large dataset to bootstrap deep learning methods.

[125]  arXiv:2205.08191 [pdf, ps, other]
Title: Semi-discretization and full-discretization with optimal accuracy for charged-particle dynamics in a strong nonuniform magnetic field
Subjects: Numerical Analysis (math.NA)

The aim of this paper is to formulate and analyze numerical discretizations of charged-particle dynamics (CPD) in a strong nonuniform magnetic field. A strategy is firstly performed for the two dimensional CPD to construct the semi-discretization and full-discretization which have optimal accuracy. This accuracy is improved in the position and in the velocity when the strength of the magnetic field becomes stronger. This is a better feature than the usual so called "uniformly accurate methods". To obtain this refined accuracy, some reformulations of the problem and two-scale exponential integrators are incorporated, and the optimal accuracy is derived from this new procedure. Then based on the strategy given for the two dimensional case, a new class of uniformly accurate methods with simple scheme is formulated for the three dimensional CPD in maximal ordering case. All the theoretical results of the accuracy are numerically illustrated by some numerical tests.

[126]  arXiv:2205.08192 [pdf, other]
Title: Moral reinforcement learning using actual causation
Authors: Tue Herlau
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)

Reinforcement learning systems will to a greater and greater extent make decisions that significantly impact the well-being of humans, and it is therefore essential that these systems make decisions that conform to our expectations of morally good behavior. The morally good is often defined in causal terms, as in whether one's actions have in fact caused a particular outcome, and whether the outcome could have been anticipated. We propose an online reinforcement learning method that learns a policy under the constraint that the agent should not be the cause of harm. This is accomplished by defining cause using the theory of actual causation and assigning blame to the agent when its actions are the actual cause of an undesirable outcome. We conduct experiments on a toy ethical dilemma in which a natural choice of reward function leads to clearly undesirable behavior, but our method learns a policy that avoids being the cause of harmful behavior, demonstrating the soundness of our approach. Allowing an agent to learn while observing causal moral distinctions such as blame, opens the possibility to learning policies that better conform to our moral judgments.

[127]  arXiv:2205.08199 [pdf, ps, other]
Title: Sharp asymptotics on the compression of two-layer neural networks
Subjects: Information Theory (cs.IT); Machine Learning (cs.LG); Machine Learning (stat.ML)

In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M < N nodes. More precisely, we consider the setting in which the weights of the target network are i.i.d. sub-Gaussian, and we minimize the population L2 loss between the outputs of the target and of the compressed network, under the assumption of Gaussian inputs. By using tools from high-dimensional probability, we show that this non-convex problem can be simplified when the target network is sufficiently over-parameterized, and provide the error rate of this approximation as a function of the input dimension and N . For a ReLU activation function, we conjecture that the optimum of the simplified optimization problem is achieved by taking weights on the Equiangular Tight Frame (ETF), while the scaling of the weights and the orientation of the ETF depend on the parameters of the target network. Numerical evidence is provided to support this conjecture.

[128]  arXiv:2205.08201 [pdf, other]
Title: A Multi-level Methodology for Behavioral Comparison of Software-Intensive Systems
Authors: Dennis Hendriks (1 and 2), Arjan van der Meer (1 and 3), Wytse Oortwijn (1) ((1) ESI (TNO), Eindhoven, The Netherlands, (2) Radboud University, Nijmegen, The Netherlands, (3) Capgemini Engineering, Eindhoven, The Netherlands)
Comments: 21 pages, 13 figures, submitted to FMICS 2022
Subjects: Software Engineering (cs.SE); Formal Languages and Automata Theory (cs.FL)

Software-intensive systems constantly evolve. To prevent software changes from unintentionally introducing costly system defects, it is important to understand their impact to reduce risk. However, it is in practice nearly impossible to foresee the full impact of software changes when dealing with huge industrial systems with many configurations and usage scenarios. To assist developers with change impact analysis we introduce a novel multi-level methodology for behavioral comparison of software-intensive systems. Our fully automated methodology is based on comparing state machine models of software behavior. We combine existing complementary comparison methods into a novel approach, guiding users step by step though relevant differences by gradually zooming into more and more detail. We empirically evaluate our work through a qualitative exploratory field study, showing its practical value using multiple case studies at ASML, a leading company in developing lithography systems. Our method shows great potential for preventing regressions in system behavior for software changes.

[129]  arXiv:2205.08202 [pdf, other]
Title: An Application of Scenario Exploration to Find New Scenarios for the Development and Testing of Automated Driving Systems in Urban Scenarios
Comments: Proceedings of the 8th International Conference on Vehicle Technology and Intelligent Transport Systems (VEHITS 2022)
Subjects: Software Engineering (cs.SE); Machine Learning (cs.LG); Robotics (cs.RO)

Verification and validation are major challenges for developing automated driving systems. A concept that gets more and more recognized for testing in automated driving is scenario-based testing. However, it introduces the problem of what scenarios are relevant for testing and which are not. This work aims to find relevant, interesting, or critical parameter sets within logical scenarios by utilizing Bayes optimization and Gaussian processes. The parameter optimization is done by comparing and evaluating six different metrics in two urban intersection scenarios. Finally, a list of ideas this work leads to and should be investigated further is presented.

[130]  arXiv:2205.08203 [pdf, other]
Title: On the Hierarchy of Distributed Majority Protocols
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)

We study the Consensus problem among $n$ agents, defined as follows. Initially, each agent holds one of two possible opinions. The goal is to reach a consensus configuration in which every agent shares the same opinion. To this end, agents randomly sample other agents and update their opinion according to a simple update function depending on the sampled opinions.
We consider two communication models: the gossip model and a variant of the population model. In the gossip model, agents are activated in parallel, synchronous rounds. In the population model, one agent is activated after the other in a sequence of discrete time steps. For both models we analyze the following natural family of majority processes called $j$-Majority: when activated, every agent samples $j$ other agents uniformly at random (with replacement) and adopts the majority opinion among the sample (breaking ties uniformly at random). As our main result we show a hierarchy among majority protocols: $(j+1)$-Majority (for $j > 1$) converges stochastically faster than $j$-Majority for any initial opinion configuration. In our analysis we use Strassen's Theorem to prove the existence of a coupling. This gives an affirmative answer for the case of two opinions to an open question asked by Berenbrink et al. [2017].

[131]  arXiv:2205.08207 [pdf, other]
Title: DynPL-SVO: A New Method Using Point and Line Features for Stereo Visual Odometry in Dynamic Scenes
Subjects: Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)

Stereo visual odometry is widely used where a robot tracks its position and orientation using stereo cameras. Most of the approaches recovered mobile robotics motion based on the matching and tracking of point features along a sequence of stereo images. But in low-textured and dynamic scenes, there are no sufficient robust static point features for motion estimation, causing lots of previous work to fail to reconstruct the robotic motion. However, line features can be detected in such low-textured and dynamic scenes. In this paper, we proposed DynPL-SVO, a stereo visual odometry with the $dynamic$ $grid$ algorithm and the cost function containing both vertical and horizontal information of the line features. Stereo camera motion was obtained through Levenberg-Marquard minimization of re-projection error of point and line features. The experimental results on the KITTI and EuRoC MAV datasets showed that the DynPL-SVO had a competitive performance when compared to other state-of-the-art systems by producing more robust and accurate motion estimation, especially in low-textured and dynamic scenes.

[132]  arXiv:2205.08209 [pdf, other]
Title: blob loss: instance imbalance aware loss functions for semantic segmentation
Comments: 23 pages, 7 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV)

Deep convolutional neural networks have proven to be remarkably effective in semantic segmentation tasks. Most popular loss functions were introduced targeting improved volumetric scores, such as the Sorensen Dice coefficient. By design, DSC can tackle class imbalance; however, it does not recognize instance imbalance within a class. As a result, a large foreground instance can dominate minor instances and still produce a satisfactory Sorensen Dice coefficient. Nevertheless, missing out on instances will lead to poor detection performance. This represents a critical issue in applications such as disease progression monitoring. For example, it is imperative to locate and surveil small-scale lesions in the follow-up of multiple sclerosis patients. We propose a novel family of loss functions, nicknamed blob loss, primarily aimed at maximizing instance-level detection metrics, such as F1 score and sensitivity. Blob loss is designed for semantic segmentation problems in which the instances are the connected components within a class. We extensively evaluate a DSC-based blob loss in five complex 3D semantic segmentation tasks featuring pronounced instance heterogeneity in terms of texture and morphology. Compared to soft Dice loss, we achieve 5 percent improvement for MS lesions, 3 percent improvement for liver tumor, and an average 2 percent improvement for Microscopy segmentation tasks considering F1 score.

[133]  arXiv:2205.08210 [pdf, other]
Title: Towards Robotic Laboratory Automation Plug & Play: Teaching-free Robot Integration with the LAPP Digital Twin
Subjects: Robotics (cs.RO)

The Laboratory Automation Plug & Play (LAPP) framework is a high-level abstraction layer that makes the autonomous operation of life science laboratory robots possible. The plug & play nature lies in the fact that the manual teaching and configuration of robots is not required. A digital twin (DT) based concept is proposed that outlines the types of information that has to be provided for each relevant component of the system. In particular, for the devices that the robot interfaces with, the robot positions have to be defined beforehand in a device-attached coordinate system (CS) by the vendor. This CS has to be detectable by the vision system of the robot by means of optical markers placed on the front side of the device. With that, the robot is capable of tending the machine by performing the pick-and-place type transportation of standard sample carriers. This basic use case is the primary scope of the LAPP-DT framework. The hardware scope is limited to simple benchtop and mobile manipulators with parallel grippers at this stage. This paper first provides an overview of relevant literature and state-of-the-art solutions, after which it outlines the framework on the conceptual level, followed by the specification of the relevant DT parameters for the robot, for the devices and for the facility. Finally, appropriate technologies and strategies are identified for the implementation.

[134]  arXiv:2205.08218 [pdf, ps, other]
Title: Is hyperinterpolation efficient in the approximation of singular and oscillatory functions?
Comments: 18 pages, 5 figures
Subjects: Numerical Analysis (math.NA)

Singular and oscillatory functions feature in numerous applications. The high-accuracy approximation of such functions shall greatly help us develop high-order methods for solving applied mathematics problems. This paper demonstrates that hyperinterpolation, a discrete projection method with coefficients obtained by evaluating the $L^2$ orthogonal projection coefficients using some numerical integration methods, may be inefficient for approximating singular and oscillatory functions. A relatively large amount of numerical integration points are necessary for satisfactory accuracy. Moreover, in the spirit of product-integration, we propose an efficient modification of hyperinterpolation for such approximation. The proposed approximation scheme, called efficient hyperinterpolation, achieves satisfactory accuracy with fewer numerical integration points than the original scheme. The implementation of the new approximation scheme is relatively easy. Theorems are also given to explain the outperformance of efficient hyperinterpolation over the original scheme in such approximation, with the functions assumed to belong to $L^1(\Omega)$, $L^2(\Omega)$, and $\mathcal{C}(\Omega)$ spaces, respectively. These theorems, as well as numerical experiments on the interval and the sphere, show that efficient hyperinterpolation has better accuracy in such approximation than the original one when the amount of numerical integration points is limited.

[135]  arXiv:2205.08220 [pdf, ps, other]
Title: Rate-Region Characterization and Channel Estimation for Cell-Free Symbiotic Radio Communications
Comments: arXiv admin note: substantial text overlap with arXiv:2106.06148
Subjects: Systems and Control (eess.SY); Signal Processing (eess.SP)

Cell-free massive MIMO and symbiotic radio communication have been recently proposed as the promising beyond fifth-generation (B5G) networking architecture and transmission technology, respectively. To reap the benefits of both, this paper studies cell-free symbiotic radio communication systems, where a number of cell-free access points (APs) cooperatively send primary information to a receiver, and simultaneously support the passive backscattering communication of the secondary backscatter device (BD). We first derive the achievable communication rates of the active primary user and passive secondary user under the assumption of perfect channel state information (CSI), based on which the transmit beamforming of the cellfree APs is optimized to characterize the achievable rate-region of cell-free symbiotic communication systems. Furthermore, to practically acquire the CSI of the active and passive channels, we propose an efficient channel estimation method based on two-phase uplink-training, and the achievable rate-region taking into account CSI estimation errors are further characterized. Simulation results are provided to show the effectiveness of our proposed beamforming and channel estimation methods.

[136]  arXiv:2205.08221 [pdf, other]
Title: Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning
Subjects: Computation and Language (cs.CL)

Sentence compression reduces the length of text by removing non-essential content while preserving important facts and grammaticality. Unsupervised objective driven methods for sentence compression can be used to create customized models without the need for ground-truth training data, while allowing flexibility in the objective function(s) that are used for learning and inference. Recent unsupervised sentence compression approaches use custom objectives to guide discrete search; however, guided search is expensive at inference time. In this work, we explore the use of reinforcement learning to train effective sentence compression models that are also fast when generating predictions. In particular, we cast the task as binary sequence labelling and fine-tune a pre-trained transformer using a simple policy gradient approach. Our approach outperforms other unsupervised models while also being more efficient at inference time.

[137]  arXiv:2205.08231 [pdf, other]
Title: Hyper-Learning for Gradient-Based Batch Size Adaptation
Subjects: Machine Learning (cs.LG)

Scheduling the batch size to increase is an effective strategy to control gradient noise when training deep neural networks. Current approaches implement scheduling heuristics that neglect structure within the optimization procedure, limiting their flexibility to the training dynamics and capacity to discern the impact of their adaptations on generalization. We introduce Arbiter as a new hyperparameter optimization algorithm to perform batch size adaptations for learnable scheduling heuristics using gradients from a meta-objective function, which overcomes previous heuristic constraints by enforcing a novel learning process called hyper-learning. With hyper-learning, Arbiter formulates a neural network agent to generate optimal batch size samples for an inner deep network by learning an adaptive heuristic through observing concomitant responses over T inner descent steps. Arbiter avoids unrolled optimization, and does not require hypernetworks to facilitate gradients, making it reasonably cheap, simple to implement, and versatile to different tasks. We demonstrate Arbiter's effectiveness in several illustrative experiments: to act as a stand-alone batch size scheduler; to complement fixed batch size schedules with greater flexibility; and to promote variance reduction during stochastic meta-optimization of the learning rate.

[138]  arXiv:2205.08232 [pdf, other]
Title: LogicSolver: Towards Interpretable Math Word Problem Solving with Logical Prompt-enhanced Learning
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)

Recently, deep learning models have made great progress in MWP solving on answer accuracy. However, they are uninterpretable since they mainly rely on shallow heuristics to achieve high performance without understanding and reasoning the grounded math logic. To address this issue and make a step towards interpretable MWP solving, we first construct a high-quality MWP dataset named InterMWP which consists of 11,495 MWPs and annotates interpretable logical formulas based on algebraic knowledge as the grounded linguistic logic of each solution equation. Different from existing MWP datasets, our InterMWP benchmark asks for a solver to not only output the solution expressions but also predict the corresponding logical formulas. We further propose a novel approach with logical prompt and interpretation generation, called LogicSolver. For each MWP, our LogicSolver first retrieves some highly-correlated algebraic knowledge and then passes them to the backbone model as prompts to improve the semantic representations of MWPs. With these improved semantic representations, our LogicSolver generates corresponding solution expressions and interpretable knowledge formulas in accord with the generated solution expressions, simultaneously. Experimental results show that our LogicSolver has stronger logical formula-based interpretability than baselines while achieving higher answer accuracy with the help of logical prompts, simultaneously.

[139]  arXiv:2205.08234 [pdf, other]
Title: Delaytron: Efficient Learning of Multiclass Classifiers with Delayed Bandit Feedbacks
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)

In this paper, we present online algorithm called {\it Delaytron} for learning multi class classifiers using delayed bandit feedbacks. The sequence of feedback delays $\{d_t\}_{t=1}^T$ is unknown to the algorithm. At the $t$-th round, the algorithm observes an example $\mathbf{x}_t$ and predicts a label $\tilde{y}_t$ and receives the bandit feedback $\mathbb{I}[\tilde{y}_t=y_t]$ only $d_t$ rounds later. When $t+d_t>T$, we consider that the feedback for the $t$-th round is missing. We show that the proposed algorithm achieves regret of $\mathcal{O}\left(\sqrt{\frac{2 K}{\gamma}\left[\frac{T}{2}+\left(2+\frac{L^2}{R^2\Vert \W\Vert_F^2}\right)\sum_{t=1}^Td_t\right]}\right)$ when the loss for each missing sample is upper bounded by $L$. In the case when the loss for missing samples is not upper bounded, the regret achieved by Delaytron is $\mathcal{O}\left(\sqrt{\frac{2 K}{\gamma}\left[\frac{T}{2}+2\sum_{t=1}^Td_t+\vert \mathcal{M}\vert T\right]}\right)$ where $\mathcal{M}$ is the set of missing samples in $T$ rounds. These bounds were achieved with a constant step size which requires the knowledge of $T$ and $\sum_{t=1}^Td_t$. For the case when $T$ and $\sum_{t=1}^Td_t$ are unknown, we use a doubling trick for online learning and proposed Adaptive Delaytron. We show that Adaptive Delaytron achieves a regret bound of $\mathcal{O}\left(\sqrt{T+\sum_{t=1}^Td_t}\right)$. We show the effectiveness of our approach by experimenting on various datasets and comparing with state-of-the-art approaches.

[140]  arXiv:2205.08240 [pdf, other]
Title: Scheduling in Wireless Networks using Whittle Index Theory
Comments: 6 pages, 9 figures. arXiv admin note: substantial text overlap with arXiv:1910.04402
Subjects: Networking and Internet Architecture (cs.NI)

We consider the problem of scheduling packet transmissions in a wireless network of users while minimizing the energy consumed and the transmission delay. A challenge is that transmissions of users that are close to each other mutually interfere, while users that are far apart can transmit simultaneously without much interference. Each user has a queue of packets that are transmitted on a single channel and mutually non interfering users reuse the spectrum. Using the theory of Whittle index for cost minimizing restless bandits, we design four index-based policies and compare their performance with that of the well-known policies: Slotted ALOHA, maximum weight scheduling, quadratic Lyapunov drift, Cella and Cesa Bianchi algorithm, and two Whittle index based policies from a recently published paper. We make the code used to perform our simulations publicly available, so that it can be used for future work by the research community at large.

[141]  arXiv:2205.08242 [pdf, other]
Title: Outage Analysis of Energy Efficiency in a Finite-Element-IRS Aided Communication System
Comments: 18 Pages, 6 Figures, 2 Tables
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP); Probability (math.PR)

In this paper, we study the performance of an energy efficient wireless communication system, assisted by a finite-element-intelligent reflecting surface (IRS). With no instantaneous channel state information (CSI) at the transmitter, we characterize the system performance in terms of the outage probability (OP) of energy efficiency (EE). Depending upon the availability of line-of-sight (LOS) paths, we analyze the system for two different channel models, viz. Rician and Rayleigh. For an arbitrary number of IRS elements $(N)$, we derive the approximate closed-form solutions for the OP of EE, using Laguerre series and moment matching methods. The analytical results are validated using the Monte-Carlo simulations. Moreover, we also quantify the rate of convergence of the derived expressions to the central limit theorem (CLT) approximations using the \textit{Berry-Esseen} inequality. Further, we prove that the OP of EE is a strict pseudo-convex function of the transmit power and hence, has a unique global minimum. To obtain the optimal transmit power, we solve the OP of EE as a constrained optimization problem. To the best of our knowledge, the OP of EE as a performance metric, has never been previously studied in IRS-assisted wireless communication systems.

[142]  arXiv:2205.08243 [pdf, other]
Title: IIsy: Practical In-Network Classification
Comments: (14 pages body, 19 pages total, 19 figures)
Subjects: Networking and Internet Architecture (cs.NI); Machine Learning (cs.LG)

The rat race between user-generated data and data-processing systems is currently won by data. The increased use of machine learning leads to further increase in processing requirements, while data volume keeps growing. To win the race, machine learning needs to be applied to the data as it goes through the network. In-network classification of data can reduce the load on servers, reduce response time and increase scalability. In this paper, we introduce IIsy, implementing machine learning classification models in a hybrid fashion using off-the-shelf network devices. IIsy targets three main challenges of in-network classification: (i) mapping classification models to network devices (ii) extracting the required features and (iii) addressing resource and functionality constraints. IIsy supports a range of traditional and ensemble machine learning models, scaling independently of the number of stages in a switch pipeline. Moreover, we demonstrate the use of IIsy for hybrid classification, where a small model is implemented on a switch and a large model at the backend, achieving near optimal classification results, while significantly reducing latency and load on the servers.

[143]  arXiv:2205.08247 [pdf, other]
Title: Monotonicity Regularization: Improved Penalties and Novel Applications to Disentangled Representation Learning and Robust Classification
Comments: Accepted to UAI 2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

We study settings where gradient penalties are used alongside risk minimization with the goal of obtaining predictors satisfying different notions of monotonicity. Specifically, we present two sets of contributions. In the first part of the paper, we show that different choices of penalties define the regions of the input space where the property is observed. As such, previous methods result in models that are monotonic only in a small volume of the input space. We thus propose an approach that uses mixtures of training instances and random points to populate the space and enforce the penalty in a much larger region. As a second set of contributions, we introduce regularization strategies that enforce other notions of monotonicity in different settings. In this case, we consider applications, such as image classification and generative modeling, where monotonicity is not a hard constraint but can help improve some aspects of the model. Namely, we show that inducing monotonicity can be beneficial in applications such as: (1) allowing for controllable data generation, (2) defining strategies to detect anomalous data, and (3) generating explanations for predictions. Our proposed approaches do not introduce relevant computational overhead while leading to efficient procedures that provide extra benefits over baseline models.

[144]  arXiv:2205.08249 [pdf, other]
Title: Learnable Optimal Sequential Grouping for Video Scene Detection
Journal-ref: Daniel Rotman, Yevgeny Yaroker, Elad Amrani, Udi Barzelay, and Rami Ben-Ari. "Learnable optimal sequential grouping for video scene detection." In Proceedings of the 28th ACM International Conference on Multimedia, pp. 1958-1966. 2020
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Video scene detection is the task of dividing videos into temporal semantic chapters. This is an important preliminary step before attempting to analyze heterogeneous video content. Recently, Optimal Sequential Grouping (OSG) was proposed as a powerful unsupervised solution to solve a formulation of the video scene detection problem. In this work, we extend the capabilities of OSG to the learning regime. By giving the capability to both learn from examples and leverage a robust optimization formulation, we can boost performance and enhance the versatility of the technology. We present a comprehensive analysis of incorporating OSG into deep learning neural networks under various configurations. These configurations include learning an embedding in a straight-forward manner, a tailored loss designed to guide the solution of OSG, and an integrated model where the learning is performed through the OSG pipeline. With thorough evaluation and analysis, we assess the benefits and behavior of the various configurations, and show that our learnable OSG approach exhibits desirable behavior and enhanced performance compared to the state of the art.

[145]  arXiv:2205.08252 [pdf, other]
Title: An Empirical Assessment of Security and Privacy Risks of Web based-Chatbots
Comments: Submitted to WISE 2020 Conference
Subjects: Cryptography and Security (cs.CR)

Web-based chatbots provide website owners with the benefits of increased sales, immediate response to their customers, and insight into customer behaviour. While Web-based chatbots are getting popular, they have not received much scrutiny from security researchers. The benefits to owners come at the cost of users' privacy and security. Vulnerabilities, such as tracking cookies and third-party domains, can be hidden in the chatbot's iFrame script. This paper presents a large-scale analysis of five Web-based chatbots among the top 1-million Alexa websites. Through our crawler tool, we identify the presence of chatbots in these 1-million websites. We discover that 13,515 out of the top 1-million Alexa websites (1.59%) use one of the five analysed chatbots. Our analysis reveals that the top 300k Alexa ranking websites are dominated by Intercom chatbots that embed the least number of third-party domains. LiveChat chatbots dominate the remaining websites and embed the highest samples of third-party domains. We also find that 850 (6.29%) of the chatbots use insecure protocols to transfer users' chats in plain text. Furthermore, some chatbots heavily rely on cookies for tracking and advertisement purposes. More than two-thirds (68.92%) of the identified cookies in chatbot iFrames are used for ads and tracking users. Our results show that, despite the promises for privacy, security, and anonymity given by the majority of the websites, millions of users may unknowingly be subject to poor security guarantees by chatbot service providers

[146]  arXiv:2205.08253 [pdf, other]
Title: Adaptive Momentum-Based Policy Gradient with Second-Order Information
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

The variance reduced gradient estimators for policy gradient methods has been one of the main focus of research in the reinforcement learning in recent years as they allow acceleration of the estimation process. We propose a variance reduced policy gradient method, called SGDHess-PG, which incorporates second-order information into stochastic gradient descent (SGD) using momentum with an adaptive learning rate. SGDHess-PG algorithm can achieve $\epsilon$-approximate first-order stationary point with $\tilde{O}(\epsilon^{-3})$ number of trajectories, while using a batch size of $O(1)$ at each iteration. Unlike most previous work, our proposed algorithm does not require importance sampling techniques which can compromise the advantage of variance reduction process. Our extensive experimental results show the effectiveness of the proposed algorithm on various control tasks and its advantage over the state of the art in practice.

[147]  arXiv:2205.08255 [pdf]
Title: Utilizing Low-Cost Linux Micro-Computer & Android Phone Solutions on Cube-Satellites
Subjects: Other Computer Science (cs.OH)

Realizing functional space systems using flight-tested components is problematic in developing economies, as such components are costly for most institutions to sponsor. The B.Sc. project, Subsystems for 2nd Iteration Cairo University Cube-Satellite, addresses technology demonstration using commercially available electronics and low cost computing platforms, such as Android phones and Raspberry Pi Linux micro-computer as computing hardware. As for software, the project makes use of open-source modules and locally developed code to implement needed functionalities, in addition to a mechanism to operate a virtual desktop Linux OS in parallel to an Android application. The paper aims to demonstrate the significance, operation design, and problem solving of such approaches. The paper concludes with future prospects for improving upon the proposed computing systems

[148]  arXiv:2205.08256 [pdf, other]
Title: Letters From the Past: Modeling Historical Sound Change Through Diachronic Character Embeddings
Comments: Accepted as long paper at ACL 2022
Subjects: Computation and Language (cs.CL)

While a great deal of work has been done on NLP approaches to lexical semantic change detection, other aspects of language change have received less attention from the NLP community. In this paper, we address the detection of sound change through historical spelling. We propose that a sound change can be captured by comparing the relative distance through time between their distributions using PPMI character embeddings. We verify this hypothesis in synthetic data and then test the method's ability to trace the well-known historical change of lenition of plosives in Danish historical sources. We show that the models are able to identify several of the changes under consideration and to uncover meaningful contexts in which they appeared. The methodology has the potential to contribute to the study of open questions such as the relative chronology of sound shifts and their geographical distribution.

[149]  arXiv:2205.08257 [pdf, other]
Title: Detection Masking for Improved OCR on Noisy Documents
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Optical Character Recognition (OCR), the task of extracting textual information from scanned documents is a vital and broadly used technology for digitizing and indexing physical documents. Existing technologies perform well for clean documents, but when the document is visually degraded, or when there are non-textual elements, OCR quality can be greatly impacted, specifically due to erroneous detections. In this paper we present an improved detection network with a masking system to improve the quality of OCR performed on documents. By filtering non-textual elements from the image we can utilize document-level OCR to incorporate contextual information to improve OCR results. We perform a unified evaluation on a publicly available dataset demonstrating the usefulness and broad applicability of our method. Additionally, we present and make publicly available our synthetic dataset with a unique hard-negative component specifically tuned to improve detection results, and evaluate the benefits that can be gained from its usage

[150]  arXiv:2205.08260 [pdf]
Title: LabVIEW is faster and C is economical interfacing tool for UCT automation
Comments: 15 pages, 9 figures, 2 tables, 23 references
Subjects: Programming Languages (cs.PL); Robotics (cs.RO)

An in-house developed 2D ultrasound computerized Tomography system is fully automated. Performance analysis of instrument and software interfacing soft tools, namely the LabVIEW, MATLAB, C, and Python, is presented. The instrument interfacing algorithms, hardware control algorithms, signal processing, and analysis codes are written using above mentioned soft tool platforms. Total of eight performance indices are used to compare the ease of (a) realtime control of electromechanical assembly, (b) sensors, instruments integration, (c) synchronized data acquisition, and (d) simultaneous raw data processing. It is found that C utilizes the least processing power and performs a lower number of processes to perform the same task. In runtime analysis (data acquisition and realtime control), LabVIEW performs best, taking 365.69s in comparison to MATLAB (623.83s), Python ( 1505.54s), and C (1252.03s) to complete the experiment. Python performs better in establishing faster interfacing and minimum RAM usage. LabVIEW is recommended for its fast process execution. C is recommended for the most economical implementation. Python is recommended for complex system automation having a very large number of components involved. This article provides a methodology to select optimal soft tools for instrument automation-related aspects.

[151]  arXiv:2205.08262 [pdf, ps, other]
Title: Lossy Computing with Side Information via Multi-Hypergraphs
Subjects: Information Theory (cs.IT)

We consider a problem of coding for computing, where the decoder wishes to estimate a function of its local message and the source message at the encoder within a given distortion. We show that the rate-distortion function can be characterized through a characteristic multi-hypergraph, which simplifies the evaluation of the rate-distortion function.

[152]  arXiv:2205.08263 [pdf, ps, other]
Title: Contact-less Material Probing with Distributed Sensors: Joint Sensing and Communication Optimization
Comments: arXiv admin note: text overlap with arXiv:1902.11117
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

The utilization of RF signals to probe material properties of objects is of huge interest both in academia as well as industry. To this end, a setup is investigated, in which a transmitter equipped with a two-dimensional multi-antenna array dispatches a signal, which hits objects in the environment and the reflections from the objects are captured by distributed sensors. The received signal at those sensors are then amplified and forwarded to a multiple antenna fusion center, which performs space-time post-processing in order to optimize the information extraction. In this process, optimal design of power allocation per object alongside sensors amplifications is of crucial importance. Here, the power allocation and sensors amplifications is jointly optimized, given maximum-ratio combining (MRC) at the fusion center. We formulate this challenge as a sum-power minimization under per-object SINR constraints, a sum-power constraint at the transmitter and individual power constraints at the sensors. Moreover, the advantage of deploying zero-forcing (ZF) and minimum mean-squared error (MMSE) at the fusion center is discussed. Asymptotic analysis is also provided for the case that large number of sensors are deployed in the sensing environment.

[153]  arXiv:2205.08264 [pdf]
Title: Working with Affective Computing: Exploring UK Public Perceptions of AI enabled Workplace Surveillance
Subjects: Human-Computer Interaction (cs.HC); Computers and Society (cs.CY)

This paper explores public perceptions around the role of affective computing in the workplace. It uses a series of design fictions with 46 UK based participants, unpacking their perspectives on the advantages and disadvantages of tracking the emotional state of workers. The scenario focuses on mundane uses of biometric sensing in a sales environment, and how this could shape management approaches with workers. The paper structure is as follows: section 1 provides a brief introduction; section 2 provides an overview of the innovative design fiction methodology; section 3 explores wider shifts around IT in the workplace; section 4 provides some legal analysis exploring emergence of AI in the workplace; and section 5 presents themes from the study data. The latter section includes discussion on concerns around functionality and accuracy of affective computing systems, and their impacts on surveillance, human agency, and worker/management interactions.

[154]  arXiv:2205.08265 [pdf, other]
Title: A two-steps approach to improve the performance of Android malware detectors
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)

The popularity of Android OS has made it an appealing target to malware developers. To evade detection, including by ML-based techniques, attackers invest in creating malware that closely resemble legitimate apps. In this paper, we propose GUIDED RETRAINING, a supervised representation learning-based method that boosts the performance of a malware detector. First, the dataset is split into "easy" and "difficult" samples, where difficulty is associated to the prediction probabilities yielded by a malware detector: for difficult samples, the probabilities are such that the classifier is not confident on the predictions, which have high error rates. Then, we apply our GUIDED RETRAINING method on the difficult samples to improve their classification. For the subset of "easy" samples, the base malware detector is used to make the final predictions since the error rate on that subset is low by construction. For the subset of "difficult" samples, we rely on GUIDED RETRAINING, which leverages the correct predictions and the errors made by the base malware detector to guide the retraining process. GUIDED RETRAINING focuses on the difficult samples: it learns new embeddings of these samples using Supervised Contrastive Learning and trains an auxiliary classifier for the final predictions. We validate our method on four state-of-the-art Android malware detection approaches using over 265k malware and benign apps, and we demonstrate that GUIDED RETRAINING can reduce up to 40.41% prediction errors made by the malware detectors. Our method is generic and designed to enhance the classification performance on a binary classification task. Consequently, it can be applied to other classification problems beyond Android malware detection.

[155]  arXiv:2205.08270 [pdf, other]
Title: Chemical Case Studies in KeYmaera X
Authors: Rose Bohrer
Comments: 17 pages. Preprint of submission to FMICS 2022
Subjects: Logic in Computer Science (cs.LO)

Safety-critical chemical processes are the backbone of multi-billion-dollar industries, thus society deserves the strongest possible guarantees that they are safe. To that end, models of chemical processes are well-studied in the formal methods literature, including hybrid systems models which combine discrete and continuous dynamics. This paper is the first to use the KeYmaera X theorem-prover to verify chemical models with differential dynamic logic. Our case studies are novel in combining the following: we provide strong general-case correctness theorems, use particularly rich hybrid dynamics, and have particularly rigorous proofs. This novel combination is made possible by KeYmaera X.
Simultaneously, we tell a general story about KeYmaera X: recent advances in automated reasoning about safety and liveness for differential equations have enabled elegant proofs about reaction dynamics.

[156]  arXiv:2205.08272 [pdf, other]
Title: NOMA-aided Joint Communication, Sensing, and Multi-tier Computing Systems
Comments: 30 pages, 8 figures
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

A non-orthogonal multiple access (NOMA)-aided joint communication, sensing, and multi-tier computing (JCSMC) framework is proposed. In this framework, a multi-functional base station (BS) carries out target sensing, while providing edge computing services to the nearby users. To enhance the computation efficiency, the multi-tier computing structure is exploited, where the BS can further offload the computation tasks to a powerful Cloud server (CS). The potential benefits of employing NOMA in the proposed JCSMC framework are investigated, which can maximize the computation offloading capacity and suppress the inter-function interference. Based on the proposed framework, the transmit beamformer of the BS and computation resource allocation at the BS and the CS are jointly optimized to maximize the computation rate subject to the communication-computation causality and the sensing quality constraints. Both partial and binary computation offloading modes are considered: 1) For the partial offloading mode, a weighted minimum mean square error based alternating optimization algorithm is proposed to solve the corresponding non-convex optimization problem. It is proved that a KKT optimal solution can be obtained; 2) For the binary offloading mode, the resultant highly-coupled mixed-integer optimization problem is first transformed to an equivalent but more tractable form. Then, the reformulated problem is solved by utilizing the alternating direction method of multipliers approach to obtain a nearly optimal solution. Finally, numerical results verify the effectiveness of the proposed algorithms and the proposed NOMA-aided JCSMC framework

[157]  arXiv:2205.08274 [pdf, other]
Title: Tackling Math Word Problems with Fine-to-Coarse Abstracting and Reasoning
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Math Word Problems (MWP) is an important task that requires the ability of understanding and reasoning over mathematical text. Existing approaches mostly formalize it as a generation task by adopting Seq2Seq or Seq2Tree models to encode an input math problem in natural language as a global representation and generate the output mathematical expression. Such approaches only learn shallow heuristics and fail to capture fine-grained variations in inputs. In this paper, we propose to model a math word problem in a fine-to-coarse manner to capture both the local fine-grained information and the global logical structure of it. Instead of generating a complete equation sequence or expression tree from the global features, we iteratively combine low-level operands to predict a higher-level operator, abstracting the problem and reasoning about the solving operators from bottom to up. Our model is naturally more sensitive to local variations and can better generalize to unseen problem types. Extensive evaluations on Math23k and SVAMP datasets demonstrate the accuracy and robustness of our method.

[158]  arXiv:2205.08285 [pdf, other]
Title: KGNN: Distributed Framework for Graph Neural Knowledge Representation
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Knowledge representation learning has been commonly adopted to incorporate knowledge graph (KG) into various online services. Although existing knowledge representation learning methods have achieved considerable performance improvement, they ignore high-order structure and abundant attribute information, resulting unsatisfactory performance on semantics-rich KGs. Moreover, they fail to make prediction in an inductive manner and cannot scale to large industrial graphs. To address these issues, we develop a novel framework called KGNN to take full advantage of knowledge data for representation learning in the distributed learning system. KGNN is equipped with GNN based encoder and knowledge aware decoder, which aim to jointly explore high-order structure and attribute information together in a fine-grained fashion and preserve the relation patterns in KGs, respectively. Extensive experiments on three datasets for link prediction and triplet classification task demonstrate the effectiveness and scalability of KGNN framework.

[159]  arXiv:2205.08287 [pdf, other]
Title: Bankrupting DoS Attackers Despite Uncertainty
Subjects: Cryptography and Security (cs.CR); Distributed, Parallel, and Cluster Computing (cs.DC); Data Structures and Algorithms (cs.DS)

On-demand provisioning in the cloud allows for services to remain available despite massive denial-of-service (DoS) attacks. Unfortunately, on-demand provisioning is expensive and must be weighed against the costs incurred by an adversary. This leads to a recent threat known as economic denial-of-sustainability (EDoS), where the cost for defending a service is higher than that of attacking.
A natural approach for combating EDoS is to impose costs via resource burning (RB). Here, a client must verifiably consume resources -- for example, by solving a computational challenge -- before service is rendered. However, prior approaches with security guarantees do not account for the cost on-demand provisioning.
Another valuable defensive tool is to use a classifier in order to discern good jobs from a legitimate client, versus bad jobs from the adversary. However, while useful, uncertainty arises from classification error, which still allows bad jobs to consume server resources. Thus, classification is not a solution by itself.
Here, we propose an EDoS defense, RootDef, that leverages both RB and classification, while accounting for both the costs of resource burning and on-demand provisioning. Specifically, against an adversary that expends $B$ resources to attack, the total cost for defending is $\tilde{O}( \sqrt{B\,g} + B^{2/3} + g)$, where $g$ is the number of good jobs and $\tilde{O}$ refers to hidden logarithmic factors in the total number of jobs $n$. Notably, for large $B$ relative to $g$, the adversary has higher cost, implying that the algorithm has an economic advantage. Finally, we prove a lower bound showing that RootDef has total costs that are asymptotically tight up to logarithmic factors in $n$.

[160]  arXiv:2205.08288 [pdf, other]
Title: Measuring Alignment Bias in Neural Seq2Seq Semantic Parsers
Comments: 5 pages, 3 figures, accepted at *SEM 2022: The 11th Joint Conference on Lexical and Computational Semantics
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Prior to deep learning the semantic parsing community has been interested in understanding and modeling the range of possible word alignments between natural language sentences and their corresponding meaning representations. Sequence-to-sequence models changed the research landscape suggesting that we no longer need to worry about alignments since they can be learned automatically by means of an attention mechanism. More recently, researchers have started to question such premise. In this work we investigate whether seq2seq models can handle both simple and complex alignments. To answer this question we augment the popular Geo semantic parsing dataset with alignment annotations and create Geo-Aligned. We then study the performance of standard seq2seq models on the examples that can be aligned monotonically versus examples that require more complex alignments. Our empirical study shows that performance is significantly better over monotonic alignments.

[161]  arXiv:2205.08289 [pdf, other]
Title: Experiments on Generalizability of User-Oriented Fairness in Recommender Systems
Comments: SIGIR 2022
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI)

Recent work in recommender systems mainly focuses on fairness in recommendations as an important aspect of measuring recommendations quality. A fairness-aware recommender system aims to treat different user groups similarly. Relevant work on user-oriented fairness highlights the discriminative behavior of fairness-unaware recommendation algorithms towards a certain user group, defined based on users' activity level. Typical solutions include proposing a user-centered fairness re-ranking framework applied on top of a base ranking model to mitigate its unfair behavior towards a certain user group i.e., disadvantaged group. In this paper, we re-produce a user-oriented fairness study and provide extensive experiments to analyze the dependency of their proposed method on various fairness and recommendation aspects, including the recommendation domain, nature of the base ranking model, and user grouping method. Moreover, we evaluate the final recommendations provided by the re-ranking framework from both user- (e.g., NDCG, user-fairness) and item-side (e.g., novelty, item-fairness) metrics. We discover interesting trends and trade-offs between the model's performance in terms of different evaluation metrics. For instance, we see that the definition of the advantaged/disadvantaged user groups plays a crucial role in the effectiveness of the fairness algorithm and how it improves the performance of specific base ranking models. Finally, we highlight some important open challenges and future directions in this field. We release the data, evaluation pipeline, and the trained models publicly on https://github.com/rahmanidashti/FairRecSys.

[162]  arXiv:2205.08290 [pdf, other]
Title: Literature Review to Collect Conceptual Variables of Scenario Methods for Establishing a Conceptual Scenario Framework
Authors: Young-Min Baek (1), Esther Cho (1), Donghwan Shin (2), Doo-Hwan Bae (1) ((1) Korea Advanced Institute of Science and Technology (KAIST), (2) University of Luxembourg)
Comments: 22 pages, 7 figures
Subjects: Software Engineering (cs.SE)

Over recent decades, scenarios and scenario-based software/system engineering have been actively employed as essential tools to handle intricate problems, validate requirements, and support stakeholders' communication. However, despite the widespread use of scenarios, there have been several challenges for engineers to more willingly utilize scenario-based engineering approaches (i.e., scenario methods) in their projects. First, the term scenario has numerous published definitions, thus lacking in a well-established shared understanding of scenarios and scenario methods. Second, the conceptual basis for engineers developing or employing scenarios is missing. To establish shared understanding and to find common denominators of scenario methods, this study leverages well-defined metamodeling and conceptualization that systematically investigate the concepts under analysis and define core entities and their relations. By conducting a semi-systematic literature review, conceptual variables are collected and conceptualized as a conceptual meta-model. As a result, this study introduces scenario variables (SVs) that represent constructs/semantics of scenario descriptions, according to 4 levels of constructs of a scenario method. To evaluate the comprehensibility and applicability of the defined variables, we analyze five existing scenario methods and their instances in automated driving system (ADS) domains. The results showed that our conceptual model and its constituent scenario variables adequately support the understanding of a scenario method and provide a means for comparative analysis between different scenario methods.

[163]  arXiv:2205.08297 [pdf, ps, other]
Title: SCL(EQ): SCL for First-Order Logic with Equality
Subjects: Logic in Computer Science (cs.LO)

We propose a new calculus SCL(EQ) for first-order logic with equality that only learns non-redundant clauses. Following the idea of CDCL (Conflict Driven Clause Learning) and SCL (Clause Learning from Simple Models) a ground literal model assumption is used to guide inferences that are then guaranteed to be non-redundant. Redundancy is defined with respect to a dynamically changing ordering derived from the ground literal model assumption. We prove SCL(EQ) sound and complete and provide examples where our calculus improves on superposition.

[164]  arXiv:2205.08300 [pdf, other]
Title: Sampling-Based Verification of CTMCs with Uncertain Rates
Subjects: Logic in Computer Science (cs.LO); Optimization and Control (math.OC)

We employ uncertain parametric CTMCs with parametric transition rates and a prior on the parameter values. The prior encodes uncertainty about the actual transition rates, while the parameters allow dependencies between transition rates. Sampling the parameter values from the prior distribution then yields a standard CTMC, for which we may compute relevant reachability probabilities. We provide a principled solution, based on a technique called scenario-optimization, to the following problem: From a finite set of parameter samples and a user-specified confidence level, compute prediction regions on the reachability probabilities. The prediction regions should (with high probability) contain the reachability probabilities of a CTMC induced by any additional sample. To boost the scalability of the approach, we employ standard abstraction techniques and adapt our methodology to support approximate reachability probabilities. Experiments with various well-known benchmarks show the applicability of the approach.

[165]  arXiv:2205.08301 [pdf, other]
Title: Centroidal Aerodynamic Modeling and Control of Flying Multibody Robots
Authors: Tong Hui (1 and 2), Antonello Paolino (1 and 4), Gabriele Nava (1), Giuseppe L'Erario (1 and 3), Fabio Di Natale (1), Fabio Bergonti (1 and 3), Francesco Braghin (2), Daniele Pucci (1 and 3) ((1) Istituto Italiano di Tecnologia, (2) Politecnico di Milano, (3) University of Manchester, (4) Università degli Studi di Napoli Federico II)
Comments: 7 pages, 6 figures, to be published in IEEE ICRA 2022. Presentation video: this https URL
Subjects: Robotics (cs.RO)

This paper presents a modeling and control framework for multibody flying robots subject to non-negligible aerodynamic forces acting on the centroidal dynamics. First, aerodynamic forces are calculated during robot flight in different operating conditions by means of Computational Fluid Dynamics (CFD) analysis. Then, analytical models of the aerodynamics coefficients are generated from the dataset collected with CFD analysis. The obtained simplified aerodynamic model is also used to improve the flying robot control design. We present two control strategies: compensating for the aerodynamic effects via feedback linearization and enforcing the controller robustness with gain-scheduling. Simulation results on the jet-powered humanoid robot iRonCub validate the proposed approach.

[166]  arXiv:2205.08303 [pdf, other]
Title: MulT: An End-to-End Multitask Learning Transformer
Comments: Accepted to CVPR 2022
Subjects: Computer Vision and Pattern Recognition (cs.CV)

We propose an end-to-end Multitask Learning Transformer framework, named MulT, to simultaneously learn multiple high-level vision tasks, including depth estimation, semantic segmentation, reshading, surface normal estimation, 2D keypoint detection, and edge detection. Based on the Swin transformer model, our framework encodes the input image into a shared representation and makes predictions for each vision task using task-specific transformer-based decoder heads. At the heart of our approach is a shared attention mechanism modeling the dependencies across the tasks. We evaluate our model on several multitask benchmarks, showing that our MulT framework outperforms both the state-of-the art multitask convolutional neural network models and all the respective single task transformer models. Our experiments further highlight the benefits of sharing attention across all the tasks, and demonstrate that our MulT model is robust and generalizes well to new domains. Our project website is at https://ivrl.github.io/MulT/.

[167]  arXiv:2205.08304 [pdf, other]
Title: Bayesian Physics-Informed Neural Networks for real-world nonlinear dynamical systems
Subjects: Machine Learning (cs.LG); Dynamical Systems (math.DS); Chaotic Dynamics (nlin.CD)

Understanding real-world dynamical phenomena remains a challenging task. Across various scientific disciplines, machine learning has advanced as the go-to technology to analyze nonlinear dynamical systems, identify patterns in big data, and make decision around them. Neural networks are now consistently used as universal function approximators for data with underlying mechanisms that are incompletely understood or exceedingly complex. However, neural networks alone ignore the fundamental laws of physics and often fail to make plausible predictions. Here we integrate data, physics, and uncertainties by combining neural networks, physics-informed modeling, and Bayesian inference to improve the predictive potential of traditional neural network models. We embed the physical model of a damped harmonic oscillator into a fully-connected feed-forward neural network to explore a simple and illustrative model system, the outbreak dynamics of COVID-19. Our Physics-Informed Neural Networks can seamlessly integrate data and physics, robustly solve forward and inverse problems, and perform well for both interpolation and extrapolation, even for a small amount of noisy and incomplete data. At only minor additional cost, they can self-adaptively learn the weighting between data and physics. Combined with Bayesian Neural Networks, they can serve as priors in a Bayesian Inference, and provide credible intervals for uncertainty quantification. Our study reveals the inherent advantages and disadvantages of Neural Networks, Bayesian Inference, and a combination of both and provides valuable guidelines for model selection. While we have only demonstrated these approaches for the simple model problem of a seasonal endemic infectious disease, we anticipate that the underlying concepts and trends generalize to more complex disease conditions and, more broadly, to a wide variety of nonlinear dynamical systems.

[168]  arXiv:2205.08305 [pdf, other]
Title: A Study of the Attention Abnormality in Trojaned BERTs
Comments: Accepted to NAACL-HTL 2022
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Trojan attacks raise serious security concerns. In this paper, we investigate the underlying mechanism of Trojaned BERT models. We observe the attention focus drifting behavior of Trojaned models, i.e., when encountering an poisoned input, the trigger token hijacks the attention focus regardless of the context. We provide a thorough qualitative and quantitative analysis of this phenomenon, revealing insights into the Trojan mechanism. Based on the observation, we propose an attention-based Trojan detector to distinguish Trojaned models from clean ones. To the best of our knowledge, this is the first paper to analyze the Trojan mechanism and to develop a Trojan detector based on the transformer's attention.

[169]  arXiv:2205.08314 [pdf, other]
Title: Self-Sovereign Identity as a Service: Architecture in Practice
Subjects: Software Engineering (cs.SE)

Self-sovereign identity (SSI) has gained a large amount of interest. It enables physical entities to retain ownership and control of their digital identities, which naturally forms a conceptual decentralized architecture. With the support of the distributed ledger technology (DLT), it is possible to implement this conceptual decentralized architecture in practice and further bring technical advantages such as privacy protection, security enhancement, high availability. However, developing such a relatively new identity model has high costs and risks with uncertainty. To facilitate the use of the DLT-based SSI in practice, we formulate Self-Sovereign Identity as a Service (SSIaaS), a concept that enables a system, especially a system cluster, to readily adopt SSI as its identity model for identification, authentication, and authorization. We propose a practical architecture by elaborating the service concept, SSI, and DLT to implement SSIaaS platforms and SSI services. Besides, we present an architecture for constructing and customizing SSI services with a set of architectural patterns and provide corresponding evaluations. Furthermore, we demonstrate the feasibility of our proposed architecture in practice with Selfid, an SSIaaS platform based on our proposed architecture.

[170]  arXiv:2205.08316 [pdf, other]
Title: Self-Supervised Learning of Multi-Object Keypoints for Robotic Manipulation
Comments: Presented at IEEE ICRA 2022 Workshop 'Reinforcement Learning for Contact-Rich Manipulation'
Subjects: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)

In recent years, policy learning methods using either reinforcement or imitation have made significant progress. However, both techniques still suffer from being computationally expensive and requiring large amounts of training data. This problem is especially prevalent in real-world robotic manipulation tasks, where access to ground truth scene features is not available and policies are instead learned from raw camera observations. In this paper, we demonstrate the efficacy of learning image keypoints via the Dense Correspondence pretext task for downstream policy learning. Extending prior work to challenging multi-object scenes, we show that our model can be trained to deal with important problems in representation learning, primarily scale-invariance and occlusion. We evaluate our approach on diverse robot manipulation tasks, compare it to other visual representation learning approaches, and demonstrate its flexibility and effectiveness for sample-efficient policy learning.

[171]  arXiv:2205.08321 [pdf, other]
Title: Finite Element Method-enhanced Neural Network for Forward and Inverse Problems
Subjects: Computational Engineering, Finance, and Science (cs.CE); Machine Learning (cs.LG); Numerical Analysis (math.NA)

We introduce a novel hybrid methodology combining classical finite element methods (FEM) with neural networks to create a well-performing and generalizable surrogate model for forward and inverse problems. The residual from finite element methods and custom loss functions from neural networks are merged to form the algorithm. The Finite Element Method-enhanced Neural Network hybrid model (FEM-NN hybrid) is data-efficient and physics conforming. The proposed methodology can be used for surrogate models in real-time simulation, uncertainty quantification, and optimization in the case of forward problems. It can be used for updating the models in the case of inverse problems. The method is demonstrated with examples, and the accuracy of the results and performance is compared against the conventional way of network training and the classical finite element method. An application of the forward-solving algorithm is demonstrated for the uncertainty quantification of wind effects on a high-rise buildings. The inverse algorithm is demonstrated in the speed-dependent bearing coefficient identification of fluid bearings. The hybrid methodology of this kind will serve as a paradigm shift in the simulation methods currently used.

[172]  arXiv:2205.08324 [pdf, other]
Title: Unified Interactive Image Matting
Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)

Recent image matting studies are developing towards proposing trimap-free or interactive methods for complete complex image matting tasks. Although avoiding the extensive labors of trimap annotation, existing methods still suffer from two limitations: (1) For the single image with multiple objects, it is essential to provide extra interaction information to help determining the matting target; (2) For transparent objects, the accurate regression of alpha matte from RGB image is much more difficult compared with the opaque ones. In this work, we propose a Unified Interactive image Matting method, named UIM, which solves the limitations and achieves satisfying matting results for any scenario. Specifically, UIM leverages multiple types of user interaction to avoid the ambiguity of multiple matting targets, and we compare the pros and cons of different annotation types in detail. To unify the matting performance for transparent and opaque objects, we decouple image matting into two stages, i.e., foreground segmentation and transparency prediction. Moreover, we design a multi-scale attentive fusion module to alleviate the vagueness in the boundary region. Experimental results demonstrate that UIM achieves state-of-the-art performance on the Composition-1K test set and a synthetic unified dataset. Our code and models will be released soon.

[173]  arXiv:2205.08325 [pdf, other]
Title: GraphMapper: Efficient Visual Navigation by Scene Graph Generation
Comments: ICPR 2022
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Understanding the geometric relationships between objects in a scene is a core capability in enabling both humans and autonomous agents to navigate in new environments. A sparse, unified representation of the scene topology will allow agents to act efficiently to move through their environment, communicate the environment state with others, and utilize the representation for diverse downstream tasks. To this end, we propose a method to train an autonomous agent to learn to accumulate a 3D scene graph representation of its environment by simultaneously learning to navigate through said environment. We demonstrate that our approach, GraphMapper, enables the learning of effective navigation policies through fewer interactions with the environment than vision-based systems alone. Further, we show that GraphMapper can act as a modular scene encoder to operate alongside existing Learning-based solutions to not only increase navigational efficiency but also generate intermediate scene representations that are useful for other future tasks.

[174]  arXiv:2205.08329 [pdf, other]
Title: Fronthaul Compression Control for shared Fronthaul Access Networks
Comments: paper to appear in IEEE Communications Magazine
Subjects: Networking and Internet Architecture (cs.NI)

There is a widely held belief that future Radio Access Network (RAN) architectures will be characterized by increased levels of virtualization, whereby base station functionalities, traditionally residing at a single location, will be scattered across different logical entities while being interfaced via high-speed fronthaul (FH) links. For the deployment of such FH links, operators are faced with the challenge of maintaining acceptable radio access performance while at the same time keeping deployment costs low. A common practice is to exploit statistical multiplexing by allowing several cells to utilize the same FH link. As a result, in order to cope with the resulting aggregated traffic, different techniques can be used to reduce the required FH data rates. Herein, we focus on FH compression control strategies for multiple-cell/multiple-user scenarios sharing a common FH link. We propose various methods for sounding reference signal (SRS) handling and analyze different FH-aware modulation data compression and scheduling strategies. Considering a full system setup, including the radio and FH access networks, numerical evaluation is conducted using a 5G NR system-level simulator implemented in ns-3. Simulation results show that, under stringent FH capacity constraints, optimized modulation compression strategies provide significant user-perceived throughput gains over baseline strategies (between 5.2x and 6.9x). On top of them, SRS handling methods achieve additional 2% to 41% gains.

[175]  arXiv:2205.08330 [pdf, other]
Title: Nonlinear Model Identification and Observer Design for Thrust Estimation of Small-scale Turbojet Engines
Comments: 6+1 pages
Subjects: Robotics (cs.RO); Systems and Control (eess.SY)

Jet-powered vertical takeoff and landing (VTOL) drones require precise thrust estimation to ensure adequate stability margins and robust maneuvering. Small-scale turbojets have become good candidates for powering heavy aerial drones. However, due to limited instrumentation available in these turbojets, estimating the precise thrust using classical techniques is not straightforward. In this paper, we present a methodology to accurately estimate the online thrust for the small-scale turbojets used on the iRonCub - an aerial humanoid robot. We use a grey-box method to capture the turbojet system dynamics with a nonlinear state-space model based on the data acquired from a custom engine test bench. This model is then used to design an extended Kalman filter that estimates the turbojet thrust only from the angular speed measurements. We exploited the parameter estimation algorithm to ensure that the EKF gives smooth and accurate estimates even at engine failures. The designed EKF was validated on the test bench where the mean absolute error in estimated thrust was found to be within 2% of rated peak thrust.

[176]  arXiv:2205.08332 [pdf, other]
Title: Scalable algorithms for physics-informed neural and graph networks
Comments: 26 pages, 13 figures. arXiv admin note: text overlap with arXiv:2104.10013
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC); Analysis of PDEs (math.AP); Dynamical Systems (math.DS)

Physics-informed machine learning (PIML) has emerged as a promising new approach for simulating complex physical and biological systems that are governed by complex multiscale processes for which some data are also available. In some instances, the objective is to discover part of the hidden physics from the available data, and PIML has been shown to be particularly effective for such problems for which conventional methods may fail. Unlike commercial machine learning where training of deep neural networks requires big data, in PIML big data are not available. Instead, we can train such networks from additional information obtained by employing the physical laws and evaluating them at random points in the space-time domain. Such physics-informed machine learning integrates multimodality and multifidelity data with mathematical models, and implements them using neural networks or graph networks. Here, we review some of the prevailing trends in embedding physics into machine learning, using physics-informed neural networks (PINNs) based primarily on feed-forward neural networks and automatic differentiation. For more complex systems or systems of systems and unstructured data, graph neural networks (GNNs) present some distinct advantages, and here we review how physics-informed learning can be accomplished with GNNs based on graph exterior calculus to construct differential operators; we refer to these architectures as physics-informed graph networks (PIGNs). We present representative examples for both forward and inverse problems and discuss what advances are needed to scale up PINNs, PIGNs and more broadly GNNs for large-scale engineering problems.

[177]  arXiv:2205.08335 [pdf, other]
Title: Explanation-Guided Fairness Testing through Genetic Algorithm
Subjects: Neural and Evolutionary Computing (cs.NE); Machine Learning (cs.LG)

The fairness characteristic is a critical attribute of trusted AI systems. A plethora of research has proposed diverse methods for individual fairness testing. However, they are suffering from three major limitations, i.e., low efficiency, low effectiveness, and model-specificity. This work proposes ExpGA, an explanationguided fairness testing approach through a genetic algorithm (GA). ExpGA employs the explanation results generated by interpretable methods to collect high-quality initial seeds, which are prone to derive discriminatory samples by slightly modifying feature values. ExpGA then adopts GA to search discriminatory sample candidates by optimizing a fitness value. Benefiting from this combination of explanation results and GA, ExpGA is both efficient and effective to detect discriminatory individuals. Moreover, ExpGA only requires prediction probabilities of the tested model, resulting in a better generalization capability to various models. Experiments on multiple real-world benchmarks, including tabular and text datasets, show that ExpGA presents higher efficiency and effectiveness than four state-of-the-art approaches.

[178]  arXiv:2205.08336 [pdf]
Title: On Uncertainty of Dynamic Systems via State Aggregation Coarse-Graining and State Decomposition Fine-Graining Ways
Comments: 13 pages are for the paper, 16 pages for supplemental material
Subjects: Information Theory (cs.IT)

Uncertainty is an important feature of dynamic systems, and entropy has been widely used to measure this attribute. In this Letter, we prove that state aggregation and decomposition can decrease and increase the entropy, respectively, of dynamic systems. More than 20 popular entropies in the literature are summarized and analyzed, and it is noted that none of them breaks this property. Finally, pertinent proofs are given for four cases.

[179]  arXiv:2205.08343 [pdf, other]
Title: Moving Stuff Around: A study on efficiency of moving documents into memory for Neural IR models
Comments: 7 pages, 2 figures. Submitted to the ReNeuIR workshop at SIGIR 2022
Subjects: Information Retrieval (cs.IR)

When training neural rankers using Large Language Models, it's expected that a practitioner would make use of multiple GPUs to accelerate the training time. By using more devices, deep learning frameworks, like PyTorch, allow the user to drastically increase the available VRAM pool, making larger batches possible when training, therefore shrinking training time. At the same time, one of the most critical processes, that is generally overlooked when running data-hungry models, is how data is managed between disk, main memory and VRAM. Most open source research implementations overlook this memory hierarchy, and instead resort to loading all documents from disk to main memory and then allowing the framework (e.g., PyTorch) to handle moving data into VRAM. Therefore, with the increasing sizes of datasets dedicated to IR research, a natural question arises: s this the optimal solution for optimizing training time? We here study how three different popular approaches to handling documents for IR datasets behave and how they scale with multiple GPUs. Namely, loading documents directly into memory, reading documents directly from text files with a lookup table and using a library for handling IR datasets (ir_datasets) differ, both in performance (i.e. samples processed per second) and memory footprint. We show that, when using the most popular libraries for neural ranker research (i.e. PyTorch and Hugging Face's Transformers), the practice of loading all documents into main memory is not always the fastest option and is not feasible for setups with more than a couple GPUs. Meanwhile, a good implementation of data streaming from disk can be faster, while being considerably more scalable. We also show how popular techniques for improving loading times, like memory pining, multiple workers, and RAMDISK usage, can reduce the training time further with minor memory overhead.

[180]  arXiv:2205.08345 [pdf, other]
Title: A compartmental model for cyber-epidemics
Comments: 6 pages, 5 figures
Subjects: Cryptography and Security (cs.CR)

In our more and more interconnected world, a specific risk is that of a cyber-epidemic (or cyber-pandemic), produced either accidentally or intentionally, where a cyber virus propagates from device to device up to undermining the global Internet system with devastating consequences in terms of economic costs and societal harms related to the shutdown of essential services. We introduce a compartmental model for studying the spreading of a malware and of the awareness of its incidence through different waves which are evolving on top of the same graph structure (the global network of connected devices). This is realized by considering vectorial compartments made of two components, the first being descriptive of the state of the device with respect to the new malware's propagation, and the second accounting for the awareness of the device's user about the presence of the cyber threat. By introducing suitable transition rates between such compartments, one can then follow the evolution of a cyber-epidemic from the moment at which a new virus is seeded in the network, up to when a given user realizes that his/her device has suffered a damage and consequently starts a wave of awareness which eventually ends up with the development of a proper antivirus software. We then compare the overall damage that a malware is able to produce in Erd\H{o}s-R\'enyi and scale-free network architectures for both the case in which the virus is causing a fixed damage on each device and the case where, instead, the virus is engineered to mutate while replicating from device to device. Our result constitute actually the attempt to build a specific compartmental model whose variables and parameters are entirely customized for describing cyber-epidemics.

[181]  arXiv:2205.08347 [pdf, other]
Title: Landing AI on Networks: An equipment vendor viewpoint on Autonomous Driving Networks
Subjects: Networking and Internet Architecture (cs.NI); Artificial Intelligence (cs.AI)

The tremendous achievements of Artificial Intelligence (AI) in computer vision, natural language processing, games and robotics, has extended the reach of the AI hype to other fields: in telecommunication networks, the long term vision is to let AI fully manage, and autonomously drive, all aspects of network operation. In this industry vision paper, we discuss challenges and opportunities of Autonomous Driving Network (ADN) driven by AI technologies. To understand how AI can be successfully landed in current and future networks, we start by outlining challenges that are specific to the networking domain, putting them in perspective with advances that AI has achieved in other fields. We then present a system view, clarifying how AI can be fitted in the network architecture. We finally discuss current achievements as well as future promises of AI in networks, mentioning a roadmap to avoid bumps in the road that leads to true large-scale deployment of AI technologies in networks.

[182]  arXiv:2205.08350 [pdf, other]
Title: RISCLESS: A Reinforcement Learning Strategy to Exploit Unused Cloud Resources
Authors: Sidahmed Yalles (UR1, IRISA-D4), Mohamed Handaoui (Hypermedia, UR1, IRISA-D4), Jean-Emile Dartois (IRT b-com, DiverSe, UR1, IRISA-D4), Olivier Barais (UR1, IRISA-D4), Laurent d'Orazio, Jalil Boukhobza (ENSTA Bretagne, Lab-STICC\_SHAKER)
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI)

One of the main objectives of Cloud Providers (CP) is to guarantee the Service-Level Agreement (SLA) of customers while reducing operating costs. To achieve this goal, CPs have built large-scale datacenters. This leads, however, to underutilized resources and an increase in costs. A way to improve the utilization of resources is to reclaim the unused parts and resell them at a lower price. Providing SLA guarantees to customers on reclaimed resources is a challenge due to their high volatility. Some state-of-the-art solutions consider keeping a proportion of resources free to absorb sudden variation in workloads. Others consider stable resources on top of the volatile ones to fill in for the lost resources. However, these strategies either reduce the amount of reclaimable resources or operate on less volatile ones such as Amazon Spot instance. In this paper, we proposed RISCLESS, a Reinforcement Learning strategy to exploit unused Cloud resources. Our approach consists of using a small proportion of stable on-demand resources alongside the ephemeral ones in order to guarantee customers SLA and reduce the overall costs. The approach decides when and how much stable resources to allocate in order to fulfill customers' demands. RISCLESS improved the CPs' profits by an average of 15.9% compared to state-of-the-art strategies. It also reduced the SLA violation time by an average of 36.7% while increasing the amount of used ephemeral resources by 19.5% on average

[183]  arXiv:2205.08354 [pdf, other]
Title: Constructive Analysis in the Agda Proof Assistant
Authors: Zachary Murray
Comments: 39 pages, undergraduate thesis of Zachary Murray at Dalhousie University
Subjects: Logic in Computer Science (cs.LO)

Proof assistant software has recently been used to verify proofs of major theorems, yet even the libraries of some of the most prominent proof assistants lack much of undergraduate mathematics. In particular, the Agda proof assistant has no formalization of the real numbers and their arithmetic. In this thesis, I present my implementation of Errett Bishop's constructive real numbers in Agda, including their arithmetic, ordering, and fundamental results, such as uncountability and Cauchy completeness. We will also survey the basic concepts of constructive analysis and the Agda proof assistant.

[184]  arXiv:2205.08355 [pdf, other]
Title: Demystifying the Data Need of ML-surrogates for CFD Simulations
Comments: Published on AI2ASE AAAI2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Computational fluid dynamics (CFD) simulations, a critical tool in various engineering applications, often require significant time and compute power to predict flow properties. The high computational cost associated with CFD simulations significantly restricts the scope of design space exploration and limits their use in planning and operational control. To address this issue, machine learning (ML) based surrogate models have been proposed as a computationally efficient tool to accelerate CFD simulations. However, a lack of clarity about CFD data requirements often challenges the widespread adoption of ML-based surrogates among design engineers and CFD practitioners. In this work, we propose an ML-based surrogate model to predict the temperature distribution inside the cabin of a passenger vehicle under various operating conditions and use it to demonstrate the trade-off between prediction performance and training dataset size. Our results show that the prediction accuracy is high and stable even when the training size is gradually reduced from 2000 to 200. The ML-based surrogates also reduce the compute time from ~30 minutes to around ~9 milliseconds. Moreover, even when only 50 CFD simulations are used for training, the temperature trend (e.g., locations of hot/cold regions) predicted by the ML-surrogate matches quite well with the results from CFD simulations.

[185]  arXiv:2205.08356 [pdf, other]
Title: DouFu: A Double Fusion Joint Learning Method For Driving Trajectory Representation
Comments: 11 pages, 7 figures
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)

Driving trajectory representation learning is of great significance for various location-based services, such as driving pattern mining and route recommendation. However, previous representation generation approaches tend to rarely address three challenges: 1) how to represent the intricate semantic intentions of mobility inexpensively; 2) complex and weak spatial-temporal dependencies due to the sparsity and heterogeneity of the trajectory data; 3) route selection preferences and their correlation to driving behavior. In this paper, we propose a novel multimodal fusion model, DouFu, for trajectory representation joint learning, which applies multimodal learning and attention fusion module to capture the internal characteristics of trajectories. We first design movement, route, and global features generated from the trajectory data and urban functional zones and then analyze them respectively with the attention encoder or feed forward network. The attention fusion module incorporates route features with movement features to create a better spatial-temporal embedding. With the global semantic feature, DouFu produces a comprehensive embedding for each trajectory. We evaluate representations generated by our method and other baseline models on classification and clustering tasks. Empirical results show that DouFu outperforms other models in most of the learning algorithms like the linear regression and the support vector machine by more than 10%.

[186]  arXiv:2205.08357 [pdf, ps, other]
Title: Minimum Tournaments with the Strong $S_k$-Property and Implications for Teaching
Comments: 9 pages, 0 figures
Subjects: Discrete Mathematics (cs.DM)

A tournament is said to have the $S_k$-property if, for any set of $k$ players, there is another player who beats them all. Minimum tournaments having this property have been explored very well in the 1960's and the early 1970's. In this paper, we define a strengthening of the $S_k$-property that we name "strong $S_k$-property". We show, first, that several basic results on the weaker notion remain valid for the stronger notion (and the corresponding modification of the proofs requires only little extra-effort). Second, it is demonstrated that the stronger notion has applications in the area of Teaching. Specifically, we present an infinite family of concept classes all of which can be taught with a single example in the No-Clash model of teaching while, in order to teach a class $\cC$ of this family in the recursive model of teaching, order of $\log|\cC|$ many examples are required. This is the first paper that presents a concrete and easily constructible family of concept classes which separates the No-Clash from the recursive model of teaching by more than a constant factor. The separation by a logarithmic factor is remarkable because the recursive teaching dimension is known to be bounded by $\log |\cC|$ for any concept class $\cC$.

[187]  arXiv:2205.08358 [pdf, other]
Title: Perturbation of Deep Autoencoder Weights for Model Compression and Classification of Tabular Data
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)

Fully connected deep neural networks (DNN) often include redundant weights leading to overfitting and high memory requirements. Additionally, the performance of DNN is often challenged by traditional machine learning models in tabular data classification. In this paper, we propose periodical perturbations (prune and regrow) of DNN weights, especially at the self-supervised pre-training stage of deep autoencoders. The proposed weight perturbation strategy outperforms dropout learning in four out of six tabular data sets in downstream classification tasks. The L1 or L2 regularization of weights at the same pretraining stage results in inferior classification performance compared to dropout or our weight perturbation routine. Unlike dropout learning, the proposed weight perturbation routine additionally achieves 15% to 40% sparsity across six tabular data sets for the compression of deep pretrained models. Our experiments reveal that a pretrained deep autoencoder with weight perturbation or dropout can outperform traditional machine learning in tabular data classification when fully connected DNN fails miserably. However, traditional machine learning models appear superior to any deep models when a tabular data set contains uncorrelated variables. Therefore, the success of deep models can be attributed to the inevitable presence of correlated variables in real-world data sets.

[188]  arXiv:2205.08362 [pdf, ps, other]
Title: LPC-AD: Fast and Accurate Multivariate Time Series Anomaly Detection via Latent Predictive Coding
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

This paper proposes LPC-AD, a fast and accurate multivariate time series (MTS) anomaly detection method. LPC-AD is motivated by the ever-increasing needs for fast and accurate MTS anomaly detection methods to support fast troubleshooting in cloud computing, micro-service systems, etc. LPC-AD is fast in the sense that its reduces the training time by as high as 38.2% compared to the state-of-the-art (SOTA) deep learning methods that focus on training speed. LPC-AD is accurate in the sense that it improves the detection accuracy by as high as 18.9% compared to SOTA sophisticated deep learning methods that focus on enhancing detection accuracy. Methodologically, LPC-AD contributes a generic architecture LPC-Reconstruct for one to attain different trade-offs between training speed and detection accuracy. More specifically, LPC-Reconstruct is built on ideas from autoencoder for reducing redundancy in time series, latent predictive coding for capturing temporal dependence in MTS, and randomized perturbation for avoiding overfitting of anomalous dependence in the training data. We present simple instantiations of LPC-Reconstruct to attain fast training speed, where we propose a simple randomized perturbation method. The superior performance of LPC-AD over SOTA methods is validated by extensive experiments on four large real-world datasets. Experiment results also show the necessity and benefit of each component of the LPC-Reconstruct architecture and that LPC-AD is robust to hyper parameters.

[189]  arXiv:2205.08363 [pdf, other]
Title: REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research
Comments: This work appears in the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22)
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)

Transparency around limitations can improve the scientific rigor of research, help ensure appropriate interpretation of research findings, and make research claims more credible. Despite these benefits, the machine learning (ML) research community lacks well-developed norms around disclosing and discussing limitations. To address this gap, we conduct an iterative design process with 30 ML and ML-adjacent researchers to develop and test REAL ML, a set of guided activities to help ML researchers recognize, explore, and articulate the limitations of their research. Using a three-stage interview and survey study, we identify ML researchers' perceptions of limitations, as well as the challenges they face when recognizing, exploring, and articulating limitations. We develop REAL ML to address some of these practical challenges, and highlight additional cultural challenges that will require broader shifts in community norms to address. We hope our study and REAL ML help move the ML research community toward more active and appropriate engagement with limitations.

[190]  arXiv:2205.08364 [pdf, other]
Title: Network Gradient Descent Algorithm for Decentralized Federated Learning
Subjects: Machine Learning (cs.LG); Methodology (stat.ME); Machine Learning (stat.ML)

We study a fully decentralized federated learning algorithm, which is a novel gradient descent algorithm executed on a communication-based network. For convenience, we refer to it as a network gradient descent (NGD) method. In the NGD method, only statistics (e.g., parameter estimates) need to be communicated, minimizing the risk of privacy. Meanwhile, different clients communicate with each other directly according to a carefully designed network structure without a central master. This greatly enhances the reliability of the entire algorithm. Those nice properties inspire us to carefully study the NGD method both theoretically and numerically. Theoretically, we start with a classical linear regression model. We find that both the learning rate and the network structure play significant roles in determining the NGD estimator's statistical efficiency. The resulting NGD estimator can be statistically as efficient as the global estimator, if the learning rate is sufficiently small and the network structure is well balanced, even if the data are distributed heterogeneously. Those interesting findings are then extended to general models and loss functions. Extensive numerical studies are presented to corroborate our theoretical findings. Classical deep learning models are also presented for illustration purpose.

[191]  arXiv:2205.08365 [pdf, other]
Title: Deep Supervised Information Bottleneck Hashing for Cross-modal Retrieval based Computer-aided Diagnosis
Comments: 7 pages, 1 figure
Journal-ref: The AAAI-22 Workshop on Information Theory for Deep Learning (IT4DL).2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Image and Video Processing (eess.IV)

Mapping X-ray images, radiology reports, and other medical data as binary codes in the common space, which can assist clinicians to retrieve pathology-related data from heterogeneous modalities (i.e., hashing-based cross-modal medical data retrieval), provides a new view to promot computeraided diagnosis. Nevertheless, there remains a barrier to boost medical retrieval accuracy: how to reveal the ambiguous semantics of medical data without the distraction of superfluous information. To circumvent this drawback, we propose Deep Supervised Information Bottleneck Hashing (DSIBH), which effectively strengthens the discriminability of hash codes. Specifically, the Deep Deterministic Information Bottleneck (Yu, Yu, and Principe 2021) for single modality is extended to the cross-modal scenario. Benefiting from this, the superfluous information is reduced, which facilitates the discriminability of hash codes. Experimental results demonstrate the superior accuracy of the proposed DSIBH compared with state-of-the-arts in cross-modal medical data retrieval tasks.

[192]  arXiv:2205.08369 [pdf]
Title: Applications of Reinforcement Learning in Deregulated Power Market: A Comprehensive Review
Subjects: Machine Learning (cs.LG); Computer Science and Game Theory (cs.GT); Systems and Control (eess.SY)

The increasing penetration of renewable generations, along with the deregulation and marketization of power industry, promotes the transformation of power market operation paradigms. The optimal bidding strategy and dispatching methodology under these new paradigms are prioritized concerns for both market participants and power system operators, with obstacles of uncertain characteristics, computational efficiency, as well as requirements of hyperopic decision-making. To tackle these problems, the Reinforcement Learning (RL), as an emerging machine learning technique with advantages compared with conventional optimization tools, is playing an increasingly significant role in both academia and industry. This paper presents a comprehensive review of RL applications in deregulated power market operation including bidding and dispatching strategy optimization, based on more than 150 carefully selected literatures. For each application, apart from a paradigmatic summary of generalized methodology, in-depth discussions of applicability and obstacles while deploying RL techniques are also provided. Finally, some RL techniques that have great potentiality to be deployed in bidding and dispatching problems are recommended and discussed.

[193]  arXiv:2205.08370 [pdf, other]
Title: Individualized Risk Assessment of Preoperative Opioid Use by Interpretable Neural Network Regression
Comments: 14 pages, 6 tables and 2 figures in main text
Subjects: Machine Learning (cs.LG); Applications (stat.AP)

Preoperative opioid use has been reported to be associated with higher preoperative opioid demand, worse postoperative outcomes, and increased postoperative healthcare utilization and expenditures. Understanding the risk of preoperative opioid use helps establish patient-centered pain management. In the field of machine learning, deep neural network (DNN) has emerged as a powerful means for risk assessment because of its superb prediction power; however, the blackbox algorithms may make the results less interpretable than statistical models. Bridging the gap between the statistical and machine learning fields, we propose a novel Interpretable Neural Network Regression (INNER), which combines the strengths of statistical and DNN models. We use the proposed INNER to conduct individualized risk assessment of preoperative opioid use. Intensive simulations and an analysis of 34,186 patients expecting surgery in the Analgesic Outcomes Study (AOS) show that the proposed INNER not only can accurately predict the preoperative opioid use using preoperative characteristics as DNN, but also can estimate the patient specific odds of opioid use without pain and the odds ratio of opioid use for a unit increase in the reported overall body pain, leading to more straightforward interpretations of the tendency to use opioids than DNN. Our results identify the patient characteristics that are strongly associated with opioid use and is largely consistent with the previous findings, providing evidence that INNER is a useful tool for individualized risk assessment of preoperative opioid use.

[194]  arXiv:2205.08371 [pdf]
Title: Evaluation of a User Authentication Schema Using Behavioral Biometrics and Machine Learning
Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

The amount of secure data being stored on mobile devices has grown immensely in recent years. However, the security measures protecting this data have stayed static, with few improvements being done to the vulnerabilities of current authentication methods such as physiological biometrics or passwords. Instead of these methods, behavioral biometrics has recently been researched as a solution to these vulnerable authentication methods. In this study, we aim to contribute to the research being done on behavioral biometrics by creating and evaluating a user authentication scheme using behavioral biometrics. The behavioral biometrics used in this study include touch dynamics and phone movement, and we evaluate the performance of different single-modal and multi-modal combinations of the two biometrics. Using two publicly available datasets - BioIdent and Hand Movement Orientation and Grasp (H-MOG), this study uses seven common machine learning algorithms to evaluate performance. The algorithms used in the evaluation include Random Forest, Support Vector Machine, K-Nearest Neighbor, Naive Bayes, Logistic Regression, Multilayer Perceptron, and Long Short-Term Memory Recurrent Neural Networks, with accuracy rates reaching as high as 86%.

[195]  arXiv:2205.08372 [pdf, other]
Title: Automatic Velocity Picking Using Unsupervised Ensemble Learning
Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV); Signal Processing (eess.SP)

In seismic data processing, accurate and efficient automatic velocity picking algorithms can significantly accelerate the processing, and the main branch is to use velocity spectra for velocity pickup. Recently, machine learning algorithms have been widely used in automatic spectrum picking. Even though deep learning methods can address the problem well in supervised cases, they are often accompanied by expensive computational costs and low interpretability. On the contrast, unsupervised learning methods based on the physical knowledge have great potential to efficiently resolve the task. In this paper, we propose an unsupervised ensemble learning (UEL) method to pick the root mean square (RMS) velocities on the spectrum. In particular, UEL utilizes the information of nearby velocity spectra and the nearest seed velocity curve to assist the selection of effective and reasonable velocity points. To increase the coherence of energy peaks, an information gain method is developed by local normalization. In addition, we designed the attention scale-space filter (ASSF) clustering method to incorporate the coherence information into the picking process. Experiments on three datasets demonstrate that compared to traditional clustering methods, UEL can recognize energy clusters better, especially with smaller blobs. Moreover, the injection of nearby spectra and interval velocity constraint in UEL significantly improves the robustness and accuracy of picking results.

[196]  arXiv:2205.08373 [pdf, other]
Title: Compositional Modeling with Stock and Flow Diagrams
Comments: 14 pages
Subjects: Logic in Computer Science (cs.LO); Category Theory (math.CT); Populations and Evolution (q-bio.PE)

Stock and flow diagrams are widely used in epidemiology to model the dynamics of populations. Although tools already exist for building these diagrams and simulating the systems they describe, we have created a new package called StockFlow, part of the AlgebraicJulia ecosystem, which uses ideas from category theory to overcome notable limitations of existing software. Compositionality is provided by the theory of decorated cospans: stock and flow diagrams can composed to form larger ones in an intuitive way formalized by the operad of undirected wiring diagrams. Our approach also cleanly separates the syntax of stock and flow diagrams from the semantics they can be assigned. We consider semantics in ordinary differential equations, although others are possible. As an example, we explain code in StockFlow that implements a simplified version of a COVID-19 model used in Canada.

[197]  arXiv:2205.08377 [pdf, other]
Title: Should attention be all we need? The epistemic and ethical implications of unification in machine learning
Comments: Forthcoming in FAccT 2022
Subjects: Machine Learning (cs.LG)

"Attention is all you need" has become a fundamental precept in machine learning research. Originally designed for machine translation, transformers and the attention mechanisms that underpin them now find success across many problem domains. With the apparent domain-agnostic success of transformers, many researchers are excited that similar model architectures can be successfully deployed across diverse applications in vision, language and beyond. We consider the benefits and risks of these waves of unification on both epistemic and ethical fronts. On the epistemic side, we argue that many of the arguments in favor of unification in the natural sciences fail to transfer over to the machine learning case, or transfer over only under assumptions that might not hold. Unification also introduces epistemic risks related to portability, path dependency, methodological diversity, and increased black-boxing. On the ethical side, we discuss risks emerging from epistemic concerns, further marginalizing underrepresented perspectives, the centralization of power, and having fewer models across more domains of application

[198]  arXiv:2205.08378 [pdf, other]
Title: Machine learning and atomic layer deposition: predicting saturation times from reactor growth profiles using artificial neural networks
Subjects: Machine Learning (cs.LG); Materials Science (cond-mat.mtrl-sci); Applied Physics (physics.app-ph)

In this work we explore the application of deep neural networks to the optimization of atomic layer deposition processes based on thickness values obtained at different points of an ALD reactor. We introduce a dataset designed to train neural networks to predict saturation times based on the dose time and thickness values measured at different points of the reactor for a single experimental condition. We then explore different artificial neural network configurations, including depth (number of hidden layers) and size (number of neurons in each layers) to better understand the size and complexity that neural networks should have to achieve high predictive accuracy. The results obtained show that trained neural networks can accurately predict saturation times without requiring any prior information on the surface kinetics. This provides a viable approach to minimize the number of experiments required to optimize new ALD processes in a known reactor. However, the datasets and training procedure depend on the reactor geometry.

[199]  arXiv:2205.08379 [pdf, other]
Title: A CMOS-based Characterisation Platform for Emerging RRAM Technologies
Comments: 5 pages. To be published in ISCAS 2022 and made available on IEEE Xplore
Subjects: Emerging Technologies (cs.ET); Systems and Control (eess.SY)

Mass characterisation of emerging memory devices is an essential step in modelling their behaviour for integration within a standard design flow for existing integrated circuit designers. This work develops a novel characterisation platform for emerging resistive devices with a capacity of up to 1 million devices on-chip. Split into four independent sub-arrays, it contains on-chip column-parallel DACs for fast voltage programming of the DUT. On-chip readout circuits with ADCs are also available for fast read operations covering 5-decades of input current (20nA to 2mA). This allows a device's resistance range to be between 1k$\Omega$ and 10M$\Omega$ with a minimum voltage range of $\pm$1.5V on the device.

[200]  arXiv:2205.08381 [pdf, other]
Title: A Wide Dynamic Range Read-out System For Resistive Switching Technology
Comments: 5 pages, To be published in ISCAS 2022 and made available on IEEE Xplore
Subjects: Emerging Technologies (cs.ET); Systems and Control (eess.SY)

The memristor, because of its controllability over a wide dynamic range of resistance, has emerged as a promising device for data storage and analog computation. A major challenge is the accurate measurement of memristance over a wide dynamic range. In this paper, a novel read-out circuit with feedback adjustment is proposed to measure and digitise input current in the range between 20nA and 2mA. The magnitude of the input currents is estimated by a 5-stage logarithmic current-to-voltage amplifier which scales a linear analog-to-digital converter. This way the least significant bit tracks the absolute input magnitude. This circuit is applicable to reading single memristor conductance, and is also preferable in analog computing where read-out accuracy is particularly critical. The circuits have been realized in Bipolar-CMOS-DMOS (BCD) Gen2 technology.

[201]  arXiv:2205.08382 [pdf, other]
Title: Compatible deep neural network framework with financial time series data, including data preprocessor, neural network model and trading strategy
Comments: 26 pages, 17 figures, 7 tables
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE)

Experience has shown that trading in stock and cryptocurrency markets has the potential to be highly profitable. In this light, considerable effort has been recently devoted to investigate how to apply machine learning and deep learning to interpret and predict market behavior. This research introduces a new deep neural network architecture and a novel idea of how to prepare financial data before feeding them to the model. In the data preparation part, the first step is to generate many features using technical indicators and then apply the XGBoost model for feature engineering. Splitting data into three categories and using separate autoencoders, we extract high-level mixed features at the second step. This data preprocessing is introduced to predict price movements. Regarding modeling, different convolutional layers, an long short-term memory unit, and several fully-connected layers have been designed to perform binary classification. This research also introduces a trading strategy to exploit the trained model outputs. Three different datasets are used to evaluate this method, where results indicate that this framework can provide us with profitable and robust predictions.

[202]  arXiv:2205.08383 [pdf, other]
Title: Bias and Fairness on Multimodal Emotion Detection Algorithms
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)

Numerous studies have shown that machine learning algorithms can latch onto protected attributes such as race and gender and generate predictions that systematically discriminate against one or more groups. To date the majority of bias and fairness research has been on unimodal models. In this work, we explore the biases that exist in emotion recognition systems in relationship to the modalities utilized, and study how multimodal approaches affect system bias and fairness. We consider audio, text, and video modalities, as well as all possible multimodal combinations of those, and find that text alone has the least bias, and accounts for the majority of the models' performances, raising doubts about the worthiness of multimodal emotion recognition systems when bias and fairness are desired alongside model performance.

[203]  arXiv:2205.08384 [pdf, ps, other]
Title: Deep Learning of Chaotic Systems from Partially-Observed Data
Subjects: Machine Learning (cs.LG); Chaotic Dynamics (nlin.CD)

Recently, a general data driven numerical framework has been developed for learning and modeling of unknown dynamical systems using fully- or partially-observed data. The method utilizes deep neural networks (DNNs) to construct a model for the flow map of the unknown system. Once an accurate DNN approximation of the flow map is constructed, it can be recursively executed to serve as an effective predictive model of the unknown system. In this paper, we apply this framework to chaotic systems, in particular the well-known Lorenz 63 and 96 systems, and critically examine the predictive performance of the approach. A distinct feature of chaotic systems is that even the smallest perturbations will lead to large (albeit bounded) deviations in the solution trajectories. This makes long-term predictions of the method, or any data driven methods, questionable, as the local model accuracy will eventually degrade and lead to large pointwise errors. Here we employ several other qualitative and quantitative measures to determine whether the chaotic dynamics have been learned. These include phase plots, histograms, autocorrelation, correlation dimension, approximate entropy, and Lyapunov exponent. Using these measures, we demonstrate that the flow map based DNN learning method is capable of accurately modeling chaotic systems, even when only a subset of the state variables are available to the DNNs. For example, for the Lorenz 96 system with 40 state variables, when data of only 3 variables are available, the method is able to learn an effective DNN model for the 3 variables and produce accurately the chaotic behavior of the system.

[204]  arXiv:2205.08385 [pdf, other]
Title: Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs
Journal-ref: AAAI 2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

The optimization with orthogonality has been shown useful in training deep neural networks (DNNs). To impose orthogonality on DNNs, both computational efficiency and stability are important. However, existing methods utilizing Riemannian optimization or hard constraints can only ensure stability while those using soft constraints can only improve efficiency. In this paper, we propose a novel method, named Feedback Gradient Descent (FGD), to our knowledge, the first work showing high efficiency and stability simultaneously. FGD induces orthogonality based on the simple yet indispensable Euler discretization of a continuous-time dynamical system on the tangent bundle of the Stiefel manifold. In particular, inspired by a numerical integration method on manifolds called Feedback Integrators, we propose to instantiate it on the tangent bundle of the Stiefel manifold for the first time. In the extensive image classification experiments, FGD comprehensively outperforms the existing state-of-the-art methods in terms of accuracy, efficiency, and stability.

[205]  arXiv:2205.08389 [pdf, other]
Title: MIDGARD: A Simulation Platform for Autonomous Navigation in Unstructured Environments
Subjects: Robotics (cs.RO)

We present MIDGARD, an open source simulation platform for autonomous robot navigation in unstructured outdoor environments. We specifically design MIDGARD to enable training of autonomous agents (e.g., unmanned ground vehicles) in photorealistic 3D environments, and to support the generalization skills of learning-based agents by means of diverse and variable training scenarios. MIDGARD differs from other major simulation platforms in that it proposes a highly configurable procedural landscape generation pipeline, which enables autonomous agents to be trained in diverse scenarios while reducing the efforts and costs needed to create digital content from scratch.

[206]  arXiv:2205.08391 [pdf, other]
Title: A High-Voltage Characterisation Platform For Emerging Resistive Switching Technologies
Comments: 5 pages. To be published in ISCAS 2022 and made available on IEEEXplore
Subjects: Emerging Technologies (cs.ET); Systems and Control (eess.SY)

Emerging memristor-based array architectures have been effectively employed in non-volatile memories and neuromorphic computing systems due to their density, scalability and capability of storing information. Nonetheless, to demonstrate a practical on-chip memristor-based system, it is essential to have the ability to apply large programming voltage ranges during the characterisation procedures for various memristor technologies. This work presents a 16x16 high voltage memristor characterisation array employing high voltage CMOS circuitry. The proposed system has a maximum programming range of $\pm22V$ to allow on-chip electroforming and I-V sweep. In addition, a Kelvin voltage sensing system is implemented to improve the readout accuracy for low memristance measurements. This work addresses the limitation of conventional CMOS-memristor platforms which can only operate at low voltages, thus limiting the characterisation range and integration options of memristor technologies.

[207]  arXiv:2205.08393 [pdf, ps, other]
Title: Full Duplex Massive MIMO Architectures: Recent Advances, Applications, and Future Directions
Comments: 7 pages, 5 figure. Under review for publication in IEEE Vehicular Technology Magazine, 2022
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

The increasingly demanding objectives for next generation wireless communications have spurred recent research activities on multi-antenna transceiver hardware architectures and relevant intelligent communication schemes. Among them belong the Full Duplex (FD) Multiple-Input Multiple-Output (MIMO) architectures, which offer the potential for simultaneous uplink and downlink operations in the entire frequency band. However, as the number of antenna elements increases, the interference signal leaking from the transmitter of the FD radio to its receiver becomes more severe. In this article, we present a unified FD massive MIMO architecture comprising analog and digital transmit and receive BeamForming (BF), as well as analog and digital SI cancellation, which can be jointly optimized for various performance objectives and complexity requirements. Performance evaluation results for applications of the proposed architecture to fully digital and hybrid analog and digital BF operations using recent algorithmic designs, as well as simultaneous communication of data and control signals are presented. It is shown that the proposed architecture, for both small and large numbers of antennas, enables improved spectral efficiency FD communications with fewer analog cancellation elements compared to various benchmark schemes. The article is concluded with a list of open challenges and research directions for future FD massive MIMO communication systems and their promising applications.

[208]  arXiv:2205.08395 [pdf]
Title: Subdivisions and Crossroads: Identifying Hidden Community Structures in a Data Archive's Citation Network
Comments: 30 pages, 7 tables, 4 figures
Subjects: Digital Libraries (cs.DL); Computers and Society (cs.CY)

Data archives are an important source of high quality data in many fields, making them ideal sites to study data reuse. By studying data reuse through citation networks, we are able to learn how hidden research communities - those that use the same scientific datasets - are organized. This paper analyzes the community structure of an authoritative network of datasets cited in academic publications, which have been collected by a large, social science data archive: the Interuniversity Consortium for Political and Social Research (ICPSR). Through network analysis, we identified communities of social science datasets and fields of research connected through shared data use. We argue that communities of exclusive data reuse form subdivisions that contain valuable disciplinary resources, while datasets at a "crossroads" broadly connect research communities. Our research reveals the hidden structure of data reuse and demonstrates how interdisciplinary research communities organize around datasets as shared scientific inputs. These findings contribute new ways of describing scientific communities in order to understand the impacts of research data reuse.

[209]  arXiv:2205.08397 [pdf, other]
Title: Improved Utility Analysis of Private CountSketch
Subjects: Data Structures and Algorithms (cs.DS); Cryptography and Security (cs.CR)

Sketching is an important tool for dealing with high-dimensional vectors that are sparse (or well-approximated by a sparse vector), especially useful in distributed, parallel, and streaming settings. It is known that sketches can be made differentially private by adding noise according to the sensitivity of the sketch, and this has been used in private analytics and federated learning settings. The post-processing property of differential privacy implies that all estimates computed from the sketch can be released within the given privacy budget.
In this paper we consider the classical CountSketch, made differentially private with the Gaussian mechanism, and give an improved analysis of its estimation error. Perhaps surprisingly, the privacy-utility trade-off is essentially the best one could hope for, independent of the number of repetitions in CountSketch: The error is almost identical to the error from non-private CountSketch plus the noise needed to make the vector private in the original, high-dimensional domain.

[210]  arXiv:2205.08399 [pdf, other]
Title: How do Variational Autoencoders Learn? Insights from Representational Similarity
Comments: The main paper is 9 pages plus an appendix; 18 pages in total
Subjects: Machine Learning (cs.LG)

The ability of Variational Autoencoders (VAEs) to learn disentangled representations has made them popular for practical applications. However, their behaviour is not yet fully understood. For example, the questions of when they can provide disentangled representations, or suffer from posterior collapse are still areas of active research. Despite this, there are no layerwise comparisons of the representations learned by VAEs, which would further our understanding of these models. In this paper, we thus look into the internal behaviour of VAEs using representational similarity techniques. Specifically, using the CKA and Procrustes similarities, we found that the encoders' representations are learned long before the decoders', and this behaviour is independent of hyperparameters, learning objectives, and datasets. Moreover, the encoders' representations up to the mean and variance layers are similar across hyperparameters and learning objectives.

[211]  arXiv:2205.08402 [pdf, ps, other]
Title: Simultaneous Multi-User MIMO Communications and Multi-Target Tracking with Full Duplex Radios
Comments: 6 pages, 5 figures. Submitted for publication in the Proceedings of IEEE Global Communications Conference (GLOBECOM), 2022
Subjects: Information Theory (cs.IT); Signal Processing (eess.SP)

In this paper, we present an Integrated Sensing and Communications (ISAC) system enabled by in-band Full Duplex (FD) radios, where a massive Multiple-Input Multiple-Output (MIMO) base station equipped with hybrid Analog and Digital (A/D) beamformers is communicating with multiple DownLink (DL) users, and simultaneously estimates via the same signaling waveforms the Direction of Arrival (DoA) as well as the range of radar targets randomly distributed within its coverage area. Capitalizing on a recent reduced-complexity FD hybrid A/D beamforming architecture, we devise a joint radar target tracking and DL data transmission protocol. An optimization framework for the joint design of the massive A/D beamformers and the Self-Interference (SI) cancellation unit, with the dual objective of maximizing the radar tracking accuracy and DL communication performance, is presented. Our simulation results at millimeter wave frequencies using 5G NR wideband waveforms, showcase the accuracy of the radar target tracking performance of the proposed system, which simultaneously offers increased sum rate compared with benchmark schemes.

[212]  arXiv:2205.08404 [pdf, other]
Title: A Comprehensive Study on Artificial Intelligence Algorithms to Implement Safety Using Communication Technologies
Subjects: Artificial Intelligence (cs.AI); Networking and Internet Architecture (cs.NI)

The recent development of artificial intelligence (AI) has increased the interest of researchers and practitioners towards applying its techniques into multiple domains like automotive, health care and air space to achieve automation. Combined to these applications, the attempt to use AI techniques into carrying out safety issues is momentarily at a progressive state. As AI problems are getting even more complex, large processing power is demanded for safety-critical systems to fulfill real-time requirements. These challenges can be solved through edge or cloud computing, which makes the communication an integral part of the solution. This study aims at providing a comprehensive picture of the state of the art AI based safety solutions that uses different communication technologies in diverse application domains. To achieve this, a systematic mapping study is conducted and 565 relevant papers are shortlisted through a multistage selection process, which are then analyzed according to a systematically defined classification framework. The results of the study are based on these main objectives: to clarify current research gaps in the field, to identify the possibility of increased usage of cellular communication in multiple domains, to identify the mostly used AI algorithms and to summarize the emerging future research trends on the topic. The results demonstrate that automotive domain is the one applying AI and communication the most to implement safety and the most used AI in this domain is neural networks, clustering and computer vision; applying cellular communication to automotive domain is highest; the use of non-cellular communication technologies is dominant however a clear trend of a rapid increase in the use of cellular communication is observed specially from 2020 with the roll-out of 5G technology.

[213]  arXiv:2205.08406 [pdf, other]
Title: Object Detection and Heading Forecasting by fusing Raw Radar Data using Cross Attention
Subjects: Information Theory (cs.IT)

Radar has been believed to be an inevitable sensor for advanced driver assistance systems (ADAS) for decades. Along with providing robust range, angle and velocity measurements, it is also cost-effective. Hence, radar is expected to play a big role in the next generation ADAS. In this paper, we propose a neural network for object detection and heading forecasting based on radar by fusing three raw radar channels with a cross-attention mechanism. We also introduce an improved ground truth augmentation method based on Bivariate norm, which represents the object labels in a more realistic form for radar measurements. Our results show 5% better mAP compared to state-of-the-art methods. To the best of our knowledge, this is the first attempt in the radar field, where cross-attention is utilized for object detection and heading forecasting without the use of object tracking and association.

[214]  arXiv:2205.08407 [pdf, ps, other]
Title: Decentralised Update Selection with Semi-Strategic Experts
Subjects: Computer Science and Game Theory (cs.GT)

Motivated by governance models adopted in blockchain applications, we study the problem of selecting appropriate system updates in a decentralised way. Contrary to most existing voting approaches, we use the input of a set of motivated experts of varying levels of expertise. In particular, we develop an approval voting inspired selection mechanism through which the experts approve or disapprove the different updates according to their perception of the quality of each alternative. Given their opinions, and weighted by their expertise level, a single update is then implemented and evaluated, and the experts receive rewards based on their choices. We show that this mechanism always has approximate pure Nash equilibria and that these achieve a constant factor approximation with respect to the quality benchmark of the optimal alternative. Finally, we study the repeated version of the problem, where the weights of the experts are adjusted after each update, according to their performance. Under mild assumptions about the weights, the extension of our mechanism still has approximate pure Nash equilibria in this setting.

[215]  arXiv:2205.08412 [pdf, other]
Title: From mean-field to complex topologies: network effects on the algorithmic bias model
Comments: 11 pages, 4 figures, Complex Networks & Their Applications X
Subjects: Social and Information Networks (cs.SI); Physics and Society (physics.soc-ph)

Nowadays, we live in a society where people often form their opinion by accessing and discussing contents shared on social networking websites. While these platforms have fostered information access and diffusion, they represent optimal environments for the proliferation of polluted contents, which is argued to be one of the co-causes of polarization/radicalization. Moreover, recommendation algorithms - intended to enhance platform usage - are likely to augment such phenomena, generating the so called Algorithmic Bias. In this work, we study the impact that different network topologies have on the formation and evolution of opinion in the context of a recent opinion dynamic model which includes bounded confidence and algorithmic bias. Mean-field, scale-free and random topologies, as well as networks generated by the Lancichinetti-Fortunato-Radicchi benchmark, are compared in terms of opinion fragmentation/polarization and time to convergence.

[216]  arXiv:2205.08416 [pdf, other]
Title: Semi-Supervised Building Footprint Generation with Feature and Output Consistency Training
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Accurate and reliable building footprint maps are vital to urban planning and monitoring, and most existing approaches fall back on convolutional neural networks (CNNs) for building footprint generation. However, one limitation of these methods is that they require strong supervisory information from massive annotated samples for network learning. State-of-the-art semi-supervised semantic segmentation networks with consistency training can help to deal with this issue by leveraging a large amount of unlabeled data, which encourages the consistency of model output on data perturbation. Considering that rich information is also encoded in feature maps, we propose to integrate the consistency of both features and outputs in the end-to-end network training of unlabeled samples, enabling to impose additional constraints. Prior semi-supervised semantic segmentation networks have established the cluster assumption, in which the decision boundary should lie in the vicinity of low sample density. In this work, we observe that for building footprint generation, the low-density regions are more apparent at the intermediate feature representations within the encoder than the encoder's input or output. Therefore, we propose an instruction to assign the perturbation to the intermediate feature representations within the encoder, which considers the spatial resolution of input remote sensing imagery and the mean size of individual buildings in the study area. The proposed method is evaluated on three datasets with different resolutions: Planet dataset (3 m/pixel), Massachusetts dataset (1 m/pixel), and Inria dataset (0.3 m/pixel). Experimental results show that the proposed approach can well extract more complete building structures and alleviate omission errors.

[217]  arXiv:2205.08425 [pdf, other]
Title: Twenty-two years since revealing cross-site scripting attacks: a systematic mapping and a comprehensive survey
Subjects: Cryptography and Security (cs.CR)

Cross-site scripting (XSS) is one of the major threats menacing the privacy of data and the navigation of trusted web applications. Since its reveal in late 1999 by Microsoft security engineers, several techniques have been developed in the aim to secure web navigation and protect web applications against XSS attacks. The problem became worse with the emergence of advanced web technologies such as Web services and APIs and new programming styles such as AJAX, CSS3 and HTML5. While new technologies enable complex interactions and data exchanges between clients and servers in the network, new programming styles introduce new and complicate injection flaws to web applications. XSS has been and still in the TOP 10 list of web vulnerabilities reported by the Open Web Applications Security Project (OWASP). Consequently, handling XSS attacks became one of the major concerns of several web security communities. In this paper, we contribute by conducting a systematic mapping and a comprehensive survey. We summarize and categorize existent endeavors that aim to protect against XSS attacks and develop XSS-free web applications. The present review covers 147 high quality published studies since 1999 including early publications of 2022. A comprehensive taxonomy is drawn out describing the different techniques used to prevent, detect, protect and defend against XSS attacks. Although the diversity of XSS attack types and the scripting languages that can be used to state them, the systematic mapping revealed a remarkable bias toward basic and JavaScript XSS attacks and a dearth of vulnerability repair mechanisms. The survey highlighted the limitations, discussed the potentials of existing XSS attack defense mechanisms and identified potential gaps.

[218]  arXiv:2205.08426 [pdf, other]
Title: Can You Still See Me?: Reconstructing Robot Operations Over End-to-End Encrypted Channels
Comments: 13 pages, 7 figures, poster presented at wisec'22
Subjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG); Robotics (cs.RO)

Connected robots play a key role in Industry 4.0, providing automation and higher efficiency for many industrial workflows. Unfortunately, these robots can leak sensitive information regarding these operational workflows to remote adversaries. While there exists mandates for the use of end-to-end encryption for data transmission in such settings, it is entirely possible for passive adversaries to fingerprint and reconstruct entire workflows being carried out -- establishing an understanding of how facilities operate. In this paper, we investigate whether a remote attacker can accurately fingerprint robot movements and ultimately reconstruct operational workflows. Using a neural network approach to traffic analysis, we find that one can predict TLS-encrypted movements with around \textasciitilde60\% accuracy, increasing to near-perfect accuracy under realistic network conditions. Further, we also find that attackers can reconstruct warehousing workflows with similar success. Ultimately, simply adopting best cybersecurity practices is clearly not enough to stop even weak (passive) adversaries.

[219]  arXiv:2205.08430 [pdf, other]
Title: Towards Resilient Access Equality for 6G Serverless p-LEO Satellite Networks
Comments: Submitted for possible publication to the IEEE Communication magazine
Subjects: Networking and Internet Architecture (cs.NI)

Low earth orbit (LEO) mega-constellations, integrating government space systems and commercial practices, have emerged as enabling technologies for the sixth generation (6G) networks due to their good merits of global coverage and ubiquitous services for military and civilian use cases. However, convergent LEO-based satellite networking infrastructures still lack leveraging the synergy of space and terrestrial systems. This paper, therefore, extends conventional serverless cloud platforms with serverless edge learning architectures for 6G proliferated LEO (p-LEO) satellite ecosystems and provides a new distributed training design from a networking perspective. The proposed design dynamically orchestrates communications and computation functionalities and resources among heterogeneous physical units to efficiently fulfill multi-agent deep reinforcement learning for service-level agreements. Innovative ecosystem enhancements, including ultrabroadband access, anti-jammed transmissions, resilient networking, and related open challenges, are also investigated for end-to-end connectivity, communications, and learning performance.

[220]  arXiv:2205.08434 [pdf, other]
Title: DNNR: Differential Nearest Neighbors Regression
Comments: published at ICML 2022
Subjects: Machine Learning (cs.LG)

K-nearest neighbors (KNN) is one of the earliest and most established algorithms in machine learning. For regression tasks, KNN averages the targets within a neighborhood which poses a number of challenges: the neighborhood definition is crucial for the predictive performance as neighbors might be selected based on uninformative features, and averaging does not account for how the function changes locally. We propose a novel method called Differential Nearest Neighbors Regression (DNNR) that addresses both issues simultaneously: during training, DNNR estimates local gradients to scale the features; during inference, it performs an n-th order Taylor approximation using estimated gradients. In a large-scale evaluation on over 250 datasets, we find that DNNR performs comparably to state-of-the-art gradient boosting methods and MLPs while maintaining the simplicity and transparency of KNN. This allows us to derive theoretical error bounds and inspect failures. In times that call for transparency of ML models, DNNR provides a good balance between performance and interpretability.

[221]  arXiv:2205.08438 [pdf]
Title: An Application of a Multivariate Estimation of Distribution Algorithm to Cancer Chemotherapy
Comments: Tech report, originally published at Missouri EDA Lab, in support of extended abstract (poster) with same title presented at GECCO 2008
Subjects: Artificial Intelligence (cs.AI); Quantitative Methods (q-bio.QM)

Chemotherapy treatment for cancer is a complex optimisation problem with a large number of interacting variables and constraints. A number of different probabilistic algorithms have been applied to it with varying success. In this paper we expand on this by applying two estimation of distribution algorithms to the problem. One is UMDA, which uses a univariate probabilistic model similar to previously applied EDAs. The other is hBOA, the first EDA using a multivariate probabilistic model to be applied to the chemotherapy problem. While instinct would lead us to predict that the more sophisticated algorithm would yield better performance on a complex problem like this, we show that it is outperformed by the algorithms using the simpler univariate model. We hypothesise that this is caused by the more sophisticated algorithm being impeded by the large number of interactions in the problem which are unnecessary for its solution.

[222]  arXiv:2205.08440 [pdf, other]
Title: Moving Smart Contracts -- A Privacy Preserving Method for Off-Chain Data Trust
Comments: 10 pages, 6 figures
Subjects: Cryptography and Security (cs.CR); Distributed, Parallel, and Cluster Computing (cs.DC); Software Engineering (cs.SE)

Blockchains provide environments where parties can interact transparently and securely peer-to-peer without needing a trusted third party. Parties can trust the integrity and correctness of transactions and the verifiable execution of binary code on the blockchain (smart contracts) inside the system. Including information from outside of the blockchain remains challenging. A challenge is data privacy. In a public system, shared data becomes public and, coming from a single source, often lacks credibility. A private system gives the parties control over their data and sources but trades in positive aspects as transparency. Often, not the data itself is the most critical information but the result of a computation performed on it.
An example is research data certification. To keep data private but still prove data provenance, researchers can store a hash value of that data on the blockchain. This hash value is either calculated locally on private data without the chance for validation or is calculated on the blockchain, meaning that data must be published and stored on the blockchain -- a problem of the overall data amount stored on and distributed with the ledger. A system we called moving smart contracts bypasses this problem: Data remain local, but trusted nodes can access them and execute trusted smart contract code stored on the blockchain. This method avoids the system-wide distribution of research data and makes it accessible and verifiable with trusted software.

[223]  arXiv:2205.08441 [pdf, other]
Title: Conditional Visual Servoing for Multi-Step Tasks
Subjects: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

Visual Servoing has been effectively used to move a robot into specific target locations or to track a recorded demonstration. It does not require manual programming, but it is typically limited to settings where one demonstration maps to one environment state. We propose a modular approach to extend visual servoing to scenarios with multiple demonstration sequences. We call this conditional servoing, as we choose the next demonstration conditioned on the observation of the robot. This method presents an appealing strategy to tackle multi-step problems, as individual demonstrations can be combined flexibly into a control policy. We propose different selection functions and compare them on a shape-sorting task in simulation. With the reprojection error yielding the best overall results, we implement this selection function on a real robot and show the efficacy of the proposed conditional servoing. For videos of our experiments, please check out our project page: https://lmb.informatik.uni-freiburg.de/projects/conditional_servoing/

[224]  arXiv:2205.08443 [pdf, other]
Title: On the Privacy of Decentralized Machine Learning
Comments: 17 pages
Subjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG)

In this work, we carry out the first, in-depth, privacy analysis of Decentralized Learning -- a collaborative machine learning framework aimed at circumventing the main limitations of federated learning. We identify the decentralized learning properties that affect users' privacy and we introduce a suite of novel attacks for both passive and active decentralized adversaries. We demonstrate that, contrary to what is claimed by decentralized learning proposers, decentralized learning does not offer any security advantages over more practical approaches such as federated learning. Rather, it tends to degrade users' privacy by increasing the attack surface and enabling any user in the system to perform powerful privacy attacks such as gradient inversion, and even gain full control over honest users' local model. We also reveal that, given the state of the art in protections, privacy-preserving configurations of decentralized learning require abandoning any possible advantage over the federated setup, completely defeating the objective of the decentralized approach.

[225]  arXiv:2205.08445 [pdf, other]
Title: Data-driven Driver Model for Speed Advisory Systems in Partially Automated Vehicles
Comments: 6 pages, 9 figures
Subjects: Robotics (cs.RO); Systems and Control (eess.SY)

Vehicle control algorithms exploiting connectivity and automation, such as Connected and Automated Vehicles (CAVs) or Advanced Driver Assistance Systems (ADAS), have the opportunity to improve energy savings. However, lower levels of automation involve a human-machine interaction stage, where the presence of a human driver affects the performance of the control algorithm in closed loop. This occurs for instance in the case of Eco-Driving control algorithms implemented as a velocity advisory system, where the driver is displayed an optimal speed trajectory to follow to reduce energy consumption. Achieving the control objectives relies on the human driver perfectly following the recommended speed. If the driver is unable to follow the recommended speed, a decline in energy savings and poor vehicle performance may occur. This warrants the creation of methods to model and forecast the response of a human driver when operating in the loop with a speed advisory system.
This work focuses on developing a sequence to sequence long-short term memory (LSTM)-based driver behavior model that models the interaction of the human driver to a suggested desired vehicle speed trajectory in real-world conditions. A driving simulator is used for data collection and training the driver model, which is then compared against the driving data and a deterministic model. Results show close proximity of the LSTM-based model with the driving data, demonstrating that the model can be adopted as a tool to design human-centered speed advisory systems.

[226]  arXiv:2205.08449 [pdf, other]
Title: Connection-minimal Abduction in EL via Translation to FOL -- Technical Report
Comments: This paper is the technical report version, including appendices, of an IJCAR 2022 paper (to appear)
Subjects: Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)

Abduction in description logics finds extensions of a knowledge base to make it entail an observation. As such, it can be used to explain why the observation does not follow, to repair incomplete knowledge bases, and to provide possible explanations for unexpected observations. We consider TBox abduction in the lightweight description logic EL, where the observation is a concept inclusion and the background knowledge is a TBox, i.e., a set of concept inclusions. To avoid useless answers, such problems usually come with further restrictions on the solution space and/or minimality criteria that help sort the chaff from the grain. We argue that existing minimality notions are insufficient, and introduce connection minimality. This criterion follows Occam's razor by rejecting hypotheses that use concept inclusions unrelated to the problem at hand. We show how to compute a special class of connection-minimal hypotheses in a sound and complete way. Our technique is based on a translation to first-order logic, and constructs hypotheses based on prime implicates. We evaluate a prototype implementation of our approach on ontologies from the medical domain.

[227]  arXiv:2205.08452 [pdf, other]
Title: A psychological theory of explainability
Comments: 14 pages, 2 figures, ICML (accepted, pre camera-ready version)
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

The goal of explainable Artificial Intelligence (XAI) is to generate human-interpretable explanations, but there are no computationally precise theories of how humans interpret AI generated explanations. The lack of theory means that validation of XAI must be done empirically, on a case-by-case basis, which prevents systematic theory-building in XAI. We propose a psychological theory of how humans draw conclusions from saliency maps, the most common form of XAI explanation, which for the first time allows for precise prediction of explainee inference conditioned on explanation. Our theory posits that absent explanation humans expect the AI to make similar decisions to themselves, and that they interpret an explanation by comparison to the explanations they themselves would give. Comparison is formalized via Shepard's universal law of generalization in a similarity space, a classic theory from cognitive science. A pre-registered user study on AI image classifications with saliency map explanations demonstrate that our theory quantitatively matches participants' predictions of the AI.

[228]  arXiv:2205.08454 [pdf, ps, other]
Title: Dynamic Optimization Fabrics for Motion Generation
Comments: Paper submitted to IEEE T-RO on 05/12/2022
Subjects: Robotics (cs.RO)

Optimization fabrics represent a geometric approach to real-time motion planning, where trajectories are designed by the composition of several differential equations that exhibit a desired motion behavior. We generalize this framework to dynamic scenarios and prove that fundamental properties can be conserved. We show that convergence to trajectories and avoidance of moving obstacles can be guaranteed using simple construction rules of the components. Additionally, we present the first quantitative comparisons between optimization fabrics and model predictive control and show that optimization fabrics can generate similar trajectories with better scalability, and thus, much higher replanning frequency (up to 500 Hz with a 7 degrees of freedom robotic arm). Finally, we present empirical results on several robots, including a non-holonomic mobile manipulator with 10 degrees of freedom, supporting the theoretical findings.

[229]  arXiv:2205.08455 [pdf, other]
Title: Utterance Weighted Multi-Dilation Temporal Convolutional Networks for Monaural Speech Dereverberation
Comments: Submitted to IWAENC 2022
Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

Speech dereverberation is an important stage in many speech technology applications. Recent work in this area has been dominated by deep neural network models. Temporal convolutional networks (TCNs) are deep learning models that have been proposed for sequence modelling in the task of dereverberating speech. In this work a weighted multi-dilation depthwise-separable convolution is proposed to replace standard depthwise-separable convolutions in TCN models. This proposed convolution enables the TCN to dynamically focus on more or less local information in its receptive field at each convolutional block in the network. It is shown that this weighted multi-dilation temporal convolutional network (WD-TCN) consistently outperforms the TCN across various model configurations and using the WD-TCN model is a more parameter efficient method to improve the performance of the model than increasing the number of convolutional blocks. The best performance improvement over the baseline TCN is 0.55 dB scale-invariant signal-to-distortion ratio (SISDR) and the best performing WD-TCN model attains 12.26 dB SISDR on the WHAMR dataset.

[230]  arXiv:2205.08458 [pdf, ps, other]
Title: Secure Summation: Capacity Region, Groupwise Key, and Feasibility
Authors: Yizhou Zhao, Hua Sun
Subjects: Information Theory (cs.IT)

The secure summation problem is considered, where $K$ users, each holds an input, wish to compute the sum of their inputs at a server securely, i.e., without revealing any information beyond the sum even if the server may collude with any set of up to $T$ users. First, we prove a folklore result for secure summation - to compute $1$ bit of the sum securely, each user needs to send at least $1$ bit to the server, each user needs to hold a key of at least $1$ bit, and all users need to hold collectively some key variables of at least $K-1$ bits. Next, we focus on the symmetric groupwise key setting, where every group of $G$ users share an independent key. We show that for symmetric groupwise keys with group size $G$, when $G > K-T$, the secure summation problem is not feasible; when $G \leq K-T$, to compute $1$ bit of the sum securely, each user needs to send at least $1$ bit to the server and the size of each groupwise key is at least $(K-T-1)/\binom{K-T}{G}$ bits. Finally, we relax the symmetry assumption on the groupwise keys and the colluding user sets; we allow any arbitrary group of users to share an independent key and any arbitrary group of users to collude with the server. For such a general groupwise key and colluding user setting, we show that secure summation is feasible if and only if the hypergraph, where each node is a user and each edge is a group of users sharing the same key, is connected after removing the nodes corresponding to any colluding set of users and their incident edges.

[231]  arXiv:2205.08459 [pdf, other]
Title: Dynamic Recognition of Speakers for Consent Management by Contrastive Embedding Replay
Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. The current version includes 32 pages, 10 figures, and 2 tables
Subjects: Sound (cs.SD); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS)

Voice assistants record sound and can overhear conversations. Thus, a consent management mechanism is desirable such that users can express their wish to be recorded or not. Consent management can be implemented using speaker recognition; users that do not give consent enrol their voice and all further recordings of these users is subsequently not processed. Building speaker recognition based consent management is challenging due to the dynamic nature of the problem, required scalability for large number of speakers, and need for fast speaker recognition with high accuracy. This paper describes a speaker recognition based consent management system addressing the aforementioned challenges. A fully supervised batch contrastive learning is applied to learn the underlying speaker equivariance inductive bias during the training on the set of speakers noting recording dissent. Speakers that do not provide consent are grouped in buckets which are trained continuously. The embeddings are contrastively learned for speakers in their buckets during training and act later as a replay buffer for classification. The buckets are progressively registered during training and a novel multi-strided random sampling of the contrastive embedding replay buffer is proposed. Buckets are contrastively trained for a few steps only in each iteration and replayed for classification progressively leading to fast convergence. An algorithm for fast and dynamic registration and removal of speakers in buckets is described. The evaluation results show that the proposed approach provides the desired fast and dynamic solution for consent management and outperforms existing approaches in terms of convergence speed and adaptive capabilities as well as verification performance during inference.

[232]  arXiv:2205.08462 [pdf, ps, other]
Title: Privacy Preserving Machine Learning for Electric Vehicles: A Survey
Comments: Submitted to Journal
Subjects: Cryptography and Security (cs.CR)

In the recent years, the interest of individual users in modern electric vehicles (EVs) has grown exponentially. An EV has two major components, which make it different from traditional vehicles, first is its environment friendly nature because of being electric, and second is the interconnection ability of these vehicles because of modern information and communication technologies (ICTs). Both of these features are playing a key role in the development of EVs, and both academia and industry personals are working towards development of modern protocols for EV networks. All these interactions, whether from energy perspective or from communication perspective, both are generating a tremendous amount of data every day. In order to get most out of this data collected from EVs, research works have highlighted the use of machine/deep learning techniques for various EV applications. This interaction is quite fruitful, but it also comes with a critical concern of privacy leakage during collection, storage, and training of vehicular data. Therefore, alongside developing machine/deep learning techniques for EVs, it is also critical to ensure that they are resilient to private information leakage and attacks. In this paper, we begin with the discussion about essential background on EVs and privacy preservation techniques, followed by a brief overview of privacy preservation in EVs using machine learning techniques. Particularly, we also focus on an in-depth review of the integration of privacy techniques in EVs and highlighted different application scenarios in EVs. Alongside this, we provide a a very detailed survey of current works on privacy preserving machine/deep learning techniques used for modern EVs. Finally, we present the certain research issues, critical challenges, and future directions of research for researchers working in privacy preservation in EVs.

[233]  arXiv:2205.08464 [pdf, other]
Title: Robust Losses for Learning Value Functions
Subjects: Machine Learning (cs.LG)

Most value function learning algorithms in reinforcement learning are based on the mean squared (projected) Bellman error. However, squared errors are known to be sensitive to outliers, both skewing the solution of the objective and resulting in high-magnitude and high-variance gradients. To control these high-magnitude updates, typical strategies in RL involve clipping gradients, clipping rewards, rescaling rewards, or clipping errors. While these strategies appear to be related to robust losses -- like the Huber loss -- they are built on semi-gradient update rules which do not minimize a known loss. In this work, we build on recent insights reformulating squared Bellman errors as a saddlepoint optimization problem and propose a saddlepoint reformulation for a Huber Bellman error and Absolute Bellman error. We start from a formalization of robust losses, then derive sound gradient-based approaches to minimize these losses in both the online off-policy prediction and control settings. We characterize the solutions of the robust losses, providing insight into the problem settings where the robust losses define notably better solutions than the mean squared Bellman error. Finally, we show that the resulting gradient-based algorithms are more stable, for both prediction and control, with less sensitivity to meta-parameters.

[234]  arXiv:2205.08473 [pdf, other]
Title: ColonFormer: An Efficient Transformer based Method for Colon Polyp Segmentation
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Identifying polyps is a challenging problem for automatic analysis of endoscopic images in computer-aided clinical support systems. Models based on convolutional networks (CNN), transformers, and combinations of them have been proposed to segment polyps with promising results. However, those approaches have limitations either in modeling the local appearance of the polyps only or lack of multi-level features for spatial dependency in the decoding process. This paper proposes a novel network, namely ColonFormer, to address these limitations. ColonFormer is an encoder-decoder architecture with the capability of modeling long-range semantic information at both encoder and decoder branches. The encoder is a lightweight architecture based on transformers for modeling global semantic relations at multi scales. The decoder is a hierarchical network structure designed for learning multi-level features to enrich feature representation. Besides, a refinement module is added with a new skip connection technique to refine the boundary of polyp objects in the global map for accurate segmentation. Extensive experiments have been conducted on five popular benchmark datasets for polyp segmentation, including Kvasir, CVC-Clinic DB, CVCColonDB, EndoScene, and ETIS. Experimental results show that our ColonFormer achieve state-of-the-art performance on all benchmark datasets.

[235]  arXiv:2205.08475 [pdf, other]
Title: Multidisciplinary Reading Patterns of Digital Documents
Subjects: Digital Libraries (cs.DL); Human-Computer Interaction (cs.HC)

Reading plays a vital role in updating the researchers on recent developments in the field, including but not limited to solutions to various problems and collaborative studies between disciplines. Prior studies identify reading patterns to vary depending on the level of expertise of the researcher on the content of the document. We present a pilot study of eye-tracking measures during a reading task with participants with different domain expertise to characterize their reading patterns.

[236]  arXiv:2205.08478 [pdf, other]
Title: An Evaluation Framework for Legal Document Summarization
Comments: 7 pages, 7 figures, 5 tables, To appear in LREC 2022
Subjects: Computation and Language (cs.CL); Information Retrieval (cs.IR); Machine Learning (cs.LG)

A law practitioner has to go through numerous lengthy legal case proceedings for their practices of various categories, such as land dispute, corruption, etc. Hence, it is important to summarize these documents, and ensure that summaries contain phrases with intent matching the category of the case. To the best of our knowledge, there is no evaluation metric that evaluates a summary based on its intent. We propose an automated intent-based summarization metric, which shows a better agreement with human evaluation as compared to other automated metrics like BLEU, ROUGE-L etc. in terms of human satisfaction. We also curate a dataset by annotating intent phrases in legal documents, and show a proof of concept as to how this system can be automated. Additionally, all the code and data to generate reproducible results is available on Github.

[237]  arXiv:2205.08479 [pdf, other]
Title: Opportunistic Routing in Quantum Networks
Comments: This paper is accepted in INFOCOM (IEEE Conference on Computer Communications) 2022
Subjects: Networking and Internet Architecture (cs.NI)

Unlike classical routing algorithms, quantum routing algorithms make use of entangled states - a type of resources that have a limited lifetime and need to be regenerated after consumption. In a nutshell, quantum routing algorithms have to use these resources efficiently, while optimizing some objectives such as the total waiting time. Current routing algorithms tend to keep a routing request waiting until all of the resources on its path are available. In this paper, we introduce a new way of managing entanglement resources in an opportunistic fashion: a request can move forward along its path as soon as possible (even if some resources on its path are not ready). We show that this opportunistic approach is fundamentally better than conventional approaches. In particular, our results indicate that this new approach achieves a 30-50% improvement in the average total waiting time and average link waiting time compared with several state-of-the-art routing algorithms. As a by-product of this work, we develop a new simulator for quantum routing, which can be used to evaluate various design choices under different scenarios.

[238]  arXiv:2205.08480 [pdf, other]
Title: Effort Informed Roadmaps (EIRM*): Efficient Asymptotically Optimal Multiquery Planning by Actively Reusing Validation Effort
Comments: Submitted to the International Symposium on Robotics Research (ISRR), Manuscript #3. 16 pages, 7 figures, 1 table. Video and code available at this https URL
Subjects: Robotics (cs.RO)

Multiquery planning algorithms find paths between various different starts and goals in a single search space. They are designed to do so efficiently by reusing information across planning queries. This information may be computed before or during the search and often includes knowledge of valid paths. Using known valid paths to solve an individual planning query takes less computational effort than finding a completely new solution. This allows multiquery algorithms, such as PRM*, to outperform single-query algorithms, such as RRT*, on many problems but their relative performance depends on how much information is reused. Despite this, few multiquery planners explicitly seek to maximize path reuse and, as a result, many do not consistently outperform single-query alternatives. This paper presents Effort Informed Roadmaps (EIRM*), an almost-surely asymptotically optimal multiquery planning algorithm that explicitly prioritizes reusing computational effort. EIRM* uses an asymmetric bidirectional search to identify existing paths that may help solve an individual planning query and then uses this information to order its search and reduce computational effort. This allows it to find initial solutions up to an order-of-magnitude faster than state-of-the-art planning algorithms on the tested abstract and robotic multiquery planning problems.

[239]  arXiv:2205.08487 [pdf, other]
Title: Systematic Mapping Protocol: Variability Management in Dynamic Software Product Lines for Self-Adaptive Systems
Comments: Systematic mapping protocol. Keywords: Self-adaptive systems, reconfiguration, Dynamic software product lines, systematic mapping. 9 pages, 2 figures, 6 tables
Subjects: Software Engineering (cs.SE)

Context: The Importance of Dynamic Variability Management in Dynamic Software Product Lines. Objective: Define a protocol for conducting a systematic mapping study to summarize and synthesize evidence on dynamic variability management for Dynamic Software Product Lines in self-adaptive systems. Method: Application the protocol to conduct a systematic mapping study according the guidelines of K. Petersen. Results: A validated protocol to conduct a systematic mapping study. Conclusions: First findings show that it is necessary to visualize new ways to manage variability in dynamic software product lines.

[240]  arXiv:2205.08489 [pdf, other]
Title: Control Interface Remapping for Bias-Aware Assistive Teleoperation
Comments: 6 pages, 4 figures, 1 table, 1 algorithm, to be published in International Consortium for Rehabilitation Robotics (ICORR) 2022
Subjects: Human-Computer Interaction (cs.HC)

Users of assistive devices vary in their extent of motor impairment, and hence their physical interaction with control interfaces can differ. There is the potential for improved utility if control interface actuation is mapped to assistive device control signals in a manner customized to each user. In this paper, we present (1) a method for creating a custom interface to assistive device control mapping based on the design of a user's bias profile, (2) a procedure and virtual task for gathering interface actuation data from which to build the bias profile and map, and (3) an evaluation of our method on 6 participants with upper limb motor impairments. Our results show that custom interface remapping based on user bias profiles shows promise in providing assistance via an improvement in the reachability of the device control space. This effect was especially pronounced for individuals who had a more limited reachable space.

[241]  arXiv:2205.08491 [pdf]
Title: Elon Musk's Twitter Takeover: Politician Accounts
Authors: Veli Safak
Comments: 2 pages, 6 figures
Subjects: Social and Information Networks (cs.SI)

While conservatives welcomed Musk's takeover of Twitter, liberals sounded the alarm bells. Based on official Twitter accounts belonging to 528 US Senate and Congress members (269 Democrats and 259 Republicans), I empirically analyze how the number of followers of these accounts changes following Twitter's acceptance of Elon Musk's offer on April 25. This study provides the first empirical analysis of changes in politician accounts on social media following this takeover agreement.

[242]  arXiv:2205.08493 [pdf, other]
Title: Faster Knapsack Algorithms via Bounded Monotone Min-Plus-Convolution
Comments: Shortened abstract. Appears at ICALP '22
Subjects: Data Structures and Algorithms (cs.DS)

We present new exact and approximation algorithms for 0-1-Knapsack and Unbounded Knapsack:
* Exact Algorithm for 0-1-Knapsack: 0-1-Knapsack has known algorithms running in time $\widetilde{O}(n + \min\{n OPT, n W, OPT^2, W^2\})$, where $n$ is the number of items, $W$ is the weight budget, and $OPT$ is the optimal profit. We present an algorithm running in time $\widetilde{O}(n + (W + OPT)^{1.5})$. This improves the running time in case $n,W,OPT$ are roughly equal.
* Exact Algorithm for Unbounded Knapsack: Unbounded Knapsack has known algorithms running in time $\widetilde{O}(n + \min\{n \cdot p_{\max}, n \cdot w_{\max}, p_{\max}^2, w_{\max}^2\})$ [Axiotis, Tzamos '19, Jansen, Rohwedder '19, Chan, He '20], where $n$ is the number of items, $w_{\max}$ is the largest weight of any item, and $p_{\max}$ is the largest profit of any item. We present an algorithm running in time $\widetilde{O}(n + (p_{\max} + w_{\max})^{1.5})$, giving a similar improvement as for 0-1-Knapsack.
* Approximating Unbounded Knapsack with Resource Augmentation: Unbounded Knapsack has a known FPTAS with running time $\widetilde{O}(\min\{n/\varepsilon, n + 1/\varepsilon^2\})$ [Jansen, Kraft '18]. We study weak approximation algorithms, which approximate the optimal profit but are allowed to overshoot the weight constraint. We present the first approximation scheme for Unbounded Knapsack in this setting, achieving running time $\widetilde{O}(n + 1/\varepsilon^{1.5})$.
Our algorithms can be seen as reductions to Min-Plus-Convolution on monotone sequences with bounded entries. These structured instances of Min-Plus-Convolution can be solved in time $O(n^{1.5})$ [Chi,Duan,Xie,Zhang '22] (in contrast to the conjectured $n^{2-o(1)}$ lower bound for the general case).

[243]  arXiv:2205.08497 [pdf, ps, other]
Title: Feature Aggregation in Zero-Shot Cross-Lingual Transfer Using Multilingual BERT
Comments: Accepted by ICPR 2022
Subjects: Computation and Language (cs.CL)

Multilingual BERT (mBERT), a language model pre-trained on large multilingual corpora, has impressive zero-shot cross-lingual transfer capabilities and performs surprisingly well on zero-shot POS tagging and Named Entity Recognition (NER), as well as on cross-lingual model transfer. At present, the mainstream methods to solve the cross-lingual downstream tasks are always using the last transformer layer's output of mBERT as the representation of linguistic information. In this work, we explore the complementary property of lower layers to the last transformer layer of mBERT. A feature aggregation module based on an attention mechanism is proposed to fuse the information contained in different layers of mBERT. The experiments are conducted on four zero-shot cross-lingual transfer datasets, and the proposed method obtains performance improvements on key multilingual benchmark tasks XNLI (+1.5 %), PAWS-X (+2.4 %), NER (+1.2 F1), and POS (+1.5 F1). Through the analysis of the experimental results, we prove that the layers before the last layer of mBERT can provide extra useful information for cross-lingual downstream tasks and explore the interpretability of mBERT empirically.

[244]  arXiv:2205.08501 [pdf, other]
Title: Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks
Comments: 23 pages, 10 figures
Subjects: Emerging Technologies (cs.ET); Machine Learning (cs.LG); Optics (physics.optics)

Neural networks are widely deployed models across many scientific disciplines and commercial endeavors ranging from edge computing and sensing to large-scale signal processing in data centers. The most efficient and well-entrenched method to train such networks is backpropagation, or reverse-mode automatic differentiation. To counter an exponentially increasing energy budget in the artificial intelligence sector, there has been recent interest in analog implementations of neural networks, specifically nanophotonic neural networks for which no analog backpropagation demonstration exists. We design mass-manufacturable silicon photonic neural networks that alternately cascade our custom designed "photonic mesh" accelerator with digitally implemented nonlinearities. These reconfigurable photonic meshes program computationally intensive arbitrary matrix multiplication by setting physical voltages that tune the interference of optically encoded input data propagating through integrated Mach-Zehnder interferometer networks. Here, using our packaged photonic chip, we demonstrate in situ backpropagation for the first time to solve classification tasks and evaluate a new protocol to keep the entire gradient measurement and update of physical device voltages in the analog domain, improving on past theoretical proposals. Our method is made possible by introducing three changes to typical photonic meshes: (1) measurements at optical "grating tap" monitors, (2) bidirectional optical signal propagation automated by fiber switch, and (3) universal generation and readout of optical amplitude and phase. After training, our classification achieves accuracies similar to digital equivalents even in presence of systematic error. Our findings suggest a new training paradigm for photonics-accelerated artificial intelligence based entirely on a physical analog of the popular backpropagation technique.

[245]  arXiv:2205.08502 [pdf, other]
Title: IEC-61850 Performance Evaluation in a 5G Cellular Network: UDP and TCP Analysis
Comments: 28 pages, 25 figures, book chapter
Journal-ref: Handbook of Smart Energy Systems. Springer, Cham.2022
Subjects: Networking and Internet Architecture (cs.NI)

This chapter summarizes the results obtained from a test bed, which is composed of a microgrid and a wireless network, both relying on real and practical premises. This test bed aims to evaluate application protocols considered in the transport layer of the standard IEC-61850. The application is a data gathering system, where IEC-61850 messages are transmitted from and between different elements in a microgrid system, such as IED, publishers, and subscribers, which are node elements that are part of the utility grid. The standard IEC-61850 defines many protocols, such as SV, GOOSE, and MMS, each of them with a variety of requirements and sharing similar transport communication protocols, such as TCP and UDP. Hence, a testing framework is used in the test bed in order to evaluate the performance of these protocols aiming to enable the communication in a real microgrid deployment. The IEC-61850 messages transported by TCP or UDP are transmitted to a centralized location, where a local database stores the information under different test scenarios and using different transmission sample times. The overall results are promising and indicate that the protocols based on TCP in combination with a 5G cellular network architecture would be suitable for most applications within the studied microgrid case. In the case of UDP, the performance indicated some specific constraints related to the multiple access nature of simultaneous transmissions and also based on the SINR.

[246]  arXiv:2205.08508 [pdf, other]
Title: A CLIP-Hitchhiker's Guide to Long Video Retrieval
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Our goal in this paper is the adaptation of image-text models for long video retrieval. Recent works have demonstrated state-of-the-art performance in video retrieval by adopting CLIP, effectively hitchhiking on the image-text representation for video tasks. However, there has been limited success in learning temporal aggregation that outperform mean-pooling the image-level representations extracted per frame by CLIP. We find that the simple yet effective baseline of weighted-mean of frame embeddings via query-scoring is a significant improvement above all prior temporal modelling attempts and mean-pooling. In doing so, we provide an improved baseline for others to compare to and demonstrate state-of-the-art performance of this simple baseline on a suite of long video retrieval benchmarks.

[247]  arXiv:2205.08510 [pdf, other]
Title: Susceptibility of Age of Gossip to Timestomping
Subjects: Information Theory (cs.IT); Networking and Internet Architecture (cs.NI); Signal Processing (eess.SP)

We consider a fully connected network consisting of a source that maintains the current version of a file, $n$ nodes that use asynchronous gossip mechanisms to disseminate fresh information in the network, and an adversary who infects the packets at a target node through data timestamp manipulation, with the intent to replace circulation of fresh packets with outdated packets in the network. We show that a single infected node increases the expected age of a fully connected network from $O(\log n)$ to $O(n)$. Further, we show that the optimal behavior for an adversary is to reset the timestamps of all outgoing packets to the current time and of all incoming packets to an outdated time. Additionally, if the adversary allows the infected node to accept a small fraction of incoming packets from the network, then a large network can manage to curb the spread of stale files coming from the infected node and pull the network age back to $O(\log n)$. Lastly, we show that if an infected node contacts only a single node instead of all nodes of the network, the system age can still be degraded to $O(n)$. These show that fully connected nature of a network can be both a benefit and a detriment for information freshness; full connectivity, while enabling fast dissemination of information, also enables fast dissipation of adversarial inputs.

[248]  arXiv:2205.08512 [pdf, other]
Title: Experimental evaluation of digitally-verifiable photonic computing for blockchain and cryptocurrency
Comments: 17 pages, 7 figures
Subjects: Emerging Technologies (cs.ET); Cryptography and Security (cs.CR); Optics (physics.optics)

As blockchain technology and cryptocurrency become increasingly mainstream, ever-increasing energy costs required to maintain the computational power running these decentralized platforms create a market for more energy-efficient hardware. Photonic cryptographic hash functions, which use photonic integrated circuits to accelerate computation, promise energy efficiency for verifying transactions and mining in a cryptonetwork. Like many analog computing approaches, however, current proposals for photonic cryptographic hash functions that promise similar security guarantees as Bitcoin are susceptible to systematic error, so multiple devices may not reach a consensus on computation despite high numerical precision (associated with low photodetector noise). In this paper, we theoretically and experimentally demonstrate that a more general family of robust discrete analog cryptographic hash functions, which we introduce as LightHash, leverages integer matrix-vector operations on photonic mesh networks of interferometers. The difficulty of LightHash can be adjusted to be sufficiently tolerant to systematic error (calibration error, loss error, coupling error, and phase error) and preserve inherent security guarantees present in the Bitcoin protocol. Finally, going beyond our proof-of-concept, we define a ``photonic advantage'' criterion and justify how recent developments in CMOS optoelectronics (including analog-digital conversion) provably achieve such advantage for robust and digitally-verifiable photonic computing and ultimately generate a new market for decentralized photonic technology.

[249]  arXiv:2205.08514 [pdf, other]
Title: Recovering Private Text in Federated Learning of Language Models
Subjects: Computation and Language (cs.CL); Cryptography and Security (cs.CR); Machine Learning (cs.LG)

Federated learning allows distributed users to collaboratively train a model while keeping each user's data private. Recently, a growing body of work has demonstrated that an eavesdropping attacker can effectively recover image data from gradients transmitted during federated learning. However, little progress has been made in recovering text data. In this paper, we present a novel attack method FILM for federated learning of language models -- for the first time, we show the feasibility of recovering text from large batch sizes of up to 128 sentences. Different from image-recovery methods which are optimized to match gradients, we take a distinct approach that first identifies a set of words from gradients and then directly reconstructs sentences based on beam search and a prior-based reordering strategy. The key insight of our attack is to leverage either prior knowledge in pre-trained language models or memorization during training. Despite its simplicity, we demonstrate that FILM can work well with several large-scale datasets -- it can extract single sentences with high fidelity even for large batch sizes and recover multiple sentences from the batch successfully if the attack is applied iteratively. We hope our results can motivate future work in developing stronger attacks as well as new defense methods for training language models in federated learning. Our code is publicly available at https://github.com/Princeton-SysML/FILM.

[250]  arXiv:2205.08515 [pdf, other]
Title: Unsupervised Segmentation in Real-World Images via Spelke Object Inference
Comments: 25 pages, 10 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)

Self-supervised category-agnostic segmentation of real-world images into objects is a challenging open problem in computer vision. Here, we show how to learn static grouping priors from motion self-supervision, building on the cognitive science notion of Spelke Objects: groupings of stuff that move together. We introduce Excitatory-Inhibitory Segment Extraction Network (EISEN), which learns from optical flow estimates to extract pairwise affinity graphs for static scenes. EISEN then produces segments from affinities using a novel graph propagation and competition mechanism. Correlations between independent sources of motion (e.g. robot arms) and objects they move are resolved into separate segments through a bootstrapping training process. We show that EISEN achieves a substantial improvement in the state of the art for self-supervised segmentation on challenging synthetic and real-world robotic image datasets. We also present an ablation analysis illustrating the importance of each element of the EISEN architecture.

[251]  arXiv:2205.08518 [pdf, other]
Title: Do Neural Networks Compress Manifolds Optimally?
Subjects: Information Theory (cs.IT); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

Artificial Neural-Network-based (ANN-based) lossy compressors have recently obtained striking results on several sources. Their success may be ascribed to an ability to identify the structure of low-dimensional manifolds in high-dimensional ambient spaces. Indeed, prior work has shown that ANN-based compressors can achieve the optimal entropy-distortion curve for some such sources. In contrast, we determine the optimal entropy-distortion tradeoffs for two low-dimensional manifolds with circular structure and show that state-of-the-art ANN-based compressors fail to optimally compress the sources, especially at high rates.

[252]  arXiv:2205.08520 [pdf]
Title: Measuring Plagiarism in Introductory Programming Course Assignments
Comments: Accepted at IEEE conference, the 2022 8th International Conference on Information Technology Trends (ITT), at Higher Colleges of Technology - Dubai Men's Campus on 25-26 May 2022, Dubai, United Arab Emirates
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)

Measuring plagiarism in programming assignments is an essential task to the educational procedure. This paper discusses the methods of plagiarism and its detection in introductory programming course assignments written in C++. A small corpus of assignments is made publically available. A general framework to compute the similarity between a solution pair is developed that uses the three token-based similarity methods as features and predicts if the solution is plagiarized. The importance of each feature is also measured, which in return ranks the effectiveness of each method in use. Finally, the artificially generated dataset improves the results compared to the original data. We achieved an F1 score of 0.955 and 0.971 on original and synthetic datasets.

[253]  arXiv:2205.08521 [pdf, other]
Title: Learning-Augmented Streaming Codes are Approximately Optimal for Variable-Size Messages
Comments: 13 pages, 8 figures, this is an extended version of the IEEE ISIT 2022 paper with the same title
Subjects: Information Theory (cs.IT)

Real-time streaming communication requires a high quality of service despite contending with packet loss. Streaming codes are a class of codes best suited for this setting. A key challenge for streaming codes is that they operate in an "online" setting in which the amount of data to be transmitted varies over time and is not known in advance. Mitigating the adverse effects of variability requires spreading the data that arrives at a time slot over multiple future packets, and the optimal strategy for spreading depends on the arrival pattern. Algebraic coding techniques alone are therefore insufficient for designing rate-optimal codes. We combine algebraic coding techniques with a learning-augmented algorithm for spreading to design the first approximately rate-optimal streaming codes for a range of parameter regimes that are important for practical applications.

[254]  arXiv:2205.08524 [pdf, other]
Title: Supervised Learning for Coverage-Directed Test Selection in Simulation-Based Verification
Comments: 8 pages (excl Refs) for IEEE AI Test
Subjects: Hardware Architecture (cs.AR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Software Engineering (cs.SE)

Constrained random test generation is one the most widely adopted methods for generating stimuli for simulation-based verification. Randomness leads to test diversity, but tests tend to repeatedly exercise the same design logic. Constraints are written (typically manually) to bias random tests towards interesting, hard-to-reach, and yet-untested logic. However, as verification progresses, most constrained random tests yield little to no effect on functional coverage. If stimuli generation consumes significantly less resources than simulation, then a better approach involves randomly generating a large number tests, selecting the most effective subset, and only simulating that subset. In this paper, we introduce a novel method for automatic constraint extraction and test selection. This method, which we call coverage-directed test selection, is based on supervised learning from coverage feedback. Our method biases selection towards tests that have a high probability of increasing functional coverage, and prioritises them for simulation. We show how coverage-directed test selection can reduce manual constraint writing, prioritise effective tests, reduce verification resource consumption, and accelerate coverage closure on a large, real-life industrial hardware design.

[255]  arXiv:2205.08525 [pdf, other]
Title: Self-supervised Neural Articulated Shape and Appearance Models
Comments: 15 pages. CVPR 2022. Project page available at this https URL
Subjects: Computer Vision and Pattern Recognition (cs.CV)

Learning geometry, motion, and appearance priors of object classes is important for the solution of a large variety of computer vision problems. While the majority of approaches has focused on static objects, dynamic objects, especially with controllable articulation, are less explored. We propose a novel approach for learning a representation of the geometry, appearance, and motion of a class of articulated objects given only a set of color images as input. In a self-supervised manner, our novel representation learns shape, appearance, and articulation codes that enable independent control of these semantic dimensions. Our model is trained end-to-end without requiring any articulation annotations. Experiments show that our approach performs well for different joint types, such as revolute and prismatic joints, as well as different combinations of these joints. Compared to state of the art that uses direct 3D supervision and does not output appearance, we recover more faithful geometry and appearance from 2D observations only. In addition, our representation enables a large variety of applications, such as few-shot reconstruction, the generation of novel articulations, and novel view-synthesis.

[256]  arXiv:2205.08527 [pdf, other]
Title: Static analysis tools in the era of cloud-native systems
Journal-ref: 4th International Conference on Microservices. Paris, 2022
Subjects: Software Engineering (cs.SE)

Microservices fuel cloud-native systems with small service sets developed and deployed independently. The independent nature of this modular architecture also leads to challenges and gaps. The intended system design might deviate far from what is eventually produced and maintained as the architecture tends to degrade over time. This paper challenges the audience on how static analysis could contribute to microservice system development and management, particularly managing architectural degradation. It elaborates on challenges and needed changes in the traditional code analysis to better fit these systems and discusses implications for practitioners once robust static analysis tools become available

[257]  arXiv:2205.08529 [pdf, other]
Title: F3B: A Low-Latency Commit-and-Reveal Architecture to Mitigate Blockchain Front-Running
Subjects: Cryptography and Security (cs.CR)

Front-running attacks, which benefit from advanced knowledge of pending transactions, have proliferated in the cryptocurrency space since the emergence of decentralized finance. Front-running causes devastating losses to honest participants$\unicode{x2013}$estimated at \$280M each month$\unicode{x2013}$and endangers the fairness of the ecosystem. We present Flash Freezing Flash Boys (F3B), a blockchain architecture to address front-running attacks by relying on a commit-and-reveal scheme where the contents of transactions are encrypted and later revealed by a decentralized secret-management committee once the underlying consensus layer has committed the transaction. F3B mitigates front-running attacks because an adversary can no longer read the content of a transaction before commitment, thus preventing the adversary from benefiting from advance knowledge of pending transactions. We design F3B to be agnostic to the underlying consensus algorithm and compatible with legacy smart contracts by addressing front-running at the blockchain architecture level. Unlike existing commit-and-reveal approaches, F3B only requires writing data onto the underlying blockchain once, establishing a significant overhead reduction. An exploration of F3B shows that with a secret-management committee consisting of 8 and 128 members, F3B presents between 0.1 and 1.8 seconds of transaction-processing latency, respectively.

[258]  arXiv:2205.08532 [pdf, ps, other]
Title: New Lower Bounds for Private Estimation and\\a Generalized Fingerprinting Lemma
Subjects: Data Structures and Algorithms (cs.DS); Cryptography and Security (cs.CR); Machine Learning (stat.ML)

We prove new lower bounds for statistical estimation tasks under the constraint of $\paren{\eps, \delta}$-differential privacy. First, we provide tight lower bounds for private covariance estimation of Gaussian distributions. We show that estimating the covariance matrix in Frobenius norm requires $\Omega\paren{d^2}$ samples, and in spectral norm requires $\Omega\paren{d^{\frac{3}{2}}}$ samples, both matching upper bounds up to logarithmic factors. We prove these bounds via our main technical contribution, a broad generalization of the fingerprinting method~\cite{BunUV14} to exponential families. Additionally, using the private Assouad method of Acharya, Sun, and Zhang~\cite{AcharyaSZ21}, we show a tight $\Omega\paren{\frac{d}{\alpha^2 \eps}}$ lower bound for estimating the mean of a distribution with bounded covariance to $\alpha$-error in $\ell_2$-distance. Prior known lower bounds for all these problems were either polynomially weaker or held under the stricter condition of $\paren{\eps,0}$-differential privacy.

[259]  arXiv:2205.08533 [pdf, ps, other]
Title: Consistent Human Evaluation of Machine Translation across Language Pairs
Comments: 10 pages
Subjects: Computation and Language (cs.CL)

Obtaining meaningful quality scores for machine translation systems through human evaluation remains a challenge given the high variability between human evaluators, partly due to subjective expectations for translation quality for different language pairs. We propose a new metric called XSTS that is more focused on semantic equivalence and a cross-lingual calibration method that enables more consistent assessment. We demonstrate the effectiveness of these novel contributions in large scale evaluation studies across up to 14 language pairs, with translation both into and out of English.

[260]  arXiv:2205.08534 [pdf, other]
Title: Vision Transformer Adapter for Dense Predictions
Subjects: Computer Vision and Pattern Recognition (cs.CV)

This work investigates a simple yet powerful adapter for Vision Transformer (ViT). Unlike recent visual transformers that introduce vision-specific inductive biases into their architectures, ViT achieves inferior performance on dense prediction tasks due to lacking prior information of images. To solve this issue, we propose a Vision Transformer Adapter (ViT-Adapter), which can remedy the defects of ViT and achieve comparable performance to vision-specific models by introducing inductive biases via an additional architecture. Specifically, the backbone in our framework is a vanilla transformer that can be pre-trained with multi-modal data. When fine-tuning on downstream tasks, a modality-specific adapter is used to introduce the data and tasks' prior information into the model, making it suitable for these tasks. We verify the effectiveness of our ViT-Adapter on multiple downstream tasks, including object detection, instance segmentation, and semantic segmentation. Notably, when using HTC++, our ViT-Adapter-L yields 60.1 box AP and 52.1 mask AP on COCO test-dev, surpassing Swin-L by 1.4 box AP and 1.0 mask AP. For semantic segmentation, our ViT-Adapter-L establishes a new state-of-the-art of 60.5 mIoU on ADE20K val, 0.6 points higher than SwinV2-G. We hope that the proposed ViT-Adapter could serve as an alternative for vision-specific transformers and facilitate future research.

[261]  arXiv:2205.08535 [pdf, other]
Title: AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars
Comments: SIGGRAPH 2022; Project Page this https URL Codes available at this https URL
Subjects: Computer Vision and Pattern Recognition (cs.CV)

3D avatar creation plays a crucial role in the digital age. However, the whole production process is prohibitively time-consuming and labor-intensive. To democratize this technology to a larger audience, we propose AvatarCLIP, a zero-shot text-driven framework for 3D avatar generation and animation. Unlike professional software that requires expert knowledge, AvatarCLIP empowers layman users to customize a 3D avatar with the desired shape and texture, and drive the avatar with the described motions using solely natural languages. Our key insight is to take advantage of the powerful vision-language model CLIP for supervising neural human generation, in terms of 3D geometry, texture and animation. Specifically, driven by natural language descriptions, we initialize 3D human geometry generation with a shape VAE network. Based on the generated 3D human shapes, a volume rendering model is utilized to further facilitate geometry sculpting and texture generation. Moreover, by leveraging the priors learned in the motion VAE, a CLIP-guided reference-based motion synthesis method is proposed for the animation of the generated 3D avatar. Extensive qualitative and quantitative experiments validate the effectiveness and generalizability of AvatarCLIP on a wide range of avatars. Remarkably, AvatarCLIP can generate unseen 3D avatars with novel animations, achieving superior zero-shot capability.

[262]  arXiv:2205.08536 [pdf, other]
Title: Disentangling Visual Embeddings for Attributes and Objects
Comments: To appear at CVPR 2022 (Oral)
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

We study the problem of compositional zero-shot learning for object-attribute recognition. Prior works use visual features extracted with a backbone network, pre-trained for object classification and thus do not capture the subtly distinct features associated with attributes. To overcome this challenge, these studies employ supervision from the linguistic space, and use pre-trained word embeddings to better separate and compose attribute-object pairs for recognition. Analogous to linguistic embedding space, which already has unique and agnostic embeddings for object and attribute, we shift the focus back to the visual space and propose a novel architecture that can disentangle attribute and object features in the visual space. We use visual decomposed features to hallucinate embeddings that are representative for the seen and novel compositions to better regularize the learning of our model. Extensive experiments show that our method outperforms existing work with significant margin on three datasets: MIT-States, UT-Zappos, and a new benchmark created based on VAW. The code, models, and dataset splits are publicly available at https://github.com/nirat1606/OADis.

Cross-lists for Wed, 18 May 22

[263]  arXiv:2205.07528 (cross-list from math.RA) [pdf, ps, other]
Title: The Smallest Hard Trees
Subjects: Rings and Algebras (math.RA); Computational Complexity (cs.CC)

We find an orientation of a tree with 20 vertices such that the corresponding fixed-template constraint satisfaction problem (CSP) is NP-complete, and prove that for every orientation of a tree with fewer vertices the corresponding CSP can be solved in polynomial time. We also compute the smallest tree that is NL-hard (assuming L is not NL), the smallest tree that cannot be solved by arc consistency, and the smallest tree that cannot be solved by Datalog. Our experimental results also support a conjecture of Bulin concerning a question of Hell, Nesetril and Zhu, namely that "easy trees lack the ability to count". Most proofs are computer-based and make use of the most recent universal-algebraic theory about the complexity of finite-domain CSPs. However, further ideas are required because of the huge number of orientations of trees. In particular, we use the well-known fact that it suffices to study orientations of trees that are cores and show how to efficiently decide whether a given orientation of a tree is a core using the arc-consistency procedure. Moreover, we present a method to generate orientations of trees that are cores that works well in practice. In this way we found interesting examples for the open research problem to classify finite-domain CSPs in NL.

[264]  arXiv:2205.07808 (cross-list from eess.SY) [pdf, other]
Title: Switch as a Verifier: Toward Scalable Data Plane Checking via Distributed, On-Device Verification
Subjects: Systems and Control (eess.SY); Networking and Internet Architecture (cs.NI); Programming Languages (cs.PL)

Data plane verification (DPV) is important for finding network errors. Current DPV tools employ a centralized architecture, where a server collects the data planes of all devices and verifies them. Despite substantial efforts on accelerating DPV, this centralized architecture is inherently unscalable. In this paper, to tackle the scalability challenge of DPV, we circumvent the scalability bottleneck of centralized design and design Coral, a distributed, on-device DPV framework. The key insight of Coral is that DPV can be transformed into a counting problem on a directed acyclic graph, which can be naturally decomposed into lightweight tasks executed at network devices, enabling scalability. Coral consists of (1) a declarative requirement specification language, (2) a planner that employs a novel data structure DVNet to systematically decompose global verification into on-device counting tasks, and (3) a distributed verification (DV) protocol that specifies how on-device verifiers communicate task results efficiently to collaboratively verify the requirements. We implement a prototype of Coral. Extensive experiments with real-world datasets (WAN/LAN/DC) show that Coral consistently achieves scalable DPV under various networks and DPV scenarios, i.e., up to 1250 times speed up in the scenario of burst update, and up to 202 times speed up on 80% quantile of incremental verification, than state-of-the-art DPV tools, with little overhead on commodity network devices.

[265]  arXiv:2205.07866 (cross-list from eess.IV) [pdf, ps, other]
Title: Primal-Dual UNet for Sparse View Cone Beam Computed Tomography Volume Reconstruction
Comments: 3 pages, 1 figure, accepted short paper submission at MIDL 2022
Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Medical Physics (physics.med-ph)

In this paper, the Primal-Dual UNet for sparse view CT reconstruction is modified to be applicable to cone beam projections and perform reconstructions of entire volumes instead of slices. Experiments show that the PSNR of the proposed method is increased by 10dB compared to the direct FDK reconstruction and almost 3dB compared to the modified original Primal-Dual Network when using only 23 projections. The presented network is not optimized wrt. memory consumption or hyperparameters but merely serves as a proof of concept and is limited to low resolution projections and volumes.

[266]  arXiv:2205.07869 (cross-list from eess.SP) [pdf, other]
Title: Near out-of-distribution detection for low-resolution radar micro-Doppler signatures
Subjects: Signal Processing (eess.SP); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Near out-of-distribution detection (OOD) aims at discriminating semantically similar data points without the supervision required for classification. This paper puts forward an OOD use case for radar targets detection extensible to other kinds of sensors and detection scenarios. We emphasize the relevance of OOD and its specific supervision requirements for the detection of a multimodal, diverse targets class among other similar radar targets and clutter in real-life critical systems. We propose a comparison of deep and non-deep OOD methods on simulated low-resolution pulse radar micro-Doppler signatures, considering both a spectral and a covariance matrix input representation. The covariance representation aims at estimating whether dedicated second-order processing is appropriate to discriminate signatures. The potential contributions of labeled anomalies in training, self-supervised learning, contrastive learning insights and innovative training losses are discussed, and the impact of training set contamination caused by mislabelling is investigated.

[267]  arXiv:2205.07880 (cross-list from stat.ML) [pdf, ps, other]
Title: A Note on the Chernoff Bound for Random Variables in the Unit Interval
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

The Chernoff bound is a well-known tool for obtaining a high probability bound on the expectation of a Bernoulli random variable in terms of its sample average. This bound is commonly used in statistical learning theory to upper bound the generalisation risk of a hypothesis in terms of its empirical risk on held-out data, for the case of a binary-valued loss function. However, the extension of this bound to the case of random variables taking values in the unit interval is less well known in the community. In this note we provide a proof of this extension for convenience and future reference.

[268]  arXiv:2205.07888 (cross-list from eess.IV) [pdf, other]
Title: Data-Driven Interpolation for Super-Scarce X-Ray Computed Tomography
Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Medical Physics (physics.med-ph)

We address the problem of reconstructing X-Ray tomographic images from scarce measurements by interpolating missing acquisitions using a self-supervised approach. To do so, we train shallow neural networks to combine two neighbouring acquisitions into an estimated measurement at an intermediate angle. This procedure yields an enhanced sequence of measurements that can be reconstructed using standard methods, or further enhanced using regularisation approaches.
Unlike methods that improve the sequence of acquisitions using an initial deterministic interpolation followed by machine-learning enhancement, we focus on inferring one measurement at once. This allows the method to scale to 3D, the computation to be faster and crucially, the interpolation to be significantly better than the current methods, when they exist. We also establish that a sequence of measurements must be processed as such, rather than as an image or a volume. We do so by comparing interpolation and up-sampling methods, and find that the latter significantly under-perform.
We compare the performance of the proposed method against deterministic interpolation and up-sampling procedures and find that it outperforms them, even when used jointly with a state-of-the-art projection-data enhancement approach using machine-learning. These results are obtained for 2D and 3D imaging, on large biomedical datasets, in both projection space and image space.

[269]  arXiv:2205.07898 (cross-list from astro-ph.CO) [pdf, other]
Title: Fast and realistic large-scale structure from machine-learning-augmented random field simulations
Comments: 13 pages, 7 figures, comments welcome
Subjects: Cosmology and Nongalactic Astrophysics (astro-ph.CO); Instrumentation and Methods for Astrophysics (astro-ph.IM); Machine Learning (cs.LG)

Producing thousands of simulations of the dark matter distribution in the Universe with increasing precision is a challenging but critical task to facilitate the exploitation of current and forthcoming cosmological surveys. Many inexpensive substitutes to full $N$-body simulations have been proposed, even though they often fail to reproduce the statistics of the smaller, non-linear scales. Among these alternatives, a common approximation is represented by the lognormal distribution, which comes with its own limitations as well, while being extremely fast to compute even for high-resolution density fields. In this work, we train a machine learning model to transform projected lognormal dark matter density fields to more realistic dark matter maps, as obtained from full $N$-body simulations. We detail the procedure that we follow to generate highly correlated pairs of lognormal and simulated maps, which we use as our training data, exploiting the information of the Fourier phases. We demonstrate the performance of our model comparing various statistical tests with different field resolutions, redshifts and cosmological parameters, proving its robustness and explaining its current limitations. The augmented lognormal random fields reproduce the power spectrum up to wavenumbers of $1 \ h \ \rm{Mpc}^{-1}$, the bispectrum and the peak counts within 10%, and always within the error bars, of the fiducial target simulations. Finally, we describe how we plan to integrate our proposed model with existing tools to yield more accurate spherical random fields for weak lensing analysis, going beyond the lognormal approximation.

[270]  arXiv:2205.07918 (cross-list from stat.ML) [pdf, other]
Title: Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flows
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

While fat-tailed densities commonly arise as posterior and marginal distributions in robust models and scale mixtures, they present challenges when Gaussian-based variational inference fails to capture tail decay accurately. We first improve previous theory on tails of Lipschitz flows by quantifying how the tails affect the rate of tail decay and by expanding the theory to non-Lipschitz polynomial flows. Then, we develop an alternative theory for multivariate tail parameters which is sensitive to tail-anisotropy. In doing so, we unveil a fundamental problem which plagues many existing flow-based methods: they can only model tail-isotropic distributions (i.e., distributions having the same tail parameter in every direction). To mitigate this and enable modeling of tail-anisotropic targets, we propose anisotropic tail-adaptive flows (ATAF). Experimental results on both synthetic and real-world targets confirm that ATAF is competitive with prior work while also exhibiting appropriate tail-anisotropy.

[271]  arXiv:2205.07957 (cross-list from q-bio.GN) [pdf]
Title: Going From Molecules to Genomic Variations to Scientific Discovery: Intelligent Algorithms and Architectures for Intelligent Genome Analysis
Comments: arXiv admin note: text overlap with arXiv:2008.00961
Subjects: Genomics (q-bio.GN); Hardware Architecture (cs.AR); Quantitative Methods (q-bio.QM)

We now need more than ever to make genome analysis more intelligent. We need to read, analyze, and interpret our genomes not only quickly, but also accurately and efficiently enough to scale the analysis to population level. There currently exist major computational bottlenecks and inefficiencies throughout the entire genome analysis pipeline, because state-of-the-art genome sequencing technologies are still not able to read a genome in its entirety. We describe the ongoing journey in significantly improving the performance, accuracy, and efficiency of genome analysis using intelligent algorithms and hardware architectures. We explain state-of-the-art algorithmic methods and hardware-based acceleration approaches for each step of the genome analysis pipeline and provide experimental evaluations. Algorithmic approaches exploit the structure of the genome as well as the structure of the underlying hardware. Hardware-based acceleration approaches exploit specialized microarchitectures or various execution paradigms (e.g., processing inside or near memory) along with algorithmic changes, leading to new hardware/software co-designed systems. We conclude with a foreshadowing of future challenges, benefits, and research directions triggered by the development of both very low cost yet highly error prone new sequencing technologies and specialized hardware chips for genomics. We hope that these efforts and the challenges we discuss provide a foundation for future work in making genome analysis more intelligent. The analysis script and data used in our experimental evaluation are available at: https://github.com/CMU-SAFARI/Molecules2Variations

[272]  arXiv:2205.07999 (cross-list from stat.ML) [pdf, other]
Title: An Exponentially Increasing Step-size for Parameter Estimation in Statistical Models
Comments: 26 pages. The authors are listed in alphabetical order
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Optimization and Control (math.OC); Statistics Theory (math.ST)

Using gradient descent (GD) with fixed or decaying step-size is standard practice in unconstrained optimization problems. However, when the loss function is only locally convex, such a step-size schedule artificially slows GD down as it cannot explore the flat curvature of the loss function. To overcome that issue, we propose to exponentially increase the step-size of the GD algorithm. Under homogeneous assumptions on the loss function, we demonstrate that the iterates of the proposed \emph{exponential step size gradient descent} (EGD) algorithm converge linearly to the optimal solution. Leveraging that optimization insight, we then consider using the EGD algorithm for solving parameter estimation under non-regular statistical models whose the loss function becomes locally convex when the sample size goes to infinity. We demonstrate that the EGD iterates reach the final statistical radius within the true parameter after a logarithmic number of iterations, which is in stark contrast to a \emph{polynomial} number of iterations of the GD algorithm. Therefore, the total computational complexity of the EGD algorithm is \emph{optimal} and exponentially cheaper than that of the GD for solving parameter estimation in non-regular statistical models. To the best of our knowledge, it resolves a long-standing gap between statistical and algorithmic computational complexities of parameter estimation in non-regular statistical models. Finally, we provide targeted applications of the general theory to several classes of statistical models, including generalized linear models with polynomial link functions and location Gaussian mixture models.

[273]  arXiv:2205.08014 (cross-list from eess.AS) [pdf, ps, other]
Title: Accented Speech Recognition: Benchmarking, Pre-training, and Diverse Data
Comments: 5 pages, 3 tables
Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)

Building inclusive speech recognition systems is a crucial step towards developing technologies that speakers of all language varieties can use. Therefore, ASR systems must work for everybody independently of the way they speak. To accomplish this goal, there should be available data sets representing language varieties, and also an understanding of model configuration that is the most helpful in achieving robust understanding of all types of speech. However, there are not enough data sets for accented speech, and for the ones that are already available, more training approaches need to be explored to improve the quality of accented speech recognition. In this paper, we discuss recent progress towards developing more inclusive ASR systems, namely, the importance of building new data sets representing linguistic diversity, and exploring novel training approaches to improve performance for all users. We address recent directions within benchmarking ASR systems for accented speech, measure the effects of wav2vec 2.0 pre-training on accented speech recognition, and highlight corpora relevant for diverse ASR evaluations.

[274]  arXiv:2205.08038 (cross-list from math.OC) [pdf, other]
Title: Newton and interior-point methods for (constrained) nonconvex-nonconcave minmax optimization with stability guarantees
Comments: Submitted for Journal publication
Subjects: Optimization and Control (math.OC); Systems and Control (eess.SY)

We address the problem of finding a local solution to a nonconvex-nonconcave minmax optimization using Newton type methods, including interior-point ones. We modify the Hessian matrix of these methods such that, at each step, the modified Newton update direction can be seen as the solution to a quadratic program that locally approximates the minmax problem. Moreover, we show that by selecting the modification in an appropriate way, the only stable points of the algorithm's iterations are local minmax points. Using numerical examples, we show that the computation time of our algorithm scales roughly linearly with the number of nonzero elements in the Hessian. For minmax control problems with per-stage costs, this generally leads to computation times that scale linearly with the horizon length.

[275]  arXiv:2205.08047 (cross-list from stat.ML) [pdf, other]
Title: Perfect Spectral Clustering with Discrete Covariates
Comments: 23 pages, 1 figure
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Social and Information Networks (cs.SI); Statistics Theory (math.ST)

Among community detection methods, spectral clustering enjoys two desirable properties: computational efficiency and theoretical guarantees of consistency. Most studies of spectral clustering consider only the edges of a network as input to the algorithm. Here we consider the problem of performing community detection in the presence of discrete node covariates, where network structure is determined by a combination of a latent block model structure and homophily on the observed covariates. We propose a spectral algorithm that we prove achieves perfect clustering with high probability on a class of large, sparse networks with discrete covariates, effectively separating latent network structure from homophily on observed covariates. To our knowledge, our method is the first to offer a guarantee of consistent latent structure recovery using spectral clustering in the setting where edge formation is dependent on both latent and observed factors.

[276]  arXiv:2205.08055 (cross-list from q-bio.BM) [pdf]
Title: HelixADMET: a robust and endpoint extensible ADMET system incorporating self-supervised knowledge transfer
Subjects: Biomolecules (q-bio.BM); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Quantitative Methods (q-bio.QM)

Accurate ADMET (an abbreviation for "absorption, distribution, metabolism, excretion, and toxicity") predictions can efficiently screen out undesirable drug candidates in the early stage of drug discovery. In recent years, multiple comprehensive ADMET systems that adopt advanced machine learning models have been developed, providing services to estimate multiple endpoints. However, those ADMET systems usually suffer from weak extrapolation ability. First, due to the lack of labelled data for each endpoint, typical machine learning models perform frail for the molecules with unobserved scaffolds. Second, most systems only provide fixed built-in endpoints and cannot be customised to satisfy various research requirements. To this end, we develop a robust and endpoint extensible ADMET system, HelixADMET (H-ADMET). H-ADMET incorporates the concept of self-supervised learning to produce a robust pre-trained model. The model is then fine-tuned with a multi-task and multi-stage framework to transfer knowledge between ADMET endpoints, auxiliary tasks, and self-supervised tasks. Our results demonstrate that H-ADMET achieves an overall improvement of 4%, compared with existing ADMET systems on comparable endpoints. Additionally, the pre-trained model provided by H-ADMET can be fine-tuned to generate new and customised ADMET endpoints, meeting various demands of drug research and development requirements.

[277]  arXiv:2205.08059 (cross-list from quant-ph) [pdf, other]
Title: Natural evolutionary strategies applied to quantum-classical hybrid neural networks
Subjects: Quantum Physics (quant-ph); Artificial Intelligence (cs.AI)

With the rapid development of quantum computers, several applications are being proposed for them. Quantum simulations, simulation of chemical reactions, solution of optimization problems and quantum neural networks are some examples. However, problems such as noise, limited number of qubits and circuit depth, and gradient vanishing must be resolved before we can use them to their full potential. In the field of quantum machine learning, several models have been proposed. In general, in order to train these different models, we use the gradient of a cost function with respect to the model parameters. In order to obtain this gradient, we must compute the derivative of this function with respect to the model parameters. For this we can use the method called parameter-shift rule. This method consists of evaluating the cost function twice for each parameter of the quantum network. A problem with this method is that the number of evaluations grows linearly with the number of parameters. In this work we study an alternative method, called Natural Evolutionary Strategies (NES), which are a family of black box optimization algorithms. An advantage of the NES method is that in using it one can control the number of times the cost function will be evaluated. We apply the NES method to the binary classification task, showing that this method is a viable alternative for training quantum neural networks.

[278]  arXiv:2205.08068 (cross-list from eess.SP) [pdf]
Title: A Framework for CSI-Based Indoor Localization with 1D Convolutional Neural Networks
Subjects: Signal Processing (eess.SP); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Modern indoor localization techniques are essential to overcome the weak GPS coverage in indoor environments. Recently, considerable progress has been made in Channel State Information (CSI) based indoor localization with signal fingerprints. However, CSI signal patterns can be complicated in the large and highly dynamic indoor spaces with complex interiors, thus a solution for solving this issue is urgently needed to expand the applications of CSI to a broader indoor space. In this paper, we propose an end-to-end solution including data collection, pattern clustering, denoising, calibration and a lightweight one-dimensional convolutional neural network (1D CNN) model with CSI fingerprinting to tackle this problem. We have also created and plan to open source a CSI dataset with a large amount of data collected across complex indoor environments at Colorado State University. Experiments indicate that our approach achieves up to 68.5% improved performance (mean distance error) with minimal number of parameters, compared to the best-known deep machine learning and CSI-based indoor localization works.

[279]  arXiv:2205.08069 (cross-list from eess.SP) [pdf]
Title: Multi-Head Attention Neural Network for Smartphone Invariant Indoor Localization
Subjects: Signal Processing (eess.SP); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)

Smartphones together with RSSI fingerprinting serve as an efficient approach for delivering a low-cost and high-accuracy indoor localization solution. However, a few critical challenges have prevented the wide-spread proliferation of this technology in the public domain. One such critical challenge is device heterogeneity, i.e., the variation in the RSSI signal characteristics captured across different smartphone devices. In the real-world, the smartphones or IoT devices used to capture RSSI fingerprints typically vary across users of an indoor localization service. Conventional indoor localization solutions may not be able to cope with device-induced variations which can degrade their localization accuracy. We propose a multi-head attention neural network-based indoor localization framework that is resilient to device heterogeneity. An in-depth analysis of our proposed framework across a variety of indoor environments demonstrates up to 35% accuracy improvement compared to state-of-the-art indoor localization techniques.

[280]  arXiv:2205.08085 (cross-list from math.OC) [pdf, ps, other]
Title: Randomized Augmented Kaczmarz Methods For Linear Systems & Linear Feasibility Problems
Subjects: Optimization and Control (math.OC); Numerical Analysis (math.NA)

In this work, we shed light on the so-called Kaczmarz method for solving Linear System (LS) and Linear Feasibility (LF) problems from a optimization point of view. We introduce well-known optimization approaches such as Lagrangian penalty and Augmented Lagrangian in the Randomized Kaczmarz (RK) method. In doing so, we propose two variants of the RK method namely the Randomized Penalty Kacmarz (RPK) method and Randomized Augmented Kacmarz (RAK) method. We carry out convergence analysis of the proposed methods and obtain linear convergence results.

[281]  arXiv:2205.08106 (cross-list from eess.IV) [pdf]
Title: Computerized Tomography Pulmonary Angiography Image Simulation using Cycle Generative Adversarial Network from Chest CT imaging in Pulmonary Embolism Patients
Comments: 23 pages, 14 figures, 6 tables
Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

The purpose of this research is to develop a system that generates simulated computed tomography pulmonary angiography (CTPA) images clinically for pulmonary embolism diagnoses. Nowadays, CTPA images are the gold standard computerized detection method to determine and identify the symptoms of pulmonary embolism (PE), although performing CTPA is harmful for patients and also expensive. Therefore, we aim to detect possible PE patients through CT images. The system will simulate CTPA images with deep learning models for the identification of PE patients' symptoms, providing physicians with another reference for determining PE patients. In this study, the simulated CTPA image generation system uses a generative antagonistic network to enhance the features of pulmonary vessels in the CT images to strengthen the reference value of the images and provide a basis for hospitals to judge PE patients. We used the CT images of 22 patients from National Cheng Kung University Hospital and the corresponding CTPA images as the training data for the task of simulating CTPA images and generated them using two sets of generative countermeasure networks. This study is expected to propose a new approach to the clinical diagnosis of pulmonary embolism, in which a deep learning network is used to assist in the complex screening process and to review the generated simulated CTPA images, allowing physicians to assess whether a patient needs to undergo detailed testing for CTPA, improving the speed of detection of pulmonary embolism and significantly reducing the number of undetected patients.

[282]  arXiv:2205.08112 (cross-list from econ.GN) [pdf, ps, other]
Title: The Fairness of Machine Learning in Insurance: New Rags for an Old Man?
Subjects: General Economics (econ.GN); Computers and Society (cs.CY)

Since the beginning of their history, insurers have been known to use data to classify and price risks. As such, they were confronted early on with the problem of fairness and discrimination associated with data. This issue is becoming increasingly important with access to more granular and behavioural data, and is evolving to reflect current technologies and societal concerns. By looking into earlier debates on discrimination, we show that some algorithmic biases are a renewed version of older ones, while others show a reversal of the previous order. Paradoxically, while the insurance practice has not deeply changed nor are most of these biases new, the machine learning era still deeply shakes the conception of insurance fairness.

[283]  arXiv:2205.08123 (cross-list from eess.IV) [pdf]
Title: Using artificial intelligence to detect chest X-rays with no significant findings in a primary health care setting in Oulu, Finland
Comments: Abstract #21318 - ECR 2022 oral presentations
Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Medical Physics (physics.med-ph)

Objectives: To assess the use of artificial intelligence-based software in ruling out chest X-ray cases, with no significant findings in a primary health care setting.
Methods: In this retrospective study, a commercially available artificial intelligence (AI) software was used to analyse 10 000 chest X-rays of Finnish primary health care patients. In studies with a mismatch between an AI normal report and the original radiologist report, a consensus read by two board-certified radiologists was conducted to make the final diagnosis.
Results: After the exclusion of cases not meeting the study criteria, 9579 cases were analysed by AI. Of these cases, 4451 were considered normal in the original radiologist report and 4644 after the consensus reading. The number of cases correctly found nonsignificant by AI was 1692 (17.7% of all studies and 36.4% of studies with no significant findings). After the consensus read, there were nine confirmed false-negative studies. These studies included four cases of slightly enlarged heart size, four cases of slightly increased pulmonary opacification and one case with a small unilateral pleural effusion. This gives the AI a sensitivity of 99.8% (95% CI= 99.65-99.92) and specificity of 36.4 % (95% CI= 35.05-37.84) for recognising significant pathology on a chest X-ray.
Conclusions: AI was able to correctly rule out 36.4% of chest X-rays with no significant findings of primary health care patients, with a minimal number of false negatives that would lead to effectively no compromise on patient safety. No critical findings were missed by the software.

[284]  arXiv:2205.08126 (cross-list from math.CO) [pdf, other]
Title: The Hamilton compression of highly symmetric graphs
Subjects: Combinatorics (math.CO); Discrete Mathematics (cs.DM)

We say that a Hamilton cycle $C=(x_1,\ldots,x_n)$ in a graph $G$ is $k$-symmetric, if the mapping $x_i\mapsto x_{i+n/k}$ for all $i=1,\ldots,n$, where indices are considered modulo $n$, is an automorphism of $G$. In other words, if we lay out the vertices $x_1,\ldots,x_n$ equidistantly on a circle and draw the edges of $G$ as straight lines, then the drawing of $G$ has $k$-fold rotational symmetry, i.e., all information about the graph is compressed into a $360^\circ/k$ wedge of the drawing. We refer to the maximum $k$ for which there exists a $k$-symmetric Hamilton cycle in $G$ as the Hamilton compression of $G$. We investigate the Hamilton compression of four different families of vertex-transitive graphs, namely hypercubes, Johnson graphs, permutahedra and Cayley graphs of abelian groups. In several cases we determine their Hamilton compression exactly, and in other cases we provide close lower and upper bounds. The cycles we construct have a much higher compression than several classical Gray codes known from the literature. Our constructions also yield Gray codes for bitstrings, combinations and permutations that have few tracks and/or that are balanced.

[285]  arXiv:2205.08127 (cross-list from physics.flu-dyn) [pdf, other]
Title: Isogeometric Hierarchical Model Reduction for advection-diffusion process simulation in microchannels
Comments: 17 pages
Subjects: Fluid Dynamics (physics.flu-dyn); Numerical Analysis (math.NA)

Microfluidics proved to be a key technology in various applications, allowing to reproduce large-scale laboratory settings at a more sustainable small-scale. The current effort is focused on enhancing the mixing process of different passive species at the micro-scale, where a laminar flow regime damps turbulence effects. Chaotic advection is often used to improve mixing effects also at very low Reynolds numbers. In particular, we focus on passive micromixers, where chaotic advection is mainly achieved by properly selecting the geometry of microchannels. In such a context, reduced order modeling can play a role, especially in the design of new geometries. In this chapter, we verify the reliability and the computational benefits lead by a Hierarchical Model (HiMod) reduction when modeling the transport of a passive scalar in an S-shaped microchannel. Such a geometric configuration provides an ideal setting where to apply a HiMod approximation, which exploits the presence of a leading dynamics to commute the original three-dimensional model into a system of one-dimensional coupled problems. It can be proved that HiMod reduction guarantees a very good accuracy when compared with a high-fidelity model, despite a drastic reduction in terms of number of unknowns.

[286]  arXiv:2205.08132 (cross-list from stat.ML) [pdf, other]
Title: Latent Variable Method Demonstrator -- Software for Understanding Multivariate Data Analytics Algorithms
Comments: 18 pages, 14 figures, code available: this https URL, preprint submitted to Computers & Chemical Engineering
Subjects: Machine Learning (stat.ML); Computers and Society (cs.CY); Machine Learning (cs.LG)

The ever-increasing quantity of multivariate process data is driving a need for skilled engineers to analyze, interpret, and build models from such data. Multivariate data analytics relies heavily on linear algebra, optimization, and statistics and can be challenging for students to understand given that most curricula do not have strong coverage in the latter three topics. This article describes interactive software -- the Latent Variable Demonstrator (LAVADE) -- for teaching, learning, and understanding latent variable methods. In this software, users can interactively compare latent variable methods such as Partial Least Squares (PLS), and Principal Component Regression (PCR) with other regression methods such as Least Absolute Shrinkage and Selection Operator (lasso), Ridge Regression (RR), and Elastic Net (EN). LAVADE helps to build intuition on choosing appropriate methods, hyperparameter tuning, and model coefficient interpretation, fostering a conceptual understanding of the algorithms' differences. The software contains a data generation method and three chemical process datasets, allowing for comparing results of datasets with different levels of complexity. LAVADE is released as open-source software so that others can apply and advance the tool for use in teaching or research.

[287]  arXiv:2205.08138 (cross-list from eess.AS) [pdf, ps, other]
Title: Composing General Audio Representation by Fusing Multilayer Features of a Pre-trained Model
Comments: 5 pages, 4 figures and 4 tables. Accepted by EUSIPCO 2022
Subjects: Audio and Speech Processing (eess.AS); Sound (cs.SD)

Many application studies rely on audio DNN models pre-trained on a large-scale dataset as essential feature extractors, and they extract features from the last layers. In this study, we focus on our finding that the middle layer features of existing supervised pre-trained models are more effective than the late layer features for some tasks. We propose a simple approach to compose features effective for general-purpose applications, consisting of two steps: (1) calculating feature vectors along the time frame from middle/late layer outputs, and (2) fusing them. This approach improves the utility of frequency and channel information in downstream processes, and combines the effectiveness of middle and late layer features for different tasks. As a result, the feature vectors become effective for general purposes. In the experiments using VGGish, PANNs' CNN14, and AST on nine downstream tasks, we first show that each layer output of these models serves different tasks. Then, we demonstrate that the proposed approach significantly improves their performance and brings it to a level comparable to that of the state-of-the-art. In particular, the performance of the non-semantic speech (NOSS) tasks greatly improves, especially on Speech commands V2 with VGGish of +77.1 (14.3% to 91.4%).

[288]  arXiv:2205.08143 (cross-list from eess.IV) [pdf, other]
Title: Brachial Plexus Nerve Trunk Segmentation Using Deep Learning: A Comparative Study with Doctors' Manual Segmentation
Comments: 9 pages
Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

Ultrasound-guided nerve block anesthesia (UGNB) is a high-tech visual nerve block anesthesia method that can observe the target nerve and its surrounding structures, the puncture needle's advancement, and local anesthetics spread in real-time. The key in UGNB is nerve identification. With the help of deep learning methods, the automatic identification or segmentation of nerves can be realized, assisting doctors in completing nerve block anesthesia accurately and efficiently. Here, we establish a public dataset containing 320 ultrasound images of brachial plexus (BP). Three experienced doctors jointly produce the BP segmentation ground truth and label brachial plexus trunks. We design a brachial plexus segmentation system (BPSegSys) based on deep learning. BPSegSys achieves experienced-doctor-level nerve identification performance in various experiments. We evaluate BPSegSys' performance in terms of intersection-over-union (IoU), a commonly used performance measure for segmentation experiments. Considering three dataset groups in our established public dataset, the IoU of BPSegSys are 0.5238, 0.4715, and 0.5029, respectively, which exceed the IoU 0.5205, 0.4704, and 0.4979 of experienced doctors. In addition, we show that BPSegSys can help doctors identify brachial plexus trunks more accurately, with IoU improvement up to 27%, which has significant clinical application value.

[289]  arXiv:2205.08176 (cross-list from math.OC) [pdf, other]
Title: On the Convergence of Policy in Unregularized Generalized Policy Mirror Descent
Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG)

In this short note, we give the convergence analysis of the policy in the recent famous policy mirror descent (PMD). We mainly consider the unregularized setting following [11] with generalized Bregman divergence. The difference is that we directly give the convergence rates of policy under generalized Bregman divergence. Our results are inspired by the convergence of value function in previous works and are an extension study of policy mirror descent. Though some results have already appeared in previous work, we further discover a large body of Bregman divergences could give finite-step convergence to an optimal policy, such as the classical Euclidean distance.

[290]  arXiv:2205.08181 (cross-list from math.CO) [pdf, other]
Title: Coloring circle arrangements: New $4$-chromatic planar graphs
Comments: 21 pages, 15 figures. An extended abstract of this work has appeared in the proceedings of EUROCOMB 2021
Subjects: Combinatorics (math.CO); Computational Geometry (cs.CG); Discrete Mathematics (cs.DM)

Felsner, Hurtado, Noy and Streinu (2000) conjectured that arrangement graphs of simple great-circle arrangements have chromatic number at most $3$. Motivated by this conjecture, we study the colorability of arrangement graphs for different classes of arrangements of (pseudo-)circles.
In this paper the conjecture is verified for $\triangle$-saturated pseudocircle arrangements, i.e., for arrangements where one color class of the 2-coloring of faces consists of triangles only, as well as for further classes of (pseudo-)circle arrangements. These results are complemented by a construction which maps $\triangle$-saturated arrangements with a pentagonal face to arrangements with 4-chromatic 4-regular arrangement graphs. This "corona" construction has similarities with the crowning construction introduced by Koester (1985). Based on exhaustive experiments with small arrangements we propose three strengthenings of the original conjecture.
We also investigate fractional colorings. It is shown that the arrangement graph of every arrangement $\mathcal{A}$ of pairwise intersecting pseudocircles is "close" to being $3$-colorable. More precisely, the fractional chromatic number $\chi_f(\mathcal{A})$ of the arrangement graph is bounded from above by $\chi_f(\mathcal{A}) \le 3+O(\frac{1}{n})$, where $n$ is the number of pseudocircles of $\mathcal{A}$. Furthermore, we construct an infinite family of $4$-edge-critical $4$-regular planar graphs which are fractionally $3$-colorable. This disproves a conjecture of Gimbel, K\"{u}ndgen, Li, and Thomassen (2019).

[291]  arXiv:2205.08187 (cross-list from stat.ML) [pdf, other]
Title: Deep neural networks with dependent weights: Gaussian Process mixture limit, heavy tails, sparsity and compressibility
Comments: 89 pages, 11 figures, 7 tables
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Probability (math.PR); Statistics Theory (math.ST)

This article studies the infinite-width limit of deep feedforward neural networks whose weights are dependent, and modelled via a mixture of Gaussian distributions. Each hidden node of the network is assigned a nonnegative random variable that controls the variance of the outgoing weights of that node. We make minimal assumptions on these per-node random variables: they are iid and their sum, in each layer, converges to some finite random variable in the infinite-width limit. Under this model, we show that each layer of the infinite-width neural network can be characterised by two simple quantities: a non-negative scalar parameter and a L\'evy measure on the positive reals. If the scalar parameters are strictly positive and the L\'evy measures are trivial at all hidden layers, then one recovers the classical Gaussian process (GP) limit, obtained with iid Gaussian weights. More interestingly, if the L\'evy measure of at least one layer is non-trivial, we obtain a mixture of Gaussian processes (MoGP) in the large-width limit. The behaviour of the neural network in this regime is very different from the GP regime. One obtains correlated outputs, with non-Gaussian distributions, possibly with heavy tails. Additionally, we show that, in this regime, the weights are compressible, and feature learning is possible. Many sparsity-promoting neural network models can be recast as special cases of our approach, and we discuss their infinite-width limits; we also present an asymptotic analysis of the pruning error. We illustrate some of the benefits of the MoGP regime over the GP regime in terms of representation learning and compressibility on simulated, MNIST and Fashion MNIST datasets.

[292]  arXiv:2205.08193 (cross-list from physics.comp-ph) [pdf, ps, other]
Title: The HEP Software Foundation Community
Authors: Graeme A Stewart, Peter Elmer, Elizabeth Sexton-Kennedy (for the HEP Software Foundation)
Subjects: Computational Physics (physics.comp-ph); Software Engineering (cs.SE)

The HEP Software Foundation was founded in 2014 to tackle common problems of software development and sustainability for high-energy physics. In this paper we outline the motivation for the founding of the organisation and give a brief history of its development. We describe how the organisation functions today and what challenges remain to be faced in the future.

[293]  arXiv:2205.08194 (cross-list from math.OC) [pdf, ps, other]
Title: Design of saturated boundary control for hyperbolic systems with in-domain disturbances
Comments: V1 matches the printed version of the paper published in Automatica
Journal-ref: Automatica, Volume 142, 2022
Subjects: Optimization and Control (math.OC); Systems and Control (eess.SY)

Boundary feedback control design is studied for 1D hyperbolic systems with an in-domain disturbance and a boundary feedback controller under the effect of actuator saturation. Nonlinear semigroup theory is used to prove well-posedness of mild solution pairs to the closed-loop system. Sufficient conditions in the form of dissipation functional inequalities are derived to establish global stability for the closed-loop system and $\mathcal{L}^2$-stability in presence of in-domain disturbances. The control design problem is then recast as an optimization problem over linear matrix inequality constraints. Numerical results are shown to validate the effectiveness of the proposed control design.

[294]  arXiv:2205.08225 (cross-list from eess.SP) [pdf, other]
Title: ROP inception: signal estimation with quadratic random sketching
Comments: 9 pages, 3 figures; part of this work has been submitted to the ESANN 2022 conference; this version contains additional proofs
Subjects: Signal Processing (eess.SP); Machine Learning (cs.LG)

Rank-one projections (ROP) of matrices and quadratic random sketching of signals support several data processing and machine learning methods, as well as recent imaging applications, such as phase retrieval or optical processing units. In this paper, we demonstrate how signal estimation can be operated directly through such quadratic sketches--equivalent to the ROPs of the "lifted signal" obtained as its outer product with itself--without explicitly reconstructing that signal. Our analysis relies on showing that, up to a minor debiasing trick, the ROP measurement operator satisfies a generalised sign product embedding (SPE) property. In a nutshell, the SPE shows that the scalar product of a signal sketch with the "sign" of the sketch of a given pattern approximates the square of the projection of that signal on this pattern. This thus amounts to an insertion (an "inception") of a ROP model inside a ROP sketch. The effectiveness of our approach is evaluated in several synthetic experiments.

[295]  arXiv:2205.08239 (cross-list from eess.IV) [pdf, other]
Title: CAS-Net: Conditional Atlas Generation and Brain Segmentation for Fetal MRI
Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)

Fetal Magnetic Resonance Imaging (MRI) is used in prenatal diagnosis and to assess early brain development. Accurate segmentation of the different brain tissues is a vital step in several brain analysis tasks, such as cortical surface reconstruction and tissue thickness measurements. Fetal MRI scans, however, are prone to motion artifacts that can affect the correctness of both manual and automatic segmentation techniques. In this paper, we propose a novel network structure that can simultaneously generate conditional atlases and predict brain tissue segmentation, called CAS-Net. The conditional atlases provide anatomical priors that can constrain the segmentation connectivity, despite the heterogeneity of intensity values caused by motion or partial volume effects. The proposed method is trained and evaluated on 253 subjects from the developing Human Connectome Project (dHCP). The results demonstrate that the proposed method can generate conditional age-specific atlas with sharp boundary and shape variance. It also segment multi-category brain tissues for fetal MRI with a high overall Dice similarity coefficient (DSC) of $85.2\%$ for the selected 9 tissue labels.

[296]  arXiv:2205.08278 (cross-list from eess.IV) [pdf, other]
Title: Multiscale reconstruction of porous media based on multiple dictionaries learning
Subjects: Image and Video Processing (eess.IV); Machine Learning (cs.LG)

Digital modeling of the microstructure is important for studying the physical and transport properties of porous media. Multiscale modeling for porous media can accurately characterize macro-pores and micro-pores in a large-FoV (field of view) high-resolution three-dimensional pore structure model. This paper proposes a multiscale reconstruction algorithm based on multiple dictionaries learning, in which edge patterns and micro-pore patterns from homology high-resolution pore structure are introduced into low-resolution pore structure to build a fine multiscale pore structure model. The qualitative and quantitative comparisons of the experimental results show that the results of multiscale reconstruction are similar to the real high-resolution pore structure in terms of complex pore geometry and pore surface morphology. The geometric, topological and permeability properties of multiscale reconstruction results are almost identical to those of the real high-resolution pore structures. The experiments also demonstrate the proposal algorithm is capable of multiscale reconstruction without regard to the size of the input. This work provides an effective method for fine multiscale modeling of porous media.

[297]  arXiv:2205.08283 (cross-list from physics.med-ph) [pdf, other]
Title: Towards the optimization of ballistics in proton therapy using genetic algorithms: implementation issues
Comments: 18 pages, 4 figures
Subjects: Medical Physics (physics.med-ph); Neural and Evolutionary Computing (cs.NE)

The dose delivered to the planning target volume by proton beams is highly conformal, sparing organs at risk and normal tissues. New treatment planning systems adapted to spot scanning techniques have been recently proposed to simultaneously optimize several fields and thus improve dose delivery. In this paper, we investigate a new optimization framework based on a genetic algorithm approach. This tool is intended to make it possible to explore new schemes of treatment delivery, possibly with future enhanced technologies. The optimization framework is designed to be versatile and to account for many degrees of freedom, without any {\it a priori} technological constraint. To test the behavior of our algorithm, we propose in this paper, as an example, to optimize beam fluences, target points and irradiation directions at the same time.
The proposed optimization routine takes typically into account several thousands of spots of fixed size. The evolution is carried out by the three standard genetic operators: mutation, crossover and selection. The figure-of-merit (or fitness) is based on an objective function relative to the dose prescription to the tumor and to the limits set for organs at risk and normal tissues. Fluence optimization is carried out via a specific scheme based on a plain gradient with analytical solution. Several specific genetic algorithm issues are addressed: (i) the mutation rate is tuned to balance the search and selection forces, (ii) the initial population is selected using a bootstrap technique and (iii) to scale down the computation time, dose calculations are carried out with a fast analytical ray tracing method and are multi-threaded.
In this paper implementation issues of the optimization framework are thoroughly described. The behavior of the proposed genetic algorithm is illustrated in both elementary and clinically-realistic test cases.

[298]  arXiv:2205.08293 (cross-list from math.PR) [pdf, ps, other]
Title: Moments, Concentration, and Entropy of Log-Concave Distributions
Comments: 22 pages
Subjects: Probability (math.PR); Information Theory (cs.IT)

We utilize and extend a simple and classical mechanism, combining log-concavity and majorization in the convex order to derive moment, concentration, and entropy inequalities for log-concave random variables with respect to a reference measure.

[299]  arXiv:2205.08295 (cross-list from stat.ML) [pdf, other]
Title: Semi-Parametric Contextual Bandits with Graph-Laplacian Regularization
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

Non-stationarity is ubiquitous in human behavior and addressing it in the contextual bandits is challenging. Several works have addressed the problem by investigating semi-parametric contextual bandits and warned that ignoring non-stationarity could harm performances. Another prevalent human behavior is social interaction which has become available in a form of a social network or graph structure. As a result, graph-based contextual bandits have received much attention. In this paper, we propose "SemiGraphTS," a novel contextual Thompson-sampling algorithm for a graph-based semi-parametric reward model. Our algorithm is the first to be proposed in this setting. We derive an upper bound of the cumulative regret that can be expressed as a multiple of a factor depending on the graph structure and the order for the semi-parametric model without a graph. We evaluate the proposed and existing algorithms via simulation and real data example.

[300]  arXiv:2205.08306 (cross-list from physics.chem-ph) [pdf, other]
Title: Accurate Machine Learned Quantum-Mechanical Force Fields for Biomolecular Simulations
Subjects: Chemical Physics (physics.chem-ph); Machine Learning (cs.LG); Biomolecules (q-bio.BM)

Molecular dynamics (MD) simulations allow atomistic insights into chemical and biological processes. Accurate MD simulations require computationally demanding quantum-mechanical calculations, being practically limited to short timescales and few atoms. For larger systems, efficient, but much less reliable empirical force fields are used. Recently, machine learned force fields (MLFFs) emerged as an alternative means to execute MD simulations, offering similar accuracy as ab initio methods at orders-of-magnitude speedup. Until now, MLFFs mainly capture short-range interactions in small molecules or periodic materials, due to the increased complexity of constructing models and obtaining reliable reference data for large molecules, where long-ranged many-body effects become important. This work proposes a general approach to constructing accurate MLFFs for large-scale molecular simulations (GEMS) by training on "bottom-up" and "top-down" molecular fragments of varying size, from which the relevant physicochemical interactions can be learned. GEMS is applied to study the dynamics of alanine-based peptides and the 46-residue protein crambin in aqueous solution, allowing nanosecond-scale MD simulations of >25k atoms at essentially ab initio quality. Our findings suggest that structural motifs in peptides and proteins are more flexible than previously thought, indicating that simulations at ab initio accuracy might be necessary to understand dynamic biomolecular processes such as protein (mis)folding, drug-protein binding, or allosteric regulation.

[301]  arXiv:2205.08340 (cross-list from stat.ML) [pdf, other]
Title: A unified framework for dataset shift diagnostics
Subjects: Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Methodology (stat.ME)

Most machine learning (ML) methods assume that the data used in the training phase comes from the distribution of the target population. However, in practice one often faces dataset shift, which, if not properly taken into account, may decrease the predictive performance of the ML models. In general, if the practitioner knows which type of shift is taking place - e.g., covariate shift or label shift - they may apply transfer learning methods to obtain better predictions. Unfortunately, current methods for detecting shift are only designed to detect specific types of shift or cannot formally test their presence. We introduce a general framework that gives insights on how to improve prediction methods by detecting the presence of different types of shift and quantifying how strong they are. Our approach can be used for any data type (tabular/image/text) and both for classification and regression tasks. Moreover, it uses formal hypotheses tests that controls false alarms. We illustrate how our framework is useful in practice using both artificial and real datasets. Our package for dataset shift detection can be found in https://github.com/felipemaiapolo/detectshift.

[302]  arXiv:2205.08349 (cross-list from stat.ML) [pdf, other]
Title: Topological Signal Processing using the Weighted Ordinal Partition Network
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Signal Processing (eess.SP)

One of the most important problems arising in time series analysis is that of bifurcation, or change point detection. That is, given a collection of time series over a varying parameter, when has the structure of the underlying dynamical system changed? For this task, we turn to the field of topological data analysis (TDA), which encodes information about the shape and structure of data. The idea of utilizing tools from TDA for signal processing tasks, known as topological signal processing (TSP), has gained much attention in recent years, largely through a standard pipeline that computes the persistent homology of the point cloud generated by the Takens' embedding. However, this procedure is limited by computation time since the simplicial complex generated in this case is large, but also has a great deal of redundant data. For this reason, we turn to a more recent method for encoding the structure of the attractor, which constructs an ordinal partition network (OPN) representing information about when the dynamical system has passed between certain regions of state space. The result is a weighted graph whose structure encodes information about the underlying attractor. Our previous work began to find ways to package the information of the OPN in a manner that is amenable to TDA; however, that work only used the network structure and did nothing to encode the additional weighting information. In this paper, we take the next step: building a pipeline to analyze the weighted OPN with TDA and showing that this framework provides more resilience to noise or perturbations in the system and improves the accuracy of the dynamic state detection.

[303]  arXiv:2205.08353 (cross-list from econ.TH) [pdf, ps, other]
Title: A General Framework for a Class of Quarrels: The Quarrelling Paradox Revisited
Subjects: Theoretical Economics (econ.TH); Computer Science and Game Theory (cs.GT)

If a measure of voting power assigns greater voting power to a player because it no longer effectively cooperates with another, then the measure displays the quarrelling paradox and violates the quarrel postulate. We provide formal criteria by which to judge whether a given conception of quarrelling is (a) reasonable and (b) fit to serve as the basis for a reasonable quarrel postulate. To achieve this, we formalize a general framework distinguishing between three degrees of quarrelling (weak, strong, cataclysmic), symmetric vs. asymmetrical quarrels, and reciprocal vs. non-reciprocal quarrels, and which thereby yields twelve conceptions of quarrelling, which encompasses the two conceptions proposed by Felsenthal and Machover and by Laruelle and Valenciano, respectively. We argue that the two existing formulations of the quarrel postulate based on these conceptions are unreasonable. In contrast, we prove that the symmetric, weak conception of quarrelling identified by our framework -- whether reciprocal or not -- is fit to serve as the basis for a reasonable quarrel postulate. Furthermore, the classic Shapley-Shubik index and Penrose-Banzhaf measure both satisfy the quarrel postulate based on a symmetric weak quarrel.

[304]  arXiv:2205.08387 (cross-list from physics.flu-dyn) [pdf, other]
Title: cfd.xyz/rom.js: An open-source framework for generating and visualizing parametric CFD results on the web
Comments: This article has been submitted for publication in OpenFOAM Journal on 2022-03-16
Subjects: Fluid Dynamics (physics.flu-dyn); Numerical Analysis (math.NA)

We present in this technical note an open-source web framework for the generation and visualization of parametric CFD results from surrogate models. It consists of a JavaScript module (rom.js) and a React JS web app (cfd.xyz) to explore fluid dynamics problems efficiently and easily for a wide range of parameters. rom.js is a JavaScript port of a set of open-source packages (Eigen, Splinter, VTK/C++ and ITHACA-FV) to solve the online stage of reduced-order models (ROM) generated by the ITHACA-FV tool. It can be executed outside a web browser within a backend JavaScript runtime environment, or in a given web solution. This methodology can also be extended to methods using machine learning. The rom.js module was used in cfd.xyz, an open-source web service to deliver a collection of interactive CFD cases in a parametric space. The framework provides a proof of technology for OpenFOAM tutorials, showing the whole process from the generation of the surrogate model to the web browser. It also includes a standalone web tool for visualizing users' ROMs by directly dragging and dropping the output folder of the offline stage. Beyond the current proof of technology, this enables a collaborative effort for the implementation of OpenFOAM surrogate models in applications demanding real-time solutions such as digital twins and other digital transformation technologies.

[305]  arXiv:2205.08390 (cross-list from eess.IV) [pdf, other]
Title: HoVer-Trans: Anatomy-aware HoVer-Transformer for ROI-free Breast Cancer Diagnosis in Ultrasound Images
Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)

Ultrasonography is an important routine examination for breast cancer diagnosis, due to its non-invasive, radiation-free and low-cost properties. However, it is still not the first-line screening test for breast cancer due to its inherent limitations. It would be a tremendous success if we can precisely diagnose breast cancer by breast ultrasound images (BUS). Many learning-based computer-aided diagnostic methods have been proposed to achieve breast cancer diagnosis/lesion classification. However, most of them require a pre-define ROI and then classify the lesion inside the ROI. Conventional classification backbones, such as VGG16 and ResNet50, can achieve promising classification results with no ROI requirement. But these models lack interpretability, thus restricting their use in clinical practice. In this study, we propose a novel ROI-free model for breast cancer diagnosis in ultrasound images with interpretable feature representations. We leverage the anatomical prior knowledge that malignant and benign tumors have different spatial relationships between different tissue layers, and propose a HoVer-Transformer to formulate this prior knowledge. The proposed HoVer-Trans block extracts the inter- and intra-layer spatial information horizontally and vertically. We conduct and release an open dataset GDPH&GYFYY for breast cancer diagnosis in BUS. The proposed model is evaluated in three datasets by comparing with four CNN-based models and two vision transformer models via a five-fold cross validation. It achieves state-of-the-art classification performance with the best model interpretability.

[306]  arXiv:2205.08398 (cross-list from physics.med-ph) [pdf, other]
Title: Variable length genetic algorithm with continuous parameters optimization of beam layout in proton therapy
Comments: 16 pages, 8 figures
Subjects: Medical Physics (physics.med-ph); Neural and Evolutionary Computing (cs.NE)

Proton therapy is a modality in fast development. Characterized by a maximum dose deposition at the end of the proton trajectory followed by a sharp fall-off, proton beams can deliver a highly conformal dose to the tumor while sparing organs at risk and surrounding healthy tissues. New treatment planning systems based on spot scanning techniques can now propose multi-field optimization. However, in most cases, this optimization only processes the field fluences whereas the choice of ballistics (field geometry) is left to the oncologist and medical physicist.
In this work, we investigate a new optimization framework based on a genetic approach. This tool is intended to explore new irradiation schemes and to evaluate the potential of actual or future irradiation systems. We propose to optimize simultaneously the target points and beam incidence angles in a continuous manner and with a variable number of beams. No \textit{a priori} technological constraints are taken into account, \textit{i.e.}~the beam energy values, incidence directions and target points are free parameters.
The proposed algorithm is based on a modified version of classical genetic operators: mutation, crossover and selection. We use the real coding associated with random perturbations of the parameters to obtain a continuous variation of the potential solutions. We also introduce a perturbation in the exchange points of the crossover to allow variations of the number of beams. These variations are controlled by introducing a beam fluence lower limit.
In this paper, we present a complete description of the algorithm and of its behaviour in an elementary test case. The proposed method is finally assessed in a clinically-realistic test case.

[307]  arXiv:2205.08409 (cross-list from eess.SP) [pdf, other]
Title: Automated Mobility Context Detection with Inertial Signals
Subjects: Signal Processing (eess.SP); Machine Learning (cs.LG)

Remote monitoring of motor functions is a powerful approach for health assessment, especially among the elderly population or among subjects affected by pathologies that negatively impact their walking capabilities. This is further supported by the continuous development of wearable sensor devices, which are getting progressively smaller, cheaper, and more energy efficient. The external environment and mobility context have an impact on walking performance, hence one of the biggest challenges when remotely analysing gait episodes is the ability to detect the context within which those episodes occurred. The primary goal of this paper is the investigation of context detection for remote monitoring of daily motor functions. We aim to understand whether inertial signals sampled with wearable accelerometers, provide reliable information to classify gait-related activities as either indoor or outdoor. We explore two different approaches to this task: (1) using gait descriptors and features extracted from the input inertial signals sampled during walking episodes, together with classic machine learning algorithms, and (2) treating the input inertial signals as time series data and leveraging end-to-end state-of-the-art time series classifiers. We directly compare the two approaches through a set of experiments based on data collected from 9 healthy individuals. Our results indicate that the indoor/outdoor context can be successfully derived from inertial data streams. We also observe that time series classification models achieve better accuracy than any other feature-based models, while preserving efficiency and ease of use.

[308]  arXiv:2205.08417 (cross-list from physics.med-ph) [pdf, other]
Title: KiT-RT: An extendable framework for radiative transfer and therapy
Comments: 28 pages, 15 figures, journal submission
Subjects: Medical Physics (physics.med-ph); Mathematical Software (cs.MS)

In this paper we present KiT-RT (Kinetic Transport Solver for Radiation Therapy), an open-source C++ based framework for solving kinetic equations in radiation therapy applications. The aim of this code framework is to provide a collection of classical deterministic solvers for unstructured meshes that allow for easy extendability. Therefore, KiT-RT is a convenient base to test new numerical methods in various applications and compare them against conventional solvers. The implementation includes spherical-harmonics, minimal entropy, neural minimal entropy and discrete ordinates methods. Solution characteristics and efficiency are presented through several test cases ranging from radiation transport to electron radiation therapy. Due to the variety of included numerical methods and easy extendability, the presented open source code is attractive for both developers, who want a basis to build their own numerical solvers and users or application engineers, who want to gain experimental insights without directly interfering with the codebase.

[309]  arXiv:2205.08418 (cross-list from eess.SP) [pdf]
Title: Fault Detection for Non-Condensing Boilers using Simulated Building Automation System Sensor Data
Authors: Rony Shohet, Mohamed Kandil (1), J.J. McArthur (1), ((1) Department Architectural Science, Ryerson University, Toronto, Canada)
Comments: 41 pages, 55106 words
Subjects: Signal Processing (eess.SP); Machine Learning (cs.LG); Systems and Control (eess.SY); Applications (stat.AP)

Building performance has been shown to degrade significantly after commissioning, resulting in increased energy consumption and associated greenhouse gas emissions. Continuous Commissioning using existing sensor networks and IoT devices has the potential to minimize this waste by continually identifying system degradation and re-tuning control strategies to adapt to real building performance. Due to its significant contribution to greenhouse gas emissions, the performance of gas boiler systems for building heating is critical. A review of boiler performance studies has been used to develop a set of common faults and degraded performance conditions, which have been integrated into a MATLAB/Simulink emulator. This resulted in a labeled dataset with approximately 10,000 simulations of steady-state performance for each of 14 non-condensing boilers. The collected data is used for training and testing fault classification using K-nearest neighbour, Decision tree, Random Forest, and Support Vector Machines. The results show that the Support Vector Machines method gave the best prediction accuracy, consistently exceeding 90%, and generalization across multiple boilers is not possible due to low classification accuracy.

[310]  arXiv:2205.08419 (cross-list from eess.SP) [pdf, other]
Title: Human Emotion Classification based on EEG Signals Using Recurrent Neural Network And KNN
Subjects: Signal Processing (eess.SP); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)

In human contact, emotion is very crucial. Attributes like words, voice intonation, facial expressions, and kinesics can all be used to portray one's feelings. However, brain-computer interface (BCI) devices have not yet reached the level required for emotion interpretation. With the rapid development of machine learning algorithms, dry electrode techniques, and different real-world applications of the brain-computer interface for normal individuals, emotion categorization from EEG data has recently gotten a lot of attention. Electroencephalogram (EEG) signals are a critical resource for these systems. The primary benefit of employing EEG signals is that they reflect true emotion and are easily resolved by computer systems. In this work, EEG signals associated with good, neutral, and negative emotions were identified using channel selection preprocessing. However, researchers had a limited grasp of the specifics of the link between various emotional states until now. To identify EEG signals, we used discrete wavelet transform and machine learning techniques such as recurrent neural network (RNN) and k-nearest neighbor (kNN) algorithm. Initially, the classifier methods were utilized for channel selection. As a result, final feature vectors were created by integrating the features of EEG segments from these channels. Using the RNN and kNN algorithms, the final feature vectors with connected positive, neutral, and negative emotions were categorized independently. The classification performance of both techniques is computed and compared. Using RNN and kNN, the average overall accuracies were 94.844 % and 93.438 %, respectively.

[311]  arXiv:2205.08422 (cross-list from eess.SP) [pdf, other]
Title: JUNO: Jump-Start Reinforcement Learning-based Node Selection for UWB Indoor Localization
Subjects: Signal Processing (eess.SP); Machine Learning (cs.LG)

Ultra-Wideband (UWB) is one of the key technologies empowering the Internet of Thing (IoT) concept to perform reliable, energy-efficient, and highly accurate monitoring, screening, and localization in indoor environments. Performance of UWB-based localization systems, however, can significantly degrade because of Non Line of Sight (NLoS) connections between a mobile user and UWB beacons. To mitigate the destructive effects of NLoS connections, we target development of a Reinforcement Learning (RL) anchor selection framework that can efficiently cope with the dynamic nature of indoor environments. Existing RL models in this context, however, lack the ability to generalize well to be used in a new setting. Moreover, it takes a long time for the conventional RL models to reach the optimal policy. To tackle these challenges, we propose the Jump-start RL-based Uwb NOde selection (JUNO) framework, which performs real-time location predictions without relying on complex NLoS identification/mitigation methods. The effectiveness of the proposed JUNO framework is evaluated in term of the location error, where the mobile user moves randomly through an ultra-dense indoor environment with a high chance of establishing NLoS connections. Simulation results corroborate the effectiveness of the proposed framework in comparison to its state-of-the-art counterparts.

[312]  arXiv:2205.08423 (cross-list from eess.SP) [pdf, ps, other]
Title: The Deployment of IRS in UAV-Empowered 6G Networks
Subjects: Signal Processing (eess.SP); Information Theory (cs.IT)

Intelligent reflecting surfaces (IRSs) with the ability to reconfigure inherent electromagnetic reflection and absorption characteristics in real-time provide unparalleled prospects to improve wireless connectivity in adverse circumstances. Unmanned aerial vehicles (UAV)-assisted wireless networks are evolved as a reliable solution to combat non-line of sight (NLoS) scenarios. Thereby, the IRS-empowered UAV-assisted cellular networks will be a significant role-player to improve the coverage and user experiences. The paper aimed to minimize the path loss and maximize the achievable data rate in IRS-UAV-assisted networks. In this context, the work analyzed path loss and achievable rate utilizing millimeter wave (mmWave) carrier considering the conventional UAV model and IRS-empowered UAV communication model. The research obtained that the IRSempowered UAV communications model can significantly minimize path loss and maximize the achievable data rate compared to the conventional UAV-assisted model.

[313]  arXiv:2205.08435 (cross-list from q-fin.RM) [pdf, other]
Title: Cyber Risk Assessment for Capital Management
Comments: This paper was first presented on July 5, 2021, at the 24th International Congress on Insurance: Mathematics and Economics
Subjects: Risk Management (q-fin.RM); Cryptography and Security (cs.CR); General Economics (econ.GN); Optimization and Control (math.OC)

Cyber risk is an omnipresent risk in the increasingly digitized world that is known to be difficult to quantify and assess. Despite the fact that cyber risk shows distinct characteristics from conventional risks, most existing models for cyber risk in the insurance literature have been purely based on frequency-severity analysis, which was developed for classical property and casualty risks. In contrast, the cybersecurity engineering literature employs different approaches, under which cyber incidents are viewed as threats or hacker attacks acting on a particular set of vulnerabilities. There appears a gap in cyber risk modeling between engineering and insurance literature. This paper presents a novel model to capture these unique dynamics of cyber risk known from engineering and to model loss distributions based on industry loss data and a particular company's cybersecurity profile. The analysis leads to a new tool for allocating resources of the company between cybersecurity investments and loss-absorbing reserves.

[314]  arXiv:2205.08451 (cross-list from q-bio.BM) [pdf]
Title: MAS2HP: A Multi Agent System to predict protein structure in 2D HP model
Subjects: Biomolecules (q-bio.BM); Artificial Intelligence (cs.AI)

Protein Structure Prediction (PSP) is an unsolved problem in the field of computational biology. The problem of protein structure prediction is about predicting the native conformation of a protein, while its sequence of amino acids is known. Regarding processing limitations of current computer systems, all-atom simulations for proteins are typically unpractical; several reduced models of proteins have been proposed. Additionally, due to intrinsic hardness of calculations even in reduced models, many computational methods mainly based on artificial intelligence have been proposed to solve the problem. Agent-based modeling is a relatively new method for modeling systems composed of interacting items. In this paper we proposed a new approach for protein structure prediction by using agent-based modeling (ABM) in two dimensional hydrophobic-hydrophilic model. We broke the whole process of protein structure prediction into two steps: the first step, which was introduced in our previous paper, is about biasing the linear sequence to gain a primary energy, and the next step, which will be explained in this paper, is about using ABM with a predefined set of rules, to find the best conformation in the least possible amount of time and steps. This method was implemented in NETLOGO. We have tested this algorithm on several benchmark sequences ranging from 20 to 50-mers in two dimensional Hydrophobic-Hydrophilic lattice models. Comparing to the result of the other algorithms, our method is capable of finding the best known conformations in a significantly shorter time. A major problem in PSP simulation is that as the sequence length increases the time consumed to predict a valid structure will exponentially increase. In contrast, by using MAS2HP the effect of increase in sequence length on spent time has changed from exponentially to linear.

[315]  arXiv:2205.08467 (cross-list from eess.IV) [pdf]
Title: Application of Graph Based Features in Computer Aided Diagnosis for Histopathological Image Classification of Gastric Cancer
Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)

The gold standard for gastric cancer detection is gastric histopathological image analysis, but there are certain drawbacks in the existing histopathological detection and diagnosis. In this paper, based on the study of computer aided diagnosis system, graph based features are applied to gastric cancer histopathology microscopic image analysis, and a classifier is used to classify gastric cancer cells from benign cells. Firstly, image segmentation is performed, and after finding the region, cell nuclei are extracted using the k-means method, the minimum spanning tree (MST) is drawn, and graph based features of the MST are extracted. The graph based features are then put into the classifier for classification. In this study, different segmentation methods are compared in the tissue segmentation stage, among which are Level-Set, Otsu thresholding, watershed, SegNet, U-Net and Trans-U-Net segmentation; Graph based features, Red, Green, Blue features, Grey-Level Co-occurrence Matrix features, Histograms of Oriented Gradient features and Local Binary Patterns features are compared in the feature extraction stage; Radial Basis Function (RBF) Support Vector Machine (SVM), Linear SVM, Artificial Neural Network, Random Forests, k-NearestNeighbor, VGG16, and Inception-V3 are compared in the classifier stage. It is found that using U-Net to segment tissue areas, then extracting graph based features, and finally using RBF SVM classifier gives the optimal results with 94.29%.

[316]  arXiv:2205.08468 (cross-list from physics.soc-ph) [pdf, other]
Title: Community Detection in networks by Dynamical Optimal Transport Formulation
Comments: 12 pages, 6 figures, 1 table
Subjects: Physics and Society (physics.soc-ph); Social and Information Networks (cs.SI)

Detecting communities in networks is important in various domains of applications. While a variety of methods exists to perform this task, recent efforts propose Optimal Transport (OT) principles combined with the geometric notion of Ollivier-Ricci curvature to classify nodes into groups by rigorously comparing the information encoded into nodes' neighborhoods. We present an OT-based approach that exploits recent advances in OT theory to allow tuning for traffic penalization, which enforces different transportation schemes. As a result, our model can flexibly capture different scenarios and thus increase performance accuracy in recovering communities, compared to standard OT-based formulations. We test the performance of our algorithm in both synthetic and real networks, achieving a comparable or better performance than other OT-based methods in the former case, while finding communities more aligned with node metadata in real data. This pushes further our understanding of geometric approaches in their ability to capture patterns in complex networks.

[317]  arXiv:2205.08494 (cross-list from math.ST) [pdf, ps, other]
Title: Covariance Estimation: Optimal Dimension-free Guarantees for Adversarial Corruption and Heavy Tails
Comments: 31 pages
Subjects: Statistics Theory (math.ST); Data Structures and Algorithms (cs.DS); Probability (math.PR)

We provide an estimator of the covariance matrix that achieves the optimal rate of convergence (up to constant factors) in the operator norm under two standard notions of data contamination: We allow the adversary to corrupt an $\eta$-fraction of the sample arbitrarily, while the distribution of the remaining data points only satisfies that the $L_{p}$-marginal moment with some $p \ge 4$ is equivalent to the corresponding $L_2$-marginal moment. Despite requiring the existence of only a few moments, our estimator achieves the same tail estimates as if the underlying distribution were Gaussian. As a part of our analysis, we prove a dimension-free Bai-Yin type theorem in the regime $p > 4$.

[318]  arXiv:2205.08528 (cross-list from stat.ML) [pdf, other]
Title: High-dimensional additive Gaussian processes under monotonicity constraints
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)

We introduce an additive Gaussian process framework accounting for monotonicity constraints and scalable to high dimensions. Our contributions are threefold. First, we show that our framework enables to satisfy the constraints everywhere in the input space. We also show that more general componentwise linear inequality constraints can be handled similarly, such as componentwise convexity. Second, we propose the additive MaxMod algorithm for sequential dimension reduction. By sequentially maximizing a squared-norm criterion, MaxMod identifies the active input dimensions and refines the most important ones. This criterion can be computed explicitly at a linear cost. Finally, we provide open-source codes for our full framework. We demonstrate the performance and scalability of the methodology in several synthetic examples with hundreds of dimensions under monotonicity constraints as well as on a real-world flood application.

Replacements for Wed, 18 May 22

[319]  arXiv:1401.6785 (replaced) [pdf, other]
Title: Non-Elementary Complexities for Branching VASS, MELL, and Extensions
Comments: Fixed Fig. 3 thanks to Hiromi Tanaka
Journal-ref: ACM Transactions on Computational Logic, vol. 16, issue 3, article 20, 2015
Subjects: Logic in Computer Science (cs.LO)
[320]  arXiv:1402.0705 (replaced) [pdf, other]
Title: Implicational Relevance Logic is 2-EXPTIME-Complete
Authors: Sylvain Schmitz
Comments: Fixed Fig. 6 as pointed out by Hiromi Tanaka
Journal-ref: Proceedings of RTA-TLCA 2014, Lecture Notes in Computer Science 8560, pp. 395--409, Springer, 2014; journal version in Journal of Symbolic Logic 81(2), pages 641--661, 2016
Subjects: Logic in Computer Science (cs.LO)
[321]  arXiv:1805.04182 (replaced) [pdf, ps, other]
Title: On the tightest interval-valued state estimator for linear systems
Comments: 7 pages, 4 figures
Journal-ref: A version of this work was presented at the IEEE Conference on Decision and Control, Miami Beach, FL, USA, 2018
Subjects: Systems and Control (eess.SY)
[322]  arXiv:1905.01605 (replaced) [pdf, ps, other]
Title: Nitsche's method for a Robin boundary value problem in a smooth domain
Comments: 17 pages, 3 figures
Subjects: Numerical Analysis (math.NA)
[323]  arXiv:2005.04015 (replaced) [pdf, ps, other]
Title: On computing the determinant, other characteristic polynomial coefficients, and inverse in Clifford algebras of arbitrary dimension
Authors: D. S. Shirokov
Comments: 24 pages
Journal-ref: Computational and Applied Mathematics, 40 (2021), 173, 29 pp
Subjects: Mathematical Physics (math-ph); Numerical Analysis (math.NA)
[324]  arXiv:2007.02938 (replaced) [pdf, other]
Title: Causal Feature Selection via Orthogonal Search
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Statistics Theory (math.ST)
[325]  arXiv:2009.05673 (replaced) [pdf, other]
Title: Applications of Deep Neural Networks with Keras
Authors: Jeff Heaton
Comments: arXiv admin note: text overlap with arXiv:1610.02357, arXiv:1603.05027, arXiv:1801.04381, arXiv:2001.02394, arXiv:1704.04861 by other authors
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[326]  arXiv:2009.07439 (replaced) [pdf, other]
Title: On the Landscape of One-hidden-layer Sparse Networks and Beyond
Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
[327]  arXiv:2010.00804 (replaced) [pdf, other]
Title: Kac-Rice formulas and the number of solutions of parametrized systems of polynomial equations
Comments: To appear in Mathematics of Computation
Subjects: Numerical Analysis (math.NA); Algebraic Geometry (math.AG); Quantitative Methods (q-bio.QM)
[328]  arXiv:2010.15745 (replaced) [pdf, other]
Title: Reinforcement Learning of Causal Variables Using Mediation Analysis
Comments: As accepted at proceedings of the AAAI Conference on Artificial Intelligence (AAAI), AAAI, 2022
Subjects: Machine Learning (cs.LG)
[329]  arXiv:2012.09102 (replaced) [pdf, other]
Title: FedADC: Accelerated Federated Learning with Drift Control
Comments: Accepted to ISIT 2021
Subjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Information Theory (cs.IT)
[330]  arXiv:2012.14702 (replaced) [pdf, other]
Title: A fast iterative algorithm for near-diagonal eigenvalue problems
Subjects: Numerical Analysis (math.NA)
[331]  arXiv:2101.09646 (replaced) [pdf, other]
Title: An Improved Level Set Method for Reachability Problems in Differential Games
Comments: 9 pages, 13 figures
Subjects: Systems and Control (eess.SY)
[332]  arXiv:2102.00865 (replaced) [pdf, ps, other]
Title: Global types and event structure semantics for asynchronous multiparty sessions
Authors: Ilaria Castellani (INDES, Inria, UCA), Mariangiola Dezani-Ciancaglini (UNITO), Paola Giannini
Subjects: Logic in Computer Science (cs.LO)
[333]  arXiv:2103.13512 (replaced) [pdf, other]
Title: Projection: A Mechanism for Human-like Reasoning in Artificial Intelligence
Authors: Frank Guerin
Comments: 29 pages, 3 figures. Some minor additions/clarifications in this revision, e.g. mathematical description
Subjects: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV); Robotics (cs.RO)
[334]  arXiv:2104.09350 (replaced) [pdf, other]
Title: A speckle filter for Sentinel-1 SAR Ground Range Detected data based on Residual Convolutional Neural Networks
Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
Subjects: Artificial Intelligence (cs.AI); Image and Video Processing (eess.IV)
[335]  arXiv:2104.14789 (replaced) [pdf, ps, other]
Title: Analyzing Semantics of Aggregate Answer Set Programming Using Approximation Fixpoint Theory
Comments: Paper presented at the 38th International Conference on Logic Programming (ICLP 2022), 22 pages (including supplementary material)
Subjects: Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)
[336]  arXiv:2104.15021 (replaced) [pdf, other]
Title: Formalizing the Face Lattice of Polyhedra
Subjects: Logic in Computer Science (cs.LO); Combinatorics (math.CO); Optimization and Control (math.OC)
[337]  arXiv:2105.06357 (replaced) [pdf, other]
Title: On Minimizing the Number of Running Buffers for Tabletop Rearrangement
Comments: 1. Add a complete version of proof for the upper bound; 2. Add a complete version of MIP model
Subjects: Robotics (cs.RO); Data Structures and Algorithms (cs.DS)
[338]  arXiv:2105.07190 (replaced) [pdf, other]
Title: A Comprehensive Taxonomy for Explainable Artificial Intelligence: A Systematic Survey of Surveys on Methods and Concepts
Comments: 71 pages, 7 figures, 6 tables, submitted to Data Mining and Knowledge Discovery
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[339]  arXiv:2105.10210 (replaced) [pdf, other]
Title: Bayesian Uncertainty Quantification of Local Volatility Model
Subjects: Applications (stat.AP); Numerical Analysis (math.NA); Other Statistics (stat.OT)
[340]  arXiv:2105.10882 (replaced) [pdf, other]
Title: Weakly-supervised 3D Human Pose Estimation with Cross-view U-shaped Graph Convolutional Network
Comments: Accepted by IEEE Transactions on Multimedia
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[341]  arXiv:2105.11225 (replaced) [pdf, other]
Title: Distantly-Supervised Long-Tailed Relation Extraction Using Constraint Graphs
Comments: Accepted by TKDE as a regular paper
Subjects: Computation and Language (cs.CL)
[342]  arXiv:2105.12120 (replaced) [pdf, other]
Title: Sampling random graphs with specified degree sequences
Comments: 18 pages, 14 figures, added references and applications, methods substantially improved, results expanded. Code available at this http URL
Subjects: Social and Information Networks (cs.SI); Data Analysis, Statistics and Probability (physics.data-an); Methodology (stat.ME)
[343]  arXiv:2106.13032 (replaced) [pdf, other]
Title: Optimization of IRS-aided Sub-THz Communications under practical design constraints
Subjects: Information Theory (cs.IT)
[344]  arXiv:2106.14948 (replaced) [pdf, other]
Title: Deep Learning for Face Anti-Spoofing: A Survey
Comments: submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[345]  arXiv:2107.02618 (replaced) [pdf]
Title: Characterizing driver heterogeneity within stochastic traffic simulation
Subjects: Physics and Society (physics.soc-ph); Systems and Control (eess.SY)
[346]  arXiv:2107.04271 (replaced) [pdf, other]
Title: FedAdapt: Adaptive Offloading for IoT Devices in Federated Learning
Comments: 13 pages
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG)
[347]  arXiv:2107.05556 (replaced) [pdf, other]
Title: DebiasedDTA: Improving the Generalizability of Drug-Target Affinity Prediction Models
Subjects: Quantitative Methods (q-bio.QM); Machine Learning (cs.LG)
[348]  arXiv:2107.10025 (replaced) [pdf, other]
Title: Fairness-aware Maximal Clique in Large Graphs: Concepts and Algorithms
Subjects: Databases (cs.DB)
[349]  arXiv:2107.11941 (replaced) [pdf, other]
Title: Computation of Reachable Sets Based on Hamilton-Jacobi-Bellman Equation with Running Cost Function
Subjects: Systems and Control (eess.SY); Optimization and Control (math.OC)
[350]  arXiv:2107.14323 (replaced) [pdf, other]
Title: Reconstruction of Random Geometric Graphs: Breaking the Omega(r) distortion barrier
Comments: v1 on arxiv was titled "Improved Reconstruction of Random Geometric Graphs." An extended abstract with the above title appeared in ICALP 2022. The current version includes the proofs that were omitted from the ICALP version and adds the section "Missing Edges."
Subjects: Computational Geometry (cs.CG); Social and Information Networks (cs.SI); Probability (math.PR); Physics and Society (physics.soc-ph); Machine Learning (stat.ML)
[351]  arXiv:2108.03152 (replaced) [pdf, other]
Title: Source-Free Domain Adaptation for Image Segmentation
Comments: 18 pages, 8 figures, submitted to Elsevier Medical Image Analysis
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[352]  arXiv:2108.03861 (replaced) [pdf, other]
Title: KGAP: Knowledge Graph Augmented Political Perspective Detection in News Media
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
[353]  arXiv:2108.04206 (replaced) [pdf, other]
Title: Classification Auto-Encoder based Detector against Diverse Data Poisoning Attacks
Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible
Subjects: Machine Learning (cs.LG)
[354]  arXiv:2108.06871 (replaced) [pdf, other]
Title: Data Efficient Human Intention Prediction: Leveraging Neural Network Verification and Expert Guidance
Comments: 9 pages, 7 figures. ICML 2021 Workshop of Human in the Loop Learning
Subjects: Machine Learning (cs.LG); Human-Computer Interaction (cs.HC)
[355]  arXiv:2109.03457 (replaced) [pdf, other]
Title: Uncertainty Quantification and Experimental Design for Large-Scale Linear Inverse Problems under Gaussian Process Priors
Comments: under review
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Applications (stat.AP); Computation (stat.CO); Methodology (stat.ME)
[356]  arXiv:2109.06838 (replaced) [pdf, other]
Title: ePiC: Employing Proverbs in Context as a Benchmark for Abstract Language Understanding
Comments: ACL 2022
Subjects: Computation and Language (cs.CL)
[357]  arXiv:2109.07018 (replaced) [pdf, other]
Title: Discretization-independent surrogate modeling over complex geometries using hypernetworks and implicit representations
Subjects: Computational Physics (physics.comp-ph); Machine Learning (cs.LG)
[358]  arXiv:2109.10341 (replaced) [pdf, other]
Title: Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents
Comments: ACL2022
Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
[359]  arXiv:2110.01450 (replaced) [pdf, ps, other]
Title: Extended dynamic mode decomposition with dictionary learning using neural ordinary differential equations
Comments: Corrigendum: The loss function in Eq. (20) is not what we have used in our code. Please replace the sum of squared error in Eq. (20) with the mean squared error
Journal-ref: Nonlinear Theory and Its Applications, IEICE, vol. 12, no. 4, pp. 626-638, 2021
Subjects: Machine Learning (cs.LG); Signal Processing (eess.SP); Dynamical Systems (math.DS); Numerical Analysis (math.NA); Chaotic Dynamics (nlin.CD); Data Analysis, Statistics and Probability (physics.data-an)
[360]  arXiv:2110.01899 (replaced) [pdf, ps, other]
Title: Random matrices in service of ML footprint: ternary random features with no performance loss
Comments: Published as a conference at ICLR2022
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
[361]  arXiv:2110.02274 (replaced) [pdf, other]
Title: Design of Spiral-Cable Forearm Exoskeleton to Assist Supination for Hemiparetic Stroke Subjects
Comments: 6 pages; Accepted to International Conference on Rehabilitation Robotics (ICORR) 2022
Subjects: Robotics (cs.RO)
[362]  arXiv:2110.04742 (replaced) [pdf, other]
Title: Sideward contact tracing and the control of epidemics in large gatherings
Comments: Main document: 12 pages, 5 figures; Supplementary Material: 14 pages, 4 figures
Journal-ref: J. R. Soc. Interface 19: 20220048 (2022)
Subjects: Physics and Society (physics.soc-ph); Social and Information Networks (cs.SI); Populations and Evolution (q-bio.PE)
[363]  arXiv:2110.06120 (replaced) [pdf, other]
Title: A fast time domain solver for the equilibrium Dyson equation
Subjects: Numerical Analysis (math.NA); Strongly Correlated Electrons (cond-mat.str-el)
[364]  arXiv:2110.06753 (replaced) [pdf, other]
Title: Learning Meta Pattern for Face Anti-Spoofing
Comments: Accepted by IEEE Transactions on Information Forensics and Security (this https URL) Source code available in this https URL
Journal-ref: IEEE Transactions on Information Forensics and Security, vol. 17, pp. 1201-1213, 2022
Subjects: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)
[365]  arXiv:2110.09202 (replaced) [pdf, other]
Title: Finding Strong Gravitational Lenses Through Self-Attention
Comments: 18 Pages, 4 tables and 19 Figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Astrophysics of Galaxies (astro-ph.GA); Instrumentation and Methods for Astrophysics (astro-ph.IM)
[366]  arXiv:2110.10422 (replaced) [pdf, other]
Title: PriorVAE: Encoding spatial priors with VAEs for small-area estimation
Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
[367]  arXiv:2111.00537 (replaced) [pdf, other]
Title: A network-based approach to QAnon user dynamics and topic diversity during the COVID-19 infodemic
Comments: accepted by APSIPA Transactions on Signal and Information Processing (Special issue: Multi-Disciplinary Dis/Misinformation Analysis and Countermeasures), 2022
Subjects: Social and Information Networks (cs.SI); Computers and Society (cs.CY)
[368]  arXiv:2111.04543 (replaced) [pdf, ps, other]
Title: Tree decompositions with bounded independence number and their algorithmic applications
Comments: 65 pages; abstract has been shortened due to arXiv requirements
Subjects: Combinatorics (math.CO); Discrete Mathematics (cs.DM); Data Structures and Algorithms (cs.DS)
[369]  arXiv:2111.04635 (replaced) [pdf, other]
Title: CORE: a COmplex event Recognition Engine
Comments: 30 pages, 11 figures
Subjects: Databases (cs.DB); Data Structures and Algorithms (cs.DS)
[370]  arXiv:2111.07380 (replaced) [pdf, other]
Title: Eluding Secure Aggregation in Federated Learning via Model Inconsistency
Comments: 17 pages
Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR)
[371]  arXiv:2111.11430 (replaced) [pdf, other]
Title: Class-agnostic Object Detection with Multi-modal Transformer
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[372]  arXiv:2111.12405 (replaced) [pdf, other]
Title: An Attack on Facial Soft-biometric Privacy Enhancement
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[373]  arXiv:2111.13196 (replaced) [pdf, other]
Title: SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning
Comments: CVPR 2022
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[374]  arXiv:2112.01006 (replaced) [pdf, other]
Title: Distributed Control for a Robotic Swarm to Pass through a Curve Virtual Tube
Comments: 18 pages, 21 figures
Subjects: Robotics (cs.RO); Multiagent Systems (cs.MA); Systems and Control (eess.SY)
[375]  arXiv:2112.02300 (replaced) [pdf, other]
Title: Unsupervised Domain Generalization by Learning a Bridge Across Domains
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[376]  arXiv:2112.06751 (replaced) [pdf, other]
Title: Role of Human-AI Interaction in Selective Prediction
Comments: Published in AAAI 2022; added link to data, small formatting corrections for camera-ready, including small changes to Fig 6-7 that do not change conclusions
Subjects: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
[377]  arXiv:2112.07708 (replaced) [pdf, other]
Title: Learning to Retrieve Passages without Supervision
Comments: NAACL 2022
Subjects: Computation and Language (cs.CL); Information Retrieval (cs.IR)
[378]  arXiv:2112.08526 (replaced) [pdf, other]
Title: Invariance Through Latent Alignment
Comments: To appear in RSS 2022. Here's our project page: this https URL
Subjects: Machine Learning (cs.LG); Robotics (cs.RO)
[379]  arXiv:2112.08837 (replaced) [pdf]
Title: Improving Unsupervised Stain-To-Stain Translation using Self-Supervision and Meta-Learning
Comments: Accepted for Journal of Pathology Informatics (JPI), 2022
Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV); Quantitative Methods (q-bio.QM)
[380]  arXiv:2112.09062 (replaced) [pdf, other]
Title: Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants
Subjects: Computation and Language (cs.CL)
[381]  arXiv:2112.09245 (replaced) [pdf, other]
Title: Automated Deep Learning: Neural Architecture Search Is Not the End
Comments: 66 pages, 10 tables, 4 figures, 325 references; improve the old version with community feedback
Subjects: Machine Learning (cs.LG)
[382]  arXiv:2112.11988 (replaced) [pdf, other]
Title: Reducing Programs to Objects
Authors: Yegor Bugayenko
Subjects: Programming Languages (cs.PL)
[383]  arXiv:2112.13416 (replaced) [pdf, other]
Title: Attribute Inference Attack of Speech Emotion Recognition in Federated Learning Settings
Subjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG); Multimedia (cs.MM)
[384]  arXiv:2201.00221 (replaced) [pdf, ps, other]
Title: Event structure semantics for multiparty sessions
Comments: arXiv admin note: text overlap with arXiv:2102.00865
Subjects: Logic in Computer Science (cs.LO)
[385]  arXiv:2201.00323 (replaced) [pdf, other]
Title: V-LinkNet: Learning Contextual Inpainting Across Latent Space of Generative Adversarial Network
Comments: 13 pages including references, 9 figures and 4 tables
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[386]  arXiv:2201.00519 (replaced) [pdf, other]
Title: Stochastic Weight Averaging Revisited
Subjects: Machine Learning (cs.LG)
[387]  arXiv:2201.00549 (replaced) [pdf, ps, other]
Title: Efficient Enumeration Algorithms for Annotated Grammars
Comments: 54 pages. Full version with proofs of the article to appear at PODS'22. Except formatting and minor differences, this article contains all the contents of the PODS'22 article, plus the technical appendices
Subjects: Formal Languages and Automata Theory (cs.FL); Data Structures and Algorithms (cs.DS)
[388]  arXiv:2201.03514 (replaced) [pdf, other]
Title: Black-Box Tuning for Language-Model-as-a-Service
Comments: Accepted by ICML 2022. Code is available at this https URL
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
[389]  arXiv:2201.07188 (replaced) [pdf, other]
Title: SoK: Blockchain Governance
Subjects: Cryptography and Security (cs.CR); Computers and Society (cs.CY); Computer Science and Game Theory (cs.GT)
[390]  arXiv:2201.07287 (replaced) [pdf, other]
Title: Polar Coded Merkle Tree: Improved Detection of Data Availability Attacks in Blockchain Systems
Comments: 9 pages, 4 figures, 2 tables, To appear in IEEE International Symposium on Information Theory (ISIT) 2022
Subjects: Information Theory (cs.IT); Cryptography and Security (cs.CR)
[391]  arXiv:2201.08322 (replaced) [pdf, other]
Title: Error-and-erasure Decoding of Product and Staircase Codes with Simplified Extrinsic Message Passing
Comments: Accepted at ISIT 2022
Subjects: Information Theory (cs.IT)
[392]  arXiv:2201.08387 (replaced) [pdf, other]
Title: Understanding and Detecting Hateful Content using Contrastive Learning
Subjects: Social and Information Networks (cs.SI); Computers and Society (cs.CY)
[393]  arXiv:2201.09047 (replaced) [pdf, other]
Title: Online Auction-Based Incentive Mechanism Design for Horizontal Federated Learning with Budget Constraint
Subjects: Computer Science and Game Theory (cs.GT); Artificial Intelligence (cs.AI)
[394]  arXiv:2201.10110 (replaced) [pdf, other]
Title: A Hybrid Quantum-Classical Algorithm for Robust Fitting
Comments: IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR) 2022
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[395]  arXiv:2201.11691 (replaced) [pdf, other]
Title: Recursive Binding for Similarity-Preserving Hypervector Representations of Sequences
Comments: 8 pages, 4, figures, 2 tables. arXiv admin note: some overlap with arXiv:2112.15475
Subjects: Artificial Intelligence (cs.AI)
[396]  arXiv:2201.12785 (replaced) [pdf, other]
Title: TransBTSV2: Towards Better and More Efficient Volumetric Segmentation of Medical Images
Subjects: Image and Video Processing (eess.IV); Computer Vision and Pattern Recognition (cs.CV)
[397]  arXiv:2201.12898 (replaced) [pdf, other]
Title: Clearing Payments in Dynamic Financial Networks
Subjects: Optimization and Control (math.OC); Computational Engineering, Finance, and Science (cs.CE); Systems and Control (eess.SY); Mathematical Finance (q-fin.MF); Risk Management (q-fin.RM)
[398]  arXiv:2201.13357 (replaced) [pdf, other]
Title: DNS: Determinantal Point Process Based Neural Network Sampler for Ensemble Reinforcement Learning
Comments: Accepted for Publication at ICML 2022
Subjects: Machine Learning (cs.LG)
[399]  arXiv:2202.01123 (replaced) [pdf, other]
Title: An ASP approach for reasoning on neural networks under a finitely many-valued semantics for weighted conditional knowledge bases
Comments: Paper presented at the 38th International Conference on Logic Programming (ICLP 2022), 16 pages
Subjects: Artificial Intelligence (cs.AI)
[400]  arXiv:2202.03051 (replaced) [pdf, ps, other]
Title: Using Partial Monotonicity in Submodular Maximization
Comments: 45 pages; 7 figures
Subjects: Machine Learning (cs.LG); Data Structures and Algorithms (cs.DS); Optimization and Control (math.OC); Machine Learning (stat.ML)
[401]  arXiv:2202.04539 (replaced) [pdf, ps, other]
Title: Dynamic self-triggered control for nonlinear systems with delays
Comments: Final version, accepted for presentation at the 9th IFAC Conference on Networked Systems (NecSys), 2022. This version contains the full proofs of Proposition 1 and Theorem 1
Subjects: Systems and Control (eess.SY)
[402]  arXiv:2202.04648 (replaced) [pdf, other]
Title: A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems
Comments: 45 pages, 14 figures
Subjects: Machine Learning (cs.LG); Machine Learning (stat.ML)
[403]  arXiv:2202.06877 (replaced) [pdf, other]
Title: A Review of zk-SNARKs
Subjects: Cryptography and Security (cs.CR)
[404]  arXiv:2202.07582 (replaced) [pdf, other]
Title: Monoidal Width: Unifying Tree Width, Path Width and Branch Width
Comments: 28 pages
Subjects: Category Theory (math.CT); Logic in Computer Science (cs.LO)
[405]  arXiv:2202.08340 (replaced) [pdf, other]
Title: A Developmentally-Inspired Examination of Shape versus Texture Bias in Machines
Comments: 7 pages, 4 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
[406]  arXiv:2202.09256 (replaced) [pdf, other]
Title: Traffic-Aware Dynamic Functional Split for 5G Cloud Radio Access Networks
Subjects: Networking and Internet Architecture (cs.NI)
[407]  arXiv:2202.10349 (replaced) [pdf, ps, other]
Title: Certified Verification of Relational Properties
Subjects: Software Engineering (cs.SE)
[408]  arXiv:2202.11099 (replaced) [pdf, other]
Title: Roto-Translation Equivariant Super-Resolution of Two-Dimensional Flows Using Convolutional Neural Networks
Authors: Yuki Yasuda
Subjects: Fluid Dynamics (physics.flu-dyn); Computer Vision and Pattern Recognition (cs.CV); Atmospheric and Oceanic Physics (physics.ao-ph)
[409]  arXiv:2202.12586 (replaced) [pdf, other]
Title: Spatio-Temporal Latent Graph Structure Learning for Traffic Forecasting
Comments: This paper has been accepted as a full paper at IJCNN 2022
Subjects: Machine Learning (cs.LG)
[410]  arXiv:2202.12666 (replaced) [pdf, ps, other]
Title: Isometry groups of formal languages for generalized Levenshtein distances
Comments: 8 pages
Subjects: Group Theory (math.GR); Formal Languages and Automata Theory (cs.FL)
[411]  arXiv:2202.13392 (replaced) [pdf, other]
Title: A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models
Comments: Accepted to ACL 2022. The code and models are available at this https URL
Subjects: Computation and Language (cs.CL)
[412]  arXiv:2202.13691 (replaced) [pdf, ps, other]
Title: On the quadrature exactness in hyperinterpolation
Comments: 11 pages, 2 figures
Subjects: Numerical Analysis (math.NA)
[413]  arXiv:2203.01578 (replaced) [pdf, other]
Title: Continual SLAM: Beyond Lifelong Simultaneous Localization and Mapping through Continual Learning
Subjects: Robotics (cs.RO)
[414]  arXiv:2203.01910 (replaced) [pdf, other]
Title: Efficient Data Structures for Exploiting Sparsity and Structure in Representation of Polynomial Optimization Problems: Implementation in SOSTOOLS
Subjects: Optimization and Control (math.OC); Mathematical Software (cs.MS)
[415]  arXiv:2203.04136 (replaced) [pdf]
Title: Cybersecurity Playbook Sharing with STIX 2.1
Comments: arXiv admin note: text overlap with arXiv:2110.10540
Subjects: Cryptography and Security (cs.CR)
[416]  arXiv:2203.10183 (replaced) [pdf, other]
Title: RoVISQ: Reduction of Video Service Quality via Adversarial Attacks on Deep Learning-based Video Compression
Subjects: Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)
[417]  arXiv:2203.10509 (replaced) [pdf, ps, other]
Title: Stability Of Matrix Polynomials In One And Several Variables
Comments: 20 pages
Subjects: Complex Variables (math.CV); Numerical Analysis (math.NA)
[418]  arXiv:2203.14343 (replaced) [pdf, other]
Title: Diagonal State Spaces are as Effective as Structured State Spaces
Comments: updated version with simpler DSS variants, RNN view for autoregressive decoding, ablation analysis, analysis of trained model parameters and kernels
Subjects: Machine Learning (cs.LG); Computation and Language (cs.CL)
[419]  arXiv:2203.17055 (replaced) [pdf, other]
Title: Certified machine learning: A posteriori error estimation for physics-informed neural networks
Subjects: Machine Learning (cs.LG); Numerical Analysis (math.NA)
[420]  arXiv:2204.02500 (replaced) [pdf, other]
Title: User-Level Differential Privacy against Attribute Inference Attack of Speech Emotion Recognition in Federated Learning
Subjects: Cryptography and Security (cs.CR); Sound (cs.SD); Audio and Speech Processing (eess.AS)
[421]  arXiv:2204.06827 (replaced) [pdf, other]
Title: How Gender Debiasing Affects Internal Model Representations, and Why It Matters
Comments: Accepted to NAACL 2022
Subjects: Computation and Language (cs.CL)
[422]  arXiv:2204.06832 (replaced) [pdf, ps, other]
Title: Self-Guided Learning to Denoise for Robust Recommendation
Comments: Accepted by SIGIR2022
Subjects: Information Retrieval (cs.IR)
[423]  arXiv:2204.08182 (replaced) [pdf, other]
Title: Modality-Balanced Embedding for Video Retrieval
Comments: Accepted by SIGIR-2022, short paper
Journal-ref: SIGIR, 2022
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (stat.ML)
[424]  arXiv:2204.09347 (replaced) [pdf, other]
Title: Active Few-Shot Learning with FASL
Subjects: Computation and Language (cs.CL)
[425]  arXiv:2204.09817 (replaced) [pdf, other]
Title: Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing
Subjects: Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL)
[426]  arXiv:2204.10380 (replaced) [pdf, other]
Title: The 6th AI City Challenge
Comments: Summary of the 6th AI City Challenge Workshop in conjunction with CVPR 2022. arXiv admin note: text overlap with arXiv:2104.12233
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[427]  arXiv:2204.10543 (replaced) [pdf, other]
Title: Zero and Few-shot Learning for Author Profiling
Subjects: Computation and Language (cs.CL)
[428]  arXiv:2204.11153 (replaced) [pdf, ps, other]
Title: Chain rules for quantum channels
Comments: v2: 6 pages, technical note, will appear at IEEE International Symposium on Information Theory 2022, final version with updated references
Subjects: Quantum Physics (quant-ph); Information Theory (cs.IT); Mathematical Physics (math-ph)
[429]  arXiv:2204.11843 (replaced) [pdf, other]
Title: A Computational Theory of Learning Flexible Reward-Seeking Behavior with Place Cells
Authors: Yuanxiang Gao
Comments: 14 pages, 23 figures
Subjects: Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
[430]  arXiv:2204.12948 (replaced) [pdf, other]
Title: Domain Knowledge-Infused Deep Learning for Automated Analog/Radio-Frequency Circuit Parameter Optimization
Comments: 7 pages, 7 figures. arXiv admin note: substantial text overlap with arXiv:2202.13185
Subjects: Machine Learning (cs.LG); Hardware Architecture (cs.AR)
[431]  arXiv:2204.12993 (replaced) [pdf, other]
Title: Counterfactual harm
Comments: Changes to definition 3. Typos corrected and document shortened. Updated Appendices A - C
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Machine Learning (stat.ML)
[432]  arXiv:2204.13060 (replaced) [pdf, other]
Title: Bisimulation Makes Analogies in Goal-Conditioned Reinforcement Learning
Comments: ICML 2022. 20 Pages, 15 Figures, 4 Tables. Website at this https URL
Subjects: Machine Learning (cs.LG)
[433]  arXiv:2204.13821 (replaced) [pdf]
Title: A Neural Network-enhanced Reproducing Kernel Particle Method for Modeling Strain Localization
Subjects: Computational Engineering, Finance, and Science (cs.CE); Machine Learning (cs.LG); Numerical Analysis (math.NA); Computational Physics (physics.comp-ph)
[434]  arXiv:2205.01252 (replaced) [pdf, other]
Title: SIMD$^2$: A Generalized Matrix Instruction Set for Accelerating Tensor Computation beyond GEMM
Comments: To Appear in the 49th International Symposium on Computer Architecture (ISCA'22), June 18--22, 2022, New York, NY, USA
Subjects: Hardware Architecture (cs.AR)
[435]  arXiv:2205.01972 (replaced) [pdf, other]
Title: Sequencer: Deep LSTM for Image Classification
Comments: updated related works, appendix, and other minors
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
[436]  arXiv:2205.02374 (replaced) [pdf, other]
Title: The composition complexity of majority
Comments: to appear in CCC 2022. Fixed typos, updated references
Subjects: Computational Complexity (cs.CC)
[437]  arXiv:2205.04411 (replaced) [pdf, other]
Title: Model-Contrastive Learning for Backdoor Defense
Subjects: Machine Learning (cs.LG)
[438]  arXiv:2205.04546 (replaced) [pdf, other]
Title: CODEC: Complex Document and Entity Collection
Comments: 10 pages, SIGIR 2022 Preprint
Subjects: Information Retrieval (cs.IR)
[439]  arXiv:2205.04930 (replaced) [pdf, other]
Title: QUANTAS: Quantitative User-friendly Adaptable Networked Things Abstract Simulator
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)
[440]  arXiv:2205.05069 (replaced) [pdf, other]
Title: Accelerating the Training of Video Super-Resolution Models
Comments: The code is available at this https URL
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM)
[441]  arXiv:2205.05335 (replaced) [pdf, other]
Title: Deep Depth Completion: A Survey
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[442]  arXiv:2205.06296 (replaced) [pdf, other]
Title: Integrating User and Item Reviews in Deep Cooperative Neural Networks for Movie Recommendation
Comments: 13 pages, typos corrected, references added
Subjects: Information Retrieval (cs.IR); Machine Learning (cs.LG)
[443]  arXiv:2205.06401 (replaced) [pdf, other]
Title: PoisonedEncoder: Poisoning the Unlabeled Pre-training Data in Contrastive Learning
Comments: To appear in USENIX Security Symposium, 2022
Subjects: Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
[444]  arXiv:2205.06445 (replaced) [pdf, other]
Title: Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition
Comments: arXiv admin note: text overlap with arXiv:2202.10290
Subjects: Audio and Speech Processing (eess.AS); Artificial Intelligence (cs.AI)
[445]  arXiv:2205.07015 (replaced) [pdf, other]
Title: Cliff Diving: Exploring Reward Surfaces in Reinforcement Learning Environments
Comments: Accepted at ICML 2022
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[446]  arXiv:2205.07216 (replaced) [pdf, other]
Title: Federated learning for LEO constellations via inter-HAP links
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Signal Processing (eess.SP)
[447]  arXiv:2205.07266 (replaced) [pdf, other]
Title: Discovering the Representation Bottleneck of Graph Neural Networks from Multi-order Interactions
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
[448]  arXiv:2205.07331 (replaced) [pdf, other]
Title: Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descent
Subjects: Numerical Analysis (math.NA); Machine Learning (cs.LG); Statistics Theory (math.ST); Computational Physics (physics.comp-ph); Machine Learning (stat.ML)
[449]  arXiv:2205.07556 (replaced) [pdf, other]
Title: An Effective Transformer-based Solution for RSNA Intracranial Hemorrhage Detection Competition
Subjects: Computer Vision and Pattern Recognition (cs.CV)
[450]  arXiv:2205.07557 (replaced) [pdf, ps, other]
Title: Heroes, Villains, and Victims, and GPT-3: Automated Extraction of Character Roles Without Training Data
Subjects: Computation and Language (cs.CL)
[451]  arXiv:2205.07686 (replaced) [pdf, other]
Title: CQR-SQL: Conversational Question Reformulation Enhanced Context-Dependent Text-to-SQL Parsers
Comments: Work in progress. 11 pages, 6 figures
Subjects: Computation and Language (cs.CL)
[452]  arXiv:2205.07829 (replaced) [pdf, ps, other]
Title: Federated Anomaly Detection over Distributed Data Streams
Comments: DSAA'2021 Conference - PhD Track
Subjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC)
[ total of 452 entries: 1-452 ]
[ showing up to 2000 entries per page: fewer | more ]

Disable MathJax (What is MathJax?)

Links to: arXiv, form interface, find, cs, recent, 2205, contact, help  (Access key information)