Showing 1–40 of 40 results for author: Xi, X

Entry-Specific Matrix Estimation under Arbitrary Sampling Patterns through the Lens of Network Flows

Authors: Yudong Chen, Xumei Xi, Christina Lee Yu

Abstract: Matrix completion tackles the task of predicting missing values in a low-rank matrix based on a sparse set of observed entries. It is often assumed that the observation pattern is generated uniformly at random or has a very specific structure tuned to a given algorithm. There is still a gap in our understanding when it comes to arbitrary sampling patterns. Given an arbitrary sampling pattern, we i… ▽ More Matrix completion tackles the task of predicting missing values in a low-rank matrix based on a sparse set of observed entries. It is often assumed that the observation pattern is generated uniformly at random or has a very specific structure tuned to a given algorithm. There is still a gap in our understanding when it comes to arbitrary sampling patterns. Given an arbitrary sampling pattern, we introduce a matrix completion algorithm based on network flows in the bipartite graph induced by the observation pattern. For additive matrices, the particular flow we used is the electrical flow and we establish error upper bounds customized to each entry as a function of the observation set, along with matching minimax lower bounds. Our results show that the minimax squared error for recovery of a particular entry in the matrix is proportional to the effective resistance of the corresponding edge in the graph. Furthermore, we show that our estimator is equivalent to the least squares estimator. We apply our estimator to the two-way fixed effects model and show that it enables us to accurately infer individual causal effects and the unit-specific and time-specific confounders. For rank-$1$ matrices, we use edge-disjoint paths to form an estimator that achieves minimax optimal estimation when the sampling is sufficiently dense. Our discovery introduces a new family of estimators parametrized by network flows, which provide a fine-grained and intuitive understanding of the impact of the given sampling pattern on the relative difficulty of estimation at an entry-specific level. This graph-based approach allows us to quantify the inherent complexity of matrix completion for individual entries, rather than relying solely on global measures of performance. △ Less

Submitted 5 September, 2024; originally announced September 2024.

SimAD: A Simple Dissimilarity-based Approach for Time Series Anomaly Detection

Authors: Zhijie Zhong, Zhiwen Yu, Xing Xi, Yue Xu, Jiahui Chen, Kaixiang Yang

Abstract: Despite the prevalence of reconstruction-based deep learning methods, time series anomaly detection remains challenging. Existing approaches often struggle with limited temporal contexts, inadequate representation of normal patterns, and flawed evaluation metrics, hindering their effectiveness in identifying aberrant behavior. To address these issues, we introduce $\textbf{SimAD}$, a… ▽ More Despite the prevalence of reconstruction-based deep learning methods, time series anomaly detection remains challenging. Existing approaches often struggle with limited temporal contexts, inadequate representation of normal patterns, and flawed evaluation metrics, hindering their effectiveness in identifying aberrant behavior. To address these issues, we introduce $\textbf{SimAD}$, a $\textbf{Sim}$ple dissimilarity-based approach for time series $\textbf{A}$nomaly $\textbf{D}$etection. SimAD incorporates an advanced feature extractor adept at processing extended temporal windows, utilizes the EmbedPatch encoder to integrate normal behavioral patterns comprehensively, and introduces an innovative ContrastFusion module designed to accentuate distributional divergences between normal and abnormal data, thereby enhancing the robustness of anomaly discrimination. Additionally, we propose two robust evaluation metrics, UAff and NAff, addressing the limitations of existing metrics and demonstrating their reliability through theoretical and experimental analyses. Experiments across $\textbf{seven}$ diverse time series datasets demonstrate SimAD's superior performance compared to state-of-the-art methods, achieving relative improvements of $\textbf{19.85%}$ on F1, $\textbf{4.44%}$ on Aff-F1, $\textbf{77.79%}$ on NAff-F1, and $\textbf{9.69%}$ on AUC on six multivariate datasets. Code and pre-trained models are available at https://github.com/EmorZz1G/SimAD. △ Less

Submitted 18 May, 2024; originally announced May 2024.

Comments: 18 pages, 12 figures,7 tables, Under review

Modeling Output-Level Task Relatedness in Multi-Task Learning with Feedback Mechanism

Authors: Xiangming Xi, Feng Gao, Jun Xu, Fangtai Guo, Tianlei Jin

Abstract: Multi-task learning (MTL) is a paradigm that simultaneously learns multiple tasks by sharing information at different levels, enhancing the performance of each individual task. While previous research has primarily focused on feature-level or parameter-level task relatedness, and proposed various model architectures and learning algorithms to improve learning performance, we aim to explore output-… ▽ More Multi-task learning (MTL) is a paradigm that simultaneously learns multiple tasks by sharing information at different levels, enhancing the performance of each individual task. While previous research has primarily focused on feature-level or parameter-level task relatedness, and proposed various model architectures and learning algorithms to improve learning performance, we aim to explore output-level task relatedness. This approach introduces a posteriori information into the model, considering that different tasks may produce correlated outputs with mutual influences. We achieve this by incorporating a feedback mechanism into MTL models, where the output of one task serves as a hidden feature for another task, thereby transforming a static MTL model into a dynamic one. To ensure the training process converges, we introduce a convergence loss that measures the trend of a task's outputs during each iteration. Additionally, we propose a Gumbel gating mechanism to determine the optimal projection of feedback signals. We validate the effectiveness of our method and evaluate its performance through experiments conducted on several baseline models in spoken language understanding. △ Less

Submitted 31 March, 2024; originally announced April 2024.

Comments: submitted to CDC2024

Multi-Objective Trajectory Planning with Dual-Encoder

Authors: Beibei Zhang, Tian Xiang, Chentao Mao, Yuhua Zheng, Shuai Li, Haoyi Niu, Xiangming Xi, Wenyuan Bai, Feng Gao

Abstract: Time-jerk optimal trajectory planning is crucial in advancing robotic arms' performance in dynamic tasks. Traditional methods rely on solving complex nonlinear programming problems, bringing significant delays in generating optimized trajectories. In this paper, we propose a two-stage approach to accelerate time-jerk optimal trajectory planning. Firstly, we introduce a dual-encoder based transform… ▽ More Time-jerk optimal trajectory planning is crucial in advancing robotic arms' performance in dynamic tasks. Traditional methods rely on solving complex nonlinear programming problems, bringing significant delays in generating optimized trajectories. In this paper, we propose a two-stage approach to accelerate time-jerk optimal trajectory planning. Firstly, we introduce a dual-encoder based transformer model to establish a good preliminary trajectory. This trajectory is subsequently refined through sequential quadratic programming to improve its optimality and robustness. Our approach outperforms the state-of-the-art by up to 79.72\% in reducing trajectory planning time. Compared with existing methods, our method shrinks the optimality gap with the objective function value decreasing by up to 29.9\%. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Comments: 6 pages, 7 figures, conference

Parcae: Proactive, Liveput-Optimized DNN Training on Preemptible Instances

Authors: Jiangfei Duan, Ziang Song, Xupeng Miao, Xiaoli Xi, Dahua Lin, Harry Xu, Minjia Zhang, Zhihao Jia

Abstract: Deep neural networks (DNNs) are becoming progressively large and costly to train. This paper aims to reduce DNN training costs by leveraging preemptible instances on modern clouds, which can be allocated at a much lower price when idle but may be preempted by the cloud provider at any time. Prior work that supports DNN training on preemptive instances employs a reactive approach to handling instan… ▽ More Deep neural networks (DNNs) are becoming progressively large and costly to train. This paper aims to reduce DNN training costs by leveraging preemptible instances on modern clouds, which can be allocated at a much lower price when idle but may be preempted by the cloud provider at any time. Prior work that supports DNN training on preemptive instances employs a reactive approach to handling instance preemptions and allocations after their occurrence, which only achieves limited performance and scalability. We present Parcae, a system that enables cheap, fast, and scalable DNN training on preemptible instances by proactively adjusting the parallelization strategy of a DNN training job to adapt to predicted resource changes before instance preemptions and allocations really happen, which significantly reduces the cost of handling these events. Parcae optimizes liveput, a novel metric that measures the expected training throughput of a DNN job under various possible preemption scenarios. Compared to existing reactive, throughput-optimized systems, Parcae's proactive, live-optimized solution considers both the throughput of a job and its robustness under preemptions. To optimize liveput, Parcae supports lightweight instance migration and uses an availability predictor to forecast future preemptions. It then uses a liveput optimizer to discover an optimal strategy to parallelize DNN training under predicted preemptions. We evaluate Parcae on a variety of DNNs and preemption traces and show that Parcae outperforms existing spot-instance DNN training systems by up to 10$\times$. More importantly, Parcae achieves near-optimal performance for training large DNNs under frequent preemptions, in which case existing approaches cannot make any progress. △ Less

Submitted 20 March, 2024; originally announced March 2024.

Comments: NSDI '24

Performance Analysis on RIS-Aided Wideband Massive MIMO OFDM Systems with Low-Resolution ADCs

Authors: Xianzhe Chen, Hong Ren, Cunhua Pan, Zhangjie Peng, Kangda Zhi, Yong Liu, Xiaojun Xi, Ana Garcia Armada, Cheng-Xiang Wang

Abstract: This paper investigates a reconfigurable intelligent surface (RIS)-aided wideband massive multiple-input multiple-output (MIMO) orthogonal frequency division multiplexing (OFDM) system with low-resolution analog-to-digital converters (ADCs). Frequency-selective Rician fading channels are considered, and the OFDM data transmission process is presented in time domain. This paper derives the closed-f… ▽ More This paper investigates a reconfigurable intelligent surface (RIS)-aided wideband massive multiple-input multiple-output (MIMO) orthogonal frequency division multiplexing (OFDM) system with low-resolution analog-to-digital converters (ADCs). Frequency-selective Rician fading channels are considered, and the OFDM data transmission process is presented in time domain. This paper derives the closed-form approximate expression of the uplink achievable rate, based on which the asymptotic system performance is analyzed when the number of the antennas at the base station and the number of reflecting elements at the RIS grow to infinity. Besides, the power scaling laws of the considered system are revealed to provide energy-saving insights. Furthermore, this paper proposes a gradient ascent-based algorithm to design the phase shifts of the RIS for maximizing the minimum user rate. Finally, numerical results are presented to verify the correctness of analytical conclusions and draw insights. △ Less

Submitted 13 March, 2024; originally announced March 2024.

Entry-Specific Bounds for Low-Rank Matrix Completion under Highly Non-Uniform Sampling

Authors: Xumei Xi, Christina Lee Yu, Yudong Chen

Abstract: Low-rank matrix completion concerns the problem of estimating unobserved entries in a matrix using a sparse set of observed entries. We consider the non-uniform setting where the observed entries are sampled with highly varying probabilities, potentially with different asymptotic scalings. We show that under structured sampling probabilities, it is often better and sometimes optimal to run estimat… ▽ More Low-rank matrix completion concerns the problem of estimating unobserved entries in a matrix using a sparse set of observed entries. We consider the non-uniform setting where the observed entries are sampled with highly varying probabilities, potentially with different asymptotic scalings. We show that under structured sampling probabilities, it is often better and sometimes optimal to run estimation algorithms on a smaller submatrix rather than the entire matrix. In particular, we prove error upper bounds customized to each entry, which match the minimax lower bounds under certain conditions. Our bounds characterize the hardness of estimating each entry as a function of the localized sampling probabilities. We provide numerical experiments that confirm our theoretical findings. △ Less

Submitted 29 February, 2024; originally announced March 2024.

Sparsity via Sparse Group $k$-max Regularization

Authors: Qinghua Tao, Xiangming Xi, Jun Xu, Johan A. K. Suykens

Abstract: For the linear inverse problem with sparsity constraints, the $l_0$ regularized problem is NP-hard, and existing approaches either utilize greedy algorithms to find almost-optimal solutions or to approximate the $l_0$ regularization with its convex counterparts. In this paper, we propose a novel and concise regularization, namely the sparse group $k$-max regularization, which can not only simultan… ▽ More For the linear inverse problem with sparsity constraints, the $l_0$ regularized problem is NP-hard, and existing approaches either utilize greedy algorithms to find almost-optimal solutions or to approximate the $l_0$ regularization with its convex counterparts. In this paper, we propose a novel and concise regularization, namely the sparse group $k$-max regularization, which can not only simultaneously enhance the group-wise and in-group sparsity, but also casts no additional restraints on the magnitude of variables in each group, which is especially important for variables at different scales, so that it approximate the $l_0$ norm more closely. We also establish an iterative soft thresholding algorithm with local optimality conditions and complexity analysis provided. Through numerical experiments on both synthetic and real-world datasets, we verify the effectiveness and flexibility of the proposed method. △ Less

Submitted 13 February, 2024; originally announced February 2024.

Comments: 7 pages, accepted to American Control Conference 2024

SpotServe: Serving Generative Large Language Models on Preemptible Instances

Authors: Xupeng Miao, Chunan Shi, Jiangfei Duan, Xiaoli Xi, Dahua Lin, Bin Cui, Zhihao Jia

Abstract: The high computational and memory requirements of generative large language models (LLMs) make it challenging to serve them cheaply. This paper aims to reduce the monetary cost for serving LLMs by leveraging preemptible GPU instances on modern clouds, which offer accesses to spare GPUs at a much cheaper price than regular instances but may be preempted by the cloud at any time. Serving LLMs on pre… ▽ More The high computational and memory requirements of generative large language models (LLMs) make it challenging to serve them cheaply. This paper aims to reduce the monetary cost for serving LLMs by leveraging preemptible GPU instances on modern clouds, which offer accesses to spare GPUs at a much cheaper price than regular instances but may be preempted by the cloud at any time. Serving LLMs on preemptible instances requires addressing challenges induced by frequent instance preemptions and the necessity of migrating instances to handle these preemptions. This paper presents SpotServe, the first distributed LLM serving system on preemptible instances. Several key techniques in SpotServe realize fast and reliable serving of generative LLMs on cheap preemptible instances. First, SpotServe dynamically adapts the LLM parallelization configuration for dynamic instance availability and fluctuating workload, while balancing the trade-off among the overall throughput, inference latency and monetary costs. Second, to minimize the cost of migrating instances for dynamic reparallelization, the task of migrating instances is formulated as a bipartite graph matching problem, which uses the Kuhn-Munkres algorithm to identify an optimal migration plan that minimizes communications. Finally, to take advantage of the grace period offered by modern clouds, we introduce stateful inference recovery, a new inference mechanism that commits inference progress at a much finer granularity and allows SpotServe to cheaply resume inference upon preemption. We evaluate on real spot instance preemption traces and various popular LLMs and show that SpotServe can reduce the P99 tail latency by 2.4 - 9.1x compared with the best existing LLM serving systems. We also show that SpotServe can leverage the price advantage of preemptive instances, saving 54% monetary cost compared with only using on-demand instances. △ Less

Submitted 27 November, 2023; originally announced November 2023.

Comments: ASPLOS 2024

OpenNet: Incremental Learning for Autonomous Driving Object Detection with Balanced Loss

Authors: Zezhou Wang, Guitao Cao, Xidong Xi, Jiangtao Wang

Abstract: Automated driving object detection has always been a challenging task in computer vision due to environmental uncertainties. These uncertainties include significant differences in object sizes and encountering the class unseen. It may result in poor performance when traditional object detection models are directly applied to automated driving detection. Because they usually presume fixed categorie… ▽ More Automated driving object detection has always been a challenging task in computer vision due to environmental uncertainties. These uncertainties include significant differences in object sizes and encountering the class unseen. It may result in poor performance when traditional object detection models are directly applied to automated driving detection. Because they usually presume fixed categories of common traffic participants, such as pedestrians and cars. Worsely, the huge class imbalance between common and novel classes further exacerbates performance degradation. To address the issues stated, we propose OpenNet to moderate the class imbalance with the Balanced Loss, which is based on Cross Entropy Loss. Besides, we adopt an inductive layer based on gradient reshaping to fast learn new classes with limited samples during incremental learning. To against catastrophic forgetting, we employ normalized feature distillation. By the way, we improve multi-scale detection robustness and unknown class recognition through FPN and energy-based detection, respectively. The Experimental results upon the CODA dataset show that the proposed method can obtain better performance than that of the existing methods. △ Less

Submitted 25 November, 2023; originally announced November 2023.

Integrating Offline Reinforcement Learning with Transformers for Sequential Recommendation

Authors: Xumei Xi, Yuke Zhao, Quan Liu, Liwen Ouyang, Yang Wu

Abstract: We consider the problem of sequential recommendation, where the current recommendation is made based on past interactions. This recommendation task requires efficient processing of the sequential data and aims to provide recommendations that maximize the long-term reward. To this end, we train a farsighted recommender by using an offline RL algorithm with the policy network in our model architectu… ▽ More We consider the problem of sequential recommendation, where the current recommendation is made based on past interactions. This recommendation task requires efficient processing of the sequential data and aims to provide recommendations that maximize the long-term reward. To this end, we train a farsighted recommender by using an offline RL algorithm with the policy network in our model architecture that has been initialized from a pre-trained transformer model. The pre-trained model leverages the superb ability of the transformer to process sequential information. Compared to prior works that rely on online interaction via simulation, we focus on implementing a fully offline RL framework that is able to converge in a fast and stable way. Through extensive experiments on public datasets, we show that our method is robust across various recommendation regimes, including e-commerce and movie suggestions. Compared to state-of-the-art supervised learning algorithms, our algorithm yields recommendations of higher quality, demonstrating the clear advantage of combining RL and transformers. △ Less

Submitted 26 July, 2023; originally announced July 2023.

Matrix Estimation for Offline Reinforcement Learning with Low-Rank Structure

Authors: Xumei Xi, Christina Lee Yu, Yudong Chen

Abstract: We consider offline Reinforcement Learning (RL), where the agent does not interact with the environment and must rely on offline data collected using a behavior policy. Previous works provide policy evaluation guarantees when the target policy to be evaluated is covered by the behavior policy, that is, state-action pairs visited by the target policy must also be visited by the behavior policy. We… ▽ More We consider offline Reinforcement Learning (RL), where the agent does not interact with the environment and must rely on offline data collected using a behavior policy. Previous works provide policy evaluation guarantees when the target policy to be evaluated is covered by the behavior policy, that is, state-action pairs visited by the target policy must also be visited by the behavior policy. We show that when the MDP has a latent low-rank structure, this coverage condition can be relaxed. Building on the connection to weighted matrix completion with non-uniform observations, we propose an offline policy evaluation algorithm that leverages the low-rank structure to estimate the values of uncovered state-action pairs. Our algorithm does not require a known feature representation, and our finite-sample error bound involves a novel discrepancy measure quantifying the discrepancy between the behavior and target policies in the spectral space. We provide concrete examples where our algorithm achieves accurate estimation while existing coverage conditions are not satisfied. Building on the above evaluation algorithm, we further design an offline policy optimization algorithm and provide non-asymptotic performance guarantees. △ Less

Submitted 24 May, 2023; originally announced May 2023.

Dialog-to-Actions: Building Task-Oriented Dialogue System via Action-Level Generation

Authors: Yuncheng Hua, Xiangyu Xi, Zheng Jiang, Guanwei Zhang, Chaobo Sun, Guanglu Wan, Wei Ye

Abstract: End-to-end generation-based approaches have been investigated and applied in task-oriented dialogue systems. However, in industrial scenarios, existing methods face the bottlenecks of controllability (e.g., domain-inconsistent responses, repetition problem, etc) and efficiency (e.g., long computation time, etc). In this paper, we propose a task-oriented dialogue system via action-level generation.… ▽ More End-to-end generation-based approaches have been investigated and applied in task-oriented dialogue systems. However, in industrial scenarios, existing methods face the bottlenecks of controllability (e.g., domain-inconsistent responses, repetition problem, etc) and efficiency (e.g., long computation time, etc). In this paper, we propose a task-oriented dialogue system via action-level generation. Specifically, we first construct dialogue actions from large-scale dialogues and represent each natural language (NL) response as a sequence of dialogue actions. Further, we train a Sequence-to-Sequence model which takes the dialogue history as input and outputs sequence of dialogue actions. The generated dialogue actions are transformed into verbal responses. Experimental results show that our light-weighted method achieves competitive performance, and has the advantage of controllability and efficiency. △ Less

Submitted 3 April, 2023; originally announced April 2023.

Comments: Accepted at SIGIR 2023 Industry Track

MetaViewer: Towards A Unified Multi-View Representation

Authors: Ren Wang, Haoliang Sun, Yuling Ma, Xiaoming Xi, Yilong Yin

Abstract: Existing multi-view representation learning methods typically follow a specific-to-uniform pipeline, extracting latent features from each view and then fusing or aligning them to obtain the unified object representation. However, the manually pre-specify fusion functions and view-private redundant information mixed in features potentially degrade the quality of the derived representation. To overc… ▽ More Existing multi-view representation learning methods typically follow a specific-to-uniform pipeline, extracting latent features from each view and then fusing or aligning them to obtain the unified object representation. However, the manually pre-specify fusion functions and view-private redundant information mixed in features potentially degrade the quality of the derived representation. To overcome them, we propose a novel bi-level-optimization-based multi-view learning framework, where the representation is learned in a uniform-to-specific manner. Specifically, we train a meta-learner, namely MetaViewer, to learn fusion and model the view-shared meta representation in outer-level optimization. Start with this meta representation, view-specific base-learners are then required to rapidly reconstruct the corresponding view in inner-level. MetaViewer eventually updates by observing reconstruction processes from uniform to specific over all views, and learns an optimal fusion scheme that separates and filters out view-private information. Extensive experimental results in downstream tasks such as classification and clustering demonstrate the effectiveness of our method. △ Less

Submitted 11 March, 2023; originally announced March 2023.

Comments: 8 pages, 5 figures, conference

A Provably Secure Strong PUF based on LWE: Construction and Implementation

Authors: Xiaodan Xi, Ge Li, Ye Wang, Yeonsoo Jeon, Michael Orshansky

Abstract: We construct a strong PUF with provable security against ML attacks on both classical and quantum computers. The security is guaranteed by the cryptographic hardness of learning decryption functions of public-key cryptosystems, and the hardness of the learning-with-errors (LWE) problem defined on integer lattices. We call our construction the lattice PUF. We construct lattice PUF with a physical… ▽ More We construct a strong PUF with provable security against ML attacks on both classical and quantum computers. The security is guaranteed by the cryptographic hardness of learning decryption functions of public-key cryptosystems, and the hardness of the learning-with-errors (LWE) problem defined on integer lattices. We call our construction the lattice PUF. We construct lattice PUF with a physically obfuscated key and an LWE decryption function block. To allow deployments in different scenarios, we demonstrate designs with different latency-area trade-offs. A compact design uses a highly serialized LFSR and LWE decryption function, while a latency-optimized design uses an unrolled LFSR and a parallel datapath. We prototype lattice PUF designs with $2^{136}$ challenge-response pairs (CRPs) on a Spartan 6 FPGA. In addition to theoretical security guarantee, we evaluate empirical resistance to the various leading ML techniques: the prediction error remains above $49.76\%$ after $1$ million training CRPs. The resource-efficient design requires only $45$ slices for the PUF logic proper, and $351$ slices for a fuzzy extractor. The latency-optimized design achieves a $148X$ reduction in latency, at a $10X$ increase in PUF hardware utilization. The mean uniformity of PUF responses is $49.98\%$, the mean uniqueness is $50.00\%$, and the mean reliability is $1.26\%$. △ Less

Submitted 5 March, 2023; originally announced March 2023.

Chaos to Order: A Label Propagation Perspective on Source-Free Domain Adaptation

Authors: Chunwei Wu, Guitao Cao, Yan Li, Xidong Xi, Wenming Cao, Hong Wang

Abstract: Source-free domain adaptation (SFDA), where only a pre-trained source model is used to adapt to the target distribution, is a more general approach to achieving domain adaptation in the real world. However, it can be challenging to capture the inherent structure of the target features accurately due to the lack of supervised information on the target domain. By analyzing the clustering performance… ▽ More Source-free domain adaptation (SFDA), where only a pre-trained source model is used to adapt to the target distribution, is a more general approach to achieving domain adaptation in the real world. However, it can be challenging to capture the inherent structure of the target features accurately due to the lack of supervised information on the target domain. By analyzing the clustering performance of the target features, we show that they still contain core features related to discriminative attributes but lack the collation of semantic information. Inspired by this insight, we present Chaos to Order (CtO), a novel approach for SFDA that strives to constrain semantic credibility and propagate label information among target subpopulations. CtO divides the target data into inner and outlier samples based on the adaptive threshold of the learning state, customizing the learning strategy to fit the data properties best. Specifically, inner samples are utilized for learning intra-class structure thanks to their relatively well-clustered properties. The low-density outlier samples are regularized by input consistency to achieve high accuracy with respect to the ground truth labels. In CtO, by employing different learning strategies to propagate the labels from the inner local to outlier instances, it clusters the global samples from chaos to order. We further adaptively regulate the neighborhood affinity of the inner samples to constrain the local semantic credibility. In theoretical and empirical analyses, we demonstrate that our algorithm not only propagates from inner to outlier but also prevents local clustering from forming spurious clusters. Empirical evidence demonstrates that CtO outperforms the state of the arts on three public benchmarks: Office-31, Office-Home, and VisDA. △ Less

Submitted 14 August, 2023; v1 submitted 19 January, 2023; originally announced January 2023.

Comments: Accepted by ACM MM2023

Multi-Label Continual Learning using Augmented Graph Convolutional Network

Authors: Kaile Du, Fan Lyu, Linyan Li, Fuyuan Hu, Wei Feng, Fenglei Xu, Xuefeng Xi, Hanjing Cheng

Abstract: Multi-Label Continual Learning (MLCL) builds a class-incremental framework in a sequential multi-label image recognition data stream. The critical challenges of MLCL are the construction of label relationships on past-missing and future-missing partial labels of training data and the catastrophic forgetting on old classes, resulting in poor generalization. To solve the problems, the study proposes… ▽ More Multi-Label Continual Learning (MLCL) builds a class-incremental framework in a sequential multi-label image recognition data stream. The critical challenges of MLCL are the construction of label relationships on past-missing and future-missing partial labels of training data and the catastrophic forgetting on old classes, resulting in poor generalization. To solve the problems, the study proposes an Augmented Graph Convolutional Network (AGCN++) that can construct the cross-task label relationships in MLCL and sustain catastrophic forgetting. First, we build an Augmented Correlation Matrix (ACM) across all seen classes, where the intra-task relationships derive from the hard label statistics. In contrast, the inter-task relationships leverage hard and soft labels from data and a constructed expert network. Then, we propose a novel partial label encoder (PLE) for MLCL, which can extract dynamic class representation for each partial label image as graph nodes and help generate soft labels to create a more convincing ACM and suppress forgetting. Last, to suppress the forgetting of label dependencies across old tasks, we propose a relationship-preserving constrainter to construct label relationships. The inter-class topology can be augmented automatically, which also yields effective class representations. The proposed method is evaluated using two multi-label image benchmarks. The experimental results show that the proposed way is effective for MLCL image recognition and can build convincing correlations across tasks even if the labels of previous tasks are missing. △ Less

Submitted 27 November, 2022; originally announced November 2022.

MUSIED: A Benchmark for Event Detection from Multi-Source Heterogeneous Informal Texts

Authors: Xiangyu Xi, Jianwei Lv, Shuaipeng Liu, Wei Ye, Fan Yang, Guanglu Wan

Abstract: Event detection (ED) identifies and classifies event triggers from unstructured texts, serving as a fundamental task for information extraction. Despite the remarkable progress achieved in the past several years, most research efforts focus on detecting events from formal texts (e.g., news articles, Wikipedia documents, financial announcements). Moreover, the texts in each dataset are either from… ▽ More Event detection (ED) identifies and classifies event triggers from unstructured texts, serving as a fundamental task for information extraction. Despite the remarkable progress achieved in the past several years, most research efforts focus on detecting events from formal texts (e.g., news articles, Wikipedia documents, financial announcements). Moreover, the texts in each dataset are either from a single source or multiple yet relatively homogeneous sources. With massive amounts of user-generated text accumulating on the Web and inside enterprises, identifying meaningful events in these informal texts, usually from multiple heterogeneous sources, has become a problem of significant practical value. As a pioneering exploration that expands event detection to the scenarios involving informal and heterogeneous texts, we propose a new large-scale Chinese event detection dataset based on user reviews, text conversations, and phone conversations in a leading e-commerce platform for food service. We carefully investigate the proposed dataset's textual informality and multi-source heterogeneity characteristics by inspecting data samples quantitatively and qualitatively. Extensive experiments with state-of-the-art event detection methods verify the unique challenges posed by these characteristics, indicating that multi-source informal event detection remains an open problem and requires further efforts. Our benchmark and code are released at \url{https://github.com/myeclipse/MUSIED}. △ Less

Submitted 25 November, 2022; originally announced November 2022.

Comments: Accepted at EMNLP 2022

Piecewise Linear Neural Networks and Deep Learning

Authors: Qinghua Tao, Li Li, Xiaolin Huang, Xiangming Xi, Shuning Wang, Johan A. K. Suykens

Abstract: As a powerful modelling method, PieceWise Linear Neural Networks (PWLNNs) have proven successful in various fields, most recently in deep learning. To apply PWLNN methods, both the representation and the learning have long been studied. In 1977, the canonical representation pioneered the works of shallow PWLNNs learned by incremental designs, but the applications to large-scale data were prohibite… ▽ More As a powerful modelling method, PieceWise Linear Neural Networks (PWLNNs) have proven successful in various fields, most recently in deep learning. To apply PWLNN methods, both the representation and the learning have long been studied. In 1977, the canonical representation pioneered the works of shallow PWLNNs learned by incremental designs, but the applications to large-scale data were prohibited. In 2010, the Rectified Linear Unit (ReLU) advocated the prevalence of PWLNNs in deep learning. Ever since, PWLNNs have been successfully applied to extensive tasks and achieved advantageous performances. In this Primer, we systematically introduce the methodology of PWLNNs by grouping the works into shallow and deep networks. Firstly, different PWLNN representation models are constructed with elaborated examples. With PWLNNs, the evolution of learning algorithms for data is presented and fundamental theoretical analysis follows up for in-depth understandings. Then, representative applications are introduced together with discussions and outlooks. △ Less

Submitted 18 June, 2022; originally announced June 2022.

Comments: 23 pages, 6 figures

A Low-Cost, Controllable and Interpretable Task-Oriented Chatbot: With Real-World After-Sale Services as Example

Authors: Xiangyu Xi, Chenxu Lv, Yuncheng Hua, Wei Ye, Chaobo Sun, Shuaipeng Liu, Fan Yang, Guanglu Wan

Abstract: Though widely used in industry, traditional task-oriented dialogue systems suffer from three bottlenecks: (i) difficult ontology construction (e.g., intents and slots); (ii) poor controllability and interpretability; (iii) annotation-hungry. In this paper, we propose to represent utterance with a simpler concept named Dialogue Action, upon which we construct a tree-structured TaskFlow and further… ▽ More Though widely used in industry, traditional task-oriented dialogue systems suffer from three bottlenecks: (i) difficult ontology construction (e.g., intents and slots); (ii) poor controllability and interpretability; (iii) annotation-hungry. In this paper, we propose to represent utterance with a simpler concept named Dialogue Action, upon which we construct a tree-structured TaskFlow and further build task-oriented chatbot with TaskFlow as core component. A framework is presented to automatically construct TaskFlow from large-scale dialogues and deploy online. Our experiments on real-world after-sale customer services show TaskFlow can satisfy the major needs, as well as reduce the developer burden effectively. △ Less

Submitted 12 May, 2022; originally announced May 2022.

Comments: Accept at SIGIR Industry Track 2022

Testing Relative Fairness in Human Decisions With Machine Learning

Authors: Zhe Yu, Xiaoyin Xi

Abstract: Fairness in decision-making has been a long-standing issue in our society. Compared to algorithmic fairness, fairness in human decisions is even more important since there are processes where humans make the final decisions and that machine learning models inherit bias from the human decisions they were trained on. However, the standard for fairness in human decisions are highly subjective and con… ▽ More Fairness in decision-making has been a long-standing issue in our society. Compared to algorithmic fairness, fairness in human decisions is even more important since there are processes where humans make the final decisions and that machine learning models inherit bias from the human decisions they were trained on. However, the standard for fairness in human decisions are highly subjective and contextual. This leads to the difficulty for testing "absolute" fairness in human decisions. To bypass this issue, this work aims to test relative fairness in human decisions. That is, instead of defining what are "absolute" fair decisions, we check the relative fairness of one decision set against another. An example outcome can be: Decision Set A favors female over male more than Decision Set B. Such relative fairness has the following benefits: (1) it avoids the ambiguous and contradictory definition of "absolute" fair decisions; (2) it reveals the relative preference and bias between different human decisions; (3) if a reference set of decisions is provided, relative fairness of other decision sets against this reference set can reflect whether those decision sets are fair by the standard of that reference set. We define the relative fairness with statistical tests (null hypothesis and effect size tests) of the decision differences across each sensitive group. Furthermore, we show that a machine learning model trained on the human decisions can inherit the bias/preference and therefore can be utilized to estimate the relative fairness between two decision sets made on different data. △ Less

Submitted 17 December, 2023; v1 submitted 21 December, 2021; originally announced December 2021.

Comments: Submitted to ICLR'24

Capturing Event Argument Interaction via A Bi-Directional Entity-Level Recurrent Decoder

Authors: Xiangyu Xi, Wei Ye, Shikun Zhang, Quanxiu Wang, Huixing Jiang, Wei Wu

Abstract: Capturing interactions among event arguments is an essential step towards robust event argument extraction (EAE). However, existing efforts in this direction suffer from two limitations: 1) The argument role type information of contextual entities is mainly utilized as training signals, ignoring the potential merits of directly adopting it as semantically rich input features; 2) The argument-level… ▽ More Capturing interactions among event arguments is an essential step towards robust event argument extraction (EAE). However, existing efforts in this direction suffer from two limitations: 1) The argument role type information of contextual entities is mainly utilized as training signals, ignoring the potential merits of directly adopting it as semantically rich input features; 2) The argument-level sequential semantics, which implies the overall distribution pattern of argument roles over an event mention, is not well characterized. To tackle the above two bottlenecks, we formalize EAE as a Seq2Seq-like learning problem for the first time, where a sentence with a specific event trigger is mapped to a sequence of event argument roles. A neural architecture with a novel Bi-directional Entity-level Recurrent Decoder (BERD) is proposed to generate argument roles by incorporating contextual entities' argument role predictions, like a word-by-word text generation process, thereby distinguishing implicit argument distribution patterns within an event more accurately. △ Less

Submitted 30 June, 2021; originally announced July 2021.

Journal ref: ACL-IJCNLP 2021

CoMP-Enabled RAN Slicing for Tactile Internet

Authors: Peng Yang, Xing Xi, Tony Q. S. Quek, Hyundong Shin

Abstract: Tactile Internet (TI) enables the omnipresence and exchange of tactile experiences across the global via the ultra-reliable and ultra-responsive connectivity. This article argues for coordinated multi-point (CoMP) enabled radio access network (RAN) slicing as an efficient solution that satisfies the stringent reliable and responsive connectivity requirements for supporting tactile applications. Th… ▽ More Tactile Internet (TI) enables the omnipresence and exchange of tactile experiences across the global via the ultra-reliable and ultra-responsive connectivity. This article argues for coordinated multi-point (CoMP) enabled radio access network (RAN) slicing as an efficient solution that satisfies the stringent reliable and responsive connectivity requirements for supporting tactile applications. This article presents the emerging challenges when accommodating CoMP-enabled RAN slicing in the TI ecosystem and expounds the functional split of CoMP-enabled RAN. Besides, this article elaborates on the implementation prototype of CoMP-enabled RAN slicing for TI with the coexistence of diverse vertical applications. Finally, this article studies a use case of enabling TI-included application multiplexing as an example of CoMP-enabled RAN slicing for TI. △ Less

Submitted 17 April, 2021; v1 submitted 17 March, 2021; originally announced March 2021.

Fresh, Fair and Energy-Efficient Content Provision in a Private and Cache-Enabled UAV Network

Authors: Peng Yang, Kun Guo, Xing Xi, Tony Q. S. Quek, Xianbin Cao, Chenxi Liu

Abstract: In this paper, we investigate a private and cache-enabled unmanned aerial vehicle (UAV) network for content provision. Aiming at delivering fresh, fair, and energy-efficient content files to terrestrial users, we formulate a joint UAV caching, UAV trajectory, and UAV transmit power optimization problem. This problem is confirmed to be a sequential decision problem with mixed-integer non-convex con… ▽ More In this paper, we investigate a private and cache-enabled unmanned aerial vehicle (UAV) network for content provision. Aiming at delivering fresh, fair, and energy-efficient content files to terrestrial users, we formulate a joint UAV caching, UAV trajectory, and UAV transmit power optimization problem. This problem is confirmed to be a sequential decision problem with mixed-integer non-convex constraints, which is intractable directly. To this end, we propose a novel algorithm based on the techniques of subproblem decomposition and convex approximation. Particularly, we first propose to decompose the sequential decision problem into multiple repeated optimization subproblems via a Lyapunov technique. Next, an iterative optimization scheme incorporating a successive convex approximation (SCA) technique is explored to tackle the challenging mixed-integer non-convex subproblems. Besides, we analyze the convergence and computational complexity of the proposed algorithm and derive the theoretical value of the expected peak age of information (PAoI) to estimate the content freshness. Simulation results demonstrate that the proposed algorithm can achieve the expected PAoI close to the theoretical value and is more 22.11% and 70.51% energy-efficient and fairer than benchmark algorithms. △ Less

Submitted 26 February, 2021; v1 submitted 25 February, 2021; originally announced February 2021.

Power Control for a URLLC-enabled UAV system incorporated with DNN-Based Channel Estimation

Authors: Peng Yang, Xing Xi, Tony Q. S. Quek, Xianbin Cao, Jingxuan Chen

Abstract: This letter is concerned with power control for a ultra-reliable and low-latency communications (URLLC) enabled unmanned aerial vehicle (UAV) system incorporated with deep neural network (DNN) based channel estimation. Particularly, we formulate the power control problem for the UAV system as an optimization problem to accommodate the URLLC requirement of uplink control and non-payload signal deli… ▽ More This letter is concerned with power control for a ultra-reliable and low-latency communications (URLLC) enabled unmanned aerial vehicle (UAV) system incorporated with deep neural network (DNN) based channel estimation. Particularly, we formulate the power control problem for the UAV system as an optimization problem to accommodate the URLLC requirement of uplink control and non-payload signal delivery while ensuring the downlink high-speed payload transmission. This problem is challenging to be solved due to the requirement of analytically tractable channel models and the non-convex characteristic as well. To address the challenges, we propose a novel power control algorithm, which constructs analytically tractable channel models based on DNN estimation results and explores a semidefinite relaxation (SDR) scheme to tackle the non-convexity. Simulation results demonstrate the accuracy of the DNN estimation and verify the effectiveness of the proposed algorithm. △ Less

Submitted 13 November, 2020; originally announced December 2020.

Attention Model Enhanced Network for Classification of Breast Cancer Image

Authors: Xiao Kang, Xingbo Liu, Xiushan Nie, Xiaoming Xi, Yilong Yin

Abstract: Breast cancer classification remains a challenging task due to inter-class ambiguity and intra-class variability. Existing deep learning-based methods try to confront this challenge by utilizing complex nonlinear projections. However, these methods typically extract global features from entire images, neglecting the fact that the subtle detail information can be crucial in extracting discriminativ… ▽ More Breast cancer classification remains a challenging task due to inter-class ambiguity and intra-class variability. Existing deep learning-based methods try to confront this challenge by utilizing complex nonlinear projections. However, these methods typically extract global features from entire images, neglecting the fact that the subtle detail information can be crucial in extracting discriminative features. In this study, we propose a novel method named Attention Model Enhanced Network (AMEN), which is formulated in a multi-branch fashion with pixel-wised attention model and classification submodular. Specifically, the feature learning part in AMEN can generate pixel-wised attention map, while the classification submodular are utilized to classify the samples. To focus more on subtle detail information, the sample image is enhanced by the pixel-wised attention map generated from former branch. Furthermore, boosting strategy are adopted to fuse classification results from different branches for better performance. Experiments conducted on three benchmark datasets demonstrate the superiority of the proposed method under various scenarios. △ Less

Submitted 7 October, 2020; originally announced October 2020.

Local Minima Structures in Gaussian Mixture Models

Authors: Yudong Chen, Dogyoon Song, Xumei Xi, Yuqian Zhang

Abstract: We investigate the landscape of the negative log-likelihood function of Gaussian Mixture Models (GMMs) with a general number of components in the population limit. As the objective function is non-convex, there can be multiple local minima that are not globally optimal, even for well-separated mixture models. Our study reveals that all local minima share a common structure that partially identifie… ▽ More We investigate the landscape of the negative log-likelihood function of Gaussian Mixture Models (GMMs) with a general number of components in the population limit. As the objective function is non-convex, there can be multiple local minima that are not globally optimal, even for well-separated mixture models. Our study reveals that all local minima share a common structure that partially identifies the cluster centers (i.e., means of the Gaussian components) of the true location mixture. Specifically, each local minimum can be represented as a non-overlapping combination of two types of sub-configurations: fitting a single mean estimate to multiple Gaussian components or fitting multiple estimates to a single true component. These results apply to settings where the true mixture components satisfy a certain separation condition, and are valid even when the number of components is over- or under-specified. We also present a more fine-grained analysis for the setting of one-dimensional GMMs with three components, which provide sharper approximation error bounds with improved dependence on the separation. △ Less

Submitted 9 March, 2024; v1 submitted 27 September, 2020; originally announced September 2020.

Comments: 73 pages, 6 figures, 2Tables. To appear in Transactions on Information Theory

Journal ref: IEEE Transactions on Information Theory, 2024

On The Optimal Number of Reflecting Elements for Reconfigurable Intelligent Surfaces

Authors: Alessio Zappone, Marco Di Renzo, Xiaojun Xi, Merouane Debbah

Abstract: This work considers a point-to-point link where a reconfigurable intelligent surface assists the communication between transmitter and receiver. The system rate, energy efficiency, and their trade-off are optimized with respect to the number of individually tunable elements of the intelligent surface. The resource allocation accounts for the communication phase and for the overhead due to channel… ▽ More This work considers a point-to-point link where a reconfigurable intelligent surface assists the communication between transmitter and receiver. The system rate, energy efficiency, and their trade-off are optimized with respect to the number of individually tunable elements of the intelligent surface. The resource allocation accounts for the communication phase and for the overhead due to channel estimation and to reporting the optimized resource allocation to the intelligent surface. Numerical results confirm the optimality of the proposed methods and show the potential gains of reconfigurable intelligent surfaces. △ Less

Submitted 15 July, 2020; originally announced July 2020.

Multicast eMBB and Bursty URLLC Service Multiplexing in a CoMP-Enabled RAN

Authors: Peng Yang, Xing Xi, Yaru Fu, Tony Q. S. Quek, Xianbin Cao, Dapeng Wu

Abstract: This paper is concerned with slicing a radio access network (RAN) for simultaneously serving two typical 5G and beyond use cases, i.e., enhanced mobile broadband (eMBB) and ultra-reliable and low latency communications (URLLC). Although many researches have been conducted to tackle this issue, few of them have considered the impact of bursty URLLC. The bursty characteristic of URLLC traffic may si… ▽ More This paper is concerned with slicing a radio access network (RAN) for simultaneously serving two typical 5G and beyond use cases, i.e., enhanced mobile broadband (eMBB) and ultra-reliable and low latency communications (URLLC). Although many researches have been conducted to tackle this issue, few of them have considered the impact of bursty URLLC. The bursty characteristic of URLLC traffic may significantly increase the difficulty of RAN slicing on the aspect of ensuring a ultra-low packet blocking probability. To reduce the packet blocking probability, we re-visit the structure of physical resource blocks (PRBs) orchestrated for bursty URLLC traffic in the time-frequency plane based on our theoretical results. Meanwhile, we formulate the problem of slicing a RAN enabling coordinated multi-point (CoMP) transmissions for multicast eMBB and bursty URLLC service multiplexing as a multi-timescale optimization problem. The goal of this problem is to maximize multicast eMBB and bursty URLLC slice utilities, subject to physical resource constraints. To mitigate this thorny multi-timescale problem, we transform it into multiple single timescale problems by exploring the fundamental principle of a sample average approximation (SAA) technique. Next, an iterative algorithm with provable performance guarantees is developed to obtain solutions to these single timescale problems and aggregate the obtained solutions into those of the multi-timescale problem. We also design a prototype for the CoMP-enabled RAN slicing system incorporating with multicast eMBB and bursty URLLC traffic and compare the proposed iterative algorithm with the state-of-the-art algorithm to verify the effectiveness of the algorithm. △ Less

Submitted 21 February, 2020; originally announced February 2020.

Analytical Modeling of the Path-Loss for Reconfigurable Intelligent Surfaces -- Anomalous Mirror or Scatterer ?

Authors: Marco Di Renzo, Fadil Habibi Danufane, Xiaojun Xi, Julien de Rosny, Sergei Tretyakov

Abstract: Reconfigurable intelligent surfaces (RISs) are an emerging field of research in wireless communications. A fundamental component for analyzing and optimizing RIS-empowered wireless networks is the development of simple but sufficiently accurate models for the power scattered by an RIS. By leveraging the general scalar theory of diffraction and the Huygens-Fresnel principle, we introduce simple for… ▽ More Reconfigurable intelligent surfaces (RISs) are an emerging field of research in wireless communications. A fundamental component for analyzing and optimizing RIS-empowered wireless networks is the development of simple but sufficiently accurate models for the power scattered by an RIS. By leveraging the general scalar theory of diffraction and the Huygens-Fresnel principle, we introduce simple formulas for the electric field scattered by an RIS that is modeled as a sheet of electromagnetic material of negligible thickness. The proposed approach allows us to identify the conditions under which an RIS of finite size can or cannot be approximated as an anomalous mirror. Numerical results are illustrated to confirm the proposed approach. △ Less

Submitted 23 January, 2020; originally announced January 2020.

RAN Slicing for Massive IoT and Bursty URLLC Service Multiplexing: Analysis and Optimization

Authors: Peng Yang, Xing Xi, Tony Q. S. Quek, Jingxuan Chen, Xianbin Cao, Dapeng Wu

Abstract: Future wireless networks are envisioned to serve massive Internet of things (mIoT) via some radio access technologies, where the random access channel (RACH) procedure should be exploited for IoT devices to access the networks. However, the theoretical analysis of the RACH procedure for massive IoT devices is challenging. To address this challenge, we first correlate the RACH request of an IoT dev… ▽ More Future wireless networks are envisioned to serve massive Internet of things (mIoT) via some radio access technologies, where the random access channel (RACH) procedure should be exploited for IoT devices to access the networks. However, the theoretical analysis of the RACH procedure for massive IoT devices is challenging. To address this challenge, we first correlate the RACH request of an IoT device with the status of its maintained queue and analyze the evolution of the queue status. Based on the analysis result, we then derive the closed-form expression of the random access (RA) success probability, which is a significant indicator characterizing the RACH procedure of the device. Besides, considering the agreement on converging different services onto a shared infrastructure, we investigate the RAN slicing for mIoT and bursty ultra-reliable and low latency communications (URLLC) service multiplexing. Specifically, we formulate the RAN slicing problem as an optimization one to maximize the total RA success probabilities of all IoT devices and provide URLLC services for URLLC devices in an energy-efficient way. A slice resource optimization (SRO) algorithm exploiting relaxation and approximation with provable tightness and error bound is then proposed to mitigate the optimization problem. Simulation results demonstrate that the proposed SRO algorithm can effectively implement the service multiplexing of mIoT and bursty URLLC traffic. △ Less

Submitted 29 January, 2021; v1 submitted 13 January, 2020; originally announced January 2020.

Energy-Efficient Resource Allocation in a Multi-UAV-Aided NOMA Network

Authors: Xing Xi, Xianbin Cao, Peng Yang, Jingxuan Chen, Dapeng Wu

Abstract: This paper is concerned with the resource allocation in a multi-unmanned aerial vehicle (UAV)-aided network for providing enhanced mobile broadband (eMBB) services for user equipments. Different from most of the existing network resource allocation approaches, we investigate a joint non-orthogonal user association, subchannel allocation and power control problem. The objective of the problem is to… ▽ More This paper is concerned with the resource allocation in a multi-unmanned aerial vehicle (UAV)-aided network for providing enhanced mobile broadband (eMBB) services for user equipments. Different from most of the existing network resource allocation approaches, we investigate a joint non-orthogonal user association, subchannel allocation and power control problem. The objective of the problem is to maximize the network energy efficiency under the constraints on user equipments' quality of service, UAVs' network capacity and power consumption. We formulate the energy efficiency maximization problem as a challenging mixed-integer non-convex programming problem. To alleviate this problem, we first decompose the original problem into two subproblems, namely, an integer non-linear user association and subchannel allocation subproblem and a non-convex power control subproblem. We then design a two-stage approximation strategy to handle the non-linearity of the user association and subchannel allocation subproblem and exploit a successive convex approximation approach to tackle the non-convexity of the power control subproblem. Based on the derived results, we develop an iterative algorithm with provable convergence to mitigate the original problem. Simulation results show that our proposed framework can improve energy efficiency compared with several benchmark algorithms. △ Less

Submitted 31 December, 2020; v1 submitted 7 December, 2019; originally announced December 2019.

Repeatedly Energy-Efficient and Fair Service Coverage: UAV Slicing (Proactive UAV Network Slicing for URLLC and Mobile Broadband Service Multiplexing)

Authors: Peng Yang, Xing Xi, Tony Q. S. Quek, Jingxuan Chen, Xianbin Cao, Dapeng Wu

Abstract: Unmanned aerial vehicle (UAV) networks are convinced as a significant part of 5G and emerging 6G wireless networks. UAV slicing is a promising proposal of converging different services onto a common UAV network without deploying individual network solution for each type of service. This paper is concerned with UAV slicing for providing energy-efficient and fair service coverage for enhanced mobile… ▽ More Unmanned aerial vehicle (UAV) networks are convinced as a significant part of 5G and emerging 6G wireless networks. UAV slicing is a promising proposal of converging different services onto a common UAV network without deploying individual network solution for each type of service. This paper is concerned with UAV slicing for providing energy-efficient and fair service coverage for enhanced mobile broadband (eMBB) users (UEs). Aiming at physically configuring UAV slices, the UAV slicing problem is formulated as a time-dependent mixed-integer-non-convex programming problem with a goal of maximizing all UEs' data rates while minimizing UAVs' total transmit power. To mitigate this challenging problem, we first decompose the original problem into two time-dependent subproblems using a Lyapunov approach. We then derive the procedure of tackling the non-convexity and the mixed-integer property of the subproblems by exploring a successive convex approximate (SCA) method and an alternative optimization scheme, respectively. Based on the derived results, we develop an algorithm with provable performance guarantees to mitigate the two subproblems repeatedly. △ Less

Submitted 21 October, 2020; v1 submitted 7 December, 2019; originally announced December 2019.

Comments: The original title is: Repeatedly Energy-Efficient and Fair Service Coverage: UAV Slicing

How Should I Orchestrate Resources of My Slices for Bursty URLLC Service Provision?

Authors: Peng Yang, Xing Xi, Tony Q. S. Quek, Jingxuan Chen, Xianbin Cao, Dapeng Wu

Abstract: Future wireless networks are convinced to provide flexible and cost-efficient services via exploiting network slicing techniques. However, it is challenging to configure network slicing systems for bursty ultra-reliable and low latency communications (URLLC) service provision due to its stringent requirements on low packet blocking probability and low codeword error decoding probability. In this p… ▽ More Future wireless networks are convinced to provide flexible and cost-efficient services via exploiting network slicing techniques. However, it is challenging to configure network slicing systems for bursty ultra-reliable and low latency communications (URLLC) service provision due to its stringent requirements on low packet blocking probability and low codeword error decoding probability. In this paper, we propose to orchestrate network resources for a network slicing system to guarantee a more reliable bursty URLLC service provision. We re-cut physical resource blocks (PRBs) and derive the minimum upper bound of bandwidth for URLLC transmission with a low packet blocking probability. We correlate coordinated multipoint (CoMP) beamforming with channel uses and derive the minimum upper bound of channel uses for URLLC transmission with a low codeword error decoding probability. Considering the agreement on converging diverse services onto shared infrastructures, we further investigate the network slicing for URLLC and enhanced mobile broadband (eMBB) service multiplexing. Particularly, we formulate the service multiplexing as an optimization problem to maximize the long-term total slice utility. The mitigation of this problem is challenging due to the requirements of future channel information and tackling a two timescale issue. To address the challenges, we develop a joint resource optimization algorithm based on a sample average approximate (SAA) technique and a distributed optimization method with provable performance guarantees. △ Less

Submitted 7 November, 2020; v1 submitted 2 December, 2019; originally announced December 2019.

Lattice PUF: A Strong Physical Unclonable Function Provably Secure against Machine Learning Attacks

Authors: Ye Wang, Xiaodan Xi, Michael Orshansky

Abstract: We propose a strong physical unclonable function (PUF) provably secure against machine learning (ML) attacks with both classical and quantum computers. Its security is derived from cryptographic hardness of learning decryption functions of public-key cryptosystems. Our design compactly realizes the decryption function of the learning-with-errors (LWE) cryptosystem. Due to the fundamental connectio… ▽ More We propose a strong physical unclonable function (PUF) provably secure against machine learning (ML) attacks with both classical and quantum computers. Its security is derived from cryptographic hardness of learning decryption functions of public-key cryptosystems. Our design compactly realizes the decryption function of the learning-with-errors (LWE) cryptosystem. Due to the fundamental connection of LWE to lattice problems, we call the construction the lattice PUF. Lattice PUF is constructed using a physically obfuscated key (POK), an LWE decryption function block, and a linear-feedback shift register (LFSR) as a pseudo-random number generator. The POK provides the secret key of the LWE decryption function; its stability is ensured by a fuzzy extractor (FE). To reduce the challenge size, we exploit distributional relaxations of space-efficient LWEs. That allows only a small challenge-seed to be transmitted with the full-length challenge generated by the LFSR, resulting in a 100X reduction of communication cost. To prevent an active challenge-manipulation attack, a self-incrementing counter is embedded into the challenge seed. We prototyped the lattice PUF with 2^136 challenge-response pairs (CRPs) on a Spartan 6 FPGA, which required 45 slices for the PUF logic proper and 233 slices for the FE. Simulation-based evaluation shows the mean (std) of uniformity to be 49.98% (1.58%), of uniqueness to be 50.00% (1.58%), and of reliability to be 1.26% (2.88%). The LWE concrete hardness estimator guarantees that a successful ML attack of the lattice PUF will require the infeasible 2^128 CPU operations. Several classes of empirical ML attacks, including support vector machine, logistic regression, and deep neural networks, are used: in all attacks, the prediction error remains above 49.76% after 1 million training CRPs. △ Less

Submitted 16 June, 2020; v1 submitted 29 September, 2019; originally announced September 2019.

Comments: 11 pages, 8 figures, 5 tables

Efficient hinging hyperplanes neural network and its application in nonlinear system identification

Authors: Jun Xu, Qinghua Tao, Zhen Li, Xiangming Xi, Johan A. K. Suykens, Shuning Wang

Abstract: In this paper, the efficient hinging hyperplanes (EHH) neural network is proposed based on the model of hinging hyperplanes (HH). The EHH neural network is a distributed representation, the training of which involves solving several convex optimization problems and is fast. It is proved that for every EHH neural network, there is an equivalent adaptive hinging hyperplanes (AHH) tree, which was als… ▽ More In this paper, the efficient hinging hyperplanes (EHH) neural network is proposed based on the model of hinging hyperplanes (HH). The EHH neural network is a distributed representation, the training of which involves solving several convex optimization problems and is fast. It is proved that for every EHH neural network, there is an equivalent adaptive hinging hyperplanes (AHH) tree, which was also proposed based on the model of HH and find good applications in system identification. The construction of the EHH neural network includes 2 stages. First the initial structure of the EHH neural network is randomly determined and the Lasso regression is used to choose the appropriate network. To alleviate the impact of randomness, secondly, the stacking strategy is employed to formulate a more general network structure. Different from other neural networks, the EHH neural network has interpretability ability, which can be easily obtained through its ANOVA decomposition (or interaction matrix). The interpretability can then be used as a suggestion for input variable selection. The EHH neural network is applied in nonlinear system identification, the simulation results show that the regression vector selected is reasonable and the identification speed is fast, while at the same time, the simulation accuracy is satisfactory. △ Less

Submitted 27 November, 2019; v1 submitted 14 May, 2019; originally announced May 2019.

Comments: submitted to Automatica

Discriminatively Boosted Image Clustering with Fully Convolutional Auto-Encoders

Authors: Fengfu Li, Hong Qiao, Bo Zhang, Xuanyang Xi

Abstract: Traditional image clustering methods take a two-step approach, feature learning and clustering, sequentially. However, recent research results demonstrated that combining the separated phases in a unified framework and training them jointly can achieve a better performance. In this paper, we first introduce fully convolutional auto-encoders for image feature learning and then propose a unified clu… ▽ More Traditional image clustering methods take a two-step approach, feature learning and clustering, sequentially. However, recent research results demonstrated that combining the separated phases in a unified framework and training them jointly can achieve a better performance. In this paper, we first introduce fully convolutional auto-encoders for image feature learning and then propose a unified clustering framework to learn image representations and cluster centers jointly based on a fully convolutional auto-encoder and soft $k$-means scores. At initial stages of the learning procedure, the representations extracted from the auto-encoder may not be very discriminative for latter clustering. We address this issue by adopting a boosted discriminative distribution, where high score assignments are highlighted and low score ones are de-emphasized. With the gradually boosted discrimination, clustering assignment scores are discriminated and cluster purities are enlarged. Experiments on several vision benchmark datasets show that our methods can achieve a state-of-the-art performance. △ Less

Submitted 23 March, 2017; originally announced March 2017.

Comments: 27 pages

A Fast and Compact Saliency Score Regression Network Based on Fully Convolutional Network

Authors: Xuanyang Xi, Yongkang Luo, Fengfu Li, Peng Wang, Hong Qiao

Abstract: Visual saliency detection aims at identifying the most visually distinctive parts in an image, and serves as a pre-processing step for a variety of computer vision and image processing tasks. To this end, the saliency detection procedure must be as fast and compact as possible and optimally processes input images in a real time manner. It is an essential application requirement for the saliency de… ▽ More Visual saliency detection aims at identifying the most visually distinctive parts in an image, and serves as a pre-processing step for a variety of computer vision and image processing tasks. To this end, the saliency detection procedure must be as fast and compact as possible and optimally processes input images in a real time manner. It is an essential application requirement for the saliency detection task. However, contemporary detection methods often utilize some complicated procedures to pursue feeble improvements on the detection precession, which always take hundreds of milliseconds and make them not easy to be applied practically. In this paper, we tackle this problem by proposing a fast and compact saliency score regression network which employs fully convolutional network, a special deep convolutional neural network, to estimate the saliency of objects in images. It is an extremely simplified end-to-end deep neural network without any pre-processings and post-processings. When given an image, the network can directly predict a dense full-resolution saliency map (image-to-image prediction). It works like a compact pipeline which effectively simplifies the detection procedure. Our method is evaluated on six public datasets, and experimental results show that it can achieve comparable or better precision performance than the state-of-the-art methods while get a significant improvement in detection speed (35 FPS, processing in real time). △ Less

Submitted 24 February, 2017; v1 submitted 2 February, 2017; originally announced February 2017.

Image Denoising via Multi-scale Nonlinear Diffusion Models

Authors: Wensen Feng, Peng Qiao, Xuanyang Xi, Yunjin Chen

Abstract: Image denoising is a fundamental operation in image processing and holds considerable practical importance for various real-world applications. Arguably several thousands of papers are dedicated to image denoising. In the past decade, sate-of-the-art denoising algorithm have been clearly dominated by non-local patch-based methods, which explicitly exploit patch self-similarity within image. Howeve… ▽ More Image denoising is a fundamental operation in image processing and holds considerable practical importance for various real-world applications. Arguably several thousands of papers are dedicated to image denoising. In the past decade, sate-of-the-art denoising algorithm have been clearly dominated by non-local patch-based methods, which explicitly exploit patch self-similarity within image. However, in recent two years, discriminatively trained local approaches have started to outperform previous non-local models and have been attracting increasing attentions due to the additional advantage of computational efficiency. Successful approaches include cascade of shrinkage fields (CSF) and trainable nonlinear reaction diffusion (TNRD). These two methods are built on filter response of linear filters of small size using feed forward architectures. Due to the locality inherent in local approaches, the CSF and TNRD model become less effective when noise level is high and consequently introduces some noise artifacts. In order to overcome this problem, in this paper we introduce a multi-scale strategy. To be specific, we build on our newly-developed TNRD model, adopting the multi-scale pyramid image representation to devise a multi-scale nonlinear diffusion process. As expected, all the parameters in the proposed multi-scale diffusion model, including the filters and the influence functions across scales, are learned from training data through a loss based approach. Numerical results on Gaussian and Poisson denoising substantiate that the exploited multi-scale strategy can successfully boost the performance of the original TNRD model with single scale. As a consequence, the resulting multi-scale diffusion models can significantly suppress the typical incorrect features for those noisy images with heavy noise. △ Less

Submitted 21 September, 2016; originally announced September 2016.

The Ho-Zhao Problem

Authors: Weng Kin Ho, Jean Goubault-Larrecq, Achim Jung, Xiaoyong Xi

Abstract: Given a poset $P$, the set, $Γ(P)$, of all Scott closed sets ordered by inclusion forms a complete lattice. A subcategory $\mathbf{C}$ of $\mathbf{Pos}_d$ (the category of posets and Scott-continuous maps) is said to be $Γ$-faithful if for any posets $P$ and $Q$ in $\mathbf{C}$, $Γ(P) \cong Γ(Q)$ implies $P \cong Q$. It is known that the category of all continuous dcpos and the category of bounded… ▽ More Given a poset $P$, the set, $Γ(P)$, of all Scott closed sets ordered by inclusion forms a complete lattice. A subcategory $\mathbf{C}$ of $\mathbf{Pos}_d$ (the category of posets and Scott-continuous maps) is said to be $Γ$-faithful if for any posets $P$ and $Q$ in $\mathbf{C}$, $Γ(P) \cong Γ(Q)$ implies $P \cong Q$. It is known that the category of all continuous dcpos and the category of bounded complete dcpos are $Γ$-faithful, while $\mathbf{Pos}_d$ is not. Ho & Zhao (2009) asked whether the category $\mathbf{DCPO}$ of dcpos is $Γ$-faithful. In this paper, we answer this question in the negative by exhibiting a counterexample. To achieve this, we introduce a new subcategory of dcpos which is $Γ$-faithful. This subcategory subsumes all currently known $Γ$-faithful subcategories. With this new concept in mind, we construct the desired counterexample which relies heavily on Johnstone's famous dcpo which is not sober in its Scott topology. △ Less

Submitted 16 January, 2018; v1 submitted 12 July, 2016; originally announced July 2016.

Comments: 19 pages, 4 figures

MSC Class: 06B35

Journal ref: Logical Methods in Computer Science, Volume 14, Issue 1 (January 17, 2018) lmcs:1529