Most of the existing learning-based dehazing methods require a diverse and large collection of paired hazy/clean images, which is intractable to obtain. Therefore, existing dehazing methods resort to training on synthetic images. This may result in a possible domain shift when treating real scenes. In this paper, we propose a novel unsupervised dehazing (lightweight) network without any reference images to directly predict clear images from the original hazy images, which consists of an interactive fusion module (IFM) and an iterative optimization module (IOM). Specifically, IFM interactively fuses multi-level features to make up for the missing information among deep and shallow features while IOM iteratively optimizes dehazed results to obtain pleasing visual effects. Particularly, based on the observation that hazy images usually suffer from quality degradation, four non-reference visual-quality-driven loss functions are designed to enable the network trained in an unsupervised way, including dark channel loss, contrast loss, saturation loss, and edge sharpness loss. Extensive experiments on two synthetic datasets and one real-world dataset demonstrate that our method performs favorably against the state-of-the-art unsupervised dehazing methods and even matches some supervised methods in terms of metrics such as PSNR, SSIM, and UQI.

Non-stationarity of EEG signals leads to high variability between subjects, making it challenging to directly use data from other subjects (source domain) for the classifier in the current subject (target domain). In this study, we propose MI-DAGSC to address domain adaptation challenges in EEG-based motor imagery (MI) decoding. By combining domain-level information, class-level information, and inter-sample structure information, our model effectively aligns the feature distributions of source and target domains. This work is an extension of our previous domain adaptation work MI-DABAN (Li et al., 2023). Based on MI-DABAN, MI-DAGSC designs Sample-Feature Blocks (SFBs) and Graph Convolution Blocks (GCBs) to focus on intra-sample and inter-sample information. The synergistic integration of SFBs and GCBs enable the model to capture comprehensive information and understand the relationship between samples, thus improving representation learning. Furthermore, we introduce a triplet loss to enhance the alignment and compactness of feature representations. Extensive experiments on real EEG datasets demonstrate the effectiveness of MI-DAGSC, confirming that our method makes a valuable contribution to the MI-EEG decoding. Moreover, it holds great potential for various applications in brain-computer interface systems and neuroscience research. And the code of the proposed architecture in this study is available under https://github.com/zhangdx21/MI-DAGSC.

In image classification, a deep neural network (DNN) that is trained on undistorted images constitutes an effective decision boundary. Unfortunately, this boundary does not support distorted images, such as noisy or blurry ones, leading to accuracy drop-off. As a simple approach for classifying distorted images as well as undistorted ones, previous methods have optimized the trained DNN again on both kinds of images. However, in these methods, the decision boundary may become overly complicated during optimization because there is no regularization of the decision boundary. Consequently, this decision boundary limits efficient optimization. In this paper, we study a simple yet effective decision boundary for distorted image classification through the use of a novel loss, called a "neural activation pattern matching (NAPM) loss". The NAPM loss is based on recent findings that the decision boundary is a piecewise linear function, where each linear segment is constructed from a neural activation pattern in the DNN when an image is fed to it. The NAPM loss extracts the neural activation patterns when the distorted image and its undistorted version are fed to the DNN and then matches them with each other via the sigmoid cross-entropy. Therefore, it constrains the DNN to classify the distorted image and its undistorted version by the same linear segment. As a result, our loss accelerates efficient optimization by preventing the decision boundary from becoming overly complicated. Our experiments demonstrate that our loss increases the accuracy of the previous methods in all conditions evaluated.

Photometric stereo methods typically rely on RGB cameras and are usually performed in a dark room to avoid ambient illumination. Ambient illumination poses a great challenge in photometric stereo due to the restricted dynamic range of the RGB cameras. To address this limitation, we present a novel method, namely Event Fusion Photometric Stereo Network (EFPS-Net), which estimates the surface normals of an object in an ambient light environment by utilizing a deep fusion of RGB and event cameras. The high dynamic range of event cameras provides a broader perspective of light representations that RGB cameras cannot provide. Specifically, we propose an event interpolation method to obtain ample light information, which enables precise estimation of the surface normals of an object. By using RGB-event fused observation maps, our EFPS-Net outperforms previous state-of-the-art methods that depend only on RGB frames, resulting in a 7.94% reduction in mean average error. In addition, we curate a novel photometric stereo dataset by capturing objects with RGB and event cameras under numerous ambient light environments.

Transparent objects widely exist in the world. The task of transparent object segmentation is challenging as the object lacks its own texture. The cue of shape information therefore gets more critical. Most existing methods, however, rely on the mechanism of simple convolution, which is good at local cues and performs weakly on global cues like shape. To solve this problem, an operation named Patch-wise Weight Shuffle is proposed to bring in the global context cue by being combined with the dynamic convolution. A network ShuffleTrans that recognizes shape better is then designed based on this operation. Besides, fitter for this task, two auxiliary modules are presented in ShuffleTrans: a Boundary and Direction Refinement Module which collects two additional information, and a Channel Attention Enhancement Module that assists the above operation. Experiments on four texture-less object segmentation datasets and two normal datasets verify the effectiveness and generality of the method. Especially, the ShuffleTrans achieved 74.93% mIoU on the Trans10k v2 test set, which is more accurate than existing methods.

Due to privacy concerns and data transmission issues, Source-free Unsupervised Domain Adaptation (SFDA) has gained popularity. It exploits pre-trained source models, rather than raw source data for target learning, to transfer knowledge from a labeled source domain to an unlabeled target domain. Existing methods solve this problem typically with additional parameters or noisy pseudo labels, and we propose an effective method named Proxy-based Mixup training with label refinery (ProxyMix) to avoid these drawbacks. To avoid additional parameters and leverages information in the source model, ProxyMix defines classifier weights as class prototypes and creates a class-balanced proxy source domain using nearest neighbors of the prototypes. To improve the reliability of pseudo labels, we further propose the frequency-weighted aggregation strategy to generate soft pseudo labels for unlabeled target data. Our strategy utilizes target features' internal structure, increases weights of low-frequency class samples, and aligns the proxy and target domains using inter- and intra-domain mixup regularization. This mitigates the negative impact of noisy labels. Experiments on three 2D image and 3D point cloud object recognition benchmarks demonstrate that ProxyMix yields state-of-the-art performance for source-free UDA tasks.

In remote sensing image classification, active learning aims to obtain an excellent classification model by selecting informative or representative training samples. However, due to the complexity of remote sensing images, the same class of ground objects usually have different spectral representations. The existing active learning methods may not take into account diverse representations of the same targets, which leads to a possible lack of intra-class diversity in the collected samples. To alleviate this problem, we propose an active learning method based on similarity level histogram (SLH) and adaptive-scale sampling to improve very high resolution remote sensing image classification. Specifically, we construct a SLH for each class of ground objects to effectively consider the intra-class diversity of the same target. To avoid the problem of sample imbalance caused by over-sampling or under-sampling, we design an adaptive-scale sampling strategy. Then, we utilize active learning to mine representative samples from each SLH warehouse according to adaptive-scale sampling strategies until the iteration condition is satisfied. Experiments show that the proposed algorithm can achieve better classification performance with limited training samples and is competitive with other methods based on four sets of publicly available data.

We predict steady-state Stokes flow of fluids within porous media at pore scales using sparse point observations and a novel class of physics-informed neural networks, called "physics-informed PointNet" (PIPN). Taking the advantages of PIPN into account, three new features become available compared to physics-informed convolutional neural networks for porous medium applications. First, the input of PIPN is exclusively the pore spaces of porous media (rather than both the pore and grain spaces). This feature diminishes required computer memory. Second, PIPN represents the boundary of pore spaces smoothly and realistically (rather than pixel-wise representations). Third, spatial resolution can vary over the physical domain (rather than equally spaced resolutions). This feature enables users to reach an optimal resolution with a minimum computational cost. The performance of our framework is evaluated by the study of the influence of noisy sensor data, pressure observations, and spatial correlation length.

An ultimate objective in continual learning is to preserve knowledge learned in preceding tasks while learning new tasks. To mitigate forgetting prior knowledge, we propose a novel knowledge distillation technique that takes into the account the manifold structure of the latent/output space of a neural network in learning novel tasks. To achieve this, we propose to approximate the data manifold up-to its first order, hence benefiting from linear subspaces to model the structure and maintain the knowledge of a neural network while learning novel concepts. We demonstrate that the modeling with subspaces provides several intriguing properties, including robustness to noise and therefore effective for mitigating Catastrophic Forgetting in continual learning. We also discuss and show how our proposed method can be adopted to address both classification and segmentation problems. Empirically, we observe that our proposed method outperforms various continual learning methods on several challenging datasets including Pascal VOC, and Tiny-Imagenet. Furthermore, we show how the proposed method can be seamlessly combined with existing learning approaches to improve their performances. The codes of this article will be available at https://github.com/csiro-robotics/SDCL.

Violent assaults and homicides occur daily, and the number of victims of mass shootings increases every year. However, this number can be reduced with the help of Closed Circuit Television (CCTV) and weapon detection models, as generic object detectors have become increasingly accurate with more data for training. We present a new semi-supervised learning methodology based on conditioned cooperative student-teacher training with optimal pseudo-label generation using a novel confidence threshold search method and improving both models by conditional knowledge transfer. Furthermore, a novel firearms image dataset of 458,599 images was collected using Instagram hashtags to evaluate our approach and compare the improvements obtained using a specific unsupervised dataset instead of a general one such as ImageNet. We compared our methodology with supervised, semi-supervised and self-supervised learning techniques, outperforming approaches such as YOLOv5 m (up to ＋19.86), YOLOv5l (up to ＋6.52) Unbiased Teacher (up to ＋10.5 AP), DETReg (up to ＋2.8 AP) and UP-DETR (up to ＋1.22 AP).

Legged robots that can instantly change motor patterns at different walking speeds are useful and can accomplish various tasks efficiently. However, state-of-the-art control methods either are difficult to develop or require long training times. In this study, we present a comprehensible neural control framework to integrate probability-based black-box optimization (PIBB) and supervised learning for robot motor pattern generation at various walking speeds. The control framework structure is based on a combination of a central pattern generator (CPG), a radial basis function (RBF) -based premotor network and a hypernetwork, resulting in a so-called neural CPG-RBF-hyper control network. First, the CPG-driven RBF network, acting as a complex motor pattern generator, was trained to learn policies (multiple motor patterns) for different speeds using PIBB. We also introduce an incremental learning strategy to avoid local optima. Second, the hypernetwork, which acts as a task/behavior to control parameter mapping, was trained using supervised learning. It creates a mapping between the internal CPG frequency (reflecting the walking speed) and motor behavior. This map represents the prior knowledge of the robot, which contains the optimal motor joint patterns at various CPG frequencies. Finally, when a user-defined robot walking frequency or speed is provided, the hypernetwork generates the corresponding policy for the CPG-RBF network. The result is a versatile locomotion controller which enables a quadruped robot to perform stable and robust walking at different speeds without sensory feedback. The policy of the controller was trained in the simulation (less than 1 h) and capable of transferring to a real robot. The generalization ability of the controller was demonstrated by testing the CPG frequencies that were not encountered during training.

In wind speed prediction technologies, deep learning-based methods have achieved promising advantages. However, most existing methods focus on learning implicit knowledge in a data-driven manner but neglect some explicit knowledge from the physical theory of meteorological dynamics, failing to make stable and long-term predictions. In this paper, we explore introducing explicit physical knowledge into neural networks and propose Physical Equations Predictive Network (PEPNet) for multi-step wind speed predictions. In PEPNet, a new neural block called the Augmented Neural Barotropic Equations (ANBE) block is designed as its key component, which aims to capture the wind dynamics by combining barotropic primitive equations and deep neural networks. Specifically, the ANBE block adopts a two-branch structure to model wind dynamics, where one branch is physic-based and the other is data-driven-based. The physic-based branch constructs temporal partial derivatives of meteorological elements (including u-component wind, v-component wind, and geopotential height) in a new Neural Barotropic Equations Unit (NBEU). The NBEU is developed based on the barotropic primitive equations mode in numerical weather prediction (NWP). Besides, considering that the barotropic primitive mode is a crude assumption of atmospheric motion, another data-driven-based branch is developed in the ANBE block, which aims at capturing meteorological dynamics beyond barotropic primitive equations. Finally, the PEPNet follows a time-variant structure to enhance the model's capability to capture wind dynamics over time. To evaluate the predictive performance of PEPNet, we have conducted several experiments on two real-world datasets. Experimental results show that the proposed method outperforms the state-of-the-art techniques and achieve optimal performance.

Modern artificial intelligence (AI) approaches mainly rely on neural network (NN) or deep NN methodologies. However, these approaches require large amounts of data to train, given, that the number of their trainable parameters has a polynomial relationship to their neuron counts. This property renders deep NNs challenging to apply in fields operating with small, albeit representative datasets such as healthcare. In this paper, we propose a novel neural network architecture which trains spatial positions of neural soma and axon pairs, where weights are calculated by axon-soma distances of connected neurons. We refer to this method as distance-encoding biomorphic-informational (DEBI) neural network. This concept significantly minimizes the number of trainable parameters compared to conventional neural networks. We demonstrate that DEBI models can yield comparable predictive performance in tabular and imaging datasets, where they require a fraction of trainable parameters compared to conventional NNs, resulting in a highly scalable solution.

Multi-view representation learning aims to capture comprehensive information from multiple views of a shared context. Recent works intuitively apply contrastive learning to different views in a pairwise manner, which is still scalable: view-specific noise is not filtered in learning view-shared representations; the fake negative pairs, where the negative terms are actually within the same class as the positive, and the real negative pairs are coequally treated; evenly measuring the similarities between terms might interfere with optimization. Importantly, few works study the theoretical framework of generalized self-supervised multi-view learning, especially for more than two views. To this end, we rethink the existing multi-view learning paradigm from the perspective of information theory and then propose a novel information theoretical framework for generalized multi-view learning. Guided by it, we build a multi-view coding method with a three-tier progressive architecture, namely Information theory-guided heuristic Progressive Multi-view Coding (IPMC). In the distribution-tier, IPMC aligns the distribution between views to reduce view-specific noise. In the set-tier, IPMC constructs self-adjusted contrasting pools, which are adaptively modified by a view filter. Lastly, in the instance-tier, we adopt a designed unified loss to learn representations and reduce the gradient interference. Theoretically and empirically, we demonstrate the superiority of IPMC over state-of-the-art methods.

Synthesizing realistic fine-grained images from text descriptions is a significant computer vision task. Although many GANs-based methods have been proposed to solve this task, generating high-quality images consistent with text information remains a difficult problem. These existing GANs-based methods ignore important words due to the use of fixed initial word features in generator, and neglect to learn semantic consistency between images and texts for discriminators. In this article, we propose a novel attentional generation and contrastive adversarial framework for fine-grained text-to-image synthesis, termed as Word Self-Update Contrastive Adversarial Networks (WSC-GAN). Specifically, we introduce a dual attention module for modeling color details and semantic information. With a new designed word self-update module, the generator can leverage visually important words to compute attention maps in the feature synthesis module. Furthermore, we contrive multi-branch contrastive discriminators to maintain better consistency between the generated image and text description. Two novel contrastive losses are proposed for our discriminators to impose image-sentence and image-word consistency constraints. Extensive experiments on CUB and MS-COCO datasets demonstrate that our method achieves better performance compared with state-of-the-art methods.

Enhancing computability of cerebral recordings and connections made with human/non-human brain have been on track and are expected to propel in our current era. An effective contribution towards said ends is improving accuracy of attempts at discerning intricate phenomena taking place within human brain. Here and in two different capacities of experiments, we attempt to distinguish cerebral perceptions shaped and affective states surfaced during observation of samples of media incorporating distinct audio-visual and emotional contents, through employing electroencephalograph/EEG recorded sessions of two reputable datasets of DEAP and SEED. Here we introduce AltSpec(E3) the inceptive form of CollectiveNet intelligent computational architectures employing collective and concurrent multi-spec analysis to exploit complex patterns in complex data-structures. This processing technique uses a full array of diversification protocols with multifarious parts enabling surgical levels of optimization while integrating a holistic analysis of patterns. Data-structures designed here contain multi-electrode neuroinformatic and neurocognitive features studying emotion reactions and attentive patterns. These spatially and temporally featured 2D/3D constructs of domain-augmented data are eventually AI-processed and outputs are defragmented forming one definitive judgement. The media-perception tracing is arguably first of its kind, at least when implemented on mentioned datasets. Backed by this multi-directional approach and in subject-independent configurations for perception-tracing on 5-media-class basis, mean accuracies of 81.00% and 68.93% were obtained on DEAP and SEED, respectively. We also managed to classify emotions with accuracies of 61.59% and 66.21% in cross-dataset validation followed by 81.47% and 88.12% in cross-subject validation settings trained on DEAP and SEED, consecutively.

Precision medicine is devoted to discovering personalized therapy for complex and difficult diseases like cancer. Many machine learning approaches have been developed for drug response prediction towards precision medicine. Notwithstanding, genetic profiles based multi-view graph learning schemes have not yet been explored for drug response prediction in previous works. Furthermore, multi-scale latent feature fusion is not considered sufficiently in the existing frameworks of graph neural networks (GNNs). Previous works on drug response prediction mainly depend on sequence data or single-view graph data. In this paper, we propose to construct multi-view graph by means of multi-omics data and STRING protein-protein association data, and develop a new architecture of GNNs for drug response prediction in cancer. Specifically, we propose hybrid multi-view and multi-scale graph duplex-attention networks (HMM-GDAN), in which both multi-view self-attention mechanism and view-level attention mechanism are devised to capture the complementary information of views and emphasize on the importance of each view collaboratively, and rich multi-scale features are constructed and integrated to further form high-level representations for better prediction. Experiments on GDSC2 dataset verify the superiority of the proposed HMM-GDAN when compared with state-of-the-art baselines. The effectiveness of multi-view and multi-scale strategies is demonstrated by the ablation study.

Graph representation is a critical technology in the field of knowledge engineering and knowledge-based applications since most knowledge bases are represented in the graph structure. Nowadays, contrastive learning has become a prominent way for graph representation by contrasting positive-positive and positive-negative node pairs between two augmentation graphs. It has achieved new state-of-the-art in the field of self-supervised graph representation. However, existing contrastive graph representation methods mainly focus on modifying (normally removing some edges/nodes) the original graph structure to generate the augmentation graph for the contrastive. It inevitably changes the original graph structures, meaning the generated augmentation graph is no longer equivalent to the original graph. This harms the performance of the representation in many structure-sensitive graphs such as protein graphs, chemical graphs, molecular graphs, etc. Moreover, there is only one positive-positive node pair but relatively massive positive-negative node pairs in the self-supervised graph contrastive learning. This can lead to the same class, or very similar samples are considered negative samples. To this end, in this work, we propose a Virtual Masking Augmentation (VMA) to generate an augmentation graph without changing any structures from the original graph. Meanwhile, a node augmentation method is proposed to augment the positive node pairs by discovering the most similar nodes in the same graph. Then, two different augmentation graphs are generated and put into a contrastive learning model to learn the graph representation. Extensive experiments on massive datasets demonstrate that our method achieves new state-of-the-art results on self-supervised graph representation. The source code of the proposed method is available at https://github.com/DuanhaoranCC/CGRA.

Visual question generation aims to focus on some target objects in an image to generate questions with certain questioning purposes. Existing studies mainly utilize an answer to extract the target object corresponding to the questioning purpose for questioning. However, answers fail to accurately and completely map to every target object, such as the objects corresponding to the answer are ambiguous or the answers are the relationship between multiple objects. To address this problem, we propose a content-controlled question generation model, which generates questions based on a given target object set specified from an image. Considering that the target objects have different contributions during the generation process, we design a recurrent generative architecture to explicitly control attention to different objects and their corresponding image information at each generative stage. Extensive experiments on the VQA v2.0 dataset and the Visual7w dataset show that the proposed model outperforms the state-of-the-art models and can controllably generate questions with specified content.

The statistical supervised learning framework assumes an input-output set with a joint probability distribution that is reliably represented by the training dataset. The learning system is then required to output a prediction rule learned from the training dataset's input-output pairs. In this work, we investigate the relationship between the sample complexity, the empirical risk and the generalization error based on the asymptotic equipartition property (AEP) (Shannon, 1948). We provide theoretical guarantees for reliable learning under the information-theoretic AEP, with respect to the generalization error and the sample size in different settings.

We introduce a large-scale neurocomputational model of spatial cognition called 'Spacecog', which integrates recent findings from mechanistic models of visual and spatial perception. As a high-level cognitive ability, spatial cognition requires the processing of behaviourally relevant features in complex environments and, importantly, the updating of this information during processes of eye and body movement. The Spacecog model achieves this by interfacing spatial memory and imagery with mechanisms of object localisation, saccade execution, and attention through coordinate transformations in parietal areas of the brain. We evaluate the model in a realistic virtual environment where our neurocognitive model steers an agent to perform complex visuospatial tasks. Our modelling approach opens up new possibilities in the assessment of neuropsychological data and human spatial cognition.

In this paper, the problem of multiplayer hierarchical decision-making problem for non-affine systems is solved by adaptive dynamic programming. Firstly, the control dynamics are obtained according to the theory of dynamic feedback and combined with the original system dynamics to construct the affine augmented system. Thus, the non-affine multiplayer system is transformed into a general affine form. Then, the hierarchical decision problem is modeled as a Stackelberg game. In the Stackelberg game, the leader makes a decision based on the information of all followers, whereas the followers do not know each other's information and only obtain their optimal control strategy based on the leader's decision. Then, the augmented system is reconstructed by a neural network (NN) using input-output data. Moreover, a single critic NN is used to approximate the value function to obtain the optimal control strategy for each player. An extra term added to the weight update law makes the initial admissible control law no longer needed. According to the Lyapunov theory, the state of the system and the error of the weights of the NN are both uniformly ultimately bounded. Finally, the feasibility and validity of the algorithm are confirmed by simulation.

The rising number of cases of human Mpox has emerged as a major global concern due to the daily increase of cases in several countries. The disease presents various skin symptoms in infected individuals, making it crucial to promptly identify and isolate them to prevent widespread community transmission. Rapid determination and isolation of infected individuals are therefore essential to curb the spread of the disease. Most research in the detection of Mpox disease has utilized convolutional neural network (CNN) models and ensemble methods. However, to the best of our knowledge, none have utilized a meta-heuristic-based ensemble approach. To address this gap, we propose a novel metaheuristics optimization-based weighted average ensemble model (MO-WAE) for detecting Mpox disease. We first train three transfer learning (TL)-based CNNs (DenseNet201, MobileNet, and DenseNet169) by adding additional layers to improve their classification strength. Next, we use a weighted average ensemble technique to fuse the predictions from each individual model, and the particle swarm optimization (PSO) algorithm is utilized to assign optimized weights to each model during the ensembling process. By using this approach, we obtain more accurate predictions than individual models. To gain a better understanding of the regions indicating the onset of Mpox, we performed a Gradient Class Activation Mapping (Grad-CAM) analysis to explain our model's predictions. Our proposed MO-WAE ensemble model was evaluated on a publicly available Mpox dataset and achieved an impressive accuracy of 97.78%. This outperforms state-of-the-art (SOTA) methods on the same dataset, thereby providing further evidence of the efficacy of our proposed model.

The problem of vanishing and exploding gradients has been a long-standing obstacle that hinders the effective training of neural networks. Despite various tricks and techniques that have been employed to alleviate the problem in practice, there still lacks satisfactory theories or provable solutions. In this paper, we address the problem from the perspective of high-dimensional probability theory. We provide a rigorous result that shows, under mild conditions, how the vanishing/exploding gradients problem disappears with high probability if the neural networks have sufficient width. Our main idea is to constrain both forward and backward signal propagation in a nonlinear neural network through a new class of activation functions, namely Gaussian-Poincaré normalized functions, and orthogonal weight matrices. Experiments on both synthetic and real-world data validate our theory and confirm its effectiveness on very deep neural networks when applied in practice.

Adversarial robustness is considered a required property of deep neural networks. In this study, we discover that adversarially trained models might have significantly different characteristics in terms of margin and smoothness, even though they show similar robustness. Inspired by the observation, we investigate the effect of different regularizers and discover the negative effect of the smoothness regularizer on maximizing the margin. Based on the analyses, we propose a new method called bridged adversarial training that mitigates the negative effect by bridging the gap between clean and adversarial examples. We provide theoretical and empirical evidence that the proposed method provides stable and better robustness, especially for large perturbations.

Deep ensemble learning, where we combine knowledge learned from multiple individual neural networks, has been widely adopted to improve the performance of neural networks in deep learning. This field can be encompassed by committee learning, which includes the construction of neural network cascades. This study focuses on the high-dimensional low-sample-size (HDLS) domain and introduces multiple instance ensemble (MIE) as a novel stacking method for ensembles and cascades. In this study, our proposed approach reformulates the ensemble learning process as a multiple-instance learning problem. We utilise the multiple-instance learning solution of pooling operations to associate feature representations of base neural networks into joint representations as a method of stacking. This study explores various attention mechanisms and proposes two novel committee learning strategies with MIE. In addition, we utilise the capability of MIE to generate pseudo-base neural networks to provide a proof-of-concept for a "growing" neural network cascade that is unbounded by the number of base neural networks. We have shown that our approach provides (1) a class of alternative ensemble methods that performs comparably with various stacking ensemble methods and (2) a novel method for the generation of high-performing "growing" cascades. The approach has also been verified across multiple HDLS datasets, achieving high performance for binary classification tasks in the low-sample size regime.

Effective exploration is the key to achieving high returns for reinforcement learning. Agents must explore jointly in multi-agent systems to find the optimal joint policy. Due to the exploration problem and the shared reward, the policy-based multi-agent reinforcement learning algorithms face policy overfitting, which may lead to the joint policy falling into a local optimum. This paper introduces a novel general framework called Learning Joint-Action Intrinsic Reward (LJIR) for improving multi-agent reinforcement learners' joint exploration ability and performance. LJIR observes agents' state and joint actions to learn to construct an intrinsic reward online that can guide effective joint exploration. With the novel combination of Transformer and random network distillation, LJIR selects the novel states to give more intrinsic rewards, which help agents find the best joint actions. LJIR can dynamically adjust the weight of exploration and exploitation during training and keep the policy invariance finally. To ensure LJIR seamlessly adopts existing MARL algorithms, we also provide a flexible combination method for intrinsic and external rewards. Empirical results on the SMAC benchmark show that the proposed method achieves state-of-the-art performance in challenging tasks.

In the 3D skeleton-based action recognition task, learning rich spatial and temporal motion patterns from body joints are two foundational yet under-explored problems. In this paper, we propose two methods for improving these problems: (I) a novel glimpse-focus action recognition strategy that captures multi-range pose features from the whole body and key body parts jointly; (II) a powerful temporal feature extractor JD-TC that enriches trajectory features by inferring different inter-frame correlations for different joints. By coupling these two proposals, we develop a powerful skeleton-based action recognition system that extracts rich pose and trajectory features from a skeleton sequence and outperforms previous state-of-the-art methods on three large-scale datasets.

Sparse deep neural networks have proven to be efficient for predictive model building in large-scale studies. Although several works have studied theoretical and numerical properties of sparse neural architectures, they have primarily focused on the edge selection. Sparsity through edge selection might be intuitively appealing; however, it does not necessarily reduce the structural complexity of a network. Instead pruning excessive nodes leads to a structurally sparse network with significant computational speedup during inference. To this end, we propose a Bayesian sparse solution using spike-and-slab Gaussian priors to allow for automatic node selection during training. The use of spike-and-slab prior alleviates the need of an ad-hoc thresholding rule for pruning. In addition, we adopt a variational Bayes approach to circumvent the computational challenges of traditional Markov Chain Monte Carlo (MCMC) implementation. In the context of node selection, we establish the fundamental result of variational posterior consistency together with the characterization of prior parameters. In contrast to the previous works, our theoretical development relaxes the assumptions of the equal number of nodes and uniform bounds on all network weights, thereby accommodating sparse networks with layer-dependent node structures or coefficient bounds. With a layer-wise characterization of prior inclusion probabilities, we discuss the optimal contraction rates of the variational posterior. We empirically demonstrate that our proposed approach outperforms the edge selection method in computational complexity with similar or better predictive performance. Our experimental evidence further substantiates that our theoretical work facilitates layer-wise optimal node recovery.

The implementation of robotic reinforcement learning is hampered by problems such as an unspecified reward function and high training costs. Many previous works have used cross-domain policy transfer to obtain the policy of the problem domain. However, these researches require paired and aligned dynamics trajectories or other interactions with the environment. We propose a cross-domain dynamics alignment framework for the problem domain policy acquisition that can transfer the policy trained in the source domain to the problem domain. Our framework aims to learn dynamics alignment across two domains that differ in agents' physical parameters (armature, rotation range, or torso mass) or agents' morphologies (limbs). Most importantly, we learn dynamics alignment between two domains using unpaired and unaligned dynamics trajectories. For these two scenarios, we propose a cross-physics-domain policy adaptation algorithm (CPD) and a cross-morphology-domain policy adaptation algorithm (CMD) based on our cross-domain dynamics alignment framework. In order to improve the performance of policy in the source domain so that a better policy can be transferred to the problem domain, we propose the Boltzmann TD3 (BTD3) algorithm. We conduct diverse experiments on agent continuous control domains to demonstrate the performance of our approaches. Experimental results show that our approaches can obtain better policies and higher rewards for the agents in the problem domains even when the dataset of the problem domain is small.

Recent studies in deep neural networks have shown that injecting random noise in the input layer of the networks contributes towards ℓp-norm-bounded adversarial perturbations. However, to defend against unrestricted adversarial examples, most of which are not ℓp-norm-bounded in the input layer, such input-layer random noise may not be sufficient. In the first part of this study, we generated a novel class of unrestricted adversarial examples termed feature-space adversarial examples. These examples are far from the original data in the input space but adjacent to the original data in a hidden-layer feature space and far again in the output layer. In the second part of this study, we empirically showed that while injecting random noise in the input layer was unable to defend these feature-space adversarial examples, they were defended by injecting random noise in the hidden layer. These results highlight the novel benefit of stochasticity in higher layers, in that it is useful for defending against these feature-space adversarial examples, a class of unrestricted adversarial examples.

Multi-label Zero-shot Learning (ZSL) is more reasonable and realistic than standard single-label ZSL because several objects can co-exist in a natural image in real scenarios. Intra-class feature entanglement is a significant factor influencing the alignment of visual and semantic features, resulting in the model's inability to recognize unseen samples comprehensively and completely. We observe that existing multi-label ZSL methods place a greater emphasis on attention-based refinement and decoupling of visual features, while ignoring the relationship between label semantics. Relying on label correlations to solve multi-label ZSL tasks has not been deeply studied. In this paper, we make full use of the co-occurrence relationship between category labels and build a directed weighted semantic graph based on statistics and prior knowledge, in which node features represent category semantics and weighted edges represent conditional probabilities of label co-occurrence. To guide the targeted extraction of visual features, node features and edge set weights are simultaneously updated and refined, and embedded into the visual feature extraction network from a global and local perspective. The proposed method's effectiveness was demonstrated by simulation results on two challenging multi-label ZSL benchmarks: NUS-WIDE and Open Images. In comparison to state-of-the-art models, our model achieves an absolute gain of 2.4% mAP on NUS-WIDE and 2.1% mAP on Open Images respectively.

Recently, deep clustering has been extensively employed for various data mining tasks, and it can be divided into auto-encoder (AE)-based and graph neural networks (GNN)-based methods. However, existing AE-based methods fall short in effectively extracting structural information, while GNN suffer from smoothing and heterophily. Although methods that combine AE and GNN achieve impressive performance, there remains an inadequate balance between preserving the raw structure and exploring the underlying structure. Accordingly, we propose a novel network named Structure-Aware Deep Clustering network (SADC). Firstly, we compute the cumulative influence of non-adjacent nodes at multiple depths and, thus, enhance the adjacency matrix. Secondly, an enhanced graph auto-encoder is designed. Thirdly, the latent space of AE is endowed with the ability to perceive the raw structure during the learning process. Besides, we design self-supervised mechanisms to achieve co-optimization of node representation learning and topology learning. A new loss function is designed to preserve the inherent structure while also allowing for exploration of latent data structure. Extensive experiments on six benchmark datasets validate that our method outperforms state-of-the-art methods.

A multilayered bidirectional associative memory neural network is proposed to account for learning nonlinear types of association. The model (denoted as the MF-BAM) is composed of two modules, the Multi-Feature extracting bidirectional associative memory (MF), which contains various unsupervised network layers, and a modified Bidirectional Associative Memory (BAM), which consists of a single supervised network layer. The MF generates successive feature patterns from the original inputs. These patterns change the relationship between the inputs and targets in a way that the BAM can learn. The model was tested on different nonlinear tasks, such as the N-bit, Double Moon and its variants, and the 3-class spiral task. Behaviors were reported through learning errors, decision zones, and recall performances. Results showed that it was possible to learn all tasks consistently. By manipulating the number of units per layer and the number of unsupervised network layers in the MF, it was possible to change the level of nonlinearity observed in the decision boundaries. Furthermore, results indicated that different behaviors were achieved from the same set of inputs by using the different generated patterns. These findings are significant as they showed how a BAM-inspired model could solve nonlinear tasks in a more cognitively plausible fashion.

Convolutional neural networks (CNNs) are one of the most successful computer vision systems to solve object recognition. Furthermore, CNNs have major applications in understanding the nature of visual representations in the human brain. Yet it remains poorly understood how CNNs actually make their decisions, what the nature of their internal representations is, and how their recognition strategies differ from humans. Specifically, there is a major debate about the question of whether CNNs primarily rely on surface regularities of objects, or whether they are capable of exploiting the spatial arrangement of features, similar to humans. Here, we develop a novel feature-scrambling approach to explicitly test whether CNNs use the spatial arrangement of features (i.e. object parts) to classify objects. We combine this approach with a systematic manipulation of effective receptive field sizes of CNNs as well as minimal recognizable configurations (MIRCs) analysis. In contrast to much previous literature, we provide evidence that CNNs are in fact capable of using relatively long-range spatial relationships for object classification. Moreover, the extent to which CNNs use spatial relationships depends heavily on the dataset, e.g. texture vs. sketch. In fact, CNNs even use different strategies for different classes within heterogeneous datasets (ImageNet), suggesting CNNs have a continuous spectrum of classification strategies. Finally, we show that CNNs learn the spatial arrangement of features only up to an intermediate level of granularity, which suggests that intermediate rather than global shape features provide the optimal trade-off between sensitivity and specificity in object classification. These results provide novel insights into the nature of CNN representations and the extent to which they rely on the spatial arrangement of features for object classification.

Visual object tracking (VOT) for intelligent video surveillance has attracted great attention in the current research community, thanks to advances in computer vision and camera technology. Meanwhile, discriminative correlation filter (DCF) trackers garnered significant interest owing to their high accuracy and low computing cost. Many researchers have introduced spatial and temporal regularization into the DCF framework to achieve a more robust appearance model and further improve tracking performance. However, these algorithms typically set fixed spatial and temporal regularization parameters, which limit flexibility and adaptability under cluttered and challenging scenarios. To overcome these problems, in this work, we propose a new dynamic spatial-temporal regularization for the DCF tracking model that emphasizes the filter to concentrate on more reliable regions during the training stage. Furthermore, we present a response deviation-suppressed regularization term for responses to encourage temporal consistency and avoid model degradation by suppressing relative response changes between two consecutive frames. Moreover, we introduce a multi-memory tracking framework to exploit various features and each memory contributes to tracking the target across all frames. Significant experiments on the OTB-2013, OTB-2015, TC-128, UAV-123, UAVDT, and DTB-70 datasets have revealed that the performance thereof outperformed many state-of-the-art trackers based on DCF and deep-based frameworks in terms of tracking accuracy and tracking success rate.

The rise of graph representation learning as the primary solution for many different network science tasks led to a surge of interest in the fairness of this family of methods. Link prediction, in particular, has a substantial social impact. However, link prediction algorithms tend to increase the segregation in social networks by disfavouring the links between individuals in specific demographic groups. This paper proposes a novel way to enforce fairness on graph neural networks with a fine-tuning strategy. We Drop the unfair Edges and, simultaneously, we Adapt the model's parameters to those modifications, DEA in short. We introduce two covariance-based constraints designed explicitly for the link prediction task. We use these constraints to guide the optimization process responsible for learning the new 'fair' adjacency matrix. One novelty of DEA is that we can use a discrete yet learnable adjacency matrix in our fine-tuning. We demonstrate the effectiveness of our approach on five real-world datasets and show that we can improve both the accuracy and the fairness of the link prediction tasks. In addition, we present an in-depth ablation study demonstrating that our training algorithm for the adjacency matrix can be used to improve link prediction performances during training. Finally, we compute the relevance of each component of our framework to show that the combination of both the constraints and the training of the adjacency matrix leads to optimal performances.

As a member of the third generation of artificial neural network models, spiking neural P systems (SN P systems) have gained a hot research spot in recent years. This work introduces the phenomenon of lateral inhibition in biological nervous systems into SN P systems, and proposes SN P systems with lateral inhibition (LISN P systems). LISN P systems add the property of synaptic length to portray the lateral distance between neurons, and adopt a new form of rules, lateral interaction rules, to describe the reception of spikes by postsynaptic neurons with different lateral distances from the presynaptic neuron. Specifically, an excited neuron produces lateral inhibition on surrounding postsynaptic neurons. Postsynaptic neurons close to the excited neuron, i.e., neurons with small lateral distances, are more susceptible to lateral inhibition and either receive a fewer number of spikes generated by the excited neuron or fail to receive spikes. As the lateral distance increases, the lateral inhibition weakens, and the number of spikes received by postsynaptic neurons increases. Based on the above mechanism, four specific LISN P systems are designed for generating arbitrary odd numbers, arbitrary even numbers, arbitrary natural numbers and arithmetic series, respectively, as examples. By designing working modules, LISN P systems provide equivalence in computational power to the universal register machines in both generating and accepting modes. This verifies the computational completeness of LISN P systems. A universal LISN P system using merely 65 neurons is devised for function computation. According to comparisons among several systems, universal LISN P systems require fewer computational resources.

This paper investigates the finite/fixed-time synchronization problem of delayed inertial memristive neural networks (DIMNNs) using interval matrix-based methods within a unified control framework. By employing set-valued mapping and differential inclusion theory, two distinct methods are applied to handle the switching behavior of memristor parameters: the maximum absolute value method and the interval matrix method. Based on these different approaches, two control strategies are proposed to select appropriate control parameters, enabling the system to achieve finite and fixed-time synchronization, respectively. Additionally, the resulting theoretical criteria differ based on the chosen control strategy, with one expressed in algebraic form and the other in the form of linear matrix inequalities (LMIs). Numerical simulations demonstrate that the interval matrix method outperforms the maximum absolute value method in terms of handling memristor parameter switching, achieving faster finite/fixed-time synchronization. Furthermore, the theoretical results are extended to the field of image encryption, where the response system is utilized for decryption and expanding the keyspace.

Convolutional neural networks (CNNs) have successfully driven many visual recognition tasks including image classification. However, when dealing with classification tasks with intra-class sample style diversity, the network tends to be disturbed by more diverse features, resulting in limited feature learning. In this article, a spatial oblivion channel attention (SOCA) for intra-class diversity feature learning is proposed. Specifically, SOCA performs spatial structure oblivion in a progressive regularization for each channel after convolution, so that the network is not restricted to a limited feature learning, and pays attention to more regionally detailed features. Further, SOCA reassigns channel weights in the progressively oblivious feature space from top to bottom along the channel direction, to ensure the network learns more image details in an orderly manner while not falling into feature redundancy. Experiments are conducted on the standard classification dataset CIFAR-10/100 and two garbage datasets with intra-class diverse styles. SOCA improves SqueezeNet, MobileNet, BN-VGG-19, Inception and ResNet-50 in classification accuracy by 1.31%, 1.18%, 1.57%, 2.09% and 2.27% on average, respectively. The feasibility and effectiveness of intra-class diversity feature learning in SOCA-enhanced networks are verified. Besides, the class activation map shows that more local detail feature regions are activated by adding the SOCA module, which also demonstrates the interpretability of the method for intra-class diversity feature learning.