Keywords: Deep Learning, Autonomous Systems, Robotics
Abstract: The intent prediction of unmanned aerial vehicles (UAVs) also known as drones is a challenging task due to the different mission profiles and tasks that the drone can perform. To alleviate this issue, this paper proposes a deep mixture of experts network to classify and predict drones trajectories measured from non-cooperative radars. Telemetry data of open-access datasets are converted to simulated radar tracks to generate a pool of heterogeneous trajectories and construct three independent datasets to train, validate, and test the proposed architecture. The network is composed of two main components: i) a deep network that predicts the class associated to the input trajectories and ii) a set of deep experts models that learns the extreme bounds of the trajectories in different future time steps. The proposed approach is tested and compared with different deep models to verify its effectiveness under different flight profiles and time-windows.

Keywords: Deep Learning, Reinforcement Learning, Graph Neural Networks
Abstract: Network analysis is responsible for taking insights or generating predictions from networked data sources where community detection finds chunks of related data in a network. The importance of community detection spans in different domain applications, from social network formation to protein interaction predictions. This work compares five state-of-the-art solutions to community detection using machine learning approaches in the context of online social networks - GraphGAN, SDNE, ComE, AC2CD, and CLARE. The experiments using real-world online social network datasets (Email-EU-Core, BlogCatalog3, Flickr) with micro-F1, macro-F1, and NMI scores demonstrate that graph neural networks and deep reinforcement learning approaches are better suited for the community detection task than others based on probabilistic or shallow networks.

Keywords: Deep Learning, Explainability, Reinforcement Learning
Abstract: This article conducts an ablation study of the Actor-Critic Architecture for Community Detection (AC2CD). The AC2CD uses Deep Reinforcement Learning (DRL) and Graph Attention Networks (GAT). Our ablation study method adheres to the principles of explainable artificial intelligence, focusing on assessing performance factors, including execution time, memory usage, and GPU utilization. We carried out experiments using two real-world datasets: Email-Eu-Core (EC), an email network among members of a European research institution (comprising 1,005 nodes, 25,571 edges, and 42 communities) available through the Stanford Snap Project, and a High School contact and friendship network (HS) in Marseilles, France, from December 2013 (comprising 329 nodes, 45,047 edges, and nine communities), obtainable from the Socio Patterns Website. We evaluated performance while considering three hyperparameters: learn_rate (LR), batch_size (BS), and n_games (NG), varying them at 10%, 30%, 50%, and 70%. The LR of 70% yielded optimal results with execution time for both EC and HS datasets. Furthermore, a BS of 70% indicated an ideal balance between execution time, GPU usage, and memory consumption for the HS dataset.

Keywords: Deep Learning, Fault Detection
Abstract: Time series anomaly detection is a binary classification problem with unbalanced data, which aims to identify data that fall outside of the normal behaviors. Since the proportion of the abnormal data is very small, the cost of labeling all data is prohibitively high. Therefore, unsupervised methods are more suitable than supervised methods. With the rapid development of deep learning, various multivariate time series anomaly detection methods based on deep learning have been proposed. However, existing methods do not fully capture the spatial-temporal correlations and are not robust to noise. To address these issues, we propose an unsupervised anomaly detection method called Orthogonal Self-Attention Variational Autoencoder Generative Adversarial Networks (OSVAE-GAN). To fully extract the spatial-temporal correlations, we use an orthogonal self-attention (OS) mechanism. Moreover, to increase the capability to deal with complex multivariate data, we integrate two generative adversarial networks (GANs) with the variational autoencoder (VAE). Finally, to reduce the influence of noise, we introduce the maximum mean discrepancy (MMD) loss. Experiments are conducted on five public datasets, which show that the proposed method is superior to the existing methods.

Keywords: Deep Learning, Financial Engineering, Decision Making
Abstract: Many cryptocurrency brokers nowadays offer a variety of derivative assets that allow traders to perform hedging or speculation. This paper proposes an effective algorithm based on neural networks to take advantage of these investment products. The proposed algorithm constructs a portfolio that contains a pair of negatively correlated assets. A deep neural network, which outputs the allocation weight of each asset at a time interval, is trained to maximize the Sharpe ratio. A novel loss term is proposed to regulate the network's bias towards a specific asset, thus enforcing the network to learn an allocation strategy that is close to a minimum variance strategy. Extensive experiments were conducted using data collected from Binance spanning 19 months to evaluate the effectiveness of our approach. The backtest results show that the proposed algorithm can produce neural networks that are able to make profits in different market situations.

Keywords: Deep Learning, Smart Grid
Abstract: The last decade has seen significant changes in the power grid complexity due to the increased integration of multiple heterogeneous distributed energy resources. Accurate and fast power flow analysis tools have then become essential to guarantee grid stability, reliable operation, strategic planning, and market strategies. State-of-the-art approaches to power flow analysis are based on iterative numerical techniques which exhibit high accuracy but slow-, or even no-, convergence. For this reason, researchers have investigated the use of data-driven techniques that, while exhibiting lower accuracy with respect to iterative numerical ones, have the advantage of being extremely fast. To address the lack of accuracy, physics-informed data-driven techniques, i.e., techniques that leverage both the data and domain knowledge to generate simultaneously fast and accurate models, have been proposed. Nevertheless, these works exhibit two main limitations: i) they do not fully leverage the physical knowledge, and ii) they do not fairly compare the different approaches. In this paper, we propose a novel physics informed data-driven model able to address both limitations by fully leveraging the physical knowledge into the data-driven, i.e., constraining the model and augmenting the available data, and proposing a framework able to fairly compare the different approaches proving the actual effectiveness of the proposal. Results on the IEEE 57 realistic power network will support the proposal.

Keywords: Ensemble Learning, Graph Neural Networks, Signal Processing
Abstract: This study proposes a novel approach that combines stacking ensemble learning with Graph Convolutional Neural Networks (GCNNs) to enhance the classification accuracy of Motor Imagery (MI) tasks in supporting individuals with injuries or impairments, enabling more effective rehabilitation and assistance. The method integrates both structural and functional connectivity information to leverage the benefits of GCNNs and ensemble learning techniques. The BCI Competition IV-2a dataset is used for evaluation. The approach employs a stacked ensemble model consisting of nine baseline models and six combining meta-models, including Logistic Regression, Neural Networks, Support Vector Machines, Random Forest, K-Nearest Neighbor, and Gradient Boosting Machines. By leveraging information from both structural and functional connectivity, the GCNNs extract meaningful features from MI data, leading to improved classification accuracy. The stacking ensemble learning technique combines multiple GCNN models trained on different connectivity aspects, resulting in a robust and accurate classifier. The fusion of structural connectivity (ADJ-CNNM) capturing anatomical connections and functional connectivity (PLV-CNNM) measuring brain activity synchronization enables a comprehensive analysis of MI data. The proposed approach effectively captures both local and global connectivity patterns, addressing the challenges associated with MI data analysis. By considering both types of connectivity, a holistic understanding of the dynamics of the underlying brain network during MI tasks is achieved. Experimental results demonstrate the effectiveness of the proposed approach, achieving an accuracy of 86.23% with K-Nearest Neighbor as the meta-model. Comparisons with state-of-the-art and baseline methods on the same dataset validate the approach’s superiority, emphasizing the importance of GCNNs and stacking ensemble learning for accurate MI task classification.

Keywords: Human-Computer Interactions, Biometric Systems, Transportation and Vehicle Systems
Abstract: Monitoring in-car VR motion sickness (VRMS) by neurophysiological signals is a formidable challenge due to unavoidable motion artifacts caused by the moving vehicle and necessary physical movements by the user to interact with the VR environment. Therefore, this paper for the first time investigates if resting-state neurophysiological features and self-reports of stress levels collected prior to exposure to a motion-simulated in-car VRMS induction platform could predict final motion sickness ratings. Our results of linear regression modeling show that the traditional EEG power spectrum was the only resting-state feature set that could predict in-car VRMS ratings. Further, the best regression result was achieved by beta power spectrum in the left parietal area with adjusted R2=22.6% versus 11.6% in the right. This result not only confirmed the left parietal involvement in motion sickness susceptibility observed in a previous resting-state fMRI study, but also advanced that methodology to mobile neurotechnologies, represented by mobile EEG, referenced by other types of resting-state features. Together, this study may offer a new mobile neurotechnology-based approach to predict passengers’ VRMS levels before they start to use VR apps in a moving vehicle.

Keywords: Deep Learning, Signal Processing, Big Data
Abstract: Transformer neural network (TNN) has demonstrated its remarkable capacity to analyze and discern complex sequential datasets. This approach has achieved unprecedented success, particularly in the domain of natural language processing (NLP). TNN has since consistently proven to perform remarkably in other fields where long-term dependencies in the data are prevalent. Electroencephalography (EEG) data has historically posed a challenge for even modern deep neural networks to classify as EEG is notably complex and noisy, making training laborious and time-consuming. Though, there has been significant research done recently into the application of TNNs in EEG classification, often the task involved does not infer the TNN’s ability for long-term dependencies. In this paper, we propose a TNN-based model for EEG-based driver vigilance monitoring, emphasizing the classification of driver vigilance states. This study utilized the data of 11 subjects taken from a public EEG dataset, focusing solely on single-channel analysis. Results indicate that the proposed TNN model can achieve average accuracies of up to 92.69% for Single-Subject analysis, 94.09% for Cross-Subject analysis and 74.74% for Leave-One-Subject-Out analysis, which surpasses state-of-the-art methods. The proposed TNN model's potential lies in not only driver vigilance state monitoring but also paving the way for broader applications of biosignal processing.

Keywords: Biometric Systems, Human-Computer Interactions, Deep Learning
Abstract: Brain-computer interfaces (BCI) allow for the brain to communicate with electronic devices. Concerning the BCI paradigms, motor imagery uses brain signals to decode an imagined movement. However, this is a hard task given the low signal-to-noise ratio. Usually, the main steps in BCI models are pre-processing, feature extraction, and classification. In recent years, Convolutional Neural Networks (CNNs) have been gaining relevance in several areas of science due to their feature extraction, translation invariance, and parameter sharing capabilities. Another, more recent way of feature extraction is using attention mechanisms, which are layers of neural networks based on human attention and have the ability to highlight important features. A variation of the attention mechanism is the Convolutional Block Attention Module, which combines the CNN structure with the attention mechanism. In this work, we propose a new model that joins the core architecture of EEGNet, a compact CNN widely used in the literature, with the Convolutional Block Attention Module and residual connections. The residual connections were introduced to lower data degradation throughout the model. The results highlight the residual connection's importance for the performance of the model. The proposed model obtained a kappa result 5.2% better than the EEGNet with a p-value less than 0.01 on BCI Competition IV dataset 2a, which is a well-known dataset for Motor Imagery. Furthermore, the proposal was better than EEGNet for most subjects and had the best-worst case.

Keywords: Signal Processing
Abstract: Electroencephalography (EEG) is an essential neuromonitoring modality, deeply integrated across scientific disciplines such as psychology, cognitive science, computational neuroscience, neurology, and psychiatry. Its relevance has surged with the rise of brain-computer interfaces. However, the potential of non-invasive EEG is hindered by compromised signal quality compared to invasive methods. The distinction between the modest EEG source amplitudes and the pronounced magnitudes of non-EEG physiological signals and environmental interferences complicates the analysis. The coexistence of subtle neural signals and prominent artifacts, both intrinsic and acquired, characterizes EEG signal processing. Various artifact management techniques have been proposed, yet the pursuit of EEG signal quality assessment remains underexplored. This mini-review addresses this gap by emphasizing the vital role of quality assessment in EEG recordings. The article highlights the significance of rigorous signal evaluation, emphasizing reliable EEG data. It also encapsulates evolving quantitative methodologies that bolster signal fidelity assessment. By delving into these aspects, the article presents a compact overview of ongoing advancements in quantitative EEG quality assessment techniques in the research field of EEG analysis and applications.

Keywords: Deep Learning
Abstract: Analysing human motions is a core topic of interest for many disciplines, from Human-Computer Interaction, to entertainment, Virtual Reality and healthcare. Deep learning has achieved impressive results in capturing human pose in real-time. Acquiring human motion datasets is highly time consuming, challenging, and expensive. Hence, human motion synthesis is a crucial research problem within deep learning and computer vision. We present a novel method for controllable human motion synthesis by applying attention-based probabilistic deep adversarial models with end-to-end training. We show that we can generate synthetic human motion over both short- and long-time horizons through the use of adversarial attention.

Keywords: Financial Engineering, Agent-Based Modeling, Multi-Agent System
Abstract: The prevention of rapidly and steeply falling market prices is vital to avoid financial crisis. To this end, some stock exchanges implement a price limit or a circuit breaker, and there has been intensive investigation into which regulation best prevents rapid and large variations in price. In this study, we examine this question using an artificial market model that is an agent-based model for a financial market. Our findings show that the price limit and the circuit breaker basically have the same effect when the parameters, limit price range and limit time range, are the same. However, the price limit is less effective when limit the time range is smaller than the cancel time range. With the price limit, many sell orders are accumulated around the lower limit price, and when the lower limit price is changed before the accumulated sell orders are cancelled, it leads to the accumulation of sell orders of various prices. These accumulated sell orders essentially act as a wall against buy orders, thereby preventing price from rising. Caution should be taken in the sense that these results pertain to a limited situation. Specifically, our finding that the circuit breaker is better than the price limit should be adapted only in cases where the reason for falling prices is erroneous orders and when individual stocks are regulated.

Keywords: Evolvable Systems, Evolving Learning
Abstract: Algorithmic trading is a topic with major developments in the last years. Investors rely mostly on indicators derived from fundamental (FA) or technical analysis (TA), while sentiment analysis (SA) has also received attention in the last decade. This has led to great financial advantages with algorithms being the main tool to create pre-programmed trading strategies. Although the three analysis types have been mainly considered individually, their combination has not been studied as much. Given the ability of each individual analysis type in identifying profitable trading strategies, we are motivated to investigate if we can increase the profitability of such strategies by combining their indicators. Thus, in this paper we propose a novel Genetic Programming (GP) algorithm that combines the three analysis types and we showcase the advantages of their combination in terms of three financial metrics, namely Sharpe ratio, rate of return and risk. We conduct experiments on 30 companies and based on the results, the combination of the three analysis types statistically and significantly outperforms their individual results, as well as their pairwise combinations. More specifically, the proposed GP algorithm has the highest mean and median values for Sharpe ratio and rate of return, and the lowest (best) mean value for risk. Moreover, we benchmark our GP algorithm against multilayer perceptron and support vector machine, and show that it statistically outperforms both algorithms in terms of Sharpe ratio and risk.

Keywords: Financial Engineering, Deep Learning
Abstract: Financial market is in general volatile with so many uncertainties and volatility is one of the main measures of uncertainty in the market among other measures. Hence, forecasting volatility is a critical component in risk management, optimizing portfolios, and in algorithmic trading among other financial problems. There have been few machine learning and artificial intelligence techniques used in the literature for the forecasting problem. Transformer Network (TN) architecture is one of newest such techniques proposed. In this work, we utilized this architecture with multi-head attention mechanism for volatility forecasting. To enhance the performance of the TN, we incorporated different variations of the feed forward layer. The performance of three distinct TN models was evaluated by implementing three different deep learning layers (CNN, LSTM, and a hybrid layer (CNN-LSTM)) in the encoder block of TN as the feed forward layer. The results clearly demonstrate that the TN model with the hybrid layer (CNN-LSTM) outperformed the other models, including a recently proposed data-driven approach.

Keywords: Financial Engineering, Advanced Optimization, Decision Making
Abstract: Diversifying asset allocation is a crucial aspect of building a profitable portfolio. The resiliency of the portfolio depends on the optimization techniques as well as algorithms used in the asset allocation. Clustering techniques would help in designing a diversified portfolio. This study investigates the resiliency of different traditional and recently proposed data-driven portfolio techniques in conjunction with four clustering techniques under varying market conditions. The novelty of the study is to present a resilient portfolio optimization using DBSCAN and Affinity Propagation clustering techniques.

Keywords: Explainability, Financial Engineering, Decision Making
Abstract: In the process of negotiating investment strategies between a fund and investors, establishing trust, transparency, traceability, and correctness among the involved parties is crucial to ensure smooth and successful outcomes. The adoption of logic-based AI, with its reliability, consistency, and explainability, can serve as a crucial catalyst to assist parties during negotiations by providing useful insights and explainable suggestions. This paper showcases how various Knowledge Representation and Reasoning (KRR) techniques can be leveraged to assist financial parties during investment negotiations. It demonstrates the use of logical definitions to represent complex financial investment strategies, allowing parties to gain a comprehensive understanding of the policies under discussion. Furthermore, automated reasoning is used to generate useful insights and actionable information enabling informed decision-making and enhancing the overall negotiation process.

Keywords: Decision Making, Data Mining, Explainability
Abstract: Sentiment lexicons are important tools for research involving opinion mining and sentiment analysis. They are highly inter-operable, and address critical limitations of learning-based or large language model-based sentiment analysis, providing better reproducibility and explainability. Existing financial sentiment lexicons, manually crafted or automatically constructed, primarily comprise single-word entries despite the fact that jargon, terminologies, and collocations in finance are often multi-word expressions. To address this gap, we present FinSenticNet, a concept-level domain-specific lexicon specifically designed for financial sentiment analysis, where over 65% entries are multi-word expressions. Our construction approach is semi-supervised: the framework consists of a concept parser, a sentiment seeds generation module, and a semantic graph construction module. Each concept (graph node) is subsequently classified in terms of its polarity using the Label Propagation Algorithm and Graph Convolutional Network. Compared to other financial sentiment lexicons, FinSenticNet captures domain-specific language features and has a broader coverage. We demonstrate this with superior evaluation results, i.e., sentiment analysis accuracy and F-scores, on multiple well-received benchmark datasets.

Keywords: Human-Like Intelligence, Autonomous Systems
Abstract: Although AI research aims to build human-level artificial intelligence, it was not clearly defined. Furthermore, many tests for HLAI have been proposed, but those are not practical and thus are not used in evaluating AI research. We conjecture that learning from others' experience with the language is the essential characteristic that distinguishes human intelligence from the rest. Humans can update the behavior policy with verbal descriptions as if they had experienced it first-hand. We present a classification of intelligence according to how individual agents learn and propose a definition and a test for HLAI. The main idea is that language acquisition without explicit rewards can be a sufficient test for HLAI. We built a simulated environment to conduct this test practically, and we hope other researchers can use it to facilitate the research on HLAI.

Keywords: Human-Like Intelligence, Explainability
Abstract: The understanding of the functional aspects of objects and tools is of paramount importance in supporting an intelligent system in navigating around in the environment and interacting with various objects, structures, and systems, to help fulfil its goals. A detailed understanding of functionalities can also lead to design improvements and novel designs that would enhance the operations of AI and robotic systems on the one hand, and human lives on the other. This paper demonstrates how a particular object – in this case, a frying pan – and its participation in the processes it is designed to support – in this case, the frying process – can be represented in a general function representational language and framework, that can be used to flesh out the processes and functionalities involved, leading to a deep conceptual understanding with explainability of functionalities that allows the system to answer “why” questions – why is something a good frying pan, say, or why a certain part on the frying pan is designed in a certain way? Or, why is something not a good frying pan? This supports the re-design and improvement on design of objects, artifacts, and tools, as well as the potential for generating novel designs that are functionally accurate, usable, and satisfactory.

Keywords: Human-Like Intelligence, Image Processing, Deep Learning
Abstract: uman eye gaze estimation is an important cognitive ingredient for successful human-robot interaction, enabling the robot to read and predict human behavior. We approach this problem using artificial neural networks and build a modular system estimating gaze from separately cropped eyes, taking advantage of existing well-functioning components for face detection (RetinaFace) and head pose estimation (6DRepNet). Our proposed method does not require any special hardware or infrared filters but uses a standard notebook-builtin RGB camera, as often approached with appearance-based methods. Using the MetaHuman tool, we also generated a large synthetic dataset of more than 57,000 human faces and made it publicly available. The inclusion of this dataset (with eye gaze and head pose information) on top of the standard Columbia Gaze dataset into training the model led to better accuracy with a mean average error below two degrees in eye pitch and yaw directions, which compares favourably to related methods. We also verified the feasibility of our model by its preliminary testing in real-world setting using the builtin 4K camera in NICO semi-humanoid robot’s eye.

Keywords: Human-Like Intelligence
Abstract: The AI in games has a large impact on the player’s experience, but the large variety of available AI implementation methods makes it difficult to determine which one(s) to use in any particular project, and the differences in their impact on players are mostly unstudied. This paper presents a comparative study to analyse the effects of Behaviour Tree AI and Hierarchical Task Network Planning AI on players experiences. The study participants (players) were given two prototypes of a third-person shooter game, each utilising different AIs, to play and give feedback on. According to the results obtained, players did not notice any major differences between the two prototypes, leading us to believe that the Behaviour Tree AI may be a better solution in most cases, as it is easier to implement.

Keywords: Human-Like Intelligence, Decision Making, Fuzzy Systems
Abstract: Human decision theory focuses on the reasoning behind the choices an individual makes. Human decision modelling is developed through mental models and can be modelled in different ways, such as fuzzy logic, deductive logic and probabilistic logic. On the other hand, machine learning techniques use a variety of statistical, probabilistic, and optimization methods to learn and detect useful patterns. In this context, this study investigates the complexities of human and machine randomness, utilizing two distinct datasets: one representing the perceived randomness of humans through the selection of nine numbers and the other encapsulating algorithmically generated random numbers from machines. The comparison of these datasets aims to understand the similarities and divergences between human (brain) randomness and machine randomness, primarily through the lens of fairness, neurocomputational, and decision-making simulations.

Keywords: Deep Learning, Financial Engineering, Decision Making
Abstract: Trading volume is an important variable to successfully capture market risks along with asset price/returns. Recently, there has been a growing interest in deep learning methods to forecast the trading volume of stocks using historical volatility as a feature. Unlike the existing work, a novel data-driven log volatility forecast is proposed in this paper as an extra feature to improve trading volume forecasts. Recently, neural networks for volatility and neural nets for electricity demand forecasting, constructed with nnetar function, have shown to be superior. The novelty of this paper is to demonstrate the neural network based on the nnetar function from the forecast package in R for trading volume forecast shows superiority over the other neural network.

Keywords: Advanced Optimization, Deep Learning, Evolvable Systems
Abstract: A fundamental problem in chemical product design is how to suitably identify chemical compounds that optimise multiple properties for a given application whilst satisfying relevant constraints. Current product synthesis generally uses trial-and-error experimentation, requiring lengthy and expensive research and development efforts. This paper introduces a novel computational chemistry approach for product design combining geometric deep learning for inference of property values and evolutionary multi-objective optimisation for identification of products of interest. Preliminary empirical results indicate that the proposed approach can be used to optimise product design considering multiple objectives and constraints given incomplete molecular attribute information.

Keywords: Deep Learning, Fuzzy Systems, Intelligent Control
Abstract: Type-2 fuzzy systems have a great adoption in different branches of engineering, due to the fact that this type of fuzzy systems are very well suited to tasks related to nonlinear systems. Data driven models like neural networks and fuzzy systems have some disadvantages, such as the high and uncertain dimensions and complex learning process. In this paper, we show the advantages of type-2 fuzzy systems over type-1 fuzzy systems in modeling nonlinear systems. We combine Type-2 Takagi-Sugeno fuzzy model with the popular deep learning model, LSTM (long-short term memory), to overcome the disadvantages fuzzy model and neural network model. We propose a fast and stable learning algorithm for this model. Comparisons with others similar black-box and grey-box models are made, in order to show the advantages of the type-2 fuzzy LSTM neural networks.

Keywords: Advanced Optimization, Decision Making
Abstract: The potential intelligence behind advanced machining systems (AMSs) offers positive contributions toward process improvement. Compared with conventional meta-heuristics, imitation learning (IL) appears to provide a more powerful tool to exploit such intelligence by observing demonstrations from technologists. This paper proposes a novel IL-based policy search algorithm that equips the agent with the optimization knowledge by executing upper-level policy learning to generate an imitation policy distribution with diverse decision behaviors. The experimental results of heavy cutting scenarios show that the proposed method rather than meta-heuristics is more viable for solving AMS optimization problems.

Keywords: Deep Learning, Model-Based, Data Mining
Abstract: Municipal solid waste incineration (MSWI) technology has developed rapidly worldwide. Carbon monoxide (CO) is one of the to be controlled key operating index of such processes. CO emission concentration prediction is a challenge problem duo to its large fluctuation range. A new CO emission concentration prediction method based on concept drift detection using kernel principal component analysis (KPCA) is proposed. The proposed approach includes off-line model construction module, on-line concept drift detection prediction and updating module. First, we construct the LSTM-based CO prediction model using historical data and KPCA-based concept drift detection model for calculating the evaluation index. Then, recursive KPCA is used to adaptive monitor the concept drift of the time-varying process. Finally, based on continuous updating of the historical LSTM mode with the concept drift samples, we achieve higher prediction accuracy. The rationality and validity are verified with the actual data of MSWI processes.

Keywords: Data Mining, Dimension Reduction, Ensemble Learning
Abstract: Municipal solid waste incineration (MSWI) is a crucial technology for waste treatment in densely populated cities. It plays a vital role in contributing to the hot concept of waste-to-energy. However, the effectively measuring of dioxins (DXN) emission from MSWI plants presents a complex challenge duo to its high economical cost and large lag time. To address the challenge, we propose a soft sensing method of DXN emission concentration based on the fast tree broad learning system (FTBLS) and the robust principal component analysis (RPCA). FTBLS can swiftly construct the DXN emission model with increment learning for obtaining accurate measuring results. RPCA is capable decomposing high-dimensional small sample data into low-rank and noise matrices, achieving robust operation condition drift detection in the presence of noise process data. The similarity estimation is used to aid the soft measuring value’s obtainment for concept drift sample. The experiment and application results demonstrate the effectiveness of our proposed online soft sensing approach.

Keywords: Data Mining, Ensemble Learning
Abstract: Concrete is an essential material ubiquitously employed in construction. Yet, deciphering the factors that influence its quality is a formidable challenge due to partially understood physical relationships, the high dimensionality of the data, and its limited availability. This study introduces an ensemble framework designed to address these challenges. It uses a combination of individual methods within an ensemble configuration to identify the critical features that determine concrete quality. Within this framework, diverse base methods are harmonized using an average-based technique, leading to a robust final verdict. After selecting the potential influencing factors, 50 experiments are conducted using the Taguchi Orthogonal Array (L-50) to generate the data points. The proposed ensemble learning framework underscores the substantial impact of storage conditions during the curing time on the final quality of concrete.

Keywords: Transportation and Vehicle Systems
Abstract: Driver behavior plays a pivotal role in preventing traffic accidents. Unlike previous studies primarily focused on observable car maneuvers, our research delves into actions that precede observable car maneuvers or remain unmanifested. We collected data from a car-sharing service frequently used by university students, meticulously analyzing the frequency of pedal changes through foot camera images. This dataset was compared with pedal depression data from the Controller Area Network (CAN) bus to detect potential safety risks attributable to cognitive load.

Our investigation revealed the existence of unrecorded driver behaviors that could potentially lead to traffic accidents. Even in the absence of recorded pedal operations in the CAN bus data, we identified locations with high pedal change frequency, signifying elevated cognitive load.

We utilized foot camera footage to track pedal changes and correlated this data with the frequency of pedal depressions recorded in the CAN bus data. This analysis evaluates the cognitive load by the gap between the frequency of pedal changes in the CAN bus data and the foot camera. As a significant outcome of our research, we developed a practical spatial map illustrating the distribution of cognitive load, as estimated through our foot placement analysis.

Notably, this cognitive load distribution map closely aligned with local knowledge and provided intuitively interpretable scenarios. It emerged as a valuable tool in identifying potential high-risk zones on the road, thereby contributing to ongoing efforts to enhance driving safety. Our findings substantially impact traffic safety measures and offer innovative insights into mitigating accidents by addressing cognitive load-induced driver behaviors.

Keywords: Deep Learning, Image Processing, Transportation and Vehicle Systems
Abstract: To develop robust and secure automated transportation systems, Traffic Sign Detection and Recognition (TSDR) is a key part. It plays a crucial role in Advanced Driver Assistance Systems (ADAS), self-driving vehicles and traffic safety. However, the task of TSDR can be challenging due to traffic signs being subject to damages, discoloration, vandalism and occlusion. Even though a lot of progress is made in both research areas of Traffic Sign Detection (TSD) and Traffic Sign Recognition (TSR), no study explicitly deals with the problem of qualitative poor traffic signs appearing in real-world scenarios. This can be assigned to the lack of an extensive traffic sign dataset containing flawless signs as well as imperfect signs. Neural networks trained exclusively on untainted data might fail at detecting flawed signs as they occur in real-world scenarios. Therefore, in this paper, a novel traffic sign dataset with condition annotations is proposed, indicating if a sign is good, discolored, vandalized, dirty or occluded. The custom dataset is created with a semi-supervised approach, in which machine learning models are trained to classify traffic signs in the condition categories. The resulting dataset can be used as basis for more precise traffic sign recognition as well as traffic sign condition classification which can be useful for maintenance planning. The dataset includes approx. 20.000 images of 10 sign classes, where 70% of data is incorporated in the training set, 10% in the validation set and 20% in the test set.

Keywords: Transportation and Vehicle Systems
Abstract: A routing and scheduling optimization approach for the airport ground movement problem considering runway spacing is introduced. An integrated modeling that considers both the routing of aircraft and runway required separations, is implemented through Aircraft Multi-Objective Optimization Algorithm AMOA* and a correct spacing validation module, coupled by a genetic algorithm in search of real-world feasible, yet optimized solutions, for a modern-day aviation setting based on London’s Stansted Airport. The proposed genetic algorithms successfully optimize taxiing time and fuel consumption for different airport traffic scenarios while fully respecting runway separation constraints. The difference between algorithms is emphasized to stress the risk of over-evaluation of savings by overlooking real-world operational conditions in the modeling phase of the problem.

Keywords: Image Processing, Autonomous Systems, Deep Learning
Abstract: We present the updated version of the HSI-Drive dataset aimed at developing automated driving systems (ADS) using hyperspectral imaging (HSI). The v2.0 version includes new annotated images from videos recorded during winter and fall in real driving scenarios. Added to the spring and summer images included in the previous v1.1 version, the new dataset contains 752 images covering the four seasons. In this paper, we show the improvements achieved over previously published results obtained on the v1.1 dataset, showcasing the enhanced performance of models trained on the new v2.0 dataset. We also show the progress made in comprehensive scene understanding by experimenting with more capable image segmentation models. These models include new segmentation categories aimed at the identification of essential road safety objects such as the presence of vehicles and road signs, as well as highly vulnerable groups like pedestrians and cyclists. In addition, we provide evidence of the performance and robustness of the models when applied to segmenting HSI video sequences captured in various environments and conditions. Finally, for a correct assessment of the results described in this work, the constraints imposed by the processing platforms that can sensibly be deployed in vehicles for ADS must be taken into account. Thus, and although implementation details are out of the scope of this paper, we focus our research on the development of computationally efficient, lightweight ML models that can eventually operate at high throughput rates. The dataset and some examples of segmented videos are available in https://ipaccess.ehu.eus/HSI-Drive/.

Keywords: Deep Learning, Autonomous Systems, Transportation and Vehicle Systems
Abstract: Abstract— Advanced Driver Assistance Systems (ADAS) are designed to assist drivers in various driving scenarios, and the object detection system is a critical component of ADAS. This paper aims to develop and evaluate an object detection system using two cameras placed on the vehicle's front and rear sides for real-time inferencing in ADAS. The real-world data set is collected under different weather and lighting conditions to evaluate the object detection system. The object detection system is further optimized using the TensorRT engine to deploy the system on the in-vehicle computing unit, NVIDIA Jetson AGX Xavier. The object detection system achieved 18 fps to process two cameras simultaneously on the in-vehicle computing unit, NVIDIA Jetson AGX Xavier. The experimental findings of this study will be useful for researchers, engineers, and manufacturers in the field of ADAS and autonomous vehicles to improve road safety and reduce accidents.

Keywords: Transportation and Vehicle Systems, Graph Neural Networks, Deep Learning
Abstract: Ensuring validation for highly automated driving poses significant obstacles to the widespread adoption of highly automated vehicles. Scenario-based testing offers a potential solution by reducing the homologation effort required for these systems. However, a crucial prerequisite, yet unresolved, is the definition and reduction of the test space to a finite number of scenarios. To tackle this challenge, we propose an extension to a contrastive learning approach utilizing graphs to construct a meaningful embedding space. Our approach demonstrates the continuous mapping of scenes using scene-specific features and the formation of thematically similar clusters based on the resulting embeddings. Based on the found clusters, similar scenes could be identified in the subsequent test process, which can lead to a reduction in redundant test runs.

Keywords: Smart Grid, Multi-Agent System, Intelligent Control
Abstract: Power grids must be stable and reliable, but the growing importance of intelligent control in smart grids creates new challenges due to the increasing dependence on communication networks. This paper investigates the influence of communication on power systems in a future scenario. The simulation environment contains a power grid of a medium-sized city in northwest Germany, in the year 2035, where the normal operation of the power system is disturbed by increased communication traffic. The power and communication networks are integrated into a co-simulation environment, that implements smart grid services in an agent-based control structure. In a case study, multiple scenarios are compared that differ in the configuration of the communication network, to show how the simulation environment can be used to study the interactions between power and communication networks.

Keywords: Deep Learning, Smart Grid, Particle Swarm Optimization
Abstract: Accurate load forecasting plays a vital role in numerous sectors, but accurately capturing the complex dynamics of dynamic power systems remains a challenge for traditional statistical models. For these reasons, time-series models (ARIMA) and deep-learning models (ANN, LSTM,GRU, etc.) are commonly deployed and often experience higher success. In this paper, we analyze the efficacy of the recently developed Transformer-based Neural Network model in load forecasting. Transformer models have the potential to improve load forecasting because of their ability to learn long-range dependencies derived from their Attention Mechanism. We apply several metaheuristics namely Differential Evolution to find the optimal hyperparameters of the Transformer-based Neural Network to produce accurate forecasts. Differential Evolution provides scalable, robust, global solutions to non-differentiable, multi-objective, or constrained optimization problems. Our work compares the proposed Transformer-based Neural Network model integrated with different metaheuristic algorithms by their performance in load forecasting based on numerical metrics such as Mean Squared Error (MSE) and Mean Absolute Percentage Error (MAPE). Our findings demonstrate the potential of metaheuristic-enhanced Transformer-based Neural Network models in load forecasting accuracy and provide optimal hyperparameters for each model.

Keywords: Smart Grid, Cybersecurity, Autonomous Systems
Abstract: In this paper, we investigate industry-led blockchain projects in the field of smart grids. Our investigation is guided by five research questions related to each industry-led project: (1) is the project active? (2) what smart grid applications does the project target? (3) what technical approach does the project take? (4) what is the maturity level of the project? and (5) what we can learn from the success or failure of the project? Our findings show that only a few projects are still active, and many have been terminated when the funding was exhausted. Nevertheless, the few active projects give us hope that sustainable technical approaches in conjunction with sound business models could lead to long-term blockchain-based projects in smart grids. Most of the active projects are targeting energy trading and using custom tokens to incentivize the production of green energy and energy savings. Furthermore, it appears that layer-2 blockchains are becoming the preferred platform for achieving high throughput with low transaction fees while preserving the security and trust of traditional large public blockchains.

Keywords: Smart Grid, Model-Based, Intelligent Control
Abstract: Renewable energy generation sources (RESs) are gaining increased popularity due to global efforts to reduce carbon emissions and mitigate effects of climate change. Planning and managing increasing levels of RESs, specifically solar photovoltaic (PV) generation sources is becoming increasingly challenging. Estimations of solar PV power generations provide situational awareness in distribution system operations. A digital twin (DT) can replicate PV plant behaviors and characteristics in a virtual platform, providing realistic solar PV estimations. Furthermore, neural networks, a popular paradigm of artificial intelligence may be used to adequately learn and replicate the relationship between input and output variables for data-driven DTs (DD-DTs). In this paper, DD-DTs are developed for Clemson University’s 1 MW solar PV plant located in South Carolina, USA to perform realistic solar PV power estimations. The DD-DTs are implemented utilizing multilayer perceptron (MLP) and Elman neural networks. Typical practical results for two DD-DT architectures are presented and validated.

Keywords: Smart Grid
Abstract: Amidst the challenges posed by the high penetration of distributed energy resources (DERs), particularly a number of distributed photovoltaic plants (DPVs), in modern electric power distribution systems (MEPDS), the integration of new technologies and frameworks become crucial for addressing operation, management, and planning challenges. Situational awareness (SA) and situational intelligence (SI) over multi-time scales is essential for enhanced and reliable PV power generation in MEPDS. In this paper, data-driven digital twins (DTs) are developed using AI paradigms to develop actual and/or virtual models of DPVs, These DTs are then applied for estimating and forecasting the power outputs of physical and virtual PV plants. Virtual weather stations are used to estimate solar irradiance and temperature at user-selected locations in a localized region, using inferences from physical weather stations. Three case studies are examined based on data availability: physical PV plant, hybrid PV plants, and virtual PV plants, generating real-time estimations and short-term forecasts of PV power production that can support distribution system studies and decision-making.

Keywords: Deep Learning, Image Processing, Particle Swarm Optimization
Abstract: Evolutionary optimization methods have been utilized to optimize a wide range of models, including many complex neural network models. Manual parameter selection requires substantial trial and error and specialist domain knowledge of the inherent structure, which does not guarantee the best outcomes. We propose a three-layered novel architecture for semantic segmentation and optimize it using two distinct evolutionary algorithm-based optimization processes namely genetic algorithm and particle swarm optimization. To fully optimize an end-to-end image segmentation framework, the network design is tested using various combinations of a few parameters. An automatic search is conducted for the optimal parameter values to maximize the performance of the image segmentation framework. Evolutionary Algorithm (EA)-based optimization of the three-layered semantic segmentation network optimizes parameter values holistically, which produces the best performance. We evaluated our proposed architecture and optimization on two benchmark datasets. The evaluation results show that the proposed optimization can achieve better accuracy than the existing approaches.

Keywords: Deep Learning, Pattern Recognition, Data Mining
Abstract: Creating an intelligent search and retrieval system for artwork images, particularly paintings, is crucial for documenting cultural heritage, fostering wider public engagement, and advancing artistic analysis and interpretation. Visual-Semantic Embedding (VSE) networks are deep learning models used for information retrieval, which learn joint representations of textual and visual data, enabling 1) cross-modal search and retrieval tasks, such as image-to-text and text-to-image retrieval; and 2) relation-focused retrieval to capture entity relationships and provide more contextually relevant search results. Although VSE networks have played a significant role in cross-modal information retrieval, their application to painting datasets, such as ArtUK, remains unexplored. This paper introduces BoonArt, a VSE-based cross-modal search engine that allows users to search for images using textual queries, and to obtain textual descriptions along with the corresponding images when using image queries. The performance of BoonArt was evaluated using the ArtUK dataset. Experimental evaluations revealed that BoonArt achieved 97% Recall@10 for image-to-text retrieval, and 97.4% Recall@10 for text-to-image Retrieval. By bridging the gap between textual and visual modalities, BoonArt provides a much-improved search performance compared to traditional search engines, such as the one provided by the ArtUK website. BoonArt can be utilised to work with other artwork datasets.

Keywords: Pattern Recognition, Image Processing, Signal Processing
Abstract: Existing crowd counting techniques have achieved significant progress with the emergence of deep learning. During development, emerging crowd counting methods have generally become more and more complex and enormous, enabling them to understand and process more prior knowledge from input data. However, they suffer from two major drawbacks: 1) they generally require a significant amount of labeled training samples, which is labor-intensive, and 2) they require increasing computational hardware resources, making it luxurious and impractical to apply directly in small-scale scenes. To address these issues, we formulate crowd counting as a classification problem and leverage least squares model with a novel semi-supervised strategy. Technically, we construct the least squares model based on only two regularization terms: a regression term and a discriminative relaxation term. Moreover, we propose a semi-supervised soft label correcting strategy incorporated in the model. As a result, a fast and accurate crowd counting method is achieved. Experimental results on five small-scale benchmarks demonstrate the proposed method outperforms the other competitors in terms of both regression metrics and consumed time.

Keywords: Image Processing, Deep Learning
Abstract: Current Defect detection methods have made significant progress on ideal datasets that typically contain a large number of defect samples. This enables the traditional defect detection methods to achieve great detection performance. However, in practical applications, the training samples obtained are often highly imbalanced, with the majority being non-defective samples and only a few being defective samples. It will generally lead to performance degradation of traditional methods if using such imbalanced samples for training. To address this issue, we propose a few-shot defect detection method based on self-supervised learning. Specifically, we propose to use a transfer learning strategy to transfer from traditional full-shot learning to few-shot learning. Next, we propose to process the training data in a self-supervised manner. As a result, the proposed method is enabled to achieve satisfactory detection performance on the industrial magnetic tile defect dataset. Experimental results verify the effectiveness of our proposed method.

Keywords: Image Processing, Bio-inspired, Swarm Intelligence
Abstract: Contrast enhancement is critical for investigating and highlighting important hidden features in a computer vision system. Continuous functions, such as incomplete beta or sigmoid functions, have traditionally been used for histogram equalization. However, histogram equalization cannot uniformly enhance the local contrast of an image, which is its main limitation. In this study, we investigate a contrast enhancement method based on a hyperbolic tangent sigmoid whose parameters can be optimized by metaheuristics. In our study, we investigated the performance of three popular metaheuristics when coupling the proposed hyperbolic tangent sigmoid to find the optimal pixel values that can intensify features of low-contrast images. The proposed method is studied on a public domain image dataset and evaluated using standard performance indicators. Preliminary results show that the proposed hyperbolic tangent sigmoid can improve image contrast and quickly adapt to other metaheuristics.

Keywords: Bio-inspired, Signal Processing, Pattern Recognition
Abstract: The field of neuromorphic vision systems aims to replicate the functionality of biological visual systems by mimicking their physical structure and electrical behaviour. Unlike traditional full-frame sensors, neuromorphic systems process data asynchronously and at the pixel level, modelling biological signalling processes. This allows for high-speed operations with lower energy consumption, making them suitable for applications like autonomous vehicles and embedded robotics. This work introduces the Neuromorphic Event Alarm Time-Series Suppression (NEATS) framework, designed to filter noise and detect outlier behaviours in event data without the need for 2-D transformations. NEATS employs rolling statistics and advanced neuromorphic data structures to minimise noise while identifying changes in scene dynamics. This framework injects attention into scene processing, similar to summarisation frameworks in traditional image processing. A novel event-vision alarm change collection (EACC) database is presented, containing controlled stimuli pattern changes captured using leading neuromorphic imaging devices. This database facilitates future benchmarking of neuromorphic attention frameworks, advancing the development of efficient and accurate artificial vision systems.

Keywords: Bio-inspired, Model-Based, Pattern Recognition
Abstract: This paper presents a progression of a popular neuromorphic memory structure by exploring advanced forgetting models for robust long-term information storage. Inspired by biological neuronal systems, neuromorphic sensors efficiently capture and transmit sensory information using event-based communication. Managing the decay of information over time is a critical aspect, and forgetting models play a vital role in this process. Building upon the foundation of an existing popular neuromorphic memory structure, this study introduces and evaluates four advanced forgetting models: ROT, adaptive, emotional memory enhancement, and context-dependent memory forgetting models. Each model incorporates different factors to modulate the rate of decay or forgetting. Through rigorous experimentation and analysis, these models are compared with the original ROT forgetting model to assess their effectiveness in retaining relevant information while discarding irrelevant or outdated data. The results provide insights into the strengths, limitations, and potential applications of these advanced forgetting models in the context of neuromorphic memory systems, thereby contributing to the progression of this popular neuromorphic memory structure.

Keywords: Deep Learning, Evolving Learning
Abstract: Bidirectional Encoder Representations from Trans- formers (BERT) has achieved state-of-the-art results on various NLP tasks. However, the size of BERT makes application in time-sensitive scenarios challenging. There are lines of research compressing BERT through different techniques and Knowledge Distillation (KD) is the most popular. Nevertheless, more recent studies challenge the effectiveness of KD from an arbitrarily large teacher model. So far, research on the negative impact of the teacher-student gap on the effectiveness of knowledge transfer has been confined mainly to computer vision. Additionally, those researches were limited to distillations between teachers and students with similar model architectures. To fill the gap in the literature, we implemented a teacher assistant (TA) model lying between a fine-tuned BERT model and non-transformer- based machine learning models, including CNN and Bi-LSTM, for sentiment analysis. We have shown that teaching-assistant- facilitated KD outperformed traditional KD while maintaining a competitive inference efficiency. In particular, a well-designed CNN model could retain 97% of BERT’s performance while being 1410x smaller for sentiment analysis. We have also found that BERT is not necessarily a better teacher model than non- transformer-based neural networks.

Keywords: Deep Learning, Defense and Security, Fuzzy Systems
Abstract: Deep learning-based methods have proved useful for adversarial attack detection. However, conventional detection algorithms exploit crisp set theory for classification boundary. Therefore, representing vague concepts is not available. Motivated by the recent success in fuzzy systems, we propose a fuzzy rule-based neural network to improve adversarial attack detection accuracy. The pre-trained ImageNet model is exploited to extract feature maps from clean and attacked images. Subsequently, the fuzzification network is used to obtain feature maps to produce fuzzy sets of difference degrees between clean and attacked images. The fuzzy rules control the intelligence that determines the detection boundaries. In the defuzzification layer, the fuzzy prediction from the intelligence is mapped back into the crisp model predictions for images. The loss between the prediction and label controls the rules to train the fuzzy detector. We show that the fuzzy rule-based network learns rich feature information than binary outputs and offers to obtain an overall performance gain. Our experiments, conducted over a wide range of images, show that the proposed method consistently performs better than conventional crisp set training in adversarial attack detection with various fuzzy system-based neural networks. The source code of the proposed method is available at https://github.com/Yukino-3/Fuzzy.

Keywords: Deep Learning, Robotics
Abstract: Current techniques in Visual Simultaneous Localization and Mapping (VSLAM) estimate camera displacement by comparing image features of consecutive scenes. These algorithms depend on scene continuity, hence requires frequent camera inputs. However, processing images frequently can lead to significant memory usage and computation overhead. In this study, we introduce SemanticSLAM, an end-to-end visual-inertial odometry system that utilizes semantic features extracted from an RGB-D sensor. This approach enables the creation of a semantic map of the environment and ensures reliable camera localization. SemanticSLAM is scene-agnostic, which means it doesn't require retraining for different environments. It operates effectively in indoor settings, even with infrequent camera input, without prior knowledge. The strength of SemanticSLAM lies in its ability to gradually refine the semantic map and improve pose estimation. This is achieved by a convolutional long-short-term-memory (ConvLSTM) network, trained to correct errors during map construction. Compared to existing VSLAM algorithms, SemanticSLAM improves pose estimation by 17%. The resulting semantic map provides interpretable information about the environment and can be easily applied to various downstream tasks, such as path planning, obstacle avoidance, and robot navigation. The code will be publicly available at https://github.com/Leomingyangli/SemanticSLAM

Keywords: Data Mining, Deep Learning, Big Data
Abstract: This paper addresses the classification of Arabic text data in the field of Natural Language Processing (NLP), with a particular focus on Natural Language Inference (NLI) and Contradiction Detection (CD). Arabic is considered a resource-poor language, meaning that there are few data sets available, which leads to limited availability of NLP methods. To overcome this limitation, we create a dedicated data set from publicly available resources. Subsequently, transformer-based machine learning models are being trained and evaluated. We find that a language-specific model (AraBERT) performs competitively with state-of-the-art multilingual approaches, when we apply linguistically informed pre-training methods such as Named Entity Recognition (NER). To our knowledge, this is the first large-scale evaluation for this task in Arabic, as well as the first application of multi-task pre-training in this context.

Keywords: Deep Learning, Image Processing, Robotics
Abstract: Uncertainty estimation is an important task for critical problems, such as robotics and autonomous driving, because it allows creating statistically better perception models and signaling the model's certainty in its predictions to the decision method or a human supervisor. In this paper, we propose a Variational Neural Network-based version of a Voxel Pseudo Image Tracking (VPIT) method for 3D Single Object Tracking. The Variational Feature Generation Network of the proposed Variational VPIT computes features for target and search regions and the corresponding uncertainties, which are later combined using an uncertainty-aware cross-correlation module in one of two ways: by computing similarity between the corresponding uncertainties and adding it to the regular cross-correlation values, or by penalizing the uncertain feature channels to increase influence of the certain features. In experiments, we show that both methods improve tracking performance, while penalization of uncertain features provides the best uncertainty quality.

Keywords: Deep Learning, Big Data, Model-Based
Abstract: Companies use users' opinions to improve their products and marketing activities. In recent years, the development of Internet technology has made it possible to extract users' opinions from text on the Web. There are many ways for users to post their opinions on the Internet, and Twitter is considered to be a platform that allows users to easily tweet their opinions. However, manually extracting opinions from Twitter is time-consuming, costly, and labor-intensive due to the relatively low percentage of opinions. Therefore, some companies aim to efficiently extract user opinions from Twitter using machine learning. However, the attempt to create a dataset for building a machine learning system produced an unbalanced dataset that does not extract opinions with sufficient accuracy because the proportion of views on Twitter is small. There are solutions to this problem of insufficient teacher data, such as utilizing knowledge from other domains through transfer learning. Although transfer learning is sometimes used to solve such problems, accuracy cannot be improved if the knowledge domains are far apart before and after the transfer. Therefore, we proposed a new method called OTR, which stands for Opinion classifier Transferred from Review data. OTR transfers knowledge of review submissions that are considered to be close in domain to opinion extraction. However, since the phrasing of review sentences and that of Social Networking Service (SNS) such as Twitter are different, there is a possibility that sufficient knowledge transfer cannot be achieved. In order to address this problem, we proposed an Opinion classifier Transferred from Review data with Pseudo-labels (OTR-P), a method that brings the domains of the source and target tasks closer. Here, the target task discriminated opinions regarding leisure facilities, and the source task estimated review ratings using Rakuten travel review data. And while performing these tasks, we attempted to bring the domains closer by attaching pseudo-labels to the tweet data. This approach improved accuracy compared to the conventional method of shifting Bidirectional Encoder Representations from Transformers (BERT).

Keywords: Automated Algorithm, Dimension Reduction
Abstract: In numerical single-objective optimization, automated algorithm selection that uses exploratory landscape analysis to describe problem features has achieved great results when the machine learning models used for prediction are trained and tested on the same problem set. However, recent work has shown that the performance of such models decreases when the training and testing sets contain different problems. In this paper, we examine a recently developed algorithm selection model trained on a set of artificial problems and tested on a well-known set of hand-made benchmark problems. This model performed poorly when it was originally presented. Here, we provide an explanation for its poor performance by analyzing the feature importance of the model using Shapley Additive Explanations. We then compare these results to an alternative algorithm selection model that was both trained and tested on the same set of hand-made benchmark problems and achieved much higher performance. This allows us to determine which features each model considers as most significant for their predictions, and where they differ. We show that the original and the alternative model use different landscape features for their predictions, which explains the difference in their performance. Further, by plotting the SHAP values on a 2D plane, we show that the original model is unable to distinguish between certain types of problems. Finally, we show that regardless of their differences in utilizing the features both the original and the alternative models perform poorly on a specific group of problems.

Keywords: Automated Algorithm, Bio-inspired
Abstract: Algorithm selection is a critical aspect of continuous black-box optimization, and various methods have been proposed to choose the most appropriate algorithm for a given problem. One commonly used approach involves employing Exploratory Landscape Analysis (ELA) features to represent optimization functions and training a machine-learning meta-model to perform algorithm selections based on these features. However, many meta-models trained on existing benchmarks suffer from limited generalizability. When faced with a new optimization function, these meta-models often struggle to select the most suitable algorithm, restricting their practical application. In this study, we investigate the generalizability of meta-models when tested on previously unseen functions that were not observed during training. Specifically, we train a meta-model on base COmparing Continuous Optimizers (COCO) functions and evaluate its performance on new functions derived as affine combinations between pairs of the base functions. Our findings demonstrate that the task of ranking algorithms becomes substantially more challenging when the functions differ from those encountered during meta-learning training. This indicates that the effectiveness of algorithm selection diminishes when confronted with problem instances that substantially deviate from the training distribution. In such scenarios, meta-models that use ELA features to predict algorithm ranks do not outperform mere predictions of the average algorithm ranks.

Keywords: Decision Making, Swarm Intelligence, Advanced Optimization
Abstract: Recently, Software-Defined Networking (SDN) is receiving much research attention due to its ability to decouple the data plane from the control architecture by associating the network switches to one (centralized) or more (distributed) controller(s). Traditionally, switches are assigned to the controllers in a static manner which results in under-utilization of the resources of the controllers and increased response delays to user requests. In this paper, we consider a practical load-balancing and agile scenario by formulating the dynamic associations of switches and controllers as an NP-hard optimization problem to minimize the maximum resource utilization of the controllers. Therefore, we propose an Ant Colony Optimization (ACO)-based algorithm to deal with the aforementioned request satisfiability issue in large SDN systems in polynomial-time. Furthermore, we envision a hybrid deep learning model consisting of Convolutional Neural Network (CNN) and Gated Recurrent Unit (GRU) structures to achieve near-optimal resource utilization for real-time SDN applications. Experimental results demonstrate that our customized CNN-GRU model outperforms the other techniques in terms of resource utilization (15%-45% optimality gap) within a significantly reduced computational running time (0.1s).

Keywords: Automated Algorithm, Randomized Algorithms, Bio-inspired
Abstract: There have been many different forms of evolutionary algorithms (EAs) designed by humans over the past 50 years, with many variants optimized for specific classes of problems. As computational resources grow, the automated design of EAs has become an increasingly viable method for improving performance. However, many components of EAs have been treated as largely immutable, for both human and automated designers, dramatically constraining design space. In particular, the evolutionary cycle (the repeating pattern of reproduction and survival) has little or no differences between most popular forms of EA. In a previous paper, we proposed a technique for automatically designing evolutionary cycles using directed graphs, greatly increasing the explorable design space. In this paper, we showcase an improved representation and evolutionary process, provide preliminary experiments demonstrating that EAs produced by this process can outperform those with a traditional cycle, and explore the phenotype landscape to show that the new space explored by our technique may contain better EAs than traditional cycles allow.

Keywords: Deep Learning, Reinforcement Learning, Automated Algorithm
Abstract: Automated Algorithm Configuration (AAC) usually takes a global perspective: it identifies a parameter configuration for an (optimization) algorithm that maximizes a performance metric over a set of instances. However, the optimal choice of parameters strongly depends on the instance at hand and should thus be calculated on a per-instance basis. We explore the potential of Per-Instance Algorithm Configuration (PIAC) by using Reinforcement Learning (RL). To this end, we propose a novel PIAC approach that is based on deep neural networks. We apply it to predict configurations for the Lin–Kernighan heuristic (LKH) for the Traveling Salesperson Problem (TSP) individually for every single instance.

To train our PIAC approach, we create a large set of 100,000 TSP instances with 2,000 nodes each --- currently the largest benchmark set to the best of our knowledge. We compare our approach to the state-of-the-art AAC method Sequential Model-based Algorithm Configuration (SMAC). The results show that our PIAC approach outperforms this baseline on both the newly created instance set and established instance sets.

Keywords: Automated Algorithm, Dimension Reduction, Particle Swarm Optimization
Abstract: The present study provides an analysis on the characteristics of single-objective optimization benchmark problems as well as the algorithms used to solve them. The target optimization domain involves the CEC competitions, each consisting a set of mathematical functions. Concerning the optimization tasks, the idea is to investigate the dis/-similarities between different competition scenarios and individual benchmarks. For the solvers, the goal is to detect the dis/-similarities between the algorithms applied to the CEC benchmarks. Those analysis missions are carried out by using the features directly and automatically extracted from the performance data, the quality of the solutions achieved by each algorithm on the benchmarks. The feature extraction process is realized through Singular Value Decomposition. Following the analysis on the algorithms, the potential of algorithm selection has been evaluated to see the performance improvement without actually developing a new algorithm, against those 20 algorithms.

Keywords: Financial Engineering, Decision Making, Advanced Optimization
Abstract: Recently there has been a growing interest in constructing portfolios with stocks and cryptocurrencies. As cryptocurrency prices increase over the years, there is a growing interest in investing in cryptocurrencies, along with diversifying portfolios by adding multiple cryptocurrencies to the existing portfolios. Even though investing in cryptocurrency leads to high returns, it also leads to high risk due to the high uncertainty of cryptocurrency price changes. Thus, more robust risk measures have been introduced to capture market risk and avoid investment loss, along with different types of portfolios to mitigate risks. Many portfolio techniques assume asset returns are normally distributed with constant variance. However, these assumptions are violated in many cases. Unlike the existing work, this study investigates the recently proposed data-driven exponentially weighted moving average (DDEWMA) covariance model to estimate the variance-covariance matrix for high frequency (hourly data) cryptocurrency returns in Markowitz portfolio optimization. The experimental results show that for high-frequency data, the DDEWMA approach outperforms the existing portfolio optimization model that uses the empirical variance-covariance matrix. Improvements have been identified in terms of the Sharpe ratio as well as risks (volatility, mean absolute deviation (MAD), Value-at-Risk (VaR), and Expected shortfall (ES)).

Keywords: Deep Learning, Financial Engineering, Multi-Agent System
Abstract: The financial market and public opinion are correlated. This means that changes in the financial market can result in changes to public opinion and changes to public opinion can result in changes to the financial market. Accordingly, it is essential for understanding and interacting with the financial market to gather text content from online sources and process it. As a result of the rapid growth of social media and other online sources, we have seen an exponential rise in data, particularly textual data, in recent years. It can be difficult for a person to read, let alone process, the massive volumes of data generated every day. This indicates that we need automated methods for processing textual data and extracting useful information. Automated text summarization is a method of shortening huge amounts of text without losing essential information. Transformers, which can efficiently manage and analyze textual data, are state-of-the-art text summarization models. However, developing such an automated text summarization model specialized in a domain (e.g. finance) can be challenging since we lack necessary domain-specific summarization datasets. In this work, we propose a pipeline for fully automating the finetuning of a text summarization model in a specific domain, namely cryptocurrency domain, without the involvement of human annotators. To this end, we introduce a novel method for self-improvement of text summarization models which relies on a model assistant which encodes domain knowledge, enabling finetuning text summarization models in specific domains in which we lack specific-domain summarization datasets. The proposed method is evaluated on a cryptocurrency-related text summarization problem and three well-known Large Language Models (LLMs) used for text summarization.

Keywords: Financial Engineering, Reinforcement Learning, Deep Learning
Abstract: High frequency trading strategies in the foreign exchange (FX) market often attempt to extract the latent signals in extremely noisy price moves to help inform trading decisions. Due to the fast-paced environments within which these decisions are made, intelligent trading is an impossible task for the human mind. Deep reinforcement learning (DRL) offers human-like intelligence and high speed computation but, due to the noisy nature of the tick data, can be prone to learning sub-optimal policies as a result of misleading feature and reward signals. In this work we use an intrinsic time sampling method referred to as directional changes (DC), which reports information whenever there is a significant change in price. By sampling tick data from nine FX currency pairs for 2250 datasets, we were able to train reinforcement learning (RL) agents using the Proximal Policy Optimisation (PPO) algorithm to identify and trade profitable strategies in high frequency FX environments. The resultant models were compared to four benchmarks including buy and hold, moving average crossover, relative strength index and a rule-based DC strategy, across three different metrics (namely returns, maximum drawdown, and Calmar ratio), with the reinforcement learning models outperforming them all.

Keywords: Deep Learning, Financial Engineering
Abstract: Credit score prediction is a crucial task in financial industry, as it helps lenders and financial institutions evaluate the creditworthiness of borrowers and manage credit risk. In this work, we present a comparative study of deep learning (DL)-based credit score prediction models. To achieve this objective, we compare the performance of DL models against traditional methods in credit scoring. We train and test the models using a real-world dataset of credit histories, containing various features such as credit card balances, payment history, and employment status. Our experimental results show that the hybrid LSTM-GRU model outperform both the LSTM and GRU models in credit score prediction, as well as traditional methods. The hybrid LSTM-GRU model demonstrates higher accuracy and better predictive power, indicating its potential for improving credit scoring models in the financial industry.

Keywords: Financial Engineering
Abstract: Dynamic portfolio optimization is inherently challenging due to the complexity of asset price dynamics and forecasts. Robust optimization is proposed as an alternative that incorporates return and risk uncertainty in portfolio optimization.

Directional change (DC) methods complement the standard, fixed time interval, and asset price data in terms of measuring the relationships and scaling laws between different types of events. DC methods can be extended for portfolio optimization using DC representations of assets and empirical scaling laws which indicate expected price changes and their duration.

In this paper, we study a robust DC-based portfolio optimization (RDC) method, for returns maximization. The proposed method uses price signals from the DC representations of multiple assets for portfolio rebalancing and optimization, together with a robust portfolio optimization rule that maximizes portfolio returns under return uncertainty.

We empirically study the effect of the robust DC-based portfolio optimization method with an application to 29 exchange-traded funds where each fund is a well-diversified asset with typically low-risk values. We compare the obtained portfolio results with benchmarks. The results indicate that the proposed method performs comparably to several benchmarks, and particularly improves a specific risk measure, maximum drawdown, in comparison to the benchmarks.

Keywords: Financial Engineering, Dimension Reduction, Decision Making
Abstract: In an increasingly digitalized commerce landscape, the proliferation of credit card fraud and the evolution of sophisticated fraudulent techniques have led to substantial financial losses. Automating credit card fraud detection is a viable way to accelerate detection, reducing response times and minimizing potential financial losses. However, addressing this challenge is complicated by the highly imbalanced nature of the datasets, where genuine transactions vastly outnumber fraudulent ones. Furthermore, the high number of dimensions within the feature set gives rise to the ``curse of dimensionality". In this paper, we investigate subspace learning-based approaches centered on One-Class Classification (OCC) algorithms, which excel in handling imbalanced data distributions and possess the capability to anticipate and counter the transactions carried out by yet-to-be-invented fraud techniques. The study highlights the potential of subspace learning-based OCC algorithms by investigating the limitations of current fraud detection strategies and the specific challenges of credit card fraud detection. These algorithms integrate subspace learning into the data description; hence, the models transform the data into a lower-dimensional subspace optimized for OCC. Through rigorous experimentation and analysis, the study validated that the proposed approach helps tackle the curse of dimensionality and the imbalanced nature of credit card data for automatic fraud detection to mitigate financial losses caused by fraudulent activities.

Keywords: Ensemble Learning, Fault Detection, Particle Swarm Optimization
Abstract: In this study, we proposed a new ensemble method to predict the magnitude of gross errors (GEs) on measurement data obtained from the hydrocarbon and stream processing industries. Our proposed model consists of an ensemble of regressors (EoR) obtained by training different regression algorithms on the training data of measurements and their associated GEs. The predictions of the regressors are aggregated using a weighted combining method to obtain the final GE magnitude prediction. In order to search for optimal weights for combining, we modelled the search problem as an optimisation problem by minimising the difference between GE predictions and corre- sponding ground truths. We used Genetic Algorithm (GA) to search for the optimal weights associated with each regressor. The experiments were conducted on synthetic measurement data generated from 4 popular systems from the literature. We first conducted experiments in comparing the performances of the proposed ensemble using GA and Particle Swarm Optimisation (PSO), nature-based optimisation algorithms to search for com- bining weights to show the better performance of the proposed ensemble with GA. We then compared the performance of the proposed ensemble to those of two well-known weighted ensemble methods (Least Square and BEM) and two ensemble methods for regression problems (Random Forest and Gradient Boosting). The experimental results showed that although the proposed ensemble took higher computational time for the training process than those benchmark algorithms, it performed better than them on all experimental datasets.

Keywords: Ensemble Learning, Deep Learning, Signal Processing
Abstract: Our research focuses on integrating Laban Movement Analysis (LMA) with neural network technologies for analyzing conducting gestures. While promising, this approach faces challenges in real-time speed and adaptability. To overcome these limitations, we propose using tree ensembles, conducting LMA classification on conducting gestures with time-invariant transforms. This study aims to outperform the previous neural network approach in terms of time and set a benchmark for comparison, contributing valuable insights to enhance real-time applications in conducting gestures.

Keywords: Ensemble Learning, Deep Learning, Image Processing
Abstract: Tumor detection has emerged as a significant aspect of neuro-oncology and neuroradiology, with critical importance in improving patient survival rates. Tumors, whether benign (non-cancerous) or malignant (cancerous), can result in severe morbidity, and their accurate detection is very important for treatment. In recent years, medical imaging modalities such as Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) have been extensively utilized for non-invasive tumor detection. These imaging techniques provide in-depth information about the tumor’s location, size, and morphology, which is pivotal for diagnosing and planning therapeutic interventions. However, the manual interpretation of these imaging modalities is time-intensive and susceptible to human inaccuracies. Moreover, the subtle features of tumors can be easily missed in the manual assessment. Hereby, we propose an ensemble deep learning approach to classify pituitary tumors, based on the weighted average technique that incorporates three base deep learning models: ResNet 152, DenseNet 201, and VGG 16. Moreover, we implement the Segment Anything Model (SAM) to perform segmentation to our dataset and then execute the ensemble model to classify pituitary tumors from normal/healthy brain images. We compare our proposed approach using segmented data and non-segmented data, finding that the segmented data outperforms the non-segmented data by a margin of 1.77%.

Keywords: Evolving Learning, Randomized Algorithms, Advanced Optimization
Abstract: Hypervolume optimal mu-distribution is the distribution of mu solutions maximizing the hypervolume indicator of mu solutions on a specific Pareto front. Most studies have focused on simple Pareto fronts such as triangular and inverted triangular Pareto fronts. There is almost no study which focuses on complex Pareto fronts such as disconnected and partially degenerate Pareto fronts. However, most real-world multi-objective optimization problems have such a complex Pareto front. Thus, it is of great practical significance to study the hypervolume optimal mu-distribution on the complex Pareto fronts. In this paper, we study this issue by empirically showing the hypervolume optimal mu-distributions on the Pareto fronts of some representative artificial and real-world test problems. Our results show that, in general, maximizing the hypervolume indicator does not lead to uniformly distributed solution sets on the complex Pareto fronts. We also give some suggestions related to the use of the hypervolume indicator for performance evaluation of evolutionary multi-objective optimization algorithms.

Keywords: Evolving Learning, Randomized Algorithms, Advanced Optimization
Abstract: In evolutionary multi-objective optimization (EMO), one important issue is to efficiently remove dominated solutions from a large number of solutions examined by an EMO algorithm. An efficient approach to remove dominated solutions from a large solution set is to partition it into small subsets. Dominated solutions are removed from each subset independently. This partition method is fast but cannot guarantee to remove all dominated solutions. To further remove remaining dominated solutions, a simple idea is to iteratively perform this approach. In this paper, we first examine three partition methods (random, objective value-based and cosine similarity-based methods) and their iterative versions through computational experiments on artificial test problems (DTLZ and WFG) and real-world problems. Our results show that the choice of an appropriate partition method is problem dependent. This observation motivates us to use a hybrid approach where different partition methods are used in an iterative manner. The results show that all dominated solutions are removed by the hybrid approach in most cases. Then, we examine the effects of the following factors on the computation time and the removal performance: the number of objectives, the shape of the Pareto front, and the number of subsets in each partition method.

Keywords: Advanced Optimization, Evolving Learning, Randomized Algorithms
Abstract: In this paper, we examine the effect of normalization in R2-based hypervolume and hypervolume contribution approximation. The fact is that the region with different scales on objective space brings approximation bias. The basic idea of normalization is to perform a coordinate transformation to make the shape of the approximated region more regular, and then transform it to obtain the final value according to the property of hypervolume and hypervolume contribution. The performance of normalization is evaluated on different datasets by comparing it with the original R2-based method. We use two different metrics to evaluate hypervolume and hypervolume contribution separately, and the results indicate that normalization does exactly improve the approximation accuracy and outperforms the original R2-based method.

Keywords: Fuzzy Systems, Operations Research
Abstract: The uncertainty in actual manufacturing systems often manifests as uncertain processing times, especially in flexible manufacturing systems. This work proposes a Decomposition-based Evolutionary Algorithm with Local Search (DLSEA) to solve flexible scheduling with fuzzy processing times by minimizing makespan and total machine workload. Considering the different scales of objectives, two normalization methods are employed on subpopulations, respectively, aiming to mitigate the potential detrimental effects of a single normalization method. This work also introduces a local search method to enhance the performance of DLSEA. The proposed DLSEA is compared with four state-of-the-art algorithms on two series of cases. The experimental results show that DLSEA exhibits superior search capabilities.

Keywords: Transportation and Vehicle Systems, Agent-Based Modeling, Image Processing
Abstract: Pedestrians and cyclists are some of the most vulnerable, but also least predictable traffic participants. Due to their ability to move in urban environments with high degrees of freedom and sudden changes of direction, their movement is still challenging to predict. We present a driver assistance system that tackles some of these challenges. Our system consists of a world model made of a variational autoencoder and a long short-term memory network. The world model takes vision and action data from the perspective of the vulnerable traffic participant and generates a visual prediction (image) of their environment up to one second in advance. The second part of our system is a transformer-based description system that takes the predicted perceptions and here, as a showcase, abstracts them down to a textual warning if a collision between car and vulnerable traffic participant seems imminent. Our description system helps contextualize the dangerous situation for the driver and could be extended to other driver assistance systems, such as blind spot detection. We evaluate our system on a dataset generated in simulations using CARLA.

Keywords: Randomized Algorithms, Swarm Intelligence, Evolving Learning
Abstract: There are significant challenges in designing optimization algorithms for constrained large-scale multiobjective optimization problems due to numerous decision variables and constraints. For example, the decision space size exponentially grows with the number of decision variables, and constraints restrict the feasible range, increasing the complexity of the search space. To solve these problems, this paper presents a constrained large-scale multiobjective optimization algorithm based on adaptive paired offspring generation (aPOCEA). Specifically, an adaptive parameter adjustment strategy is proposed to determine the number of solutions in each subpopulation and balance the exploration and exploitation ability of the algorithm, enhancing the convergence speed of aPOCEA. Meanwhile, we propose a parent selection strategy to select high-quality parent solutions, increasing the probability of generating high-quality offspring solutions. Experimental results on ten benchmarks, each with two to three objectives, multiple constraints, and hundreds of decision variables, demonstrate that aPOCEA outperforms other representative optimization algorithms.

Keywords: Deep Learning, Transportation and Vehicle Systems, Model-Based
Abstract: The trajectory prediction of heavy heavy trains (HHTs) is crucial for ensuring the safe and automatic operation. It is inevitably to design a train model for predicting train operation trajectory for HHTs. However, large capacity, complex line parameters, and the existence of many unmodeled dynamics in the operation process including air resistance, working condition switching, external environment make it difficult to establish accurate speed trajectory prediction models (STPMs) using traditional mechanism-based modeling methods. Most research now consider using data-driven model to learn information from data, but they have on information about the physical characteristic of the STMP model. This makes it difficult to accurately describe the relationship between the control force and the running speed during the train operation. To overcome these issues, this study combines the mechanism-driven mechanism model with the deep learning model to construct a new long and short-term memory hybrid (LSTMM) model. Specifically, the mechanism model describes the change of control force, while the LSTM captures the unmodel dynamic characteristics of long time series of running process. The effectiveness of the proposed method is demonstrated while the performance is compared with the traditional LSTM and mechanism models using the real data.

Keywords: Operations Research, Model-Based, Automated Algorithm
Abstract: Steel production involves many energy-intensive processes. Under the goal of carbon peak and carbon neutrality, it is essential to study the steel production scheduling problems aiming at energy saving to realize the green manufacturing of steel production processes. Aiming at the hot rolling process of high-quality steel products, a novel energy-saving production scheduling problem is studied in this paper. Unlike existing research, this paper additionally considers temperature constraints and the optimization of temperature-keeping equipment assignment in producing high-quality steel products. To solve it efficiently, this paper presents an improved simulated annealing algorithm where destruction and construction strategies from iterated greedy algorithms are embedded into it. Experimental results show that the presented algorithm has obvious advantages compared with other competitive peers. Its excellent solution performance means its great application potential.

Keywords: Deep Learning, Autonomous Systems, Model-Based
Abstract: In recent years there have been remarkable advancements in autonomous driving. While autonomous vehicles demonstrate high performance in closed-set conditions, they encounter difficulties when confronted with unexpected situations. At the same time, world models emerged in the field of model-based reinforcement learning as a way to enable agents to predict the future depending on potential actions. This led to outstanding results in sparse reward and complex control tasks. This work provides an overview of how world models can be leveraged to perform anomaly detection in the domain of autonomous driving. We provide a characterization of world models and relate individual components to previous works in anomaly detection to facilitate further research in the field.

Keywords: E-health, Deep Learning, Reinforcement Learning
Abstract: One of the most effective ways to help patients at the dangerous levels of diabetes is an artificial pancreas, a device that constantly monitors the patient's blood sugar level and injects insulin based on this level. Patient's reaction to insulin is highly individualized, so the artificial pancreas needs to be trained on each patient. It turns out that the best training results are attained when instead of the usual ReLU neurons, we use their minor modification known as Exponential Linear Units (ELU). In this paper, we provide a theoretical explanation for the empirically observed effectiveness of ELUs.

Keywords: Fuzzy Systems, Intelligent Control
Abstract: In many practical situations, practitioners use easier-to-compute fuzzy control to approximate the more-difficult-to-compute optimal control. As expected, for many characteristics, this approximate control is slightly worse than the optimal control it approximates. However, with respect to robustness or smoothness, the approximating fuzzy control is often better than the original one. In this paper, we provide a theoretical explanation for this somewhat mysterious empirical phenomenon.

Keywords: Model-Based, Transportation and Vehicle Systems
Abstract: This paper presents a novel technique for the approximation of the survival signature for very large systems. In recent years, the survival signature has seen promising applications for the reliability analysis of critical infrastructures. It outperforms traditional techniques by allowing for complex modelling of dependencies, common causes of failures and imprecision. However, as an inherently combinatorial method, the survival signature suffers greatly from the curse of dimensionality. Computation for very large systems, as needed for critical infrastructures, is mostly infeasible. New advancements have applied Monte Carlo simulation to approximate the signature instead of performing a full evaluation. This allows for significantly larger systems to be considered. Unfortunately, these approaches will also quickly reach their limits with growing network size and complexity. In this work, instead of approximating the full survival signature, we will strategically select key values of the signature to accurately approximate it using a surrogate radial basis function network. This surrogate model is then extended to an interval predictor model (IPM) to account for the uncertainty in the prediction of the remaining unknown values. In contrast to standard models, IPMs return an interval bounding the survival signature entry. The resulting imprecise survival signature is then fed into the reliability analysis, yielding upper and lower bounds on the reliability of the system. This new method provides a significant reduction in numerical effort enabling the analysis of larger systems where the required computational demand was previously prohibitive.

Keywords: Human-Like Intelligence, Decision Making, Intelligent Control
Abstract: To integrate and assist the system and automation design phases of complex marine vessels, this paper proposes a two-level semantically enhanced scheme. At the design level, the system components are described and automatically connected by a developed graph-making tool using semantic “knowledge“. Decisions regarding the system selection are made based on certain Quality of Service Criteria (QoS) and enforced in the final semantic database using a dedicated cognitive agent. The automation level leverages the selected systems semantic information with that of the associated automation components and reuses the graph-making tool to update the connection graph. The resulting knowledge-graph is then used to “reason“ for the creation of feasible closed-loop control architectures while a cognitive agent determines which closed-loop architecture to use based on various QoS criteria. The chosen closed-loop architecture can then change in an online manner during the vessel operation in case that system reconfiguration is required either due to malfunctioning components, or aiming to satisfy mission’s goals. The applicability and efficiency of the proposed method are shown using a case study for marine propulsion.

Keywords: Explainability, Pattern Recognition, Image Processing
Abstract: Artificial intelligence (AI) reasoning and explainable AI tasks have gained popularity recently, enabling users to explain the predictions or decision processes of AI models. This paper introduces Forest Monkey (FM), a toolkit designed to reason the outputs of any AI-based defect detection and/or classification model with data explainability. Implemented as a Python package, the FM takes input in the form of dataset folder paths (including original images, ground truth labels, and predicted labels) and provides a set of charts and a text file to illustrate the reasoning results and suggest possible improvements. The FM toolkit consists of processes such as feature extraction from predictions to reasoning targets, feature extraction from images to defect characteristics, and a decision tree-based AI-Reasoner. Additionally, this paper investigates the time performance of the FM toolkit when applied to four AI models with different datasets. Lastly, a tutorial is provided to guide users in performing reasoning tasks using the FM toolkit.

Keywords: Reinforcement Learning, Intelligent Control, Transportation and Vehicle Systems
Abstract: Design space exploration is a crucial aspect of engineering and optimization, focused on identifying optimal design configurations for complex systems with a high degree of freedom in the actor set. It involves systematic exploration while considering various constraints and requirements. One of the key challenges in design space exploration is the need for a control strategy tailored to the particular design. In this context, reinforcement learning has emerged as a promising solution approach for automatically inferring control strategies, thereby enabling efficient comparison of different designs. However, learning the optimal policy is computationally intensive, as the agent determines the optimal policy through trial and error. The focus of this study is on learning a single strategy for a given design and scenario, enabling the evaluation of numerous architectures within a limited time frame. The study also highlights the importance of plant modeling considering different modeling approaches to effectively capture the system complexity on the example of vehicle dynamics. In addition, a careful selection of an appropriate hyperparameter set for the reinforcement learning algorithm is emphasized to improve the overall performance and optimization process.

Keywords: Fault Detection, Internet of Things
Abstract: This paper proposes refrigerated showcase fault detection by an autoencoder with coin betting and Maximum Correntropy Criterion (MCC). In actual situations, showcase data may include outliers which are incorrectly stored data. Radio frequency interference or incorrect sensor setting cause the outliers. When the outliers are included in learning data, the conventional autoencoders using least square error (LSE) may be influenced by the outliers. On the other hand, even when the outliers are included in learning data, autoencoders using MCC can reduce influence from the outliers. Moreover, the conventional artificial neural networks utilize various learning algorithms such as stochastic gradient descent with momentum (SGDM), adaptive moment estimation (Adam), adaptive gradient algorithm (AdaGrad). These methods have hyperparameters related to a learning rate. Since the hyperparameters affect learning strongly, it is required to tune the hyperparameters appropriately and the tuning requires engineering costs. On the other hand, coin betting can automatically tune a learning rate appropriately while learning. Therefore, the coin betting is expected to reduce the engineering costs for parameter tuning. Practicability of the proposed method is verified by comparison with an autoencoder with SGD and LSE, an autoencoder with SGDM and MCC, an autoencoder with Adam and MCC, and an autoencoder with AdaGrad and MCC. The results are verified by the Friedman test, a post hoc test using the Wilcoxson signed-rank sum test with the Holm correction, and parameter sensitivity analysis.

Keywords: Decision Making, Internet of Things, Reinforcement Learning
Abstract: Edge computing offers cloud-like services closer to users and IoT devices, providing high speed and accessibility for network users. Edge computing, often called Mobile Edge Computing (MEC), is a distributed paradigm that utilizes heterogeneous computational and storage resources with well-provisioned capabilities rather than relying on the ample resources of the cloud. In addition, edge users usually refer to portable and mobile devices that connect to and disconnect from the network at will. Therefore, scheduling tasks at the appropriate time and allocating the right resources can be modeled as a multi-objective optimization problem in MEC. Moreover, each task has specific requirements, further adding to the complexity of the optimization problem. In this study, we formulate the scheduling problem as a Markov Decision Process (MDP) to schedule the tasks. The learning time of the task scheduler is minimized when it faces new users and edge servers. Subsequently, we employ the Q-learning (QL) algorithm from the Reinforcement Learning (RL) paradigm to address the optimization problem and effectively adapt the proposed scheduler to the dynamic nature of MEC. Accordingly, we designed the valid state space, action space, and reward function with appropriate conditions and proper rewards for the proposed QL-based technique. We conducted comprehensive experiments to validate the results of the proposed solution, taking into account the inherent randomness of the QL-based technique. The experimental results demonstrate that the proposed technique achieves the lowest learning time compared to Deep learning-based and Deep RL-based approaches. Furthermore, on average, the proposed technique obtains a 72% faster runtime compared to previous works, using 58% fewer computation cycles and 50% less memory. These improvements make the proposed approach an efficient and lightweight task scheduler for MEC.

Keywords: Federated Learning, Autonomous Systems, Cybersecurity
Abstract: Smart cities embrace unmanned autonomous vehicles (UxVs) for urban mobility and addressing challenges. UxVs include UAVs, UGVs, USVs, and UUVs, empowered by AI, particularly deep learning (DL), for autonomous missions. However, traditional DL has limitations in adapting to dynamic environments and raises data privacy concerns. Limited data availability and starting from scratch to adapt to a new environment during missions pose challenges. Additionally, cyber threats, particularly in terms of communication and data security, can jeopardize the missions performed by UxVs. This paper proposes a federated transfer learning scheme for UxVs, sharing prior knowledge and training with limited data while ensuring security through blockchain. Domain adaptation with maximum mean discrepancy enhances the DL model's performance in target domains. The proposed scheme's feasibility is demonstrated in an empirical environment, and it outperforms existing works.

Keywords: Ambient Intelligence, Image Processing, Internet of Things
Abstract: Moths pose a significant threat to agricultural crops, and identifying them accurately is crucial for effective pest monitoring and crop conservation efforts. However, manually evaluating glue traps is a time-consuming and labor-intensive process, which has led to the development of automated solutions. In this study, we present a deep learning-based automated detection pipeline that can detect moths in images captured by field traps with pheromone-emitting glue pads. To train our model, we collected a comprehensive dataset that includes moths from various environments, such as agricultural plants, homes, and food production facilities. We augmented this dataset and included additional glue pad datasets, enabling the model to detect moths regardless of the species. We base our model on the YOLOv5 algorithm and fine-tune it using transfer learning, which enables us to identify moths in real-time and on embedded hardware. Our evaluation of the algorithm reveals that it achieves an average precision of 98.2 % on a test dataset, which outperforms reference models from previous research. We also assess the model's ability to handle disturbances such as other insects, varying lighting conditions, and foreign objects. Importantly, our solution maintains a tiny memory footprint and low inference time of 2.3 ms, making it a highly efficient and effective tool for moth detection in the field.

Keywords: Deep Learning, Internet of Things, Image Processing
Abstract: JPEG image compression algorithm is a widely used technique for image size reduction in edge and cloud computing settings. However, applying such lossy compression on images processed by deep neural networks can lead to significant accuracy degradation. Inspired by the curriculum learning paradigm, we propose a training approach called curriculum pre-training (CPT) for crowd counting on compressed images, which alleviates the drop in accuracy resulting from lossy compression. We verify the effectiveness of our approach by extensive experiments on three crowd counting datasets, two crowd counting DNN models and various levels of compression. The proposed training method is not overly sensitive to hyper-parameters, and reduces the error, particularly for heavily compressed images, by up to 19.70%.

Keywords: Advanced Optimization, Internet of Things, Operations Research
Abstract: This paper proposes an optimal production scheduling method using a practical production simulator and integer form of population-based incremental learning (IF-PBIL) with two initial probability matrix setting methods. There are three parameters for decision variables in a target factory. It is necessary to optimize these three parameters at the same time in order to evaluate them. Moreover, IF-PBIL is one of the cooperative metaheuristics and determines integer values based on probability values and generates solutions. Initial integer values are determined by equal probability values, and various solutions are generated. Hence, there is a possibility that it is difficult to search high-quality solutions from initial stages of the search. Furthermore, since the production simulator requires long execution time, the execution number of the production simulator should be reduced as much as possible. In order to tackle the challenge, the proposed method applies two initial probability matrix setting methods. It is confirmed that the proposed method can search high-quality solutions from initial stages of the search and can reduce the production costs with the fewer execution number of the production simulator using actual factory data of a polishing process of an assembly processing factory.

Keywords: Deep Learning, Pattern Recognition, Image Processing
Abstract: Anomaly detection in video data plays a crucial role in numerous applications, such as industrial monitoring and automated surveillance. This paper presents a novel method for video anomaly detection (VAD) using Generative Adversarial Networks (GANs). The proposed method called VALT-GAN combines two separate branches, one for spatial information and the other for temporal information, to capture relevant features from video data. The framework is utilized to learn the normal features from the training video dataset, enabling the generator to produce realistic samples. However, existing GAN-based methods face challenges in detecting subtle or unseen anomalies. To address this, we introduce latent mining for adversarial training which allows us to train a robust GAN model with high anomaly detection (AD) capability. We exploit the latent space following the continuous nature of the generator using the Iterative Fast Gradient Signed Method (IFGSM) which improves the quality of the generated images. Experimental evaluations show the effectiveness of VALT-GAN as compared to traditional methods on UCSD (University of California, San Diego) Peds2, CUHK (Chinese University of Hong Kong) Avenue, and ShanghaiTech datasets.

Keywords: Image Processing, Particle Swarm Optimization, Swarm Intelligence
Abstract: Copy-move forgery is one of the most used ma- nipulations for tampering with digital images. The authenticity of the image becomes more crucial when the images are used in important processes. keypoints-based algorithms have been reported to be very effective in revealing copy-move evidence due to their robustness against various attacks. However, these approaches sometimes fail to make good prediction because of different factors such small number of keypoints detected, or wrongly detected keypoints. Matching the correct keypoints and filtering the wrong keypoints are other difficult tasks. One reason behind these issues is the parameters used to configure the key point detection algorithm. In this paper, another CMF (copy-move forgery) detection algorithm is proposed, by applying particle swarm optimization to find the best parameters for the algorithm for all different phases. Furthermore, filtering is achieved through two stages to remove most of the wrong keypoints detected. Additionally, triangulation is used as another technique applied to the algorithm in order to increase the detection area. Experimental results shows that the algorithm has good performance.

Keywords: Deep Learning, Data Mining, Remote Sensing
Abstract: Sound-based insect wingbeat classification presents a unique challenge with implications for areas such as mosquito control and the prevention of mosquito-borne diseases. This paper introduces a straightforward modified ResNet-9 model to address this challenge by utilizing one-dimensional convolutional layers. The architecture of the proposed ResNet-9 model is outlined in detail. Impressively, the model can accurately classify fruitflies and mosquitoes using raw audio data instead of relying on spectrograms. Its performance surpasses the majority of preceding models while concurrently reducing the number of trainable parameters by 90%. The results from this research carry notable significance for practical applications in insect control and disease prevention.

Keywords: Image Processing, Pattern Recognition, Signal Processing
Abstract: Augmented reality is a growing technology with potential applications in education, medicine, entertainment, and tourism, among others. Basically, what this technology seeks is to combine information from the real world with virtual information, without the user perceiving the difference between the two. To achieve this, the augmented reality system must be able to dimension the real-world objects in real time, to generate realistic virtual scenarios. To carry out this dimensioning, a good alternative is to use an artificial vision system that provides a good compromise between cost and performance. In this work a method is presented to calculate the distances among known reference objects in real world and the camera, using a monocular artificial vision system.

Keywords: Deep Learning, Autonomous Systems, Big Data
Abstract: Object detection plays a crucial role in automated image analysis by identifying and localizing objects within an image. One-stage Deep Learning (DL)-based object detectors have achieved impressive results, primarily due to large-scale datasets available for training them. However, these approaches rely heavily on abundant labeled data, posing challenges when only a few samples per class are available. To this end, few-shot object detection approaches have been proposed. Among them, fine-tuning the final detection head while keeping the feature extractor/backbone frozen is a commonly used approach for few-shot object detection. This approach effectively utilizes pre-existing knowledge encoded in the backbone, using a small number of samples to learn new object categories. However, in this paper, we argue that fine-tuning only the last layers may limit accuracy and lead to overfitting if the initial layers of the detection head are not adapted for the new task. The data processing inequality, which states that information lost in early network layers cannot be recovered in subsequent ones, supports this argument. To address this issue, we propose a symmetric fine-tuning method that involves both the first and last layers of the detection head, aiming to maintain a fixed trainable parameter budget while strategically selecting parameters for fine-tuning. Experimental results demonstrate the effectiveness and efficiency of this approach and open up several interesting future research directions.

Keywords: Big Data, Data Mining, Transportation and Vehicle Systems
Abstract: According to the data from the Bureau of Transportation Statistics (BTS), the number of passengers and flights has been increasing year by year. However, flight delay has become a pervasive problem in the United States in recent years due to various factors, including human factors such as security regulations, as well as natural factors such as bad weather. Flight delay not only affects the profits of airlines but also affects the satisfaction of passengers. Therefore, a model that can predict the arrival time of airplanes needs to be developed. Machine learning methods have been widely applied to prediction problems. In this paper, a variety of machine learning and computational intelligence methods, including linear regression, decision tree (DT), random forest (RF), gradient boosting (GB), gaussian regression models and genetic programming were trained on the U.S. Department of Transportation's (DOT) BTS dataset. The results show that genetic programming performs best and can be used to predict the arrival time of the U.S. flights in advance, which is beneficial for airlines and passengers to make timely decisions.