Keywords: Application of Artificial Intelligence, Intelligent Internet Systems, Neural Networks and their Applications
Abstract: Accurately identifying network intrusion cannot only help individuals and enterprises better deal with network security problems but also maintain the Internet environment. Currently, classification methods with autoencoders for feature learning have been proved to be suitable for the network intrusion detection. This work proposes a new hybrid classification method named SABD for network intrusion detection. SABD integrates Stacked sparse contractive autoencoders, Attention-based Bidirectional long-term and short-term memory (LSTM), and Decision fusion. SABD integrates the feature extraction of stacked sparse contractive autoencoders with the classification ability of attention-based bidirectional LSTM. Specifically, stacked sparse contractive autoencoders are used for extracting features, which are sent to the attention-based bidirectional LSTM for the classification. Finally, the decision fusion algorithm is adopted to integrate classification results of multiple classifiers and yield the final results. Experimental results based on real-life UNSWNB15 data demonstrate that the proposed SABD outperforms its state-of-the-art peers in terms of classification accuracy.

Keywords: Machine Learning, Application of Artificial Intelligence, Neural Networks and their Applications
Abstract: Throughout the years, many attempts have been made at creating music procedurally. However, very few of those attempts were concerned with the actual meaning of the music that was generated. The goal of this research is to implement a pipelined model of neural networks capable of generating homophonic music without input. The generated music should be meaningful, that is to say, it should sound like it has a purpose. This is similar to how a human composer would write music. This idea of meaning makes a composition tell a story or express feelings. This is the reason humans write music, as well as create other forms of arts. As the goal of artificial creativity is to approach human creativity as close as possible, Artificial Intelligence should try to imitate humans as close as possible. Therefore, it is important for a music generating AI to understand meaning in music. In order to introduce meaning in AI composition, the process is approached step-by-step. Dedicated neural models trained on melodies that express purpose through the use of motifs are first used to generate a melody. That melody serves as input to another neural model, trained on chord progressions that contextualize the melodies they accompany, to generate harmony. Besides the model being symbolic, there are no musical constraints or guidelines for generation. By using this pipelined but unconstrained approach, it is possible to procedurally generate music that sounds as if composed by a human being.

Keywords: Application of Artificial Intelligence, Neural Networks and their Applications
Abstract: With increasingly more sophisticated phishing campaigns in recent years, phishing emails lure people using more legitimate-looking personal contexts. To tackle this problem, instead of traditional heuristics-based algorithms, more adaptive detection systems such as natural language processing (NLP)-powered approaches are essential to understanding phishing text representations. Nevertheless, concerns surrounding the collection of phishing data that might cover confidential information hinder the effectiveness of model learning. We propose a decentralized phishing email detection framework called Federated Phish Bowl (FedPB) which facilitates collaborative phishing detection with privacy. In particular, we devise a knowledge-sharing mechanism with federated learning (FL). Using long short-term memory (LSTM) for phishing detection, the framework adapts by sharing a global word embedding matrix across the clients, with each client running its local model with Non-IID data. We collected the most recent phishing samples to study the effectiveness of the proposed method using different client numbers and data distributions. The results show that FedPB can attain a competitive performance with a centralized phishing detector, with generality to various cases of FL retaining a prediction accuracy of 83%.

Keywords: Application of Artificial Intelligence, Neural Networks and their Applications, Deep Learning
Abstract: Predicting the Arctic sea ice concentration,(SIC) has an essential guiding role in understanding climate change trends, resource extraction and route planning. Existing deep learning models still have the problem that it is challenging to utilize the global spatial information of SIC, and the predictions of boundary regions are not accurate enough. In this paper, we propose a new deep learning model, namely ConvLSTM-CRF, to predict the monthly sea ice concentration in the Arctic. We add a dense conditional random fields to the ConvLSTM, which further extracts global spatial information and can predict SIC more accurately. The experimental results show that compared with ConvLSTM, our model has a great improvement in the overall prediction accuracy and has more accurate predictions in the SIC boundary region, especially in the melting and freezing seasons when the SIC changes drastically. Our model also shows better prediction performance when making iterative predictions. In addition, ConvLSTM-CRF can be applied to similar time series forecasting problems, such as precipitation forecasting and snowfall forecasting.

Keywords: Application of Artificial Intelligence, Deep Learning, Media Computing
Abstract: Driving simulators are utilized for many applications including basic driver training, human factor studies, human-machine interaction, and vehicle prototyping in automobile industries. The main purpose of using driving simulator is to provide realistic driving experience. Since simulator platforms have physical limitations, Motion Cueing Algorithms (MCAs) are used to generate driving sensation for the simulator user while considering the simulator's physical and dynamical constraints. When using a model predictive control (MPC)-based MCA, the principle of MPC is leveraged to predict an optimized future behavior of the simulator where a series of control actions is developed across a defined future horizon using the explicitly specified process model. Corresponding to the pre-positioning or time-varying reference MPC, it is crucial to predict the future vehicle motion signals for the simulator accurately. The existing methods for predicting vehicle motion signals do not excel in predicting time-series of a long sequence due to the missing feedback loop or limited memory size. To address this issue, the Long Short-Term Memory (LSTM) model is developed to predict motion signals using Python. The performance of LSTM is compared with those from different traditional methods using several measurements criteria, which include the root mean squared error (RMSE), mean absolute error (MAE), and Pearson’s correlation coefficient (r). The results indicate that LSTM outperforms RNN by producing more accurate motion allowing the MCA to deliver realistic motion sensations, the LSTM model can be employed in a wide range of applications including autonomous vehicles trajectory prediction, and other prediction problems.

Keywords: Assistive Technology, Human Factors, Human-Machine Interface
Abstract: Assistive robots can be found in hospitals and rehabilitation clinics, where they help patients maintain a positive disposition. Our proposed robotic mobility solution combines state of the art hardware and software to provide a safer, more independent, and more productive lifestyle for people with some of the most severe disabilities. New hardware includes, a retractable roof, manipulator arm, a hard backpack, a number of sensors that collect environmental data and processors that generate 3D maps for a hands-free human-machine interface.

The proposed new system receives input from the user via head tracking or voice command, and displays information through augmented reality into the user's field of view. The software algorithm will use a novel cycle of self-learning artificial intelligence that achieves autonomous navigation while avoiding collisions with stationary and dynamic objects. The prototype will be assembled and tested over the next three years and a publicly available version could be ready two years thereafter.

Keywords: Assistive Technology, Human Factors, Human-Machine Interface
Abstract: The direct connection of the jaw to the brain allows it to retain its motor and sensory capabilities even after severe spinal cord injuries. As such, it can be an accessible means of providing inputs for people with paralysis to manipulate their environment. This paper explores the potential for using the jaw, specifically the mandible, as an alternative input to human-machine interface systems. Two tests were developed to test the mandible's ability to respond to visual stimuli. First, a visual response time test to measure the precision and accuracy of user input through a mandible-actuated button. Second, a choice response test to observe coordination between the mandible and a finger.

Study results show that the mean response time of mandible inputs is 8.3% slower than the corresponding mean response time of performing the same task with a thumb. The delay in response after making a decision is statistically insignificant between the mandible- and finger-actuated inputs with the mandible being 2.67% slower.

Based on these results, the increase in response time while using the mandibular input is minimal for new users. Coordination is feasible in tasks involving both the mandible and thumb. Extensive training with a made-to-fit device has the potential to enable a visual response time equivalent to the fingers in more complex tasks. The mandible is a viable option for accessible HMI for discreet inputs. Further testing into continuous input is needed to explore the mandible's potential as an input for body augments.

Keywords: Assistive Technology, Human-Computer Interaction, Human-Machine Interface
Abstract: Assistive navigation for blind or visually impaired (BVI) individuals is of significance to extend their mobility and safety in traveling, enhancing their employment opportunities and fostering personal fulfillment. Conventional research is mainly based on robotic navigation approaches through localization, mapping, and path planning frameworks. They require heavy manual annotation of semantic information in maps and its alignment with sensor mapping. Inspired by the fact that we human beings naturally rely on language instruction inquiry and visual scene understanding to navigate in an unfamiliar environment, this paper proposes a novel vision-language model-based approach for BVI navigation. It does not need heavy-labeled indoor maps and provides a Safe and Efficient E-Wayfinding (SeeWay) assistive solution for BVI individuals. The system consists of a scene-graph map construction module, a navigation path generation module for global path inference by vision-language navigation (VLN), and a navigation with obstacle avoidance module for real-time local navigation. The SeeWay system was deployed on portable iPhone devices with cloud computing assistance for the VLN model inference. The field tests show the effectiveness of the VLN global path finding and local path re-planning. Experiments and quantitative results reveal that heuristic-style instruction outperforms direction/detailed-style instructions for VLN success rate (SR), and the SR decreases as the navigation length increases.

Keywords: Assistive Technology, Human-Computer Interaction, Human-Machine Interface
Abstract: Upper-limb exoskeletons (ULEs) have the potential to assist severely paralyzed individuals. Many rehabilitative upper-limb exoskeletons have been developed in the recent years, where the prominent issue of bulkiness and actuator placement has been effectively resolved by the cable-transmission systems, allowing the actuators to be mounted away from the joints. This results in increasing the compactness and wearability of the exoskeleton. Thus, this article summarizes twenty-five cable-driven upper-limb exoskeletons designed for rehabilitating disabled individuals. This paper also presents a comparison based on usability of ULEs by end users considering the design characteristics and user-based experiments for real-time applications of the arm exoskeletons.

Keywords: Assistive Technology, Human-Machine Interface
Abstract: This work presents an adaptive prosthetic training gripper with a compact, variable stiffness differential mechanism and a vision-based shared control scheme that relies on a Lightmyography (LMG) interface to trigger the selected grasps. The gripper incorporates three monolithic adaptive fingers manufactured using the concept of Hybrid Deposition Manufacturing (HDM) and includes a gear drive system that allows two fingers bases to rotate, implementing abduction / adduction and thereby increasing the available grasping workspace. The fingers are actuated through a compact, series- elastic differential mechanism that reduces the total number of required actuators to two. The developed gripper is operated using a vision-based myoelectric control framework that utilizes an RGB camera and a Convolutional Neural Network (CNN) for object detection and classification as well as for grasp selection and an LMG muscle machine interface for grasp triggering. The efficiency of the proposed gripper and the control framework have been experimentally validated through a series of complex grasping experiments executed using a plethora of everyday life objects.

Keywords: Robotic Systems
Abstract: Autonomous robot systems are often designed to achieve specific goals. This paper restricts attention to a specific type of goal, namely reaching a desired state within a certain time bound. For such goals, a robot system's competency and resilience can be defined as the probability of reaching the desired state as a function of the time bound under a nominal unperturbed condition and under known perturbation conditions, respectively. Two metrics taken from prior work for measuring competency and resilience, power and efficiency, are modified so that they do not require subjective parameters. This paper formalizes the adapted metrics for discrete time Markov chains. The adapted metrics are applied to a best-of-N case study that is solved by a graph-based approach and modeled as a discrete time Markov chain. The case study demonstrates that the modified metrics allow power-efficiency trade-offs to be more easily visualized than the cluttered visualizations produced by the original metrics.

Keywords: Robotic Systems, System Modeling and Control, Modeling of Autonomous Systems
Abstract: We propose a fast optimization algorithm to find feasible solutions for a special class of switching structured problems. A type of continuous optimization problem that has switching structured constraints is called mathematical programs with switching constraints (MPSC). Relaxation methods are well known gradient descent-based approaches for solving MPSC. However, with the use of these conventional algorithms, we reveal that the sequence of solutions easily converges to an infeasible stationary point due to a special class of switching constraint whose feasible set is geometrically separated into mutually exclusive sets in variable space. We define this kind of switching constraint as the disjunctive allowable set constraint (DAS-constraint). To force the sequence of solutions to escape from such infeasible stationary points, we construct a new algorithm that introduces random sampling if convergence to an infeasible point is detected. Furthermore, to reduce the computation cost due to random sampling, we randomize only specific variables that are relevant to DAS-constraints. Numerical experiments show that feasible solutions can be found by using our proposed algorithm, even for large-scale optimization problems where no solutions are found within a practical time limit when using a conventional algorithm.

Keywords: Robotic Systems, System Modeling and Control, Mechatronics
Abstract: A Motion Cueing Algorithm (MCA) is an algorithm that transforms the movement of a simulated vehicle into movement that can be reproduced with a Motion Simulator (MS) while respecting its physical constraints. Crucially, MCAs aim to provide a realistic driving experience to simulator users. Adaptive MCAs are a type of MCA that is flexible, computationally light and designed to adjust behaviour based on the current MS state. However, adaptive MCAs require extensive manual tuning which is difficult, time consuming and a sub-optimal process. This paper presents an optimisation-based method using Particle Swarm Optimisation (PSO) for automatically tuning the free parameters of the Adaptive Gradient Descent -based MCA (AGDA) while accounting for MS physical constraints and motion fidelity. The cost function of the tuning routine considers the RMSE, correlation coefficient (CC) and error oscillation of the motion sensation signals of the MS driver with respect to those of the simulated vehicle driver. The displacement, velocity and acceleration of the MS are also considered. The proposed method was implemented using MATLAB and Simulink and the effectiveness of the approach was tested with a Rigs of Rods simulation of a ground vehicle. Compared to the existing manually tuned AGDA, the optimally tuned AGDA obtained with the proposed method performs 32.6% and 23.7% better in terms of RMSE and CC of the motion sensation signals, respectively. The observed performance improvement and moderate computational load of the AGDA renew its relevance in the context of modern MCAs.

Keywords: Robotic Systems, System Modeling and Control, Consumer and Industrial Applications
Abstract: This paper exploits the use of Ultra Wide Band (UWB) technology to improve the localization of robots in both indoor and outdoor environments. In order to efficiently integrate the UWB technology in existing multi-sensor architectures, such as Kalman-based, we propose two approaches to estimate the UWB position covariance values. The first approach uses statistical methods to estimate static covariance values based on data acquired a priori. The second approach adopts a neural network (NN) to capture the relationship between the positional error of the UWB data and the signal quality information, such as the Estimate Of Precision (EOP) and Received Signal Strength Indicator (RSSI). The GPS-RTK is used as ground truth and RGB-D odometry is adopted for both benchmarking and integration purposes. Position sources are fused by means of an Extended Kalman Filter (EKF). Real world experiments are conducted with a tracked mobile robot driving outdoors in a closed-loop trajectory. Results show that the NN is able to efficiently model the sensor covariances and adapt the trustworthiness of the EKF estimation, overcoming data loss by relying on the other available estimation source.

Keywords: Robotic Systems, Intelligent Transportation Systems, System Modeling and Control
Abstract: A Nonlinear Model Predictive Control (NMPC) strategy aimed at controlling a small-scale car model for autonomous racing competitions is presented in this paper. The proposed control strategy is concerned with minimizing the lap time while keeping the vehicle within track boundaries. The optimization problem considers both the vehicle’s actuation limits and the lateral and longitudinal forces acting on the car modeled through the Pacejka’s magic formula and a simple drivetrain model. Furthermore, the approach allows to safely race on a track populated by static obstacles generating collision-free trajectories and tracking them while enhancing the lap timing performance. Gazebo simulations using the F1/10 simulator showcase the feasibility and validity of the proposed control strategy. The code is released as open-source making it possible to replicate the obtained results.

Keywords: Machine Learning, Biometric Systems and Bioinformatics, Computational Life Science
Abstract: The Hierarchical Redundancy Eliminated Tree Augmented Naive Bayes (HRE-TAN) classifier is a semi-naive Bayesian model that learns a type of hierarchical redundancy-free tree-like feature representation to estimate the data distribution. In this work, we propose two new types of positive feature values prioritized hierarchical redundancy eliminated tree augmented naive Bayes classifiers that focus on features bearing positive instance values. The two newly proposed methods are applied to 28 real-world bioinformatics datasets showing better predictive performance than the conventional HRE-TAN classifier.

Keywords: Machine Learning, Information Assurance and Intelligence, Application of Artificial Intelligence
Abstract: Error-Correcting Output Coding (ECOC) is a method for constructing a multi-valued classifier using a combination of binary classifiers. The effectiveness of ECOC for multi-valued classification problems has been demonstrated by many experimental evaluations. Therefore, classification performance have strongly depended on the data under consideration, and it is not clear what kind of combinations of binary classifiers have good performance. Motivated by this fact, the authors have clarified the best combination of binary classifiers that makes ECOC, assuming a situation in which each binary classifier can estimate the true posterior probability. They also have proposed a total framework for analytical evaluation when a binary classifier outputs an estimated posterior probability that approximates the true posterior probability. These studies established a framework for evaluating the theoretical performance of ECOC.

Based on these findings, this study discusses the theoretical performance of ECOC from the upper bound perspective. The results showed that increasing the Hamming distance between code words can blackuce the error rate. We then evaluate various combinations of binary classifiers based on analytical evaluation.

Keywords: Transfer Learning, Deep Learning, Machine Learning
Abstract: Continual learning requires the ability to reliably transfer previously learned knowledge to new tasks without disrupting established competencies. Methods such as Deepmind's progressive neural network accomplish high-quality transfer learning while nullifying the insidious problem of catastrophic forgetting. However, most module-based continual learning systems require task labels during operation -- a constraint that limits their application in many real-world conditions where task indicators are opaque. This paper proposes a task detector neural algorithm to acquire task information while maintaining immunity to forgetting. Our proposed task detector allows progressive neural networks (and many similar systems) to operate without task labels during test-time. Our task detector is built from familiarity autoencoders which recognize the nature of the required task from input data. We demonstrate the generality and effectiveness of this approach through experiments in video game playing and automated image repair. Our results show near-perfect task recognition in all domains (>99% F1), rewards above published single-task scores in MinAtar, and realistic image repairs on damaged human face pictures. The performance of our integrated approach is nearly identical to the progressive systems equipped with ground-truth task labels. Code is available at: https://github.com/arcosin/Task_Detector.

Keywords: Machine Vision, Deep Learning, Image Processing and Pattern Recognition
Abstract: Vehicle detection is a challenging task in computer vision. Since the differences among different vehicle shapes are small and not easy to recognize, the common object detection models usually do not work well for vehicle recognition. To select positive samples, the existing anchor-based detection methods usually use high-quality anchor boxes which are very close to the shapes of the vehicles. Such methods often make the prediction strongly dependent on high-quality anchor boxes. At present, the ATSS (Adaptive Training Sample Selection) method has also been used to select high-quality anchor boxes. The main purpose of this method is to automatically set the shapes of the anchor boxes according to the training datasets. However, for vehicle detection, the shapes of the vehicles are usually similar while the scales of the vehicles are quite different, which may influence the performance of the ATSS based vehicle detection. To solve the above problems, we propose a vehicle detection model PSLTNet (Positives Selection with Low Threshold Network). To reduce the prediction instability due to relying on high-quality anchor boxes, PSLTNet applies the positive samples selection with low threshold to enhance the prediction ability for different quality anchor boxes. To improve the identification ability of different vehicle types, PSLTNet adopts a pyramid structure and cascades dilated convolutions at each level. In addition, in order to make the bounding box regression loss be independent of scales, PSLTNet uses GIOU (Generalized Intersection Over Union) to supervise the position regression task of candidate bounding boxes. Finally, to reduce the loss of a large number of simple negatives, PSLTNet uses Focal Loss to supervise the classification task of the background candidate bounding boxes. The Experimental results upon the BIT-Vehicle dataset show that the proposed method can obtain better performance than that of the existing methods.

Keywords: Application of Artificial Intelligence, Computational Intelligence, Evolutionary Computation
Abstract: Protecting the high voltage transmission lines has been one of the most significant problems in the power systems. Precise and timely detection, identification, and location estimation of line-to-ground, line-to-line, line-to-line-to-ground, and line-to-line-to-line faults can considerably enhance the speed of a recovery process of transmission lines and hence reduce the costs associated with the downtime of a power system. Consequently, having a robust, affordable, and accurate fault diagnosis system is crucial to perform these tasks within an acceptable time window after a fault occurs in the presence of system uncertainties. Mistakenly detected or undetected faults can be expensive in the conventional techniques and this fact has motivated us to present a robust detection, identification, and location estimation system by using a machine learning method called generalized regression neural networks. The robustness of this technique is tested with respect to the variations of fault resistance, phase difference between two connected buses, fault inception angle, local bus voltage fluctuations, source inductance fluctuations, and measurement noise. Besides, the effect of noise on the GRNN method is revealed in this paper. Its comparison with the existing state-of-the-art methods shows its outstanding performance in the accurate fault classification and location estimation for transmission lines.

Keywords: Computational Intelligence, Intelligent Internet Systems, Application of Artificial Intelligence
Abstract: Advances in the frontier of intelligence and system sciences have triggered the emergence of Autonomous AI (AAI) systems. AAI is cognitive intelligent systems that enable non-programmed and non-pretrained inferential intelligence for autonomous intelligence generation by machines. Basic research challenges to AAI are rooted in their transdisciplinary nature and trustworthiness among interactions of human and machine intelligence in a coherent framework. This work presents a theory and a methodology for AAI trustworthiness and its quantitative measurement in real-time context based on basic research in autonomous systems and symbiotic human-robot coordination. Experimental results have demonstrated the novelty of the methodology and effectiveness of real-time applications in hybrid intelligence systems involving humans, robots, and their interactions in distributed, adaptive, and cognitive AI systems.

Keywords: Cloud, IoT, and Robotics Integration, Evolutionary Computation, Swarm Intelligence
Abstract: Unmanned aerial vehicle (UAV) communication is a promising technology in 5G/6G wireless communications. However, there are several challenges for ensuring secure communications in practical scenarios. In this paper, we consider a UAV-enabled communication scenario that a UAV needs to maintain secure communication with the ground communication nodes (GCNs), subject to the known ground eavesdropping nodes (GENs). UAV needs to select optimal communication positions and avoid obstacles. We formulate a UAV secrecy scheduling optimization problem (USSOP) to maximize the average secrecy rate and the minimum secrecy rate jointly. Then, we propose a particle swarm optimization with normal distribution initialization, differential mechanism and avoiding obstacles operator (PSONDA) to solve the USSOP. Simulation results show that this method performs better than other comparison algorithms.

Keywords: Cloud, IoT, and Robotics Integration, Evolutionary Computation, Swarm Intelligence
Abstract: Automated guided vehicle (AGV) has the advantages of high endurance, autonomy and security, which render them appealing for applications such as monitoring and sensing networks.However, AGVs may be limited in energy and transmission range. Unmanned Aerial Vehicle (UAV) is a promising platform that can assist terrestrial networks. In this work, we aim to adopt a UAV swarm to assist the data forwarding of AGVs and propose a novel data transmission framework based on collaborative beamforming (CB) where the AGVs and UAVs jointly construct a virtual antenna array (VAA) to transmit data to the remote air base stations (ABSs). Specifically, we formulate a terrestrial and air collaboratively data transmission multi objective optimization problem (TATFMOP) to optimize the excitation current weights and locations of the AGVs and UAVs, and the communication order of different remote ABSs. Since TATFMOP is an NP-hard problem, we present an extended multi-objective ant-lion optimization (EMOALOIB) with oppositionbased population initialization and black hole update operators to solve the problem. Simulations results demonstrate that the proposed EMOALOIB outperforms other existing benchmark algorithms and can obtain more valuable solutions.

Keywords: Cloud, IoT, and Robotics Integration, Evolutionary Computation, Swarm Intelligence
Abstract: Automated guided vehicle (AGV) communications and networks have attracted extensive attention and have broad prospects in the field of wireless transmission. Unmanned aerial vehicle (UAV) can be used as flight base station (BS) to receive information from the ground AGVs. In this paper, we study an AGV-UAV communication scenario, in which a group of AGVs form an AGV-based virtual antenna array (AVAA) and communicate with different UAVs based on distributed collaborative beamforming (DCB). We formulate an AGV-UAV communication multi-objective optimization problem (AUCMOP) to simultaneously maximize the total transmission rate, minimize the total repositioning time of AGVs and minimize the total motion energy consumptions of AGVs by optimizing the positions, excitation current weights and moving speeds of AGVs, as well as the sequence for communicating with different UAVs. The formulated AUCMOP is complicated, and thus we propose an improved multi-objective ant lion optimization algorithm with Chebyshev chaos-opposition based learning solution initialization and hybrid solution update method (IMOALOCH). Simulation results show that the proposed IMOALOCH can solve the formulated AUCMOP effectively and it is superior to other comparison algorithms.

Keywords: Application of Artificial Intelligence, Machine Learning
Abstract: Crime occurrence prediction is an important task for public security and urban governance. In real world, some crimes are quite harmful, thus it’s necessary to predict these crimes in the next few days (weeks). However, these harmful crime events may have relatively lower frequency, and their records are discontinuous and sporadic, making the multi-step occurrence prediction difficult. Due to the sparsity and irregularity of the crime sequences, most of the existing crime prediction techniques do not distinguish crime types and have difficulties to predict low-frequency crimes’ occurrences. In addition, they cannot accurately predict the occurrence of crimes over multiple recent time slots due to sporadic crime records. In this paper, we propose Multi-Time-Slot Crime Occurrence Prediction (MCOP) model , which aims to (1) predict crime types separately and (2) predict crime occurrences in the next multiple time slots. Specifically, MCOP uses a GRU-ODE-Bayes model to handle the discontinuous and sporadic crime events sequences. MCOP is enhanced by a data augmentation to address the data sparsity problem, and exploits the near repeat phenomenon to enable continuous crime predictions. We evaluated our model using real-world urban data, and the results showed our model outperforms the baseline techniques.

Keywords: Evolutionary Computation, Machine Learning, Expert and Knowledge-Based Systems
Abstract: Machine learning applications related to high-stakes decisions are often surrounded by significant amounts of controversy. This has led to increasing interest in interpretable machine learning models. A well-known class of interpretable models is that of decision trees (DTs), which mirror a common strategy used by humans to arrive at solutions through a series of well-defined decisions. However, much of previous research on DTs for criminal justice predictions has focused primarily on collections (ensembles) of DTs whose results are aggregated together. Such DT ensembles are used to help improve accuracy; however, their increased complexity and deviation from human decision-making processes makes them much less interpretable compared to single-DT approaches. In this paper, we present a new DT model for criminal recidivism prediction that is designed with high interpretability, accuracy, and fairness as core objectives. The interpretability of the model stems from its formulation in terms of a single DT structure, while accuracy is achieved through an intensive optimization process of DT parameters that is carried out using a novel evolutionary algorithm. Through extensive experiments, we analyze the performance of our proposed EADTC (Evolutionary Algorithm Decision Tree for Crime prediction) method on relevant datasets. Our experiments show that the EADTC approach achieves competitive accuracy and fairness with respect to state-of-the-art ensemble DT models, while achieving higher interpretability due to the simpler, single-DT structure.

Keywords: Application of Artificial Intelligence, Deep Learning, Machine Learning
Abstract: Water quality prediction refers to the prediction of future water quality changes based on past data. Traditional prediction models cannot capture intricate and nonlinear features. Typical machine learning methods extract nonlinear characteristics, but they suffer from overfitting problems due to data noise. Most current deep learning models have problems of gradient disappearance and explosion, and often fail to capture long-term dependence. To solve the above-mentioned problems, this work proposes a multi-indicator time series prediction method named SG-Informer for river water quality prediction. SG-Informer integrates the Savitsky-Golay filter, the ProbSparse self-attention mechanism of an encoder, and a generative style decoder, serving as data smoothing and noise elimination, network scale reduction, and prediction speed improvement, respectively. SG-Informer establishes a high-quality water quality time prediction model, which effectively predicts the future water quality time series trend. Based on real-life data sets of water quality, multi-indicator and single-indicator prediction experiments are performed. Experimental results demonstrate that the proposed SG-Informer outperforms several state-of-the-art prediction methods in terms of prediction accuracy.

Keywords: Application of Artificial Intelligence, Knowledge Acquisition, Image Processing and Pattern Recognition
Abstract: Predicting human trajectory is one of the key knowledge required for autonomous driving and social robots in real scenarios. Recent studies based on Transformer networks have shown a great ability to model social behaviors. As far as we know, global trajectory information has an essential influence on prediction at a certain step. However, these methods only rely on the previous trajectory states/attention but ignore the important following states/attention of the trajectory for each pedestrian, which will generally collapse on some irregular movements (e.g.acceleration, deceleration, and motionless). To solve this issue, we propose a Global Temporal Attention Optimization model (GTAO), which activates the utilization of the following states/attention of the trajectory, and jointly and iteratively optimizes the preliminary trajectory prediction through a global temporal attention (GTA) module. To effectively address the decline in the generalizability and abnormal processing of the model, we further introduce global temporal guidance (GTG) module to instruct the GTA to learn the features closer to realistic trajectories. Experimental results on commonly used real-world human trajectory prediction datasets (ETH and UCY) indicate that our GTAO can achieve better performance in terms of prediction accuracy.

Keywords: Application of Artificial Intelligence, Machine Learning
Abstract: The early identification of the patients’ hospitalization at triage level within the Emergency Department (ED) presents a potential solution to reduce the risk of overcrowding and improve the quality of care. Thus, predicting patient out-come on arrival assists medical staff in the make of the appropriate patient pathway decision and so reduces the risk of medical error and complication of the patient's condition. Previous works don’t consider the uncertainty of medical data while the management of this uncertainty is one of the most important and crucial tasks of medical information systems. Thus, we present in this paper an improved version of the classical prediction model by taking into account the uncertainty and by managing properly the missing information. In this context, we propose a new approach based on Dempster-Shafer theory and Dynamic Multi-Layer Perceptron algorithm. Our proposed approach deploys the correspondent neural network as follows: 1) computes for each input parameter the Basic Belief Assignment (BBA) that provides an assessment of the uncertainty pattern using the Dempster’s rule; 2) deduces the correspondent weights based on the computed BBA, and 3) uses an appropriate transfer function to activate the next layer neurons. In this paper, we demonstrate the effectiveness of our proposed method by using a real ED database. We prove that our proposed approach manages efficiently the uncertainty of the medical data sources and missing information, so improves the decision making and reduces errors and complexity.

Keywords: Application of Artificial Intelligence, Deep Learning, Machine Learning
Abstract: Human Activity Recognition (HAR) has emerged as a potential research topic for smart healthcare owing to the fast growth of wearable and smart devices in recent years. The significant applications of HAR in ambient assisted living environments include monitoring the daily activities of elderly and cognitively impaired individuals to assist them by observing their health status. In this research, we present a deep learning-based fusion approach for multimodal HAR that fuses the different modalities of data to obtain robust outcomes. Here, Convolutional Neural Networks (CNNs) retrieve the high-level attributes from the image data, and the Convolutional Long Short Term Memory (ConvLSTM) is utilized to capture significant patterns from the multi-sensory data. Finally, the extracted features from the modalities are fused through self-attention mechanisms that enhance the relevant activity data and inhibit the superfluous and possibly confusing information by measuring their compatibility. Lastly, extensive tests have been performed to measure the efficiency and robustness of the developed fusion approach using the UP-Fall detection dataset. It is evident from the experimental findings that the proposed fusion technique outperforms the existing state-of-the-art and achieves relatively better performance.

Keywords: Image Processing and Pattern Recognition, Machine Vision, Application of Artificial Intelligence
Abstract: Pedestrian Attribute Recognition (PAR) is an indispensable topic in smart video analysis. The recognition of fine-grained attributes is challenging work as they are indistinguishable in surveillance images. In this study, we propose an Attention Auxiliary Spatial Fusion (AASF) model to improve the performance of PAR from the following two aspects: (1) We employ an Embedded Attention (EA) module to encode position information into channel information so that it can aggregate features in two different spatial directions to enhance the small-scale visual clues. (2) We propose a Feature Pyramid Adaptive Fusion (FPAF) module to adaptively select useful features for multiple attributes from different levels with contradictory information. Extensive experiments conducted on two large public indoor and outdoor PAR datasets demonstrate that our model achieves state-of-the-art results, especially obtaining a better performance on fine-grained attributes.

Keywords: Image Processing and Pattern Recognition, Machine Vision, Deep Learning
Abstract: Facial Expression Recognition (FER) in the wild is a significant yet challenging topic in computer vision due to the feature inconsistency caused by the individual specificity of facial expressions. In addition to variations of facial expressions caused by identity, pose, and occlusion, gender also affects the face of human emotions. Even though males, females, and infants share the same facial expressions, their characteristics are vastly different. To capture the effect of gender on facial expressions, we propose a novel multi-task FER method with joint gender learning. First, in addition to the original emotion labels of face images, we annotate gender labels, including male, female, and infant. Second, we introduce a gender-aware multi-task convolutional neural network for FER, which can learn the emotion and gender features of faces. Compared with single-task expression recognition methods, our proposed framework for introducing gender feature learning can significantly achieve higher performance on FER in the wild. Finally, we verify the effectiveness of our framework on two public wild FER datasets, RAF-DB and FER2013. And the results show that the gender learning auxiliary task is beneficial to the improvement of the performance of FER.

Keywords: Machine Vision, Deep Learning
Abstract: This paper describes a method of online refinement of a scene recognition model for robot navigation considering traversable plants, flexible plant parts which a robot can push aside while moving. In scene recognition systems that consider traversable plants growing out to the paths, misclassification may lead the robot to getting stuck due to the traversable plants recognized as obstacles. Yet, misclassification is inevitable in any estimation methods. In this work, we propose a framework that allows for refining a semantic segmentation model on the fly during the robot's operation. We introduce a few-shot segmentation based on weight imprinting for online model refinement without fine-tuning. Training data are collected via observation of a human's interaction with the plant parts. We propose novel robust weight imprinting to mitigate the effect of noise included in the masks generated by the interaction. The proposed method was evaluated through experiments using real-world data and shown to outperform an ordinary weight imprinting and provide competitive results to fine-tuning with model distillation while requiring less computational cost.

Keywords: Application of Artificial Intelligence, Deep Learning
Abstract: Recognizing key entities on texts of water environment accurately and rapidly cannot only extract important information of the water environment, but also improve the water quality. In recent years, Chinese named entity recognition becomes a research focus and many methods based on neural networks have been proven effective on entity recognition. This work proposes an improved hybrid prediction model named LBPSC for Chinese named entity recognition for the water environment data, which combines Lattice structure, Bi-directional long short-term memory (BiLSTM), Positional feature encoding, Sentence self-attention and conditional random field (CRF). LBPSC employs a three-phase end-to-end methodology for Chinese named entity recognition. It first adopts a BiLSTM with lattice structure to extract both character and word features from two directions, thereby avoiding word segmentation errors. It then innovatively combines a sentence self-attention mechanism with positional feature encoding to better handle sentences and add the position information to the trained features after BiLSTM. Then, a CRF layer is adopted to decode features and finally output the predicted tag of the data. Experimental results with real-life dataset demonstrate that LBPSC outperforms other deep learning algorithms in terms of prediction accuracy.

Keywords: Manufacturing Automation and Systems, System Architecture, System Modeling and Control
Abstract: Co-design of control and communication is critical to enhancing the control performance of industrial Cyber-Physical System (ICPS). However, the stochastic features of non-deterministic communication strategies in existing works limit control performance improvement. This paper proposes a centralized-control and deterministic-transmission co-design architecture for multi-subsystem ICPS over Time-Sensitive Network (TSN). Packet-wise no-wait transmission scheduling is devised to enable fine-grained scheduling for deterministic delays of the control system data. An integrated system model for the co-design architecture is constructed by integrating scheduling variables into the control system model, which enables simultaneous optimization of the control and transmission strategies for improving the feasible solution space and control performance. Co-design problem based on the integrated model is formulated as mixed-integer non-linear programming (MINLP) problem. A Tabu-search-based heuristic algorithm is proposed to obtain near-optimal solutions efficiently for the co-design problem. Simulation results demonstrate that the proposed method effectively improves control performance compared with best-effort transmission without co-design.

Keywords: Manufacturing Automation and Systems, Robotic Systems, System Architecture
Abstract: Industry 4.0 production systems have to support flexibility in products, processes, and production resources. To meet the required level of flexibility, Industry 4.0 production systems have to be capable of interpreting and executing production plans, which consist of generic actions (such as robotic manipulations or transport of material and products). In industrial practice, Manufacturing Execution Systems (MES) together with Supervisory Control and Data Acquisition (SCADA) systems are usually responsible for such tasks. This paper proposes a new architecture of MES implemented in Python that is able to verify, interpret, and execute production plans automatically generated by an AI planner. Moreover, the proposed MES supports running in a distributed way in several instances, where each instance is able to interpret a location-specific part of a production plan. All MES instances are synchronized and the current global or partial (per each instance) production progress can be observed via HTTP/REST API. The proposed approach is demonstrated in practice on the Industry 4.0 Testbed use-case, utilized for evaluation.

Keywords: Manufacturing Automation and Systems, Consumer and Industrial Applications, Quality and Reliability Engineering
Abstract: The fourth industrial revolution and the digital transformation, commonly known as Industry 4.0, is exponentially progressing in recent years. Connected computers, devices, and intelligent machines communicate with each other and interact with the environment to require only a minimum of human intervention. An important issue in Industry 4.0 is the evaluation of the quality of the process in terms of Key Performance Indicators (KPIs). Monte Carlo simulations can play an important role to improve the estimations. However, there is still a lack of clear workflow to conduct the Monte Carlo simulations to improve such estimations. This paper, therefore, proposes a simulation flow for conducting Monte Carlo methods comparison in Industry 4.0 applications. Based on the simulation flow, we compare Monte Carlo (MC) and Markov Chain Monte Carlo (MCMC) methods on the efficiency KPI of Smart Manufacturing data. The experimental results show the applicability of MC and MCMC with Industry 4.0 data and possible limitations of the two simulation methods.

Keywords: Manufacturing Automation and Systems, Consumer and Industrial Applications, Fault Monitoring and Diagnosis
Abstract: Distinct defects will inevitability incur due to multiple layers of production are required to manufacture TFT-LCD panels. Localization and segmentation of defects are vital in monitoring the panels to improve the yields. But accurate segmentation of defect areas is challenging due to the complex/diverse defects, illumination artifacts, the similarity of the defective pixels with the neighboring pixels, and excessive overall colors in the image. Furthermore, defects have distinct shapes, types, sizes, and locations. This study proposed a singular value decomposition Kernelized Neutrosophic entropy (SVDKNE) method to resolve these challenges that can enhance the inhomogeneous defect images adaptively. Finally, the comparative analysis on a dataset with 309 images validates that the SVDKNE method outperforms other five methods in locating and segmenting defects with higher values of average Jaccard similarity of 0.93, structural similarity of 0.99, and peak signal-to-noise ratio of 38.78.

Keywords: Fault Monitoring and Diagnosis, Intelligent Power Grid, Decision Support Systems
Abstract: Circuit breakers (CBs) play an important role in modern society because they make the power transmission and distribution systems reliable and resilient. Therefore, it is important to maintain their reliability and to monitor their operation. A key to ensure a reliable operation of CBs is to monitor their condition. In this work, we performed an accelerated life testing for mechanical failures of a vacuum circuit breaker (VCB) by performing close-open operations continuously until failure. We recorded data for each operation and made the collected run-to-failure dataset publicly available. In our experiments, the VCB operated more than 26000 close-open operations without current load with the time span of five months. The run-to-failure long-term monitoring enables us to monitor the evolution of the VCB condition and the degradation over time. To monitor CB condition, closing time is one of the indicators, which is usually measured when the CB is taken out of operation and is completely disconnected from the network. We propose an algorithm that enables to infer the same information on the closing time from a non-intrusive sensor. By utilizing the short-time energy (STE) of the vibration signal, it is possible to identify the key moments when specific events happen including the time when the latch starts to move, and the closing time. The effectiveness of the proposed algorithm is evaluated on the VCB dataset and is also compared to the binary segmentation (BS) change point detection algorithm. This research highlights the potential for continuous online condition monitoring, which is the basis for applying future predictive maintenance strategies.

Keywords: Human Factors, Human-centered Learning, Human-Machine Interface
Abstract: Apparent color and pixel color are known to be different due to color constancy and contexts of images. An apparent color dataset for photographs and images will be a basis for developing perception-based color extraction and transfer system. In this study, we collected the apparent color dataset for a given image and extracted position through an online experiment that mimics the situation of using a color extraction system. Additionally, we analyzed 1110 data and verified the difference between apparent color and pixel color. As a result, approximately 80% data has ∆E_NBS > 6.0 color difference that is defined “Much” or “Very Much” difference by the national bureau of standards (NBS). Luminance (L*) of the apparent color was on average ∆E_00 = [9.90, 11.9] (95% confidence interval) greater than that of the pixel color. These results indicate that the color deviations cannot be ignored when apparent colors are extracted in images.

Keywords: Human-centered Learning, Human-Computer Interaction, Human Performance Modeling
Abstract: Riding bicycles with only one hand on the handlebar can severely undermine the operator’s steering capability and threaten road and transportation safety. Prior studies have exploited motion sensors to detect riding contexts and recognize related behaviors. Nevertheless, they fail to integrate a scheme to account for single-handed riding with elements crucial to danger prevention: awareness of the surroundings, response to danger, and convenient adoption. In this work, we proposed, designed, and implemented DoubleCheck: a smartphone-based real-time framework for cycling hand detection and distraction recognition. The method monitors handlebar holding on different road surfaces and recognizes hazardous distraction activities related to single-handed cycling using motion signals captured by a built-in inertial measurement unit in a handlebar-borne smartphone. It was designed on the premise that single-handed cycling enabled operators to adapt their body movements to different (often distracting) activities. We conducted an evaluation experiment using 22 participants on asphalt and pavement. The results indicate that DoubleCheck achieves an F1-score of 0.96 for hand detection and 0.69 for distraction recognition, demonstrating its efficacy as a candidate rider-safety precautionary measure.

Keywords: Interactive and Digital Media, Intelligence Interaction, Human-centered Learning
Abstract: Detecting human-object interactions is essential for comprehensive understanding of visual scenes. In particular, spatial connections between humans and objects are important cues for reasoning interactions. To this end, we propose a skeleton-aware graph convolutional network for human-object interaction detection (SGCN4HOI). SGCN4HOI exploits the spatial connections between human keypoints and object keypoints to capture their fine-grained structural interactions via graph convolutions and make use of visual features and spatial configuration features from human-object pairs. To better preserve the object skeleton structure information and facilitate human-object interaction detection, we propose a novel skeleton-based object keypoints representation that utilizes the K-means algorithm. The performance of SGCN4HOI is evaluated in the public benchmark V-COCO dataset. Experimental results show that the proposed approach outperforms the state-of-the-art pose-based models and achieves competitive performance against other models.

Keywords: Human-centered Learning, Human-Computer Interaction, Assistive Technology
Abstract: In highly competitive markets, customer loyalty plays an increasingly important role for companies. An important aspect is the recording and processing of customer complaints, on the one hand for problem-solving and on the other hand for internal process optimization. The complaint handling process extends over several phases, in which different people with varying expertise and experience may be involved, which poses a major challenge in order to achieve Consistent quality in the recording process of complaints. To address this issue, this work presents a robust active learning based AI system that allows using existing expert knowledge to create more reliable classifications in a shorter amount of time. The implemented prototype shows a decrease of up to 86% in the average processing time of complaints, with an average increase of up to 37% in the classification accuracy.

Keywords: Human-centered Learning, Assistive Technology, Human Performance Modeling
Abstract: Many first-year university students do not complete the study plan and drop out. By investigating how students earn ECTS credits we create a model that makes it possible to predict students, who are at risk of failure. Weekly analysis of student data allows us to identify patterns important for prediction. Early predictions inform students about the potential danger of failure and also allow tutors to intervene. On the other hand, from the data of successful students, it is possible to derive study trajectories leading to the successful completion of the academic year and offer these trajectories to students. The described techniques for student support are demonstrated by examples.

Keywords: Human-Machine Interface, Virtual and Augmented Reality Systems, Wearable Computing
Abstract: Wearable haptic technology plays a key role to enhance the feeling of immersion in virtual reality, telepresence, telehealth and entertainment systems. This work presents a wearable fingertip capable of providing touch, sliding and vibration feedback while the user interacts with virtual objects. This multimodal feedback is applied to the human fingertip using an array of servo motors, a coin vibration motor and 3D printed components. The wearable fingertip uses a 3D printed cylinder that moves up and down to provide touch feedback, and rotates in left and right directions to deliver sliding feedback. The direction of movement and speed of rotation of the cylinder are controlled by the exploration movements performed by the user hand and finger. Vibration feedback is generated using a coin vibration motor with the frequency controlled by the type of virtual material explored by the user. The Leap Motion module is employed to track the human hand and fingers to control the feedback delivered by the wearable device. This work is validated with experiments for exploration of virtual objects in Unity. The experiments show that this wearable haptic device offers an alternative platform with the potential of enhancing the feeling and experience of immersion in virtual reality environments, exploration of objects and telerobotics.

Keywords: Wearable Computing, Virtual and Augmented Reality Systems, Human-Machine Interface
Abstract: Tactile sensation is essential for recognizing mechanical interactions between the human body and its environment; it is generally uniquely experienced by individuals. Haptic sharing technologies enable to share of other person's (or robot's) tactile sensations, and it is expected to apply for skill learning, collaborative work, and robot operation. While wearable tactile sensors are useful for tactile measurements, they are affected by differences in physical characteristics between individuals. Therefore, to accurately convey the tactile information felt by others and utilize it for work, it is necessary to remove individual differences due to differences in individual physical characteristics. In this study, based on the frequency based intensity of skin vibration of an individual, the vibration detected by the acceleration sensor affixed to the fingertip was modulated to remove individual differences. To confirm the effectiveness of the proposed method, we investigated both physical and perceptual similarities for skin vibrations collected from three persons with three different fine textures. The results showed that vibrations individually modulated with the proposed method was similar in the intensity and power spectrum density than those using the uniform modulation function, and was perceived similar when presenting those with a vibrator.

Keywords: Human-Machine Cooperation and Systems, Human-Machine Interface, Virtual and Augmented Reality Systems
Abstract: 3D reconstruction from images obtained by RGB-D camera provides a 3D model that reflects the current conditions of nuclear power plants (NPPs), and it is useful for field work support in NPPs. However, machinery equipment and pipes are intricately connected in NPPs and a lot of areas are not photographed because they are occluded by peripheral machines or pipes. It is feasible, but still challenging photographing those areas with RGB-D camera-mounted drones, due to the battery capability and the weight of drones. In this study, 3D reconstruction in cooperation with a light-weight mirror-mounted autonomous drone has been proposed and a novel method for determining drone's flight path to scan those areas with a mirror has been developed. In this method, suitable locations for scanning the occluded areas with a mirror are detected and connected using multiple paths that are verified as safe to fly. The developed method was evaluated in the simulation environment. The results suggested that the developed method could obtain more points and photograph more occluded areas than the method of repeating random movements and the method to locally search for the optimal direction.

Keywords: Human-Machine Interface, Human-Computer Interaction, Virtual and Augmented Reality Systems
Abstract: In application use of electronically controllable ultrasound fields, spatial calibration of multiple airborne ultrasonic phased arrays forming a synthetic aperture must be completed with subwavelength accuracy for extending workspaces. However, manually completing the calibration requires a lot of time and effort, and there has been no established automatic method for this problem. Here we propose an acoustic calibration method using spatiotemporal structured ultrasound field sequentially generated by each array and a single microphone for acoustic measurement. We form a point cloud of microphone positions by the measurement and then register them into Gaussian model in the global coordinate system common across all arrays. We apply likelihood maximization approach to estimate the positions and postures of all arrays that minimizes the discrepancy of global microphone positions retrieved from local observations. We conducted numerical experiments to find that the calibration with no error was achieved with no measurement errors in microphone position, whereas mean estimation errors of less than a quarter of the wavelength in position and one degree in posture were observed when assuming a possible magnitude of measurement errors estimated in previous works.

Keywords: Human-Machine Interface, Virtual and Augmented Reality Systems, Human-Computer Interaction
Abstract: Direct cortical stimulation of the somatosensory cortex (S1-DCS) has been shown to evoke distinct and localizable percepts, exploitable as neurohaptic feedback. In this study, we leveraged a novel virtual reality (VR) experimental platform to evaluate S1-DCS neurohaptic feedback during naturalistic object interaction. Two human subjects implanted with intracranial electrodes for seizure localization were asked to discriminate between visually identical virtual objects based on their distinct S1-DCS neurohaptic profiles. In a binary discrimination task, neurohaptic feedback was either present or absent while grasping a virtual object. In the ternary discrimination task, neurohaptic feedback was either present in one of two distinct neurohaptic sequences or absent. Both subjects performed significantly above chance in binary and ternary discrimination, demonstrating the efficacy of S1-DCS as neurohaptic feedback. Successful ternary discrimination also demonstrated that different sequences of amplitude-modulated S1-DCS at a single pair of electrodes can evoke discriminable neurohaptic percepts. Moreover, amplitude-modulated S1-DCS sequences were shown to elicit sensorimimetic percepts described as “bumpy” and “smooth” in Subject 1, and as a sensation of movement in the paralyzed hand of Subject 2. Our study demonstrates the reliability and discriminability of both simple and complex S1-DCS for neurohaptic feedback during immersive VR object interaction and supports the use of immersive VR for neurohaptic design towards the development of functional brain computer interface.

Keywords: Trust in Autonomous Systems, Intelligent Transportation Systems, Robotic Systems
Abstract: Tremendous progress in deep learning over the last years has led towards a future with autonomous vehicles on our roads. Nevertheless, the performance of their perception systems is strongly dependent on the quality of the utilized training data. As these usually only cover a fraction of all object classes an autonomous driving system will face, such systems struggle with handling the unexpected. In order to safely operate on public roads, the identification of objects from unknown classes remains a crucial task. In this paper, we propose a novel pipeline to detect unknown objects. Instead of focusing on a single sensor modality, we make use of lidar and camera data by combining state-of-the art detection models in a sequential manner. We evaluate our approach on the Waymo Open Perception Dataset and point out current research gaps in anomaly detection.

Keywords: Intelligent Transportation Systems, Consumer and Industrial Applications, Service Systems and Organizations
Abstract: The increasing demand for freight transport requires logistic companies to improve their competitiveness by ensuring high service levels at limited costs. This paper investigates the problem of defining delivery plans with the aim to support logistic companies in reducing planning times and freight delivery costs. In delivery planning, given a set of delivery requests, both the routes and load configurations of Transport Units (TUs) are to be established. In the literature, this problem is defined as Three-dimensional Loading Capacitated Vehicle Routing Problem with Time Windows (3L-CVRPTW). However, these problems are generally tackled separately and referred to as the vehicle routing problem and the container loading problem, respectively. Moreover, only a few contributions present solution approaches for real logistic systems, and these methods are mainly based on heuristics. In this work, we define a novel matheuristic algorithm for the integrated solution of the vehicle routing problem and container loading problem. The proposed method is suitable for real logistic applications and combines the advantages of exact solutions with the rapidity of heuristics. The approach aims at minimizing the total travel costs and the clients’ time windows violations in the routes’ definition, while optimizing the configuration of the cargo inside each TU. The developed matheuristic algorithm is tested both on a well-known literature benchmark and on a real dataset provided by the Italian company Elettric80. The obtained results show that the proposed method succeeds in determining in a short computational time both feasible routes and loading plans, minimizing the related costs while fulfilling logistics constraints

Keywords: Intelligent Transportation Systems
Abstract: This paper proposes GraphPro, a short-term link speed prediction framework for signalized urban traffic networks. Different from other traditional approaches that adopt only reactive inputs (i.e., surrounding traffic data), GraphPro also accepts proactive inputs (i.e., traffic signal timing). This allows GraphPro to predict link speed more accurately, depending on whether or not there is a contextual change in traffic signal timing. A Wasserstein generative adversarial network (WGAN) structure, including a generator (prediction model) and a discriminator, is employed to incorporate unprecedented network traffic states and ensures a high level of generalizability for the prediction model. A hybrid graph block, comprised of a reactive cell and a proactive cell, is implemented into each neural layer of the generator. In order to jointly capture spatiotemporal influences and signal contextual information on traffic links, the two cells adopt several key neural network-based components, including graph convolutional network, recurrent neural architecture, and self-attention mechanism. The double-cell structure ensures GraphPro learns from proactive input only when required. The effectiveness and efficiency of Graph-Pro are tested on a short-term link speed prediction task using real-world traffic data. Due to the capabilities of learning from real data distribution and generating unseen samples, GraphPro offers a more reliable and robust prediction when compared with state-of-the-art data-driven models.

Keywords: Intelligent Transportation Systems, Communications, Smart Sensor Networks
Abstract: Vehicle Ad-hoc Networks have laid an essential technical foundation for realizing intelligent transportation. Unexpected mobility change of a specific node often causes a communication link failure. Thus, this work proposes a node backup strategy for routing protocol in software-defined vehicular networks. The core of the strategy is to promote communication link stability through backup nodes. A node backup routing algorithm is designed to search for alternative nodes for each node in a communication link. A node can flexibly select the next-hop node during packet transmission based on actual conditions. When an unexpected mobility change of a specific node causes a communication link failure, we can restore the link by enabling the alternative nodes. The influence of various factors on packet reception rate and communication delay is studied through simulation experiments. By comparing with two existing routing protocols, the effectiveness of the proposed approach is verified.

Keywords: Intelligent Transportation Systems, System Modeling and Control
Abstract: Kinematics of N-Trailer vehicles is characterized by two kinds of parameters, namely, the trailer lengths and the hitching offsets. In the case of automated or robotic N-Trailers, which can often exchange a subset of trailers during operations (e.g., when used in logistic stations or transhipment ports), kinematic parameters of the vehicle can become unknown or at least uncertain. Moreover, in the multi-axle trailers with fixed wheels the resultant trailer length and hitching offset are inherently uncertain or even time-varying during cornering motion. Therefore, we address the problem of parametric estimation of the N-Trailer kinematics using only (noisy) measurements of the vehicle's joint (articulation) angles. A scalable sequential estimation procedure is proposed which can be flexibly applied to vehicles equipped with an~arbitrary number of trailers interconnected with the off-axle and/or on-axle hitches. Numerical results presented in the paper illustrate large-sample statistical properties of the proposed estimation procedure.

Keywords: Fuzzy Systems and their applications, Computational Intelligence
Abstract: In decision-making with fuzzy preference relations, pairwise comparisons among all the attributes or alternatives in question are done by decision-makers to generate a matrix. The validity and reliability of the decision made is ensured if the consistency of this matrix is high, which usually requires alterations of the entries of the matrix. This research introduces a new consistency improvement procedure by allowing an information granularity level in the decision-making process. Concretely, to improve the consistency, it invokes a process of an optimal information granularity distribution across the related alterations of the entries of the matrix. To demonstrate its effectiveness, we complete some detailed experiments.

Keywords: Image Processing and Pattern Recognition, Deep Learning, Neural Networks and their Applications
Abstract: Temporal Action Localization (TAL) is a task of identifying a set of actions in a video, which involves localizing the start and end frames and classifying each action instance. Existing methods have addressed this task by using predefined anchor windows or heuristic bottom-up boundary-matching strategies, which are major bottlenecks in inference time. Additionally, the main challenge is the inability to capture long-range actions due to a lack of global contextual information. In this paper, we present a novel anchor-free framework, referred to as HTNet, which predicts a set of triplets from a video based on a Transformer architecture. After the prediction of coarse boundaries, we refine it through a Background Feature Sampling (BFS) module and hierarchical Transformers, which enables our model to aggregate global contextual information and effectively exploit the inherent semantic relationships in a video. We demonstrate how our method localizes accurate action instances and achieves state-of-the-art performance on two TAL benchmark datasets: THUMOS14 and ActivityNet 1.3.

Keywords: Neural Networks and their Applications, Representation Learning, Deep Learning
Abstract: Sentence embeddings are commonly used in text clustering and semantic retrieval tasks. State-of-the-art sentence representation methods are based on artificial neural networks fine-tuned on large collections of manually labeled sentence pairs. Sufficient amount of annotated data is available for high-resource languages such as English or Chinese. In less popular languages, multilingual models have to be used, which offer lower performance. In this publication, we address this problem by proposing a method for training effective language-specific sentence encoders without manually labeled data. Our approach is to automatically construct a dataset of paraphrase pairs from sentence-aligned bilingual text corpora. We then use the collected data to fine-tune a Transformer language model with an additional recurrent pooling layer. Our sentence encoder can be trained in less than a day on a single graphics card, achieving high performance on a diverse set of sentence-level tasks. We evaluate our method on eight linguistic tasks in Polish, comparing it with the best available multilingual sentence encoders.

Keywords: Image Processing and Pattern Recognition, Neural Networks and their Applications, Deep Learning
Abstract: In mission-critical systems such as self-driving, medical, and infrastructure systems, hardware faults can lead to serious accidents. Therefore, we need a method to detect hardware faults of artificial intelligence (AI) with high accuracy. A low-cost fault detection method with less computation is required to reduce AI's chip area and/or energy consumption. In this paper, we propose an approximate Dual Modular Redundancy (DMR) unit using a Random Forest approximation method, which can significantly reduce the computation for inference in the convolutional neural networks (CNNs). We assume various scenarios of faults and evaluate the fault effects. In our experiments, we demonstrate that the proposed approximate DMR unit achieves high fault detection for three types of fault models. In addition, we report a 42.8% to 48.3% reduction in the computation for inference compared to the conventional method.

Keywords: Expert and Knowledge-Based Systems, Knowledge Acquisition, Fuzzy Systems and their applications
Abstract: Dempster-Shafer evidence theory is an important technique to fuse uncertain information and make decisions. As the first step of applying the evidence theory in practical applications, the generation of basic probability assignment(BPA) directly affects the subsequent fusion and decision-making. However, it is still a difficult and open issue. This paper proposed a new method to generate BPA with the normal membership function. First, we construct a normal membership function for each class with its mean value and standard deviation of a certain attribute, whose function value represents the probability of a sample belonging to it. According to the rule of the intersection of fuzzy sets, the probability of a sample belonging to the proposition with multiple classes is the minimum function value of the normal membership functions for all relevant classes. Therefore, the BPA for the attribute can be obtained with the rule. Similarly, the BPAs for other attributes can also obtained with the same method. To make decision, the BPAs for all attributes will be combined into a fused BPA with the Dempster combination rule. The final classification result is the proposition with the largest value in it. Finally, we conduct the classification experiments on nine datasets from the UCI dataset and the result shows the superiority and robustness of the proposed method.

Keywords: Assistive Technology, Intelligence Interaction
Abstract: Under the recent trend of unmanned economy, the retail stores have reduced the manpower for service and cashier gradually. The retail product recognition becomes one essential problem for unmanned shopping. Although the success of deep neural network makes the object recognition feasible in various applications, it is still difficult to perform well on a large number of classes. This paper presents an ensemble learning approach to deal with recognition for the growing number of retail products. In the proposed technique, the object classification networks are first improved with feature extraction and block attention. The ensemble model is then constructed by integrating the multiple network models with the loss selection as model weights. In the experiments, the feasibility of our ensemble recognition method is validated with a number of production items. The results have demonstrated the effectiveness compared to the state-of-the-art recognition algorithms.

Keywords: Assistive Technology
Abstract: Patients with knee arthropathy suffer from pain owing to damage at the knee joint cartilage and experience difficulty in walking as the condition worsens. As one of treatments of knee arthropathy, an implantation of cellular cartilage has been performed. After implantation of cellular cartilage, the knee joint should avoid overweight to prevent cell damage form body weight and apply appropriate mechanical stress to articular cartilage to promote cartilage regeneration. Therefore, a weight support system which can adjust the amount of weight in everyday use is required. This study aims to develop an in-clothes body weight support system to protect the knee joint during daily walking to support knee arthropathy treatment and recovery in regenerative medicine, and confirm basic performance of the system through experiments with able- bodied person. The system had frame structure with two-node link from the groin to the foot with the user's knee joint as the center of rotation. It consisted of load-bearing seat, height adjustment mechanism, thigh cuff, mechanical knee joint, and, ground reaction force sensor shoes, and was designed to be sufficiently thin to fit inside clothes. In the 10-Meter walking test, the system could adjust the amount of supporting weight and support two-thirds, one-half, and one-third of total body weight of an able-bodied participant during walking. In conclusion, we confirmed that the developed system had basic performance of weight support.

Keywords: Assistive Technology, Human-Computer Interaction, Human Factors
Abstract: Human walking is a complex activity with a high level of cooperation and interaction between different systems in the body. Accurate detection of the phases of the gait in real-time is crucial to control lower-limb assistive devices like exoskeletons and prostheses. There are several ways to detect the walking gait phase, ranging from cameras and depth sensors to the sensors attached to the device itself or the human body. Electromyography (EMG) is one of the input methods that has captured lots of attention due to its precision and time delay between neuromuscular activity and muscle movement. This study proposes a few Machine Learning (ML) based models on lower-limb EMG data for human walking. The proposed models are based on Gaussian Naive Bayes (NB), Decision Tree (DT), Random Forest (RF), Linear Discriminant Analysis (LDA) and Deep Convolutional Neural Networks (DCNN). The traditional ML models are trained on hand-crafted features or their reduced components using Principal Component Analysis (PCA). On the contrary, the DCNN model utilises convolutional layers to extract features from raw data. The results show up to 75% average accuracy for traditional ML models and 79% for Deep Learning (DL) model. The highest achieved accuracy in 50 trials of the training DL model is 89.5%.

Keywords: Human Performance Modeling, Entertainment Engineering, Assistive Technology
Abstract: The three times increase of SonyLiv viewers during the Tokyo Olympic, the 10% hike of YouTube users during the isolation era of covid-pandemic, and the 19% growth in Netflix user count due to the fastest growth of OTT, etc. have made the digital platform's mode all-time active and specific. The hourly increase of users' interactions and the e-commerce platform's desire of letting users engage on their sites are pushing researchers to shape the virtual digital web as user specific and revenue-oriented. This paper develops a deep learning-based approach for building a movie recommendation system with three main aspects: (a) using a knowledge graph to embed text and meta information of movies, (b) using multi-modal information of movies like audio, visual frames, text summary, meta data information to generate movie/user representations without directly using rating information to represent user/movie; this multi-modal representation can help in coping up with cold-start problem of recommendation system (c) a graph attention network based approach for developing regression system. For meta encoding, we have built knowledge graph from the meta information of the movies directly. For movie-summary embedding, we extracted nouns, verbs, and object to build a knowledge graph with head-relation-tail relationships. A deep neural network, as well as Graph attention networks, are utilized for measuring performance in terms of RMSE score. The proposed system is tested on an extended MovieLens-100K data-set having multi-modal information. Experimental results establish that only rating based embeddings in the current setup outperform the state-of-the-art techniques but usage of multi-modal information in embedding generation performs better than its single-modal counterparts.

Keywords: Assistive Technology, Human-Machine Interface
Abstract: This work proposes an eco-driving assistance system (EDAS) based on model predictive control (MPC) with a primary objective to improve the driver’s driving style in an energy-efficient manner. To improve the efficiency of an EDAS, a learning-based approach to model the driver behavior from urban driving data collected using a dynamic driving simulator is presented. To cluster the driving data of thirty-four participants, unsupervised learning techniques such as principal component analysis (PCA) and hierarchical cluster analysis (HCA) were used. Furthermore, to predict the driver speed error while tracking an advisory speed, both stochastic and deterministic models, namely Stochastic Volatility (SV) and Gated Recurrent Unit (GRU) respectively, are trained. Six new drivers evaluated the proposed concept, whose driving style is classified using a trained temporal convolution network (TCN). Using the predicted driver speed error, the eco-driving advisory speed is compensated and provided as a feedback to the driver via a human-machine interface (HMI). The results reveal that the deterministic model has been able to achieve higher prediction accuracy as compared to the stochastic model. Furthermore, the results also suggest that the drivers using EDAS with driver error compensation have been able to perform better advisory speed tracking and achieve improved energy savings.

Keywords: Robotic Systems, Mechatronics
Abstract: Finger dexterity and tactile perception are key capabilities for humans to manipulate objects within hand, as well as robots. Inspired by the thumb-forefinger dexterous manipulative movement, we devised a novel robotic finger with an active rotational tactile sensor (i.e. TacRot), and mounted the finger on a parallel-jaw gripper. By processing the high-resolution images of the vision-based tactile sensor, we achieved depth reconstruction of the surface and localization of the contact area. To improve gripping flexibility and stability, we applied a self-adaptive grasping strategy with real-time contact detection feedback, which performed 94% success rate in experiment. Based on the rotational actuator at the fingertip, we proposed two in-hand manipulation primitives: (1) pivot: fingertips co-rotating for object reorientation; (2) twist: fingertips contra-rotating for object spin. The primitives are theoretically analyzed and experimentally verified in two practical tasks: pivoting a paper cup under vertical constraints and twisting a screw with spin angle estimation. Our design and experiments demonstrate a feasible way to enhance the active tactile manipulation ability for common parallel-jaw grippers.

Keywords: Robotic Systems
Abstract: Existing Simultaneous Localization and Mapping (SLAM) approaches are limited in their scalability due to growing map size in long-term robot operation. Moreover, processing such maps for localization and planning tasks leads to the increased computational resources required onboard. To address the problem of memory consumption in long-term operation, we develop a novel real-time SLAM algorithm, MeSLAM, that is based on neural field implicit map representation. It combines the proposed global mapping strategy, including neural networks distribution and region tracking, with an external odometry system. As a result, the algorithm is able to efficiently train multiple networks representing different map regions and track poses accurately in large-scale environments. Experimental results show that the accuracy of the proposed approach is comparable to the state-of-the-art methods (on average, 6.6 cm on TUM RGB-D sequences) and outperforms the baseline, iMAP*. Moreover, the proposed SLAM approach provides the most compact-sized maps without details distortion (1.9 MB to store 57 m^3) among the state-of-the-art SLAM approaches.

Keywords: Robotic Systems, Mechatronics, System Modeling and Control
Abstract: Nowadays, design and development of legged quadruped robots is a quite active area of scientific research. In fact, the legged robots have become popular due to their capabilities to adapt to harsh terrains and diverse environmental conditions in comparison to other mobile robots. With the higher demand for legged robot experiments, more researches and engineers need an affordable and quick way of locomotion algorithm development. In this paper, we present a new open source quadruped robot HyperDog platform, which features 12 RC servo motors, onboard NVIDIA Jetson nano computer and STM32F4 Discovery board. HyperDog is an open-source platform for quadruped robotic software development, which is based on Robot Operating System 2 (ROS2) and micro-ROS. Moreover, the HyperDog is a quadrupedal robotic dog entirely built from 3D printed parts and carbon fiber, which allows the robot to have light weight and good strength. The idea of this work is to demonstrate an affordable and customizable way of robot development and provide researches and engineers with the legged robot platform, where different algorithms can be tested and validated in simulation and real environment. The developed project with code is available on GitHub.

Keywords: Robotic Systems, Soft Robotics, Service Systems and Organizations
Abstract: Inventory management is very important in any industry. In the past, people were directly involved in inventory management. However, this method is time consuming and requires a lot of labor. Therefore, in recent years, inventory management methods using drones are increasing. In this paper, we introduce an inventory management system using RFID and QR code, and a method of estimating the relative position of a drone based on the QR code, which is essential in the system. As a result of the experiment, when the center point of the QR code and the center point of the camera coincide, the error rate was very small, 0-1%. When the center points do not coincide, an error rate of up to 4% occurred.

Keywords: Robotic Systems, Cooperative Systems and Control, Consumer and Industrial Applications
Abstract: In this paper we introduce a novel optimal trajectory planning method for robots in static and dynamic environments while accounting for the robot’s kinematic and dynamic constraints. The proposed method is called Danger Field Sampling Based Model Predictive Optimization (DF-SBMPO). This new method includes the advantages of SBMPO such as dealing with non-linear constraints and non-invertible models and planning in the input space while improving its computational efficiency in local minima situations. By introducing a new cost function and combining the Danger Field method in the collision checking process, our method can efficiently find a near time-optimal and smooth trajectory to the goal. We verify the method in simulation using a car-like robot model in an environment containing both static and moving obstacles. By comparing it systematically to other sampling-based path planning algorithms and the Time-Scaling time-optimal trajectory planning method, the results show that our method generally produces collision-free, smoother, near time-optimal trajectories that satisfy constraints with less computation time and fewer samples.

Keywords: Computational Intelligence, Optimization and Self-Organization Approaches
Abstract: In the freemium business model, how to price value-added services and design effective strategy to achieve the sustainability of value-added services promotion is of great significance to enterprises. By constructing a monopolistic freemium enterprise, this paper uses a Multinational Logit model (MNL model) to analyze value-added services pricing and two kinds of value-added services promotion strategies (the quality reduction strategy of basic product and the price discount strategy of value-added services) with heterogeneous sticky-users demand, and then discusses the optimal promotion strategy. The results show that: Both the quality reduction strategy of basic product and the price discount strategy of value-added services can have positive impacts on the profit of enterprise. The sticky users demand valuation plays a positive role in promoting the optimal profit of enterprise. The optimal promotion strategy is the price discount strategy of value-added services. This study can provide a theoretical basis and decision support for the operation and management of the freemium enterprises.

Keywords: Application of Artificial Intelligence, Cloud, IoT, and Robotics Integration
Abstract: Emergency vehicles (EVs), such as ambulances, police vehicles, and fire-fighting trucks, play an essential role in delivering emergency services in our society. To decrease the negative impact of EVs on normal traffic, a traffic guidance strategy is proposed for the evacuation of regular vehicles on the path of the EV. Queueing theory is used to provide equilibrium guidance for the evacuation. Furthermore, lane-changing and traffic light preemption strategies are used to prioritize the EV. A simulation experiment is conducted on a map of Huangdao District, Qingdao City, China with the platform of SUMO. The proposed method is validated on three types of traffic flow density. Compared with the existing state-of-the-art strategies, the superiority of our approach is verified from the aspects of EV’s average waiting time and the time loss of other vehicles.

Keywords: Application of Artificial Intelligence, Deep Learning, Swarm Intelligence
Abstract: Virtual Coupling (VC) is an emerging topic in the railway industry. It breaks the long-distance limitation and enables trains to operate at closer distances with new safety distance constraints. The new safety distance has high-order nonlinear time-varying because it is closer to the characteristics of virtually coupled trains (VCTs). This makes it difficult for the Automatic Train Operation (ATO) system to track the front virtually coupled train (VCT) in real-time and cannot guarantee stop synchronization. VCTs are equivalent to a physically connected train. So synchronous operation is very important. In order to solve the above problems, this paper takes the entire VCTs as optimization object and takes synchronized stopping, punctuality, and precise stopping as optimization indicators. The reinforcement learning Deep Q Network (DQN) algorithm is used to solve the optimization problem to seek the reference speed curve of each VCT. ATO performs real-time control on the basis of reference speed curve, which reduces calculation pressure and ensures synchronous operation of VCTs. Moreover, this paper improves DQN algorithm according to operation scenarios of VCTs to make the solution faster. Finally, we conduct a simulation experiment with the scene of Beijing Metro Line 11. The experiment demonstrates the effectiveness of this method for stopping synchronization and the superiority of the improved DQN algorithm for VCTs running scenarios.

Keywords: Cybernetics for Informatics, Heuristic Algorithms
Abstract: Virtual Coupling (VC) is a hot topic in railways. A virtually coupled train set operates as a single train and consists of multiple unit trains (UTs). Each UT keeps a desired gap distance with its predecessor by following a spacing policy. Constant time headway (CTH) spacing policy is widely adopted in the existing studies about VC. Arrival time difference (ATD), i.e., the difference between the time instants of two successive UTs arriving at a platform, is large when using CTH. It is not expected because the passengers loading of all UTs should operate at the same time. This paper firstly analyzes the causes of ATD mathematically and proves ATD is inevitable when adopting CTH. Then, the change tendency of ATD with respect to each parameter is analyzed. The above analysis indicates that CTH is not suitable for the arrival process. Thus, a quadratic spacing policy is designed to reduce ATD. Experimental results show that ATD can hardly be reduced by only optimizing the parameters of CTH, but the quadratic spacing policy greatly reduces it.

Keywords: Evolutionary Computation, Swarm Intelligence, Heuristic Algorithms
Abstract: Electric Vehicle Routing Problem (EVRP) is considerably challenging due to the capacity and electricity constraints of electric vehicles (EVs). Most existing studies on EVRP consider no limits on charging times when optimizing the routes of EVs. However, due to the long time of charging, the charging times of EVs are usually limited due to the urgent service demands of customers. To simulate this practical problem, this paper first formulates the EVRP with both capacity and charging time constraints (EVRP-CC). To tackle this new optimization problem, this paper further devises a two-stage solution construction method for ant colony optimization (ACO) to build feasible solutions to EVRP-CC. Subsequently, we embed the proposed method into five popular and classical ACO algorithms, namely ant system (AS), ranking based ant system (Rank-AS), elite ant system (EAS), max-min ant system (MMAS), and ant colony system (ACS), to solve EVRP-CC. Extensive experiments conducted on several instances generated from the widely used EVRP benchmark set demonstrate that the proposed solution construction method is effective to help ACO to solve EVRP-CC. In particular, Rank-AS with the proposed solution construction method achieves the best overall performance in solving EVRP-CC.

Keywords: Deep Learning, Neural Networks and their Applications, Machine Learning
Abstract: Optimization of per-layer or/and total inference time without accuracy loss in convolutional neural networks (CNNs) is significantly crucial in resource-constrained Edge-AI devices and embedded systems. To do this, this work 1) introduces a CNN inference micro-benchmark (mbNet) for performance analysis and optimization and 2) proposes a simple yet effective performance model for adaptive kernel selection to optimize per- layer CNN inference time. Considering the convolutional layer is the core part of CNNs, the two mainstream convolutional strategies of unrolling based convolution (UNROLL) and direct convolution (DIRECT) are adopted/implemented, compared and analyzed in terms of per-layer convolutional time. Using the obtained data from our mbNet, we build an accurate and interpretable tree-based performance model, with which our adaptive kernel selection approach shows significant convolu- tional performance improvement up to 5.4x speedup (on average, 2.7x and 1.6x speedup, compared to the default UNROLL and DIRECT respectively).

Keywords: Optimization and Self-Organization Approaches, Machine Learning, Agent-Based Modeling
Abstract: In two earlier papers presented at the 2019 and 2020 American Control Conferences, the concept of “Mutual Learning” was introduced by the authors and applied to learning in static and dynamic stochastic environments. In this paper, we extend the concept of mutual learning to optimization. Two agents attempting to optimize the same performance index “learn” from each other to reach the solution more efficiently. Since optimization is a well investigated mathematical area in systems theory, it is particularly well suited to the original objective of the authors to study “Mutual Learning” in a systems theoretic framework. The two agents involved in mutual learning can use any of the methods well-known in the literature to optimize the given function. The initial conditions and the period over which the optimization is carried out, may be different for the two agents before they communicate with each other for the first time. The principal conclusion of the paper is that mutual learning should be viewed as a general research area, and not as a specific procedure used in different system theoretic problems.

Keywords: Heuristic Algorithms, Cloud, IoT, and Robotics Integration, Optimization and Self-Organization Approaches
Abstract: The edge-cloud computing systems are widely used to support various computation services. In this paper, we consider the dynamic task offloading problem in the edge-cloud computing system with multiple independent and stochastic arriving tasks. The system periodically schedules and offloads tasks to heterogenous resources in consideration of the required transmission delays and computation times. Our goal is to minimize the total weighted response time over all the tasks. A greedy local search based online offloading framework is proposed for the problem under study, which dynamically assigns tasks to the appropriate destination (edge servers or cloud servers) and preemptively allocates computing resources to each task according to its latency-sensitivity. Evaluation experiments are delicately designed on a number of testing instances with various parameter settings. Experimenta results show that the proposed algorithm is more effective than the compared algorithms.

Keywords: Cloud, IoT, and Robotics Integration, Computational Intelligence, Heuristic Algorithms
Abstract: Fog computing extends a paradigm of cloud computing to a network edge, thus reducing transmission delay of users’ tasks and providing fast services. Dramatic increase of mobile devices brings a big challenge of how to keep low-response processing of tasks. To solve this challenge, this work aims to minimize the latency of tasks while meeting energy limits of mobile devices, and formulates a constrained optimization problem. It designs an improved optimization algorithm by following core steps of whale optimization algorithm (WOA) inspired by whale hunting behaviors. It is named Chaotic Differential WOA with Levy flight (CDWL). CDWL integrates merits of WOA, chaotic differential evolution, and random walks of Levy flight. In this way, CDWL achieves strong global search and quick convergence. Real-life experiments demonstrate CDWL outperforms its six state-of-the-art peers.

Keywords: Image Processing and Pattern Recognition, Machine Learning, Application of Artificial Intelligence
Abstract: Artificial intelligence and machine learning have become increasingly popular in recent years, applying them to different applications are also vital in prediction. As facial emotion is one of the important expressions in our daily life, emotion recognition helps in conversation and understanding more effectively. Although emotion recognition may seem to be an easy task to humans, it is challenging for the machine. Currently, many facial emotion detection technologies are based on the features extracted from the whole face and do further analysis to detect emotions. However, there are some limitations to this method. For example, it may require considerable time in the training and prediction with the high-resolution image input. It is a major issue if we apply the model to a mobile device that gives high-resolution photos. To deal with the problem, reducing the input size while remaining the important information would be a solution. To achieve it, the dimension reduction method may be an option. On the other hand, there is a lot of psychological research on how humans recognize emotion by facial expression. Improvements can be made if we can make good use of the results for emotion recognition tasks. Thus, in this paper, we proposed a psychological-oriented method for reducing the input size on the emotion detection task which is based on the eyes region. It is found that the proposed method gives compatible performance compared to the face-based model. With the proposed method, the time used for training and prediction is significantly reduced.

Keywords: Transfer Learning, Deep Learning, Machine Learning
Abstract: Transfer learning performance in convolutional neural networks (CNNs) depends on the selection of retraining and fixation layers. The conventional method requires source data, which increases the cost proportionally to the data's size. To solve this problem, this study proposed a source-data-less layer importance index that indicates the layers that should be retrained. Consequently, using the maximum activated feature maps generated by the model itself instead of the activated feature maps, the proposed method eliminates the need for any source data. Further, the experimental results on the five image classification datasets demonstrated a strong positive correlation of the proposed layer importance index compared with the conventional method. In addition, the transfer learning method using the layer importance index significantly improved the average classification accuracy compared with fine-tuning. CNNs have become indispensable owing to various applications, and thus, this study aimed at further enhancing its capabilities is relevant to the present and coming times.

Keywords: Transfer Learning, Deep Learning, Image Processing and Pattern Recognition
Abstract: Fine-grained visual categorization is challenged by limited training data by localizing discriminative regions and learning diverse features. We propose an effective regularization method that simultaneously imposes spatial consistency and feature diversity on CNN feature maps from a unified perspective. The former guides different feature map channels to concentrate collaboratively on the discriminative areas while the latter ensures that the feature maps are diverse. The proposed method does not require additional supervision, and leverages the covariance matrix of multi-channel feature maps to regularize the loss at the last convolutional layer where the semantic information is the richest. This allows the influence to be backpropagated to update all convolutional layers. We perform experiments using four network architectures for transfer learning from two source domains to three target domains, and demonstrate that our regularization method improves accuracy in all different settings. The proposed regularization method achieves state-of-the-art performance on CUB-200-2011, Stanford-Cars and Stanford-Dogs datasets with 89.8%, 94.6% and 88.5% accuracy, respectively.

Keywords: Machine Learning, Computational Intelligence, Expert and Knowledge-Based Systems
Abstract: This paper proposes Random Walk-steered Majority Undersampling (RWMaU), an undersampling approach to address the class imbalance problem for binary classifiers. RWMaU is focused to find the majority points which lie at the overlapped region of the minority and the majority classes. Such points meddle with the learning and detection of the minority points. RWMaU uses random walks to mark the majority points satisfying the above characteristic in a non-parametric fashion. For each majority point, a proximity score is calculated on the basis of --- the visit frequencies and the order of visits of the majority points in the random walks. This score is used to perceive the closeness of the majority class points and the minority class. The points lying close to the minority class are subsequently undersampled. Empirical evaluations on 21 datasets using 3 classifiers demonstrate substantial improvement in performance of RWMaU over existing methods for addressing class imbalance and show that it is an efficient and effective way to address class imbalance in binary classification problems.

Keywords: Image Processing and Pattern Recognition, Deep Learning, Neural Networks and their Applications
Abstract: The presented work is guided with the motivation of understanding the deep-learning based 3D reconstruction process for applications in aerial close-range photogrammetry. Given the highly dynamic nature of such a setting, the accuracy and understanding of traditional reconstruction methods as well as the generalization capabilities of deep learning-based methods is required. However, the state-of-the-art methods are typically inadequate. The presented work demonstrates a two-part machine learning-based approach that rely on autoencoder-like models. The first is a Sparse AutoEncoder (SAE) that takes a single image as input and reconstructs a 3D voxel grid. The input images are then sorted based on the reconstruction quality of the SAE output. The second is a Variational AutoEncoder (VAE) that processes multiple images sampled from the ordered set to generate an enhanced 3D voxel grid. The work highlights a novel approach to 3D model reconstruction and presents insights to the process of 3D reconstruction from single image inputs. The autoencoders are trained on a dataset comprised of multiple objects with images captured from different zenith and azimuth angles, simulating an aerial vehicle viewpoint. We show the efficacy of the proposed approach by reconstructing a 3D voxel grid on a ModelNet40 dataset class.

Keywords: Cloud, IoT, and Robotics Integration, Intelligent Internet Systems, Multimedia Computation
Abstract: Precise real-time prediction of the number of future network attacks cannot only prompt cloud infrastructures to fast respond to them and protect network security, but also prevent economic and business losses. In recent years, neural networks, e.g., Long and Short Term Memory (LSTM) and Temporal Convolutional Network (TCN), have been proven to be suitable for predicting time series data. In addition, attention mechanisms are also widely used for time series prediction. In this work, we propose a novel hybrid deep learning prediction method by combining the capabilities of the Savitzky-Golay (SG) filter, TCN, Multi-head self-attention, and BiLSTM for the prediction of network attacks. This work first adopts the SG filter to eliminate noise in the raw data. In addition, it applies TCN to extract short-term features from the sequences. Furthermore, it adopts multi-head self-attention to capture intrinsic connections among features. Finally, this work adopts BiLSTM to extract bi-directional and long-term correlations in the sequences. Experimental results with a real dataset show that the proposed method outperforms several typical algorithms in terms of prediction accuracy.

Keywords: Cloud, IoT, and Robotics Integration, Intelligent Internet Systems, Application of Artificial Intelligence
Abstract: Currently, cloud computing service providers face big challenges in predicting large-scale workload and resource usage time series. Due to the difficulty in capturing nonlinear features, traditional forecasting methods usually fail to achieve high performance in predicting resource usage and workload sequences. Much noise implicit in original sequences of resources and workloads to predict is another reason for their low performance. To address these problems, this work proposes a hybrid prediction model named SABG that integrates an adaptive Savitzky-Golay (SG) filter, Attention mechanism, Bidirectional and Grid versions of Long and Short Term Memory (LSTM) networks. SABG adopts an adaptive SG filter in the data preprocessing to eliminate noise and extreme points in the original time series. It uses bidirectional and grid LSTM networks to capture bidirectional features and dimension ones, respectively. Then, it utilizes an attention mechanism to explore the importance of different data dimensions. SABG aims to predict resource usage and workloads in highly variable traces in cloud computing systems. Extensive experimental results demonstrate that SABG achieves higher-accuracy prediction than several benchmark prediction approaches with datasets from Google cluster traces.

Keywords: Cloud, IoT, and Robotics Integration, Intelligent Internet Systems, Application of Artificial Intelligence
Abstract: With the rapid development of cloud computing technologies, more and more individual users and enterprises choose to deploy their key applications in green data centers (GDCs), and the scale of GDCs is increasing rapidly. To ensure service quality and maximize the revenue, cloud service providers in GDCs need to reasonably and efficiently allocate computing resources and schedule tasks of users. Traditional heuristic algorithms face challenges of uncertainty and complexity in GDCs for scheduling tasks. To solve them, this work establishes an improved resource allocation and task scheduling method based on deep reinforcement learning. It considers the dependency among different tasks, and builds a workload model based on the real-life data in Google cluster trace. In addition, a deep reinforcement learning-based scheduling model is proposed to reasonably allocate and schedule resources (CPU and memory) in GDCs. Based on two models, an Improved Deep Q-learning Network (IDQN) is proposed to autonomously learn the changing environment of GDCs, and yield the optimal strategy for resource allocation and task scheduling. Real-life data-based experiments demonstrate that IDQN achieves lower task rejection rates and energy cost than several typical task scheduling methods.

Keywords: Computational Intelligence, Evolutionary Computation, Heuristic Algorithms
Abstract: Abstract—With the rapid development of science and technology, a large amount of electronic waste is inevitably generated from various discarded and End-Of-Life electronic products. If these products are not handled properly, they can cause environmental pollution as well as loss of resources. As an important part of remanufacturing, disassembly is usually done manually with low efficiency and high labor cost. In this paper, parallel disassembly lines with multiple robots are proposed. These robots can run automatically and be used to perform disassembly in an optimal disassembly mode. A multi- type robot can be flexibly set with multiple functions. A mathematical model is established to assign disassembly tasks to the robots such that a line can achieve the maximum profit and minimum carbon emissions. An improved multi-objective multi-verse optimizer is proposed and applied to a set of instances. Experimental results show that the algorithm has an overwhelming performance advantage over the other three commonly-used algorithms in solving this problem. It has better performance than the other peer algorithms in solving parallel disassembly line balancing problems.

Keywords: Computational Intelligence, Evolutionary Computation, Heuristic Algorithms
Abstract: 摘要 - 如果人们只是简单地丢弃他们的旧产品，他们将面临许多问题，例如对环境的污染和资源浪费。因此，回收和再制造废旧产品是必要的，这使得研究拆卸线平衡问题变得非常重要。目前，手动拆卸很流行，在发生危险的拆卸零件时，它并不能保证人身安全。针对这一问题，提出一种人机混合拆解方法。采用基于强化学习的改进Q-learning算法，求解双侧拆解线平衡问题，最大限度缩短总拆装时间。将改进后的算法与Sarsa算法进行了比较。结果表明，它可以找到比Sarsa更好的解决方案，并且在大规模病例中表现优于Sarsa。

Keywords: Kansei (sense/emotion) Engineering, Information Visualization
Abstract: To differentiate from competitors, it has become increasingly important to design products based on the “Kansei value,” which impresses and inspires consumers. However, the perceptual indices of products are generally designed qualitatively, which is not only time-consuming but also costly. Therefore, in recent years, studies have been conducted to discuss the relationship between some perceptions by applying multivariate analysis and machine learning to the adjectives related to perception and affective response. Nonetheless, determining the structure that expresses the relationship of the adjectives is obtained by trial and error, based on the hypothesis of the researchers. Therefore, we aimed to investigate a method for mechanically exploring some semi-optimal structures without a hypothesis of the model structure by applying a genetic algorithm to the construction of adjective relationships using structural equation modeling. In this study, we prepared a four-layered model according to the human perception process for eight categories of material samples and constructed a relationship from a total of 20 words perceived from each sample. Consequently, we could construct a perceptual model that can be interpreted quantitatively and semantically using the proposed method. The advantage of this technique is that it can be used to construct important structures that might be overlooked.

Keywords: Kansei (sense/emotion) Engineering
Abstract: With the blossoming of online social networks (OSN), personality analysis based on OSN texts has gained much research attention in recent years. The previous methods mainly focus on human-designed features extracted through psychological dictionaries or semantic features extracted through language models. However, the shallow statistics features can not fully convey the personality information and the language models can not capture enough psychological background knowledge. Besides, the lack of large labeled datasets has been a serious obstacle impending further research. To tackle these problems, we propose a personality analysis model, namely PerKG, which combines personality knowledge graph and heterogeneous graph representation learning to exploit external knowledge from psycholinguistics and learn the group-level information to predict users' personalities accurately. Specifically, we construct a personality knowledge graph based on existing psycholinguistics knowledge. And then, for each user, we align the user information with the knowledge graph to obtain the personality heterogeneous graph. Finally, the personality vector of each entity node is learned for prediction by designing a walk strategy on the personality heterogeneous graph. Detailed experimentation shows that our proposed PerKG architecture can effectively improve the performance and alleviate the label sparsity problem of personality analysis.

Keywords: Kansei (sense/emotion) Engineering, Human Factors
Abstract: The quantitative evaluation of human skill is an important research topic in the development of autonomous ship systems, which must possess the ability and skill of an individual. Furthermore, a physiological index cannot be used for evaluation because the individual character includes their results. However, in this paper, we propose a quantitative evaluation based on saliva. The evaluation of inside response is an important factor in understanding the skills and abilities of an individual. We measured saliva cortisol, which is an ideal index for evaluating stress, and applied it in a simulator-based experiment.

Keywords: Human-centered Learning, Human-Machine Cooperation and Systems, Assistive Technology
Abstract: To maintain ship navigation safety, the navigator must understand and predict the movements of other ships. Radar images include both ship (target) and non-ship images (noise), such as sea clutter. To understand the movements of other ships, navigators must detect ship images from radar images manually. To maintain the optimum situation awareness of navigators while reducing the workload, a function that can automatically detect and track ship images, including small ships, is desired on the radar. Therefore, a system to predict ship movements using machine learning and radar images is proposed in this study. The proposed prediction system is based on a learning model developed using a denoising convolutional autoencoder. The learning and validation data are radar images processed via image processing in advance. The prediction accuracy of ship movements in this study is 90.97%, and the loss is 0.0396.

Keywords: Interactive Design Science and Engineering, Kansei (sense/emotion) Engineering, Assistive Technology
Abstract: When driving becomes fully automated, media browsing and PC work in the car are expected to increase, which is likely to cause car sickness. In this study, we investigated a method to assess car sickness objectively and quantitatively by inducing sickness with a driving simulator (DS) and examining the changes in physiological indices with low-burden measurement devices. Ten participants were asked to ride DS and read a book there, and subjective ratings of sickness and drowsiness, electrocardiogram, respiration, and skin conductance were obtained. Experiments were conducted under two conditions: with sinusoidal vibration at a roll angle of 0.2 Hz (A) and without vibration (B). Nine out of ten participants experienced mild subjective sickness, although no statistically significant differences between conditions were obtained. The principal component analysis on physiological indices was conducted on seven participants across conditions; two were excluded due to arrhythmia and one was due to no sickness. In addition, skin conductance was excluded from the analysis because of measurement deficits. As a result, the first component related to respiratory instability was commonly obtained in all, and the principal component loadings were similar for all but one of the participants. This component increased with moderate sickness and drowsiness. The second component was related to mean heart rate and heart rate variability. Not only there were large individual differences in the loading patterns of this component and its principal component scores showed complex changes depending on the degree of sickness and drowsiness. Thus, although the process of sickness is complex and differs among individuals, the combination of changes in these two principal components may be used to assess the process. Further studies with a larger number of participants are needed to develop a method to assess sickness and drowsiness separately and to address individual differences.

Keywords: Human-Computer Interaction, Human-Machine Interface, User Interface Design
Abstract: Conversational Intelligent Tutoring Systems (CITS) in learning environments are capable of providing personalized instruction to students in different domains, to improve the learning process. This interaction between the Intelligent Tutoring System (ITS) and the user is carried out through dialogues in natural language. In this study, we use an open source framework called Rasa to adapt the original button-based user interface of an algebraic/arithmetic word problem-solving ITS to one based primarily on the use of natural language. We conducted an empirical study showing that once properly trained, our conversational agent was able to recognize the intent related to the content of the student’s message with an average accuracy above 0.95.

Keywords: Human-Computer Interaction, Human-Machine Cooperation and Systems, Intelligence Interaction
Abstract: The 2-dimensional cost map has been widely used for the navigation of ground mobile robot. Although it is effective when the ground is flat, it becomes clumsy and ineffective when the ground has some slopes, where such slopes are often misjudged as the forbidden area by the cost map. For this reason, we propose a slope-adaptive navigation approach based on multi-layer cost map in this paper. Instead of taking the point cloud of slope as obstacles, we actively construct a multi-layer cost map that takes slope information into the map in the stage of building environment map. A slope detection algorithm is developed to switch the cost map during the robot navigation. The slope is then considered as a passable road, only with extra cost. In the case that the slope leads the robot to a new floor, we adopt the Aruco code to switch the map information, such that the navigation can still keep working. Both simulation and real-world experimental results demonstrate the high effectiveness of our proposed approach.

Keywords: Human Factors, Human-centered Learning, Human-Computer Interaction
Abstract: Advanced language models like BERT have great performances in various Natural Language Processing tasks. However, recent researches have shown that language models can learn gender biases from corpus, and lead to discriminatory decisions and unfair allocation of resources. We proposed a measure of gender bias in BERT resume retrieval system, by performing job searches on a group of resumes with different genders but consistent abilities. We proposed to calculate the average ranking and Discounted Cumulative Gain (DCG) scores of male and female resumes, and found that men outperformed women even though the two resumes were identical except for gender. This shows BERT has gender stereotypes, and its resume retrieval systems prefer male candidates. Therefore, we also proposed a regularized debiasing method to promote gender equity. Referring to Densifier method, we can get the subspace vector encoding bias semantics through matrix transformation of word vector difference between occupational and gender words. By defining the correlation between BERT word vector and gender bias subspace as the loss term, we can remove the bias semantics in BERT, and also avoid it learning stereotypes even trained on an unfair data set. After regularized debiased, the gender ranking gap of BERT was reduced by an average of 61.8%, while DCG scores were reduced by 53.9%.

Keywords: Intelligence Interaction, Human-Computer Interaction, Wearable Computing
Abstract: Scoliosis is a complex three-dimensional deformity of the spine and trunk, which often occurs in healthy adolescents. Although most adolescent scoliosis doesn’t have clinical symptoms, scoliosis may develop into rib deformity and respiratory system damage, which seriously affects the life quality of patients. At present, the scoliosis rehabilitation training based on Schroth method has been proved to be effective in slowing down the progression of scoliosis. In this work, we present a real-time practical wearable system called scoliosis rehabilitation assistant (SRA). The system can monitor patients for scoliosis rehabilitation training through inertial measurement unit (IMU), and provides supporting rehabilitation training games. After the training, the training data will be transmitted to the cloud database in real time, providing insights for the future development of patients. The result from a user study demonstrated that our system has great potential to become an effective solution to help patients with spinal rehabilitation training and reduce the workload of scoliosis doctors.

Keywords: Brain-based Information Communications, Human-Machine Interface, Human-Computer Interaction
Abstract: The susceptibility of electroencephalography (EEG) signal to artifacts is considered a major obstacle preventing the deployment of relevant non-invasive neurotechnology. In spite of a large body of literature dedicated to the identification, rejection and removal of artifactual components in EEG, the study of the impact that different artifacts may have on the EEG signal properties has been mostly qualitative and focused on the source (e.g. muscle activity, electromagnetic interference) rather than the function generating them. This work takes advantage of a unique dataset where EEG of 12 participants elicited during the execution of 9 common human activities (e.g., speaking, blinking, etc.) is co-registered with electromyography (EMG), electrooculography (EOG), accelerometer and gyroscope sensors, and baselined to "resting" (artifact-free) intervals to allow an exact, quantified assessment of the impact of artifacts. We examine several metrics capturing different facets of the influence of artifacts on EEG and measure the extent to which a state-of-the-art artifact removal method is able to eliminate them. In addition to an in-depth, quantified profiling of functional EEG artifacts, our work provides valuable information for precisely tuning the hyper-parameters of artifact rejection and removal algorithms and for designing realistic brain-computer interface (BCI) applications.

Keywords: Electric Vehicles and Electric Vehicle Supply Equipment, Cooperative Systems and Control, Consumer and Industrial Applications
Abstract: The autonomous vehicle (AV) has been studied by many researchers recently because of its valuable points in transportation, aviation, military, smart city, and aerospace. The model predictive control (MPC) is employed to track the artificial intelligent regenerated motion signals with higher accuracy than other error- and model-based controllers as it can consider the constraints of the system in extracting the optimal solution. However, the accuracy and applicability of the MPC rely on the MPC horizons, including prediction and control horizons. The higher prediction horizons mean a higher computational load of the system, which reduces the real-time applicability of the system. On the other hand, a higher prediction horizon increases the system's stability in facing abrupt motion signals. In addition, higher control horizons mean more dexterity in the system facing an unknown situation. On the other hand, a longer control horizon increases the computational load of the system exponentially. This study employs particle swarm optimisation (PSO) to extract the optimal MPC horizons considering the accuracy and computational load. The cost function is defined to increase the accuracy of the longitudinal time-varying velocity tracking, decrease the lateral deviation, decrease the relative yaw angle and decrease the computational load of the system. It should be noted that the lateral deviation and relative yaw angle are extracted using the vehicle four wheels dynamic model in order to evaluate the AVs’ passenger motion comfort. The proposed method is designed and developed under MATLAB/Simulink. The extracted optimal MPC horizon is compared with some other arrangements of the MPC horizons to prove the efficiency of the proposed method compared with the trial-and-error method.

Keywords: Intelligent Transportation Systems
Abstract: Accurate traffic prediction is critical to the effectiveness of intelligent transportation systems. However, traffic data are highly nonlinear with complicated dynamic spatio-temporal correlations, accurate traffic forecasting, particularly long-term forecasting, remains a difficulty. Existing models generally learn a fixed graph to capture spatial correlations, which makes them difficult to effectively capture changing spatial dynamics over time, resulting in poor prediction outcomes. To tackle these challenges, we propose a new spatio-temporal graph convolution network model, named Spatio-Temporal Attention-based Graph Convolution Network (STAGCN), which jointly fuses dynamic evolving spatial correlations and long-term temporal correlations to improve the performance. First, we design a spatio-temporal multi-head self-attention module to capture both spatial heterogeneity and temporal correlations. Second, we propose an adaptive evolving graph convolution module that can learn a new graph at each time step and make STAGCN evolve dynamically. Meanwhile, self-attention is used to construct a dynamic adjacency matrix to further capture spatial correlations. In addition, we optimize the original Transformer by employing relation-aware attention mechanism to make it better suitable for time series prediction, thereby improving the long-term prediction performance of the model. Extensive experiments are conducted on two real-world datasets, demonstrating that our proposed model achieves state-of-the-art performance and consistently outperforms other baselines.

Keywords: Intelligent Transportation Systems
Abstract: A large volume of commuting private cars cause serious traffic congestion, especially during morning and evening rush hours, and the low occupancy rate of commuting private cars bring a huge waste of resources. Carpooling among commuting private cars can reduce vehicle volume and alleviate traffic congestion. Moreover, commuters matching is the key issue to solve for realizing stable carpooling. This paper designs a commuting trajectory based stable carpooling model (CT-CSC) for commuting private cars, and proposes a particle swarm-based commuter matching approach for stable carpooling (PSCMA). The objective of CT-CSC model is minimizing carpooling cost. In the proposed PSCMA, the particle swarm and fitness calculation rules are redesigned to make it suitable for the CT-CSC model. The real-world RFID electronic identification data in Chongqing is used for experimental verification. Experimental results show the effect of the parameter inertia factor ω for the PSCMA. In addition, compared with genetic algorithm, hill climbing algorithm and simulation annealing algorithm, the performance of the PSCMA is outstanding. Furthermore, 1,003 commuters passing through the Huanghuayuan Bridge in Chongqing are selected to carpool, and we analyzed the reduction of the number of commuters, mileages and gasoline

Keywords: Trust in Autonomous Systems, Modeling of Autonomous Systems, Intelligent Transportation Systems
Abstract: The 21st century is the age of automation. The automotive industry is converging towards deployment of complete automation by 2030. But are humans ready for it, or will they be hesitant to adopt it due to the lack of trust? To safeguard future autonomous mobility, robust run-time trust assurance and assessment is necessary. One strategy that is so far under-explored is rooted in involving the intelligence inside the autonomous agents, which could be directed towards detection of trust-breaking behavior in other agents so that problematic vehicles are reported before they can engage in harmful behavior.

To support the progress in this direction, we propose a peer-to-peer model-based run-time trust assessment method, employing the model in terms of a Digital Twin for an autonomous vehicle (drone in our case) to ensure the trusted execution of intelligent agents. In this research, we examine the role of the Digital Twin in the trust-building scenario and design a proof-of-concept model of a Digital Twin. To illustrate the approach, we present a case study of an autonomous-drone food delivery system and use formal approaches such as Petri Nets and Finite State Machines (FSM) to evaluate the scenario and demonstrate how trust could be built among autonomous drones or other vehicles.

Keywords: Homeland Security, Decision Support Systems, Intelligent Transportation Systems
Abstract: Trajectory prediction of a moving object has imperative significance in both research and practical applications, ranging from target tracking, security surveillance and autonomous vehicle driving. For improving the efficacy of such prediction, a novel approach of data stream learning coupled with Kalman-filter and evolving correlated horizons feature selection (KF-ECH-FS) is proposed. KF has traditionally been used as a control-feedback-loop mechanism that corrects the errors from past trials, to predict the next-step position in trajectory prediction. In our fusion model here, KF and its windowed version are being used as predictor variables in a multivariate time series forecasting process. The predictor variables which serve as additional features aid in improving the trajectory prediction when only the relevant features are being selected in the incremental learning manner by multi-variate data stream analytics. Our proposed ECH-FS solves the problem of model overfitting when many features after expansion by time-series windowing are evaluated and selected along the learning process. A simple and efficient feature selection heuristics, Auto-encoder is used, along with data stream learning by Gate Recurrent Unit. The results, through an experimentation over a sample case of camera surveillance of accident prevention, show that our proposed KF-ECH-FS is superior to either KF or windowing alone in 1-step horizon trajectory prediction.

Keywords: Application of Artificial Intelligence, Deep Learning, Neural Networks and their Applications
Abstract: Thermal Comfort Data is critical to generate machine learning models for efficient heating and cooling systems. However, thermal comfort datasets are often highly imbalanced due to subjective user feedback, thus making it challenging to accurately predict both majority and minority classes. This demands the use of data synthesis techniques prior to training classification models to balance the datasets. Commonly used techniques like Synthetic Minority Over-sampling Technique (SMOTE) or Adaptive Synthetic Sampling Method (ADASYN) often compromise testing accuracy and more sophisticated techniques like Conditional Wasserstein Generative Adversarial Network with gradient penalty (cWGAN-GP) are significantly expensive to train. In this paper we propose a novel data augmentation algorithm called Conditional Classifier-Generator (cCGen) to address these two issues. We evaluated the performance of cCGen with real thermal comfort data against SMOTE, ADASYN and cWGAN-GP at different imbalance ratios. Our experiments reveal that our approach can produce better F1 scores than other sampling methods while being more than 10 times faster than cWGAN-GP and not compromising test accuracy.

Keywords: Machine Learning, Deep Learning, Evolutionary Computation
Abstract: We address adversarial attacks on the actuators at the joints of legged robots trained by deep reinforcement learning. The vulnerability to the joint attacks can significantly impact the safety and robustness of legged robots. In this study, we demonstrate that the adversarial perturbations to the torque control signals of the actuators can significantly reduce the rewards and cause walking instability in robots. To find the adversarial torque perturbations, we develop black-box adversarial attacks, where the adversary cannot access the neural networks trained by deep reinforcement learning. The black box attack can be applied to legged robots regardless of the architecture and algorithms of deep reinforcement learning. We employ three search methods for the black-box adversarial attacks: random search, differential evolution, and numerical gradient descent methods. In experiments with the quadruped robot Ant-v2 and the bipedal robot Humanoid-v2, in OpenAI Gym environments, we find that differential evolution can efficiently find the strongest torque perturbations among the three methods. In addition, we realize that the quadruped robot Ant-v2 is vulnerable to the adversarial perturbations, whereas the bipedal robot Humanoid-v2 is robust to the perturbations. Consequently, the joint attacks can be used for proactive diagnosis of robot walking instability.

Keywords: Evolutionary Computation, Deep Learning, Machine Learning
Abstract: We propose an evolutionary computation method based on deep reinforcement learning to determine the vulnerability to adversarial attacks (such as corrosion and defects caused by collisions) on the length and thickness of parts of legged robots. This type of attack changes the robot's body shape and interferes with walking; we call the attacked body the adversarial body shape. The proposed evolutionary computation method searches adversarial body shape by minimizing the expected cumulative reward earned through walking simulation. To evaluate the effectiveness of the proposed method, we performed experiments with three different legged robots (Walker2d, Ant-v2, and Humanoid-v2) in OpenAI Gym. The experimental results reveal that Walker2d and Ant-v2 are more vulnerable to attack on the length than on the thickness of the body parts, whereas Humanoid-v2 is vulnerable to attack on both the length and thickness. We further identified that the adversarial body shapes break left-right symmetry or shift the center of gravity of the legged robots. This method of finding adversarial body shapes can be used to proactively diagnose the vulnerability of legged robot walking.

Keywords: Image Processing and Pattern Recognition, Machine Vision, Media Computing
Abstract: The emotion of human beings tends to be complex in real conditions, generating compound expressions in human faces. Compound expression recognition is an important challenge for the assessment of human complex emotion. The recognition system based on six basic expressions cannot meet the demand of compound expressions recognition. The recognition performance of models learned from the basic expressions is poor due to the small number of compound expression datasets with highly accurate labels and insufficient sample diversity.Making full use of domains outside of compound expressions in small sample datasets will help promote diversity. We propose the Multi-Domain Fusion Generative Adversarial Network (MDFGAN), which innovatively fuses the face domain, compound expression domain and basic expression domain to obtain rich expression generation capability and high accuracy recognition. Pairing the face domain and the contour-unrelated compound expression domain in the generator will expand the sample diversity. The contour-related compound expression domain and the basic expression domain will jointly improve the expression recognition accuracy of the discriminator. Finally, we conducted comprehensive experiments on CFEE-26, CFEE-7 and CK+. Compared with the baseline approach of CFEE-26, the results of MDFGAN improved 6.79% on UF1 and 8.5% on UAR.

Keywords: Deep Learning, Representation Learning, Machine Learning
Abstract: Teaching machines to understand human language is one of the most elusive challenges in artificial intelligence. Machine reading comprehension is a crucial task in evaluating how computer systems understand natural language. This study presents a machine reading comprehension model based on external knowledge. We use a framework named K-Adapter to infuse two kinds of external knowledge with two specific adapters. This model can capture richer semantic information, which is more suitable for real application scenarios. The proposed model is evaluated on the COSMOS QA dataset and outperforms the competitive baselines.

Keywords: Human Factors, Assistive Technology
Abstract: Recent advances in autonomous driving systems raise new questions about how to enhance the communication and takeover control between the system and the driver. Eye tracking technologies have shown their feasibility to recognize whether the driver’s gaze is directed ‘on-road’ or ‘off-road’. However, this binary information alone is not sufficient to infer the driver’s engagement in the driving task. In the present work, we take the next step and investigate how driving modes (autopilot, navigation system, and printed map) associated with different levels of engagement can be categorized from drivers’ gaze patterns. Using gaze data recorded in these three driving tasks along with several state-of-the-art machine learning methods, we demonstrate that the driving modes are associated with different gaze patterns. We achieved an average accuracy of 90.1% for binary and 80.3% for multi-class driving mode classification. Our findings pave the way for enhancing driver monitoring systems in (semi-) autonomous cars.

Keywords: Human-Computer Interaction, Intelligence Interaction, Assistive Technology
Abstract: Sensing human activity via WiFi Channel State Information (CSI) has considerable application prospects in future intelligent interaction scenarios such as virtual reality, intelligent games, metaverse, etc. Recently, many Deep Learning-based WiFi sensing schemes have been proposed in the literature, which gained high accuracy for a wide range of simple activities such as standing, squatting, and bending. However, the performance will be suffered when existing approaches are used to recognize drastic activities, such as actions in vigorous sports. This is mainly due to the reason that the spatiotemporal information of these actions is not well utilized. To overcome this drawback, we propose a novel DL-based WiFi sensing method for drastic activity recognition by combining the Convolutional Neural Network (CNN) and the Bidirectional Long Short-Term Memory (BiLSTM) network. The designed CNN-BiLSTM architecture is in parallel with feature extraction, which can simultaneously extract sufficient spatiotemporal features of action data and establish the mapping relationship between actions and CSI streams, thereby improving the accuracy of activity recognition. The CNN is used to extract information on the spatial dimension, while the BiLSTM extracts information on the time dimension. To verify the performance of the proposed scheme, we build a hardware experiment platform and constrain a dataset with 1400 pieces of records for 7 classes of basketball actions. After training over the dataset, the proposed CNN-BiLSTM scheme achieves 96% experimental accuracy on the test set, which is better than the benchmark methods.

Keywords: Human Factors, Assistive Technology
Abstract: Various kinds of level 2 driver assistance systems have been commercialized in the market to reduce traffic fatalities. However, there is a concern that if drivers ware assisted by such systems for a long of time, their level of alertness would tend to decline. To resolve these kinds of issues, most existing research has focused on preventing excessive dependence on driving assistance and arousal reduction caused by the assistance. In this paper, we focus on the driver excitement in the level 2 automated driving (advanced driving support) and propose a method for estimating the excitement using the SD (semantic differential) method. We also show that there is a clear correlation between excitement and arousal levels during driving with a level 2 driver assistance system. We evaluate the effectiveness of our proposal through a driving simulator-based experiment on the Tokyo Metropolitan Expressway.

Keywords: Information Visualization, Assistive Technology, User Interface Design
Abstract: With the development of big data, people focus on data security and privacy, blockchain has been paid more and more attention. Compared to pre-2013, the number of blockchain literature publications has increased. Scholars from different research fields have deeply explored blockchain technology. In this paper, we collected blockchain literature data from 2013 to 2020, and then we conducted a quantitative analysis of publications, author groups, research content and future research content. For the current research status, we also discussed the problems facing blockchain. We started by mining representative authors, and revealed current status of blockchain research, author's team cooperation, and blockchain's development content. We can provides coordinate and inspiration for new starters, and give coordinate for the future development of blockchain.

Keywords: Human Factors, Assistive Technology, Human Performance Modeling
Abstract: Quality of emergency services depend on the effectiveness of emergency vehicle fleets (e.g., police, fire trucks, and ambulances). Traditionally, these fleets are composed of emergency crews that must attend highly stochastic incident demand within a short target arrival time. Incidents must be attended immediately, which prevents that an a priori computed work break schedule is executed as planned and many crews may be left without a break during prolonged periods of time. Extended focused work periods decrease efficiency with related decline of attention and performance. Therefore, break schedule should be regularly updated on the fly to allow frequent and sufficiently long time for rest. In this paper, we propose a dynamic algorithm for on the fly work and break balancing for crews in emergency fleets. Based on the historical intervention data, the algorithm (re)arranges vehicles' crews' work breaks as the time evolves considering individual crews' preferences. Moreover, it dynamically reallocates stand-by vehicles for improved coverage of a region of interest. We show the performance of the proposed algorithm on two simple functional examples.

Keywords: Cooperative Systems and Control
Abstract: In safety-critical applications, where several mobile robots and autonomous agents are being utilized for a mission, a fault-resilience behavior of the system is necessary. The fault resilience mechanism mostly uses the robot's redundancy and tasks reassignment to recover malfunctioning and increase operating efficiency. The E-CARGO (Environments - Classes, Agents, Roles, Groups, and Objects) model designed for the Role-Based Collaboration (RBC) approach has been used successfully on cooperative Multi-Robot Systems (MRSs). Role-based characteristics of E-CARGO will facilitate cooperative decision-making and simplify handling failure. This paper develops an extended E-CARGO model for a fault resilience role engine. Agents use factor graphs to update the process role and manage the potential failure in each time step. We apply hybrid control in this paper. By "hybrid" we mean that evaluating and assigning initial roles are centralized, and role-playing is decentralized based on the local observations. The RBC life cycle and a Bayesian consensus will maintain fault resilience behaviors. Potential failure can be identified in a Bayesian way by updating agents' reliability and calling the central unit to assign new process roles to guarantee robustness. Simulation experiments show that the proposed role engine can increase performance and tolerate failures in multi-robot path planning scenarios.

Keywords: Quality and Reliability Engineering, Distributed Intelligent Systems
Abstract: The task of assessing, grouping and arranging data into meaningful groups or clusters based on their similarities / dissimilarities measures known as cluster analysis. Thereby, there are numerous clustering algorithms: hierarchical and partitional. In the last decade, clustering using bio-inspired algorithms received more attention, specifically the ant clustering algorithms. Regardless, they have required a lot of processing power due to the massive amount of data that has been generated during the last years. As a consequence, determining the computational cost of these algorithms is one of the most interesting tasks in the quest for optimal clustering solutions in a real-time system. This study presents a research guide for the researchers working in the same field. A series of experiments are elaborated to investigate the computational complexity of the most promising algorithms applied to students grouping problem. The results indicate two challenges that arise when using ant clustering algorithms: the difficulty in adjusting parameters and extended computation time.

Keywords: Cooperative Systems and Control, Decision Support Systems
Abstract: With the frequent occurrence of disasters, such as war, storms, hurricanes, and earthquakes, post-disaster rescue is particularly important. Based on the principle of “life first”, life detection and rescue have become the primary task of post-disaster rescue. In this paper, the E-CARGO (Environments – Classes, Agents, Roles, Groups, and Objects) model is used to formalize the life detection and rescue problem. With the help of the idea of the convex hull algorithm, a Hierarchical Coverage Based on Square Algorithm (HCBS) is proposed to achieve the maximum coverage of the rescue area with the minimum number of detection areas, and then a rapid assignment of UAV life detection and rescue with minimized rescue cost is realized through group role assignment (GRA). Using the PuLP extension library of Python, we implement the proposed algorithm. The experiments show that the proposed method is fast and effective.

Keywords: Decision Support Systems
Abstract: In many practical cases, the original task is divided into smaller, easier-to-complete tasks and assigned to different groups. Group role assignment (GRA) is dedicated to optimizing the performance of a group, which is not applicable to the multi-group role assignment (MGRA). Moreover, in dynamic scenes, the agents' capabilities change over time, further complicating the problem. Based on the emerging and promising role-based collaboration (RBC) theory and its E-CARGO (Environments - Classes, Agents, Roles, Groups, and Objects) model, we formulate the adaptive MGRA problem, and propose a novel current state-based MGRA (CSB_MGRA) algorithm to keep the entire team productive. The constraints of the tasks are not the same due to their diverse characteristics and needs. Moreover, team members do not necessarily remain the same in the whole process, and staff transfers may occur between groups. A constant assignment scheme is not guaranteed to maximize team performance. Therefore, the constraints of different groups are set to be different, and re-assignments of the whole team are considered in the construction of CSB_MGRA. The experimental results prove the practicality of the solution proposed in this paper.

Keywords: Control of Uncertain Systems, Consumer and Industrial Applications, Manufacturing Automation and Systems
Abstract: In order to meet the growing demand for curtain wall cleaning UAVs in the market and improve the shortcomings of current manual cleaning and robot and UAV cleaning design. In this paper, a new type of curtain wall cleaning UAV is designed and developed. Firstly, the system structure of curtain wall cleaning UAV is designed, and the dynamic and kinematic models of curtain wall cleaning UAV are established. The cascade PID control algorithm is designed, and the outer position loop and inner attitude loop are used to control the curtain wall cleaning UAV. Finally, the control algorithm is verified by simulation experiment with MATLAB / Simulink. The simulation curve shows that the cascade PID control algorithm can meet the requirements of fast response speed and high stability of UAV, and has good anti-interference ability.

Keywords: Biometric Systems and Bioinformatics, Machine Learning, Application of Artificial Intelligence
Abstract: Malocclusion has a high prevalence in the population, which seriously affects patients' oral and mental health. Angle's classification is a widely accepted diagnostic standard for malocclusion, either requiring professional intervention and complicated procedures, or increasing radiation risks. This paper proposes a new method of Angle's classification based on occlusal contact information to realize the automatic Angle's classification. Firstly, a novel bite force measurement device is used to record the occlusal data of subjects with different occlusal categories, Meta-analysis evaluated several occlusion quantitative evaluation indicators. Then, the imbalance of the data set is improved by oversampling, and popular machine learning models are used for training and performance evaluation. The result shows that the accuracy of the random forest model combined with occlusal contact information reaches 87.83%, and the performance of other evaluation indexes is good. It is demonstrated that machine learning models can be applied to Angle's classification and shows the great potential of occlusal contact information in the aided diagnosis of oral diseases.

Keywords: Neural Networks and their Applications, Deep Learning, Application of Artificial Intelligence
Abstract: Product backorder is a common problem in supply chain management systems. It is essential for entrepreneurs to predict the likelihood of backorder accurately to minimize a company's losses. However, existing methods are hard to achieve satisfactory results since the number of backorders and non-backorders are extremely imbalanced. Besides, the backorder data's attributes are complex to oversample them effectively. To address these problems, a novel hybrid sampling method is proposed to help predict extremely imbalanced backorder. The Randomized Undersampling (RUS) and a Conditional Wasserstein Generative Adversarial Network (CWGAN) are innovatively introduced into backorder prediction. First, RUS is used to reduce the majority non-backorder samples. Second, CWGAN is served as an oversampling technique to generate high-quality backorder samples. It utilizes unique structures in the generator and the discriminator to effectively model both numerical and categorical variables. Finally, the training dataset is balanced, and the Random Forest Classifier (RFC) is adopted to make backordering prediction. In the experiments of Kaggle's dataset 'Can you predict product backorder?', our proposed method is superior to all benchmark methods in terms of standard evaluation metrics. The results show that our proposed product backorder prediction model is effective.

Keywords: Evolutionary Computation, Deep Learning
Abstract: Scientific articles contain rich knowledge that can significantly assists scientific research, but it is difficult to precisely extract knowledge information due to the complexity of the discourse structure of scientific articles. To provide more accurate scientific research knowledge for researchers in a specific academic domain, it is necessary to study the discourse structure of domain scientific articles and to propose an automatic annotation approach to automatically annotate discourse information from articles. Unfortunately, few works have studied the discourse structure of domain scientific articles and the corresponding automatic discourse annotation. To fill this gap, we take scientific articles of the wastewater-based epidemiology domain as a case to study how to automatically and efficiently annotate discourse information. This paper has three contributions. Firstly, we propose a hierarchical discourse model with two layers to cover all potential discourses in various domain scientific articles. Specifically, the first layer defines four core discourse concepts to describe the main process of a scientific research which can be applied in all scientific articles in various domains. The second layer defines fine-granular domain-specific structure, which can accurately describe the entire research contents of a specific domain. Secondly, based on the proposed model, we build a corpus dataset of 100 annotated scientific articles in the wastewater-based epidemiology domain. Thirdly, based on the model and dataset, we propose a simple yet efficient Top-K resampling-based approach to train a more effective classifier for automatic annotation. Extensive experiments verify the effectiveness and efficiency of our proposed hierarchical discourse model and the Top-K resampling-based classification approach.

Keywords: Application of Artificial Intelligence, Neural Networks and their Applications, Multimedia Computation
Abstract: Long-tail music genre classification is a crucial task in the Music Information Retrieval (MIR) field for identifying the long-tail, data-poor genre based on the related music audio segments, which is very prevalent in real-world scenarios. Most of the existing models are designed for class-balanced music datasets, resulting in poor performance in accuracy and generalization when identifying the music genres at the tail of the distribution. Inspired by the success of introducing Multi-instance Learning (MIL) in various classification tasks, we propose a novel mechanism named Multi-instance Attention (MATT) to boost the performance for identifying tail classes. Specifically, we first construct the bag-level datasets by generating the album-artist pair bags. Second, we leverage neural networks to encode the music audio segments. Finally, under the guidance of a multi-instance attention mechanism, the neural network-based models could select the most informative genre to match the given music segment. Comprehensive experimental results on a large-scale music genre benchmark dataset with long-tail distribution demonstrate MATT significantly outperforms other state-of-the-art baselines.

Keywords: Neural Networks and their Applications, Deep Learning, Representation Learning
Abstract: Data augmentation has been widely introduced into graph-based tasks to improve the generalizability of models. Based on empirical hypothesis, we show that the node in a graph has different importance, the important one is critical for classification task while the unimportant one hurts the performance. However, there are few works in data augmentation addressing the information propagation of nodes with different importance. In this work, we propose a novel graph data augmentation algorithm for graph classification task, called PassAugment, aiming to pass these importance in graph data augmentation. After distinguishing the importance of all nodes in each graph using the saliency map, we design a data augmentation approach including two strategies: (i) randomly adding edges between the important nodes and the other nodes to globally improve the effective information passing, and (ii) randomly removing edges between the unimportant nodes and their neighbors to locally reduce the ineffective information passing. More importantly, our proposed approach as a stand-alone module can be combined with many GNNs architectures. Experimental results on graph classification task show that our approach consistently improves the accuracy and achieves or closely matches the state-of-the-art performance.

Keywords: Machine Learning, Application of Artificial Intelligence
Abstract: Machine learning is broadly used in many intelligent cybernetic systems. With the burgeoning of the communities of AI, the number of machine learning-based models is rapidly increasing, but picking a suitable and optimal (or relatively good) model from overwhelming options has become a conundrum when deploying a new system. Therefore, we are motivated by an intriguing question: Can we automatically select a proper model for new data? However, unsupervised model selection poses two main challenges: (i) Evaluation and comparison of candidate models on the new data are infeasible due to the lack of labels; and (ii) It is non-trivial to build relationships between model performance and data characteristics when the interaction between these characteristics should be considered. In light of these limitations, this paper proposes a factorization machine-based unsupervised model selection method. Following mainstream model selection protocols, we also leverage model performance on prior known datasets. Differently, we learn higher-order complex relationships between model performance and dataset characteristics. Specifically, our method transfers the historical performance into a second-order function of meta-features and embedding weights by harnessing the power of factorization machine. This function can be subsequently used to select a proper model when given a new dataset. Extensive experiments show that our method obtains more superior model selection performance than five state-of-the-art approaches, and our method executes faster than its competitors by approximate three magnitudes.

Keywords: Deep Learning
Abstract: Face detection tasks under the current epidemic prevention situation often acquire images with partial occlusion. General face detectors ignore the challenge brought by occlusion, making it difficult to meet daily needs. In order to address this problem, this paper proposes a real-time occluded face detection network based on the improved CenterNet with information dropping strategy. First, depth separable convolution and attention mechanism are introduced into the backbone to reduce parameters and extract occlusion-robust features. Second, a feature fusion neck is designed to improve the performance of multi-scale face detection. In addition, the data augmentation method with information removal strategy enriches the diversity of occlusion samples. Experiments indicate that our model improves the fps as well as maintains the accuracy.

Keywords: Deep Learning, Machine Vision, Machine Learning
Abstract: Pollen classification plays an essential role in many fields such as medicine and palynology. Notably, manual pollen identification via observing key pollen information (e.g., their contours) is time-consuming and laborious. To date, deep learning methods can extract complex features in an end-to-end manner. However, deep learning based automatic classification methods on pollen grains are still rare, and their performances remain unsatisfactory owing to limitation of interference from irrelevant information (such as impurities and bubbles) and the lack of pollen attention. Based on the above considerations, we propose a contour-guided network called CG-Net, which contains three modules. Image pre-processing module first removes impurities and bubbles in pollen images according to the color information. Then, contour awareness module is designed to generate contour features and these features are served as attention maps for next module. Finally, contour guidance module weights the yielded contour attention maps to both original images and feature maps of different convolution layers, making the CNN focus on discriminative features of pollen grains. Extensive experiments are conducted on several real-world pollen datasets, and the results demonstrate the effectiveness of our proposed method with the accuracy and F1-score over 84%.

Keywords: Evolutionary Computation, Heuristic Algorithms, Optimization and Self-Organization Approaches
Abstract: This paper proposes a novel population-based optimization algorithm to solve the multi-modal optimization problem. We call it the chaotic evolution deterministic crowding (CEDC) algorithm. Since the genetic algorithm is difficult to find all optimal solutions and the accuracy is not high when searching for multi-modal optimization problems, we use the ergodicity of chaos to implement the exploration and fitness comparison of the deterministic crowding algorithm. Through the tests of several multi-modal benchmark functions, it is shown that the algorithm can effectively and accurately find the most optimal solutions to the multi-modal problem. It does not need to set the niche radius in advance, so it can better solve multi-modal optimization problems. We test it with nine multi-modal benchmark functions ranging from one-dimension (1-D) to ten-dimension (10-D), and we compare it with a genetic algorithm and evaluate from peak ratio, max peak ratio, and running time. The experimental results show that the CEDC algorithm is better than conventional algorithms in both runtime and peak ratio.

Keywords: Computational Intelligence
Abstract: Non-negative latent factor analysis models such as tensor factorization have achieved significant success in collaborative-filtering-based recommendation tasks because they can perform representation learning to high-dimensional and incomplete data efficiently. However, they also suffer from either slow computational speed or representation accuracy loss. To address these issues, this paper presents a Partition-based Localized Tensor Factorization (PLTF) approach for predicting the missing values in the user-item-time rating tensors. First, a large sparse tensor is constructed to model users’ rating behavior. Then, it is transformed into recursive bordered block diagonal form by using the graph partitioning technology. Smaller and denser sub-tensors are extracted and factorized by using CP decomposition algorithm. Experimental results on sparse tensors from real applications show the efficiency of the proposed PLTF approach.

Keywords: Application of Artificial Intelligence, Deep Learning, Image Processing and Pattern Recognition
Abstract: While several methods for predicting uncertainty on deep networks have been recently proposed, they do not always readily translate to large and complex datasets without significant overhead. In this paper we utilize a special instance of the Mixture Density Networks (MDNs) to produce an elegant and compact approach to quantify uncertainty in regression problems. When applied to standard regression benchmark datasets, we show an improvement in predictive log-likelihood and root-mean-square-error when compared to existing state-of-the-art methods. We demonstrate the efficacy and practical usefulness of the method for (i) predicting future stock prices from stochastic, highly volatile time-series data; (ii) anomaly detection in real-life highly complex video segments; and (iii) the task of age estimation and data cleansing on the challenging IMDb-Wiki dataset of half a million face images.

Keywords: Application of Artificial Intelligence, Machine Learning
Abstract: Real-time estimation of travel time is a key traffic parameter for designing and planning for transportation systems, particularly when providing mobility-on-demand (MOD) services. However, the analysis and prediction of travel time can be delayed significantly due to the complexity and huge computational requirements of microsimulation models. Thus, as an alternative solution, we propose a data-driven approach for the efficient and reliable prediction of travel time. Our approach takes advantage of the strengths of SVM and ARIMA for fully capturing the traffic patterns in the traffic data. We introduce a new parameter κ into the SVM-ARIMA model to adjust the weight of the ARIMA component, which significantly improves the performance. We validate the performance of the proposed approach using data generated from a microsimulation platform. Our experimental results and comparisons with the existing ML-based methods demonstrates the efficacy of the proposed data-driven approach.

Keywords: Machine Learning, Deep Learning, Computational Intelligence
Abstract: Federated learning (FL) provides the possibility to solve the problem of data privacy, but it suffers much from the data heterogeneity among different participants. Currently, some promising FL algorithms improve the effectiveness of learning under the non independent-and-identically-distributed (Non-IID) data settings. However, they require a large number of communication rounds between the server and clients for an acceptable accuracy. Inspired by the training paradigm of gossip learning, this paper proposes a new FL framework, named FedGosp. It first classifies the clients into different categories based on the model weights trained by the locally stored data. Then FedGosp utilizes the communication not only between clients and the server, but also between different classes of clients themselves. This training process enables instilling knowledge about various data distributions in the passed models. We evaluate the performance of FedGosp in multiple Non-IID settings on CIFAR10 and MNIST datasets, and compare it with the recently popular algorithms such as SCAFFOLD, FedAvg and FedProx. The experimental results show that FedGosp can improve the model accuracy by 6.53% and save 5.6× communication costs at best compared to the second-ranked baseline.

Keywords: Evolutionary Computation, Computational Intelligence
Abstract: The purpose of this paper is to devise an interpretable hybrid classification model for Convolutional Neural Networks (CNN) and a Learning Classifier System (LCS). The presented hybrid system integrates the fundamental attributes from both types of these classifiers. In the proposed hybrid model CNN works as an automatic feature extractor, and LCS works to provide interpretable rule-based classification results. Although LCS has limitations working on higher dimensional datasets, we resolve this limitation by using CNN as a feature extractor. The other concept of the non-interpretability of CNN is addressed by using the LCS rule. Furthermore, our experiment with higher dimensional datasets like CIFAR-10 and Fashion-MNIST shows that extended LCS provides comparable performance to the standard neural network model while also providing interpretable results. We named this extended LCS method Convolutional Learning Classifier System (C-LCS).

Keywords: Swarm Intelligence, Machine Learning, Optimization and Self-Organization Approaches
Abstract: Extracting the latent information in High-dimensional and Incomplete matrices is an important and challenging issue. The Latent Factor Analysis (LFA) model can well handle the high-dimensional matrices analysis. Recently, Particle Swarm Optimization (PSO)-incorporated LFA models have been proposed to tune the hyper-parameters adaptively with high efficiency. However, the incorporation of PSO causes the premature problem. To address this issue, we propose a sequential Adam-adjusting-antennae BAS (A²BAS) optimization algorithm, which refines the latent factors obtained by the PSO-incorporated LFA model. The A²BAS algorithm consists of two sub-algorithms. First, we design an improved BAS algorithm which adjust beetles' antennae and step-size with Adam; Second, we implement the improved BAS algorithm to optimize all the row and column latent factors sequentially. With experimental results on two real high-dimensional matrices, we demonstrate that our algorithm can effectively solve the premature convergence issue.

Keywords: Computational Intelligence, Evolutionary Computation, Heuristic Algorithms
Abstract: Abstract—Nowadays, the recycling of waste products has attracted extensive attention in academia and industry. In the layout design of disassembly lines, single-row and U-shaped hybrid disassembly lines have different application scenarios. Considering workstation resource configuration, disassembly line cycle time, and disassembly task precedence relationship, we address a Multi-product Hybrid-disassembly-line-balancing Problem (MHP), and establish its mathematical model with the objective of disassembly profit maximization. In addition, the union variable neighborhood descent (U-VND) algorithm is used to solve the problem, in which two kinds of neighborhood structures composed of different actions is designed. Experimental results and comparative analysis show that the proposed algorithm can quickly obtain stable and high-quality solutions, which verifies the validity of the neighborhood structure and the correctness of the model.

Keywords: Computational Intelligence, Evolutionary Computation, Heuristic Algorithms
Abstract: Abstract—With the development of science and technology, a large number of electronic products have been discarded and become waste products. To obtain economic benefits and protect the environment, disassembly lines are designed to disassemble valuable parts from waste products. This paper proposes a mathematical model for the human-robot collaborative multiproduct hybrid disassembly line balancing problem with the disassembly revenue being the objective. A hybrid line combines a single-row line and a U-shaped line. We use the multi-neighborhood parallel greedy search algorithm to solve the model. Based on the algorithm, an alternate neighborhood search scheme consisting of different actions is designed. Some real-world cases are used to examine the feasibility of the proposed algorithm. The experimental results show that the multi-neighborhood parallel greedy search algorithm can solve the multi-product hybrid disassembly line balancing problem effectively.

Keywords: Computational Intelligence, Evolutionary Computation, Heuristic Algorithms
Abstract: Abstract—The remanufacturing, recycling, and reusing of waste products are particularly important to solve the problem of the resource shortage. Disassembly is a key step in the recycling process. How to minimize the negative impact of greenhouse gases on the environment has attracted extensive attention. This paper studies the disassembly line balancing problem to minimize the carbon emissions generated in the disassembly process. A Q-learning algorithm in reinforcement learning is applied to solve the disassembly line balancing problem. Through the analysis and comparison with the stateaction-reward-state’-action algorithm to address the same reallife cases, it is proved that the Q-learning algorithm has good performance in most cases. In terms of solution speed, the proposed method is faster in both small-scale and large-scale cases.

Keywords: Heuristic Algorithms, Optimization and Self-Organization Approaches, Cloud, IoT, and Robotics Integration
Abstract: Task scheduling in heterogenous and distributed systems for the directed acyclic graph (DAG) based applications has been widely studied. In DAG task scheduling problems, a set of distributed tasks with dependencies are dispatched to appropriate computing instances. The objective is to obtain the feasible schedule with the minimal schedule length, i.e., makespan. In this paper, we propose a task duplication based clustering framework for the problem under study. We employ the task duplication scheme in the framework which allows task clusters contain duplicated tasks in order to reduce the communication cost. A selection matrix is introduced to record the candidate tasks to generate clusters. Multiple feasible task clustering solutions are obtained based on the selection matrix and among which the best one with the minimal makespan is output. Experimental results indicate that the proposal outperforms compared algorithms on both effectiveness and robustness.

Keywords: Machine Learning, Neural Networks and their Applications, Deep Learning
Abstract: Flexible job shop scheduling (FJSP) is one of the most important problems in the domain of machining process optimization. This paper proposes a deep reinforcement learning approach to resolve the FJSP. In the approach, the FJSP is formulated as a Markov decision process where disjunctive graph is used to represent the state, operation set and machine allocation are used as the actions, the reward function is established based on the optimization objective (i.e. makespan). To obtain the embedding representation of the disjunctive graph of the FJSP, the corresponding graph neural network (GNN）is used to extract the state features. The multi-layer perceptron (MLP) decision network and scheduling rules cooperate to achieve the selection of actions (i.e. operations and machines). The multi-threaded asynchronous advantage actor-critic (A3C) algorithm is employed to optimize the model parameters to shorten the training time. The approach has been tested on the benchmarks. The results prove that this approach is superior to scheduling rules and meta-heuristic algorithms in results and computing time respectively.

Keywords: Communications, Consumer and Industrial Applications, Conflict Resolution
Abstract: The dramatic increase in the number of students enrolled in higher education programs outside their country of citizenship during the last half-century has created a huge demand for study abroad-related information circulation. Nonetheless, media nowadays attempts to break information barriers by gathering and processing data from multiple sources, mainly focusing on university academic competency. Although important, academic competency cannot represent international students' overall quality of life when spending their time in unfamiliar foreign cities. This research is designed to provide solutions to the problem from another perspective – Utilizing the Multiple Criteria Decision Making (MCDM) method to evaluate international study destinations through various dimensions comprehensively. The research integrates various aspects such as economic development, culture inclusiveness, and personal safety into account. It then adopts the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) to obtain a detailed index for each destination in the rank. Our work-Global Ranking of Study Destination (GRSD) by cities, is accomplished to contribute to the international student community. With a ranking system that considers vital facets of life in certain cities, prospective international students can make assessments and decisions better for their future.

Keywords: Communications, Consumer and Industrial Applications, Smart Sensor Networks
Abstract: Acoustic-communication-based intelligent devices attract attention for their simplicity and cost-effectiveness. Still, there is a lack of research on acoustic communication solutions that can simultaneously balance low Bit Error Rate(BER), inaudible sound, low power consumption, high throughput, and operate on embedded devices. This paper proposes an acoustic communication framework based on Chirp Spread Spectrum (CSS) modulation technology, which can accomplish reliable acoustic communication on common MCU such as stm32 and ESP32. Our proposed Custom Rolling Matching Encoding (CRME) protocol matches complex acoustic channel states with a vote-queue algorithm. It performs reliable low-rate acoustic communication in a low signal-to-noise ratio (SNR) environment. Results show that the framework has surpassed technologies such as Bluetooth Low Energy (BLE) and Radio-frequency Identification (RFID) in terms of BER, power consumption for device usage, and communication distance: BER is below 0.5% at a 272 bps data reception rate. Also, the framework can be deployed on embedded devices, consumes less power than RFID, and has similar performance to BLE, providing an alternative to cost-effective acoustic communication.

Keywords: Communications, Technology Assessment
Abstract: Multidisciplinary teams are often a necessity for software projects as they provide the required expertise to effectively solve complex problems. However, efficient collaboration between teams with different disciplines is challenging due to several factors such as considering gaps in knowledge areas, establishing a development process, and understanding different requirements from various groups or stakeholders. The agile methodology, such as scrum, offers a powerful approach to managing the software development process effectively. As part of the agile methodology, some techniques and tools are used to manage requirements change, which is a common practice in multidisciplinary teams. This research aims to leverage process-mining techniques to analyze data from Jira and GitHub to analyze the efficacy of software development process, particularly in multidisciplinary teams. This approach is applied to a case study of a virtual healthcare intervention system to measure the team's productivity. The results indicate several deficiencies in the process with respect to requirements engineering task that cause loss of time and increase rework rates. Results indicate that there are some challenges in the development process that contribute to some deficiencies. The rework rate is high and the number of tasks that are intended to be completed is less than what was planned. These factors can contribute to the lengthening of the software development process. Most of these challenges can be addressed by improving the requirement engineering process in order to obtain the requirements and manage change requests more efficiently.

Keywords: Communications, Control of Uncertain Systems, Robotic Systems
Abstract: Providing an efficient service of time-sensitive data transfer plays an essential role in entertainment, personal, and business communication, and industry, especially in the systems of monitoring and robotic automation. Those applications require high network throughput and short latency, rarely available within one data channel. Until recently, nonhomogeneous channels could not have been concurrently engaged owing to the protocol legacy restrictions. In order to answer the current challenges, new protocols, e.g., Multipath TCP (MPTCP), have been designed. MPTCP splits the data stream over a number of channels in the way dictated by a low-level component – the scheduler. However, the default scheduler aims at increasing throughput rather than constraining the transfer delay. Thus, for time-sensitive applications, other solutions are needed. In this work, a new scheduler, explicitly targeting the transmission aspects related to protocol delay, is designed. The scheduler properties are analyzed formally and its performance is tested experimentally.

Keywords: Communications, Large-Scale System of Systems, Cooperative Systems and Control
Abstract: The opinions of multi-agent in the project-based interorganizational networks (PIONs) are identified as major roles in the success of interorganizational cooperation. Existing models of opinion dynamics consider only the influence of the heterogeneous bounded confidence of neighbors on agents, ignoring the combined effect of the global reputation of the entire network, neighbors' similar opinions, and their degree of stubbornness. In this paper, a perspective exchange framework for project-based interorganizational networks is used to identify agents who can proactively raise the level of their opinions. A modified agent-based opinion model of PIONs based Hegselmann–Krause (HK) model is then proposed. Extensive case experiments are conducted on an artificially generated network dataset, and the theory of Shannon-like entropy is applied to verify the stability and convergence of the proposed model. The simulation results show that the increase in the proportion of global reputation leads the group to evolve toward consensus and converge slowly, while the increase in the proportion of neighbor's opinions makes the group evolve toward polarization and converge quickly.

Keywords: Human-Machine Cooperation and Systems, Human-Machine Interface, Human Factors
Abstract: This paper addresses the problem of transfer of control authority between a robot's AI and a remote human operator, when controlling a Mixed-Initiative (MI) robotic system. We propose a negotiation-theoretic method that enables the robot's AI and the human operator to cooperatively and dynamically determine (i.e. negotiate) the transfer of control authority between these two agents. An experimental study is presented in which a state-of-the-art Expert-guided Mixed-Initiative Control Switcher (EMICS) method is compared with our proposed Negotiation-Enabled Mixed-Initiative Control Switcher (NEMICS) algorithm. Results suggest that the NEMICS framework is able to successfully avoid conflicts for control, which is a fundamental challenge encountered with previous MI control methods. Comparing NEMICS with the EMICS, we provide evidence of improved navigational safety (i.e. fewer collisions). Additionally, our usability study suggests that human operators perceived their interactions with NEMICS as less intrusive than with EMICS.

Keywords: Human-Machine Cooperation and Systems, Design Methods, Information Visualization
Abstract: With increasing autonomous capabilities of machines the cooperation with humans becomes more and more important. In cooperative guidance and control, humans and machines either share or trade control in a cooperative way. Applied to vehicles, patterns of cooperative movements found in human-to-human interaction can be transferred to the cooperation and interaction of humans and highly automated vehicles. Starting with an overview of the concept of cooperative movement, the concept of interaction patterns is elaborated, on the one hand from an observation perspective and on the other hand from a design perspective. This includes image schemas, design patterns and interaction patterns.

Keywords: Human-Machine Cooperation and Systems, Brain-based Information Communications, Human-Machine Interface
Abstract: Brain-Computer Interface technology will be soon becoming accessible to the general public. It would be to replace or compensate physical disability in this case of handicap. However, it would also be for all people searching for peaceful consciousness, to control environment from no other types of command but directly from the brain. Such a device that could be easily and quickly handled by untrained and novice users has been proposed by Mentalista. The company designed and trained a Brain-Computer Interface and in this paper we propose a method to identify how such a device may be used to control a robot. The method proposes to identify the cooperative patterns that suit users’ states and robot’s abilities regarding the environment constraints. The Human-Machine Cooperation principles support the methodological approach.

Keywords: Human-Machine Cooperation and Systems, Assistive Technology, Supervisory Control
Abstract: An assist-as-needed semi-autonomous control algorithm is designed to address the problem of safely driving a vehicle (a power wheelchair) in an environment with static obstacles. The main idea is to maximize the human driver's control experience while allowing them to navigate safely (the inputs do not lead to collisions). The proposed physically-inspired model-based obstacle avoidance algorithm relies on optimal maps of the expected time for executing a safe stop manoeuvre. These maps are pre-computed using policy iteration in the case of an experienced driver stochastic model. As the burden of complex calculations is handled offline, the online implementation of the algorithm requires little computing resources. Its efficiency is tested experimentally in a study with healthy participants: a statistically significant result confirmed that the proposed algorithm outperforms a baseline rule-based control. A discussion with pros and cons ends this paper.

Keywords: Human-Machine Cooperation and Systems, Design Methods, Assistive Technology
Abstract: This paper presents the validation and the comparative study of a shared control concept for a large vehicle manipulator (LVM). The state-of-the-art controlling a LVM is manual control: The operator controls the manipulator to carry out a specific task and keeps the vehicle on the road. Easing the work for the operator, an automatic lane-keeping of the vehicle can be taken into account: An automation of the vehicle which keeps it on its reference, but without taking into consideration of the manipulator's specific task. However, the operator has his specific task with the manipulator, and therefore, such automation may not be satisfying. Therefore, this paper presents the validation and compares the Limited Information Shared Controller (LISC) proposed previously with the manual control mode. This step is crucial, showing the concept's applicability and benefits compared to the state-of-the-art solution. Thus, the LISC is compared with a non-cooperative controller (NCC) and the manual mode on a real-time simulator with test subjects. It has a more realistic experimental setup than in other studies because there is no predefined manipulator reference. The study results indicate that the NCC can lead to undesired motions of the overall system because the test subjects cannot carry out their specific task. On the other hand, the proposed the LISC of the vehicle can reduce the working load while supporting the operator in carrying out the manipulator's specific task.

Keywords: Human-Computer Interaction, Human-Machine Cooperation and Systems, Human-Machine Interface
Abstract: Digitization of handwritten documents preserving the author’s style is important because it maintains the authenticity of the document. However, generating personalized digitized text is challenging and is an unsolved problem. In this paper, we propose a novel handwriting generation approach which surpasses previous state of the art results by ~16% in objective study and by ~17% in subjective study. Our contribution is twofold: (1) developing a new objective function based on a novel reparameterization technique for mixture models (2) developing a Fréchet Inception Distance (FID)-based metric for evaluating digitized handwriting styles. Moreover, to the best of our knowledge, our work is the first to show application of Optimization-Based Model Agnostic Meta Learning (MAML), a few shot learning technique, to adapt our model to new author styles with few training examples. Our method presented in this paper establishes the new state of the art in this field.

Keywords: Human-Computer Interaction, Human-Machine Interface
Abstract: Current vision-based tactile sensors have several limitations, such as their size and measurable surface. Therefore, we propose a novel vision-based tactile sensor based on the re-propagated frustrated total internal reflection (FTIR). The part of the FTIR generated by the contact is re-propagated through the medium, and the FTIR spectra are observed from the side of the medium. We validate the physical principle of observation, including multiple contact images by simulations. In addition, a prototype system is developed to estimate the contact position through observations and regression algorithms. Finally, several experiments were performed to confirm the feasibility of the proposed contact estimation based on the re-propagated FTIR.

Keywords: Human-Computer Interaction, Intelligence Interaction, Information Visualization
Abstract: Hand actions classification is an imperative field for acquiring smart functionality in modern electronic de- vices because hand actions classification offers interactive and innovative methods to communicate and interact. Therefore, we develop a novel architecture based on you only looking at coefficients (YOLACT), a real-time instance segmentation approach, and a temporal relation network (TRN) for hand actions understanding. In addition, our framework consists of a face recognition-based security network (FRB-SN) for user identification. We trained the YOLACT and the TRN models using the segmented version of the 20BN jester dataset composed of hand actions images and ground truths while the FRB-SN is trained using the VGGFace2 dataset. For testing, the YOLACT is used to segment the object from the given image sequence and then passed to the TRN-trained model to predict the corresponding action. Our experimental results showed that the accuracy and frame rate of the proposed framework are competitive.

Keywords: Human-Computer Interaction, Human Factors
Abstract: This article discusses the segmentation of indoor daily living environments, such as rooms in a house, into regions that are habitually used for different purposes by the residents living in the environment. Because these regions usually include purpose-related furnishings, previous studies have focused on the functionalities of these furnishings or their predefined parts to characterize the purposes of the regions, including those furnishings used by the residents. However, the regions used for specific purposes do not necessarily correspond to the furnishings or their predefined parts. Even for a single dining table, its central area typically serves as the region used for placing food, drink, salt shakers etc., whereas the peripheral area typically constitutes a region for dining together. This article attempts to extract the regions used for different purposes as a spatial range from the given environment by segmenting it based on the similarity in the variation of postures observed in each position and the occupancy of its surrounding space. In the experiments using two datasets for indoor daily living environments observed by RGB-D cameras, the obtained regions primarily correspond to our understanding of the regions used for different purposes.

Keywords: Human-Computer Interaction, Intelligence Interaction, Human-Machine Cooperation and Systems
Abstract: With a growing demand for new technologies, concepts such as the Internet of Everything (IoE) — in which smart sensors (humans and machines) connect, communicate, and share information from the surrounding environment — are gaining notoriety. Thus, this study focuses on developing a knowledge flow model in the IoE scenario for Smart Cities, including how knowledge is disseminated in these intelligent environments in human-to-machine interactions. The study considers a knowledge-based taxonomy defined for IoE environments and supports the full realization of IoE applications through a knowledge management strategy. The proposed IoE Knowledge Flow Model contributes to identification of knowledge flows related to IoE architecture layers in smart city applications.

Keywords: Assistive Technology, Human-Machine Cooperation and Systems, Systems Safety and Security
Abstract: In this paper, a steering action-aware Adaptive Cruise Control (ACC) approach for teleoperated road vehicles is proposed. In order to keep the vehicle in a safe state, the ACC approach can override the human operator's velocity control commands. The safe state is defined as a state from which the vehicle can be stopped safely, no matter which steering actions are applied by the operator. This is achieved by first sampling various potential future trajectories. In a second stage, assuming the trajectory with the highest risk, a safe and comfortable velocity profile is optimized. This yields a safe velocity control command for the vehicle. In simulations, the characteristics of the approach are compared to a Model Predictive Control-based approach that is capable of overriding both, the commanded steering angle as well as the velocity. Furthermore, in teleoperation experiments with a 1:10-scale vehicle testbed, it is demonstrated that the proposed ACC approach keeps the vehicle safe, even if the control commands from the operator would have resulted in a collision.

Keywords: Systems Safety and Security, Human-Machine Cooperation and Systems, Design Methods
Abstract: Safety is a central challenge in human-robot collaboration. Particularly in higher collaboration levels, separating safety devices, such as fences, are no longer needed and must be replaced by intelligent sensor-based systems. Of particular interest is the adaptive speed control of the robot. This work presents a methodology to adaptively control the end-effector velocity of the robot based on the distances to dynamic environmental objects. The method combines distance measurement and environmental subtraction with conservative velocity estimation using robot-specific stopping distances and is available in real-time. Data acquisition is performed using a co-moving 3D camera sensor attached to the robot structure.

Keywords: Human-Machine Cooperation and Systems, Human-Computer Interaction, Systems Safety and Security
Abstract: Frauds in supply chain finance not only result in substantial loss for financial institutions (e.g., banks, trust company, private funds), but also are detrimental to the reputation of the ecosystem. However, such frauds are hard to detect due to the complexity of the operating environment in supply chain finance such as involvement of multiple parties under different agreements. Traditional instruments of financial institutions are time-consuming yet insufficient in countering fraudulent supply chain financing. In this study, we propose a novel blockchain-driven framework for deterring fraud in supply chain finance. Specifically, we use inventory financing in jewelry supply chain as an illustrative scenario. The blockchain technology enables secure and trusted data sharing among multiple parties due to its characteristics of immutability and traceability. Consequently, information on manufacturing, brand license, and warehouse status are available to financial institutions in real time. Moreover, we develop a novel rule-based fraud check module to automatically detect suspicious fraud cases by auditing documents shared by multiple parties through a blockchain network. To validate the effectiveness of the proposed framework, we employ agent-based modeling and simulation. Experimental results show that our proposed framework can effectively deter fraudulent supply chain financing as well as improve operational efficiency.

Keywords: Systems Safety and Security, Information Systems for Design
Abstract: With the rapid development of the Internet of Things (IoT), a large amount of data is exchanged between various communicating devices. Since the data should be communicated securely between the communicating devices, the network security is one of the dominant research areas for the 6LoWPAN IoT applications. Meanwhile, 6LoWPAN devices are vulnerable to attacks inherited from both the wireless sensor networks and the Internet protocols. Thus intrusion detection systems have become more and more critical and play a noteworthy role in improving the 6LoWPAN IoT networks. However, most intrusion detection systems focus on the attacked areas in the IoT networks instead of precisely on certain IoT nodes. This may lead more resources to further detect the compromised nodes or waste resources when detaching the whole attacked area. In this paper, we therefore proposed a new precisional detection strategy for 6LoWPAN Networks, named as PDS-6LoWPAN. In order to validate the strategy, we evaluate the performance and applicability of our solution with a thorough simulation by taking into account the detection accuracy and the detection response time.

Keywords: Assistive Technology, Human Performance Modeling, Brain-based Information Communications
Abstract: We present three user studies that gradually prepare our prototype system SmartHelm for use in the field, i.e. supporting cargo cyclists on public roads for cargo delivery. SmartHelm is an attention-sensitive smart helmet that integrates none-invasive brain and eye activity detection with hands-free Augmented Reality (AR) components in a speech-enabled outdoor assistance system. The described studies systematically increased in ecological validity from lab to field. The first study consisted of an Augmented Reality preparation examination in the lab. The second study then investigated simulated attention distraction modeling, whereas the third study examined real world attention distraction modeling while cycling in traffic. During these three studies, multimodal data (EEG, eye-tracking, video, GPS and speech) has been collected synchronously and analyzed in offline and online experiments. Machine Learning models were trained and optimized for attention modeling.

Results: Analyses of self-report and objective data during the simulation study show the plausibility of the simulated internal and external distractions. The analysis of behavioral data captured by multimodal biosignals recorded in the field study further shows that real visual attention distractions can be automatically identified using synchronized video and eye tracking data. Machine Learning methods based on long short term memory models (LSTMs) indicate that simulated attention distractions can be automatically detected from EEG data, with the best detection performance for mental distractions. Finally, the self-report data suggest that the comfort of the SmartHelm helmet should be further improved for permanent use in road traffic.

Keywords: Application of Artificial Intelligence, Cloud, IoT, and Robotics Integration, Computational Intelligence
Abstract: Edge AI in contrast to cloud-based approaches enables low-latency and low-power AI applications in smart factories. AI manufacturing platforms based on the combination of Edge and Sky Computing transform legacy factories into resource-efficient Industrie 4.0 facilities. Active digital twins allow the permanent tracking of the CO2 footprint for every instance of a product. In the next decade of Industrie 4.0, one major goal is the automatic interpretation of industrial data using AI to enable zero-defect and sustainable production. This means that all phases of the production chain can be supported by machine learning. We illustrate the latest research breakthroughs of industrial Edge AI with practical examples from major European industry initiatives.

Keywords: Deep Learning, Machine Learning, Neural Networks and their Applications
Abstract: In a recent day, we could witness an explosive growth of artificial intelligence and deep learning in medical applications. With the increased availability of medical images, deep learning tools can provide a necessary diagnostic utility. However, current DNN models have shown high variations in their performance to each medical image dataset. In this study, we proposed ensemble learning to achieve synergistic improvements in model accuracy and thereby provide highly stabilized performance on diverse medical datasets. We first investigated the model performance of the latest deep learning architectures, e.g., Inception, VGGNet, MobileNet, Xception, ResNet50, and selected 7 state-of-the-art models to the diverse open CT datasets (SARS-COV-2 CT-Scan, USCD CT, and COVID-X dataset). The model parameters were transferred from the other domain and fine-tuned based on medical image sets. The last convolutional layers were stacked and a fully-connected neural network is employed to find generalized feature space. The peak accuracy of the fine-tuned single CNN models were InceptionV3 - 0.96, VGG16 - 0.94, VGG19 - 0.94, MobileNetV2 - 0.98, Xception - 0.9, ResNet - 0.96, DenseNet201 - 0.97. The proposed ensemble model achieves the peak accuracy of 0.99%, outperforming each individual model and achieving the highest performance in all three open CT datasets. Experimental results demonstrated that the proposed ensemble model is able to represent the hierarchical features and thereby it improves the stability and reproducibility of the classifier models.

Keywords: Image Processing and Pattern Recognition, Machine Learning, Deep Learning
Abstract: Due to fixed convolution kernel structure, it is intractable for Convolutional Neural Networks (CNN) to adequately deal with intact instance geometric transformations, which may paralyze the CNN's paradigmatic feature extraction. Inspired by Deformable Convolutional Networks, in this work, we introduce stochastic deformatble kernels, where kernel shape is assumed to be random vaiable. Stochastic deformable kernels may extract non-local alienated structural features, which slightly condones geometric transformations. Variational learning of the model based on two approximate posteriors is derived. And experiments over transformed MNIST dataset demonstrate the validity of this approach.

Keywords: Deep Learning
Abstract: The application of convolutional neural networks (CNNs) is sometimes limited by a large number of parameters and floating-point operations. Pruning methods have been proved to be effective to solve this problem. These methods improve the efficiency and storage occupancy of CNNs by removing weights connected with certain neurons/channels. The key issue is the selection of suitable neurons/channels to be pruned. Then, fine-tuning is usually applied to restore the performance of a pruned model to that before the pruning. However, existing neurons/channels selection methods do not explicitly consider the impact of the pruning on the model output. Moreover, the performance of a fine-tuned model may suffer from the information loss problem caused by the pruned neurons/channels. In this work, a stochastic sensitivity measure-based neurons/channels selection criterion is proposed to choose and prune insensitive neurons/channels, which effectively reduces the degradation of model performance. Moreover, a compensation operation followed by fine-tuning is proposed to relieve the information loss problem and restore model performance. Experimental results show that our method yields comparable compression and acceleration rates with less accuracy degradation compared with existing pruning methods for CNNs.

Keywords: Image Processing and Pattern Recognition, Neural Networks and their Applications, Deep Learning
Abstract: Remote speech reconstruction is widely used in counter-terrorism, medical science and engineering. In order to obtain reconstructed speech with high accuracy, we propose a speech reconstruction method. This method consists of two parts. Firstly, some optical devices are used to collect speckle images. Secondly, the convolutional neural network is used to detect the subtle motion of speckles. The results show that the lowest mean absolute error of the sinusoidal signal reconstructed by the method is 0.0489, and the lowest mean absolute error of the real speech is 0.0271. Compared with the convolutional neural network proposed before, the error of reconstructed speech is small, and the number of parameters is significantly reduced, with 0.73M for our model compared to 11.45M for the previous model. Besides, the time cost of training on some datasets is reduced to less than 1 hour, which is much lower than the previous model. The experimental results prove that our model is a lightweight, high-accuracy model for remote speech reconstruction with short training time.

Keywords: Application of Artificial Intelligence, Biometric Systems and Bioinformatics, Deep Learning
Abstract: In recent years, facial expression recognition has always been a popular research topic. Since the variation among human facial expressions is huge, facial expression recognition is still one of the challenging topics in computer vision. In the facial expression recognition, the key challenge is to capture the dynamic variation of the physical structure of the face from the videos. However, traditional machine learning based systems may have large errors due to the feature extraction on environmental factors such as postures, angles, light, occlusion or backgrounds, which can easily lead to unrecognized or error identification results. In this work, we propose a deep learning based framework, and add residual in residual dense block and triple attention mechanism to our model. On several benchmark facial expression datasets, we demonstrated the effectiveness of our method and compare our model with other state-of-the-art modalities.

Keywords: Medical Informatics, Human-Computer Interaction, Virtual and Augmented Reality Systems
Abstract: Cardiopulmonary resuscitation (CPR) is the most effective quick responsive method to avoid the risk of death due to a heart attack. However, no practical CPR education has been established regarding the citizens’ correct posture while performing CPR. Herein, we developed a training system using an Azure Kinect DK sensor camera to visualise the upper and lower body posture while performing the CPR operation from both the front and side. Assuming its use under COVID-19 restrictions, we implemented a noncontact voice-activated interface and functionality for accurately detecting system’ pressure. For evaluating the training system, after using it for the airport and life insurance employees training, a comparative investigation was conducted to determine whether the system’s applicability for training. As per B business office experiment, when the number of data items was n = 44, a correlation coefficient with a strong correlation of 0.662 was obtained. For regression analysis performed on B business office, total posture score and compression frequency significantly differed (significant probability P < 0.001 and significance level 5%).

Keywords: Human Factors, Human-Computer Interaction, Virtual and Augmented Reality Systems
Abstract: Vection is commonly defined as the illusory sensation of self-motion. Research on vection can assist in improving the fidelity of motion simulators. Vection can be influenced through top-down factors, such as attention, but previous research on the effect of a secondary task on vection presented conflicting findings. We investigated the effect of a visual discrimination reaction time task on vection. Twenty-nine participants were visually and audibly immersed in virtual environments with different levels of ecological relevance wherein they used a joystick to continuously report on their vection experience. In contrast to previous research, our results showed no significant effect of a secondary task on vection measures nor an effect of sensory cues and environment context on secondary task performance. We conclude that participants’ ability to report their vection experience was unaffected whilst performing a visual attention reaction time task.

Keywords: Virtual and Augmented Reality Systems, Human-Computer Interaction, Human Factors
Abstract: The illusory sensation of self-motion is defined as vection. Vection research can help enhance Virtual Reality applications and improve simulator fidelity as vection appears to be a desired sensation in motion simulators. The experience of vection can be modulated by cognitive factors and potentially personal traits, such as the vividness of imagination. Previous research investigating the relationship between auditory vection and kinesthetic imagery presented conflicting findings. However, the relationship between visually-induced vection and imagination has not been investigated. Herein we investigated the relationship between kinesthetic imagery and unimodal visual and bimodal visual-auditory vection. Twenty-nine participants were visually and audibly immersed in virtual environments with varying degrees of ecological relevance wherein they reported on their vection experience. No differences were found for vection intensity and latency measures between participants with high and low kinesthetic imagery. We conclude that imagery does not appear to play a role in the experience of visually-induced vection.

Keywords: Virtual and Augmented Reality Systems, Multimedia Systems, Information Visualization
Abstract: Metaverse was recently introduced as an online virtual world or world of worlds that incorporates the augmented/virtual/mixed/extended reality (AR/VR/MR/XR), Internet of Things (IoT), personalization with virtual avatars, interaction and communication interlinking real and/or virtual “things” with “things”. Metaverse users are supposed to live in a digital realm in which such living universes are created either as imaginative or realistic environments. This brings a new creative space for architects in the Architecture- Engineering- Construction (AEC) sector to design new buildings and cities in a semi-realistic digital world as they can be inspired by actual urban settings and augment these real or realistic environments towards a cyber environment. This paper aims to present such a cyber environment, called BIMyVerse, that blends IoT-enabled situational awareness, VR-supported integrated semantic digital twin models incorporating Building Information Model (BIM), Geographical Information System (GIS) and sensory cyberspace and explainable interpretations for better interaction with buildings and urban contexts. The proof of concept studies demonstrate how BIMyVerse can be used for i) situational fire regulation check and monitoring the maintenance status (building-scale); ii) circular economy provisioning through the digital twin of an urban transformation area (urban-scale).

Keywords: Virtual and Augmented Reality Systems, Human-Machine Interface, Human-Machine Cooperation and Systems
Abstract: To achieve high fidelity haptic rendering of soft objects in a high mobility virtual environment, we propose a novel haptic display DandelionTouch. The tactile actuators are delivered to the fingertips of the user by a swarm of drones. Users of DandelionTouch are capable of experiencing tactile feedback in a large space that is not limited by the device's working area. Importantly, they will not experience muscle fatigue during long interactions with virtual objects. Hand tracking and swarm control algorithm allow guiding the swarm with hand motions and avoid collisions inside the formation.

Several topologies of impedance connection between swarm units were investigated in this research. The experiment, in which drones performed a point following task on a square trajectory in real-time, revealed that drones connected in a Star topology performed the trajectory with low mean positional error (RMSE decreased by 20.6% in comparison with other impedance topologies and by 40.9% in comparison with potential field-based swarm control). The achieved velocities of the drones in all formations with impedance behavior were 28% higher than for the swarm controlled with the potential field algorithm.

Additionally, the perception of several vibrotactile patterns was evaluated in a user study with 7 participants. The study has shown that the proposed combination of temporal delay and frequency modulation allows users to successfully recognize the surface property and motion direction in VR simultaneously (mean recognition rate of 70%, maximum of 93%).

DandelionTouch suggests a new type of haptic feedback in VR systems where no hand-held or wearable interface is required.

Keywords: Discrete Event Systems, System Modeling and Control, Intelligent Transportation Systems
Abstract: The intelligence of transportation has developed rapidly in recent years, and its reliability and safety have also attracted a lot of attention. The Adaptive Cruise control (ACC) system is a significant achievement of traffic intelligence. The principle of the ACC system is the process of balance feedback between the relative speed and distance of the front and current vehicles. In this paper, the running principle of ACC system is abstracted, and the Balanced Feedback Net (BFN) is proposed to model and analyze it based on Petri nets. The reachable marking graph and the incidence matrix of Petri nets are used to analyze the BFN model. The analysis results show a certain risk of rear-end collision in the balance feedback process of the ACC system. In this regard, we give a relevant risk identification algorithm to reduce the risk of rear-end collision and improve the reliability of the ACC system.

Keywords: System Modeling and Control, Discrete Event Systems
Abstract: The paper proposes a novel analysis algorithm of the control part of cyber-physical systems specified by an interpreted Petri net. In particular, the three essential properties of Petri nets are studied: boundedness, safeness, and liveness. The presented idea combines linear algebra technique with reachability tree analysis. To clarify the presented concept, the method is illustrated by a real-life example of a cyber-physical system. Moreover, the experimental verification of the proposed technique was performed to examine its effectiveness and efficiency.

Keywords: Manufacturing Automation and Systems, Discrete Event Systems, System Modeling and Control
Abstract: The paper deals with the modelling and analysis of physical resilient manufacturing systems (RMS) specified by the interpreted Petri net. Nowadays a need to improve manufacturing resilience is evident due to several unexpected situations currently on the market. The production strategy of mass customization application of RMS on the physical layer enables quick and cost-effective reaction to changing market and social conditions. Such systems should be capable of operating under any changes in production. Therefore, in this paper, a novel modelling concept of Petri net-based physical RMS for mass customisation production is proposed. The idea is based on the interpreted Petri net, which permits for additional specification of input and output signals of the system. Moreover, such a net ought to be live and bounded (or even safe), therefore a boundedness and safeness verification algorithm is developed. The proposed techniques are illustrated by the real-life case-study example of a RMS at the physical layer operating according to a mass customisation production strategy.

Keywords: Decision Support Systems, Consumer and Industrial Applications, Service Systems and Organizations
Abstract: The spread of COVID-19 has led many people to turn to O2O platforms to buy daily supplies leading to a boom in the O2O e-commerce industry. How to improve the service quality of O2O platforms to attract more customers has become an important concern for service providers. This study differs from previous statistical analysis studies in that it applies the data mining methodology to extract the key factors that affect the service quality of O2O e-commerce platforms. A hybrid multi-criteria decision-making method is then utilized to obtain the influence relationships and weights of the dimensions and criteria. The results suggest that privacy security, and reliability have a positive impact on social interaction, recommendation quality, efficiency and empathy. Empathy, social interaction and recommendation quality are the three most important factors for evaluating the service quality of O2O e-commerce platforms. Finally, the theoretical implications and management implications based on the findings are discussed.