Keywords: BMI Emerging Applications, Other Neurotechnology and Brain-Related Topics
Abstract: Neurotechnology has increased its contact with various fields of engineering, and the market for neurotechnology is expanding due to the fusion of big data and AI technologies, as well as the lightweight and sophisticated devices for measuring brain activity. Thus, the social implementation of neurotechnology has become a reality, and the development of an international code of ethics as well as international standard to conduct "responsible research and innovation (RRI)" that contributes to the ethical development, use, and diffusion of neurotechnology is now actively discussed from the perspective of neuroethics. It is required to develop human resources who understand the elements and neurotechnology and have knowledge of neuroethics as well as RRI. However, it is not clear what kind of engagement and educational opportunities should be provided in undergraduate and graduate school education. In this presentation, the author investigated the current situation of undergraduate and graduate education on neuroethics in Japan and the United States. The author also conducted a web-based survey of Japanese students, graduate students, and faculty regarding desirable educational topics and learning formats. A series of studies suggest that the introduction of neuroethics and RRI into engineering education requires appropriate awareness-raising and educational opportunities based on more advanced technological development trends and the formulation of international rulemaking, which might be different from the conventional structure of educational courses in philosophy and general education.

Keywords: BMI Emerging Applications, Other Neurotechnology and Brain-Related Topics, Passive BMIs
Abstract: In the fNIRS-based neural decoding problem, fNIRS measures local cortico-activity, so it is necessary to configure the input space to encompass all spatiotemporal features. We propose a neural decoding model with input embedding reflecting fNIRS data's spatial and temporal traits together. Our model embeds fNIRS channels corresponding to 3x3 and 5x5 sized grids based on fNIRS optodes into new channels using the depth-wise convolution layers. The embedded data is passed to the classifier through the 2D convolution layer and the transformer encoder. The transformer-based model is better for evaluating the correlated features between non-adjacent and distant channels because it inherently possesses the multi-head self-attention layer. The open dataset (mental arithmetic, BNCI Horizon 2020 [1]) was trained to verify model performance. As a data preprocessing process [2], the frequency band of 0.09Hz or more was removed through the 4th-order low-pass Butterworth filter, data segmentation was performed, and then z-score standardization was applied. As a result of the learning, the average accuracy of leave-one-subject-out (LOSO) cross-validation (CV) was 90.19%, and the average accuracy of k-fold cross-validation was 81.78%. A five-fold CV was performed five times, and the average accuracy of 25 results was used. Typically, the accuracy of the k-fold CV is supposed to be higher than that of the LOSO CV. Still, the proposed model resulted in the k-fold accuracy being lower than the LOSO accuracy. We checked the training loss graph and revealed an overfitting problem. The reason could be that the model parameter size is too large compared to the complexity of the problem (i.e., binary classification between rest state and task state) to be solved in the open dataset. Another one could be that the trained feature patterns are not very different because fNIRS data were measured only in the forebrain area, a functionally and structurally identical cortical region. Therefore, the proposed model should be further applied and updated to other datasets [3, 4] measured in the whole brain region or different cortical areas.

Keywords: Passive BMIs
Abstract: Due to its high performance, filter-bank canonical correlation analysis (FBCCA) is widely used in steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs). The FBCCA has a broad subband bandwidth for detecting harmonic components. Nevertheless, the SSVEP signal responds most strongly at fundamental frequency. This study proposed a target spectral band canonical correlation analysis(TSB-CCA) that focuses on the target frequency(which is assumed to have a high fundamental frequency component). Using existing low- and high-frequency SSVEP dataset, we assessed the accuracy of the canonical correlation analysis (CCA), the FBCCA, and the proposed TSB-CCA. Consequently, it was determined that FBCCA performed better with the low frequency dataset, while TSB-CCA performed better with the high frequency dataset.

Keywords: Passive BMIs
Abstract: A representative approach is event-related potential (ERP) based paradigm among the various brain-computer interface (BCI) sytems. The primary objective of this study is to investigate the effect of ventilation pattern manipulation on character recognition accuracy in the ERP-based speller paradigm. To check the possibility of the proposed methods, five subjects participated in the experiment, and each subject experienced two sessions. This study demonstrates that the proposed ventilation method is more effective than a normal method on average and lead our assumptions in a positive direction.

Keywords: Brain-Computer Interfaces
Abstract: In the field of multimedia, various electroencephalogram (EEG) studies have been reported in the context of brain-machine interface research. Recently, Neuralink has demonstrated a prototype wherein they invasively measured the brainwaves of a primate and showed that it was possible to control the bar in the Pong Game merely by thought. However, this method, which involves the surgical installation of electrodes inside the skull for measuring brainwaves, poses difficulties for application to the people. Therefore, in this study, we designed a non-invasive EEG-based brain-machine interface and applied it to the Pong Game. In particular, we demonstrated that it is possible to decode the intention of manipulation directly, not through motor imagery, by measuring signals solely from the prefrontal cortex, unlike traditional brainwave decoders that use sensory-motor signals.

Keywords: Large-Scale System of Systems, Cyber-physical systems, Decision Support Systems
Abstract: Rapidly increasing usage of nuclear-integrated energy units has created new challenges in terms of cybersecurity. This paper discusses the potential cyberthreat challenges and cyber risks associated with the widespread adoption of these units, and the role of artificial intelligence (AI) and machine learning (ML) techniques in enhancing the security and resilience of these systems.

Keywords: Neural Networks and their Applications, Soft Computing, Socio-Economic Cybernetics
Abstract: We consider a simple three-layer dynamical systems related to recurrent neural networks. The input to hidden layers construct an elementary cellular automaton and the hidden to output layers are one-to-one connection described by a permutation. Depending on the permutation connections, the network can generate various periodic orbits of binary vectors. Especially, we have discovered globally stable periodic orbits such that almost all initial points fall into the orbits. Based on numerical analysis, we present an important conjecture for property of globally stable periodic orbits. This is a first step to consider various periodic orbits and their engineering applications.

Keywords: Application of Artificial Intelligence, Cybernetics for Informatics, Image Processing and Pattern Recognition
Abstract: The biological immune systems of organisms provide a function of “never again” of diseases by responding to new sources of risk that arise in everyday life by dealing with both the diversity and specificity of diseases. In this paper, the author describes the concept of a societal immunization system in which the biological function of an immune system protecting from pathogens is reproduced informatic and robotic and applied to protect from the risk of fatal accidents in the living environment. To proceed with the system concept empirically, the author focuses on unintentional childhood accidents, the second most frequent cause of child fatalities. This paper reports a developing system for demonstrating proof of concept through implementing the societal immune system that completes from the creation of knowledge on fatal situations to integration with fatal situations recognition technology. Fundamental evaluations were conducted with a dummy child in a living laboratory environment, and the feasibility of the developed system was evaluated through tests among children of two years and five years in an actual home.

Keywords: BMI Emerging Applications, Other Neurotechnology and Brain-Related Topics, Active BMIs
Abstract: This study investigates the impact of visual perception, specifically emojis, on speech imagery decoding in brain-computer interfaces for emotional representation. Nine participants took part in two sessions: a speech imagery session and speech imagery with visual perception session. The results show that deep learning, specifically EEGNet, achieved the highest accuracy in the speech imagery session. Interestingly, the inclusion of visual perception was found to reduce the user’s focus on expressing emotions through speech imagery. These findings support the notion that speech imagery decoding provides a more intuitive approach to natural emotional expression and user intention communication.

Keywords: Intelligent Transportation Systems, Smart Buildings, Smart Cities and Infrastructures
Abstract: In this paper, a Vision Transformer Model with Compressive Sensing for crowd density level classification is proposed. Crowd density level classification is an important crowd monitoring task that is widely used in public places. However, the performance of existing methods degrades when dealing with heavily occluded scenes because they have difficulty in extracting complete and accurate crowd features. To solve this problem, we perform compressed perception on selected image blocks after removing occlusions, and then feed the results into the transformer backbone network, which outputs classification results from the density classification task head. We conducted experiments in a typical occlusion scenario of subway cars, and the results show that our approach achieves relatively good results.

Keywords: Cybernetics for Informatics, Deep Learning, Image Processing and Pattern Recognition
Abstract: The focus of this paper is palmprint recognition. Using the database of Hong Kong Polytechnic University. A method is proposed to generate synthetic palmprint images by using different types of GANs. These images are evaluated by a MesoNet algorithm to achieve an AUC score of 0.66. The results demonstrate the algorithm’s accuracy in detecting deepfake palmprints.

Keywords: Machine Learning, Big Data Computing,, Deep Learning
Abstract: This paper deals wiith a low-rank matrix completion, which is a problem of estimating missing entries in a given low-rank matrix. Several singular value decomposition (SVD) based matrix shrinkage algorithms have been proposed and achieve good performance. While the SVD based algorithm takes high computational cost derived from computation of the singular value decomposition (SVD), it can be speeded up by using GPU computing. However, GPU computing is not available for very huge matrices due to the limitation of memory on GPU. In order to utilize GPU computing, this paper provides distributed singular value shrinkage algorithm. This paper also deals with a low-rank tensor completion and applies the proposed distributed algorithm to a tensor completion. Numerical examples show that the proposed distributed algorithm can recover a low-rank tensor well.

Keywords: Image Processing and Pattern Recognition, AI and Applications, Deep Learning
Abstract: The field of medical imaging analysis has undergone a revolution thanks to the incorporation of automated methods for medical image annotation that are powered by artificial intelligence (AI). The lack of interpretability and transparency in these systems, however, raises questions about how they make decisions and restricts their use in therapeutic settings. Using explainable AI (XAI) tools to illustrate the decision-making procedure of medical picture annotation systems, we propose to address this difficulty in this study. We investigate numerous XAI methods, including Grad-CAM, LRP, and attention mechanisms, to shed light on the key elements and justifications for the system's annotations. We seek to improve the interpretability, reliability, and therapeutic utility of these automated systems by the visualization and explanation of the decision-making process.

Keywords: Neural Networks and their Applications, Deep Learning, Computational Intelligence
Abstract: Current deep learning techniques have led to spectacular results, but they still have limitations. One of them is that, in contrast to humans who can learn from a few examples and learn fast, modern deep learning techniques require a large amount of data to learn, and they take a long time to train. In this paper, we show that neural networks do have a potential to learn from a small number of examples -- and learn fast. We speculate that the corresponding idea may already be implicitly implemented in Large Language Models -- which may partially explain their (somewhat mysterious) success.

Keywords: Cognitive Computing
Abstract: Context plays a crucial role in understanding and representing the meaning of words. In previous work [1], we proposed a context awareness approach to enhance knowledge discovery from textual data. In this paper, we will present the system architecture to enable a hybrid approach to extract knowledge and its context from text.

Keywords: System Modeling and Control
Abstract: This paper proposes novel interval type-2 (IT-2) T-S fuzzy control systems and designs H-infinity controllers for a DC/DC converter. The novel IT-2 T-S fuzzy control systems proposed in this paper can divide the nonlinear converter into two subsystems. Therefore, regardless of the number of premise parameters the state-space equations have, the novel IT-2 T-S fuzzy control systems only need two control rules. It does not only reduce the number of the control rules, but it also reduces the calculation time of the microprocessor and has more relaxed stability conditions. The H-infinity performance index is used to suppress the external interference and the forward diode bias of buck mode and boost mode. The experimental results show that the novel IT-2 T-S fuzzy control systems have better performance than the existing IT-2 T-S fuzzy control systems.

Keywords: Digital Twin, Intelligent Transportation Systems, Discrete Event Systems
Abstract: A battery swapping station has been attracting attention to reduce long battery charging times of electric scooters. For quality service, effective charging management is necessary to be able to provide charged batteries on request. This study proposes a level 3 digital twin system to optimize the charging management. The proposed system synchronizes an agent-based simulation model, describing battery exchange process between drivers and stations, with the collected actual data via a preprocessor. Then, an optimal charging schedule is derived based on the synchronized model through an efficient simulation-based optimization tool and applied to the physical system.

Keywords: Communications, Smart Buildings, Smart Cities and Infrastructures, Smart Sensor Networks
Abstract: BT mesh network adopts channel hopping and advertising method to transmit data packets. In early specification, three advertising channels and fixed hopping sequence (Ch37, Ch38, Ch39) are designed for BT mesh networks. However, when there are a large number of relay nodes deployed in the network and a large number of data packets needed to be transmitted, the advertising method and fixed hopping sequence may cause serious packet collision and reduce packet forwarding efficiency. Hence, this paper will focus on the influence study of channel hopping sequences on packet collision in an advertising environment. Based on the random distribution mesh network, this paper simulates and compares the performances of fixed hopping sequence and three proposed random hopping sequences. The results show that the proposed random hopping sequences can effectively reduce the probability of packet collision and further improve packet forwarding efficiency in the BT mesh networks.

Keywords: Autonomous Vehicle, Cooperative Systems and Control, Intelligent Transportation Systems
Abstract: The automated driving control system plays an important role in autonomous vehicles. So far, control parameters in such systems mainly rely on modeling the interaction between the subsystems to derive the proper adaptive law. In this study, we propose an adaptive steering compensation scheme based on the type-2 fuzzy neural network (T2FNN) with the sliding mode learning algorithm to yield effective compensation control by learning online interactions from the measured feedback error. The conducted simulation will verify the adaptability and robustness of the proposed approach and test under several driving conditions.

Keywords: Manufacturing Automation and Systems, System Modeling and Control
Abstract: Dynamic scheduling can be used in scenarios in which jobs may arrive irregularly, and therefore, the schedule of jobs may need to be changed. In actuality, due to various reasons, demand for a service can suddenly change. In previous papers, various methods using genetic programming (GP) have been proposed. Using GP, an appropriate dispatching rule that determines the job sequence can be derived in a short amount of time, and derived rules are better than existing dispatching rules. However, the derived dispatching rules are not analyzed in previous papers. In this paper, dispatching rules that use GP are analyzed; the characteristics of the rules are described, and they are compared with previous dispatching rules.

Keywords: Machine Learning, AI and Applications, Machine Vision
Abstract: We propose a data analysis method that combines the objectives of nonlinear principal component analysis and nonlinear discriminant analysis with the kernel method in a reproducing kernel Hilbert space. This method addresses nonlinear data analysis problems in high-dimensional spaces, specifically the reproducing kernel Hilbert space, through the use of the kernel trick. Our proposed method can be considered as a semi-supervised data analysis approach. We evaluate our proposed method using various kernel functions and datasets, both visually and quantitatively. The evaluation results demonstrate that our proposal outperforms kernel principal component analysis and generalized discriminant analysis in terms of classification performance. This indicates the advantages and originality of our proposed method. Furthermore, we analyze and discuss our findings based on the evaluation results, and highlight potential areas for further research and future work related to our proposal.

Keywords: Machine Learning, AI and Applications
Abstract: In the context of predicting activity on a social network, data for any individual will be limited. Also, low levels of activity for a topic of interest may make it difficult to build a strong predictive model. This work examines how augmentation by oversampling all activity data used to build a predictive model of user activity on Twitter can be used to improve the fidelity of predictions. Our features are counts of activity by de-identified users at the granularity of hours. It is shown that for some topics oversampling the data by creating new synthetic examples provides an effective way to increase the accuracy of predictions of future activity.

Keywords: Machine Learning, Deep Learning, Image Processing and Pattern Recognition
Abstract: When training neural networks, the weights of the model are updated at each optimization step, and the older weights are discarded}. In this paper, we propose a method called, Training Knowledge Inheritance (TKI), to use the knowledge about the progression of weight and loss data in reducing overfitting and improving the generalization in the later stages of training. We reformulate the traditional gradient optimization problem as a reinforcement learning task. In particular, by treating the trainable weight space as an environment, the learning rate as action, and the validation accuracies as the rewards, we train a Q-network (controller) to learn the discounted future validation accuracy and guide the later training of another network (worker). We conduct the experiments on the MNIST, CIFAR-10, and CIFAR-100 datasets with naive dense neural networks and ResNet-56. Our results show that TKI could emph{rediscover} learning rate schedule rules similar to previous works, including increasing, decaying, and cyclical repeating.

Keywords: Machine Learning, Image Processing and Pattern Recognition, Neural Networks and their Applications
Abstract: Mutual learning algorithm can be an efficient mechanism for improving machine learning and neural network efficiency in a multi-agent system. Specifically, in many cases, where the system cannot be trained using a big training dataset, the data exchange in the teacher-student network system can lead to efficient learning. Usually, in mutual learning algorithms, a big network plays the role of a static teacher and passes the data to smaller networks, known as student networks, to improve the efficiency of the latter. In this paper, we will show that two small networks can dynamically play the changing roles of teacher and student to share their knowledge and hence, the efficiency of both networks improves simultaneously. We demonstrate the concept and the proposed mutual learning algorithm using convolutional neural networks (CNNs) to recognize the benchmark Modified National Institute of Standards and Technology (MNIST) hand-writing dataset

Keywords: Machine Learning
Abstract: The minimal complexity support vector machine (ML1 SVM), which is a fusion of the standard support vector machine (L1 SVM) and the minimal complexity machine (MCM), works to improve the generalization ability over the L1 SVM. In this paper, we introduce a three-kernel structure into the ML1 SVM. We call it ML1 SVM (MK). We speed up model selection by three-stage cross-validation: In the first stage, we determine the hyperparameters for the L1 SVM (MK) fixing the hyperparameters for multiple kernels. In the second stage we determine the hyperparameters for the multiple kernels, and in the third stage, we determine the hyperparameter for the ML1 SVM (MK) for controlling the maximum margin. We also discuss a method for accelerating the second stage of model selection. In the computer experiment using two-class and multiclass problems, the ML1 SVM (MK) showed statistically comparable to, or better than the L1 SVM (MK), ML1 SVM, and L1 SVM.

Keywords: Machine Vision, Deep Learning, Representation Learning
Abstract: This paper addresses the problem of class-incremental learning. The model is trained to recognize the classes added incrementally. It thus suffers from the challenging issue of catastrophic forgetting. Stemming from the knowledge distillation idea of attempting to retain the model's knowledge on seen classes while learning the newly-added ones, we advance to further alleviate the catastrophic forgetting via our proposed multi-anchor distillation objective, which is realized by constraining the spatial relationship between the input data and the multiple class embeddings of each seen class in the feature space while training the model. Moreover, since the knowledge distillation for incremental learning generally relies on keeping a replay buffer to store the samples of seen classes, the buffer of limited size brings another issue of class imbalance: the number of samples from each seen class decreases gradually, thus being much smaller than the number of samples from each new class. We therefore propose to introduce the continually-adapted margin into the classification objective for tackling the prediction bias towards new classes caused by the class imbalance. Experiments are conducted on various datasets and settings to demonstrate the effectiveness and superior performance of our proposed techniques in comparison to several state-of-the-art baselines.

Keywords: Machine Vision, Image Processing and Pattern Recognition, Deep Learning
Abstract: Fusion techniques are frequently utilized in the realm of multimodal object detection tasks. While many current studies showcase their proficiency in generating visually pleasing fused images, only a limited number of them focused on the object detection performance. This study addresses the issue by presenting an end-to-end framework for object detection through the fusion of visible and infrared features (VIFF). Specifically, our approach involves the use of two distinct processing units that independently extract features from visible and infrared images, followed by the fusion of these features using a novel fusion strategy. While the visible feature processing unit preserves the direction of the gradient of visible images, the infrared feature processing unit focuses on extracting the contrast and semantic features of infrared images. Both features are aggregated by attention mechanisms and then fed into the backbone of the object detection networks. Our fusion network achieved superior object detection accuracy compared to existing state-of-the-art approaches on various datasets. We have also demonstrated that the proposed visible feature and infrared feature processing units are capable of enhancing the performance of various object detection models.

Keywords: Machine Vision, Representation Learning, Deep Learning
Abstract: Online continual learning (OCL) aims to enable model learning from a non-stationary data stream to continuously acquire new knowledge as well as retain the learnt one, under the constraints of having limited system size and computational cost, in which the main challenge comes from the ``catastrophic forgetting'' issue -- the inability to well remember the learnt knowledge while learning the new ones. With the specific focus on the class-incremental OCL scenario, i.e. OCL for classification, the recent advance incorporates the contrastive learning technique for learning more generalised feature representation to achieve the state-of-the-art performance but is still unable to fully resolve the catastrophic forgetting. In this paper, we follow the strategy of adopting contrastive learning but further introduce the textit{semantically distinct augmentation} technique, in which it leverages strong augmentation to generate more data samples, and we show that considering these samples semantically different from their original classes (thus being related to the out-of-distribution samples) in the contrastive learning mechanism contributes to alleviate forgetting and facilitate model stability. Moreover, in addition to contrastive learning, the typical classification mechanism and objective (i.e. softmax classifier and cross-entropy loss) are included in our model design for utilising the label information, but particularly equipped with a sampling strategy to tackle the tendency of favouring the new classes (i.e. model bias towards the recently learnt classes). Upon conducting extensive experiments on CIFAR-10, CIFAR-100, and Mini-Imagenet datasets, our proposed method is shown to achieve superior performance against various baselines.

Keywords: Haptic Systems, Kansei (sense/emotion) Engineering, Human Perception in Multimedia
Abstract: In this paper, we propose a method for generating height-map images that can achieve surface texture with the haptic sensation desired by the user. A texture plate was generated based on the height-map image, and a haptic data set was generated through questionnaire evaluation when subjects freely touched the plate. Next, we constructed a haptic estimation model using features based on image decomposition methods and confirmed that haptic sensation can be estimated from image features. Then, we set up a method to calculate evaluation values for haptic and visual impressions and set up an objective function using these values. Finally, we confirmed that we can generate height-map images with the desired haptic sensation by optimizing the objective function to minimize it through verification with a test set.

Keywords: Biometrics and Applications,, Human Factors, Human Perception in Multimedia
Abstract: Eyeblink dynamics, including eyeblink rate, duration, and timings, are widely recognized to reflect internal states such as cognitive and psychological states. Gaze movement strategies have been widely investigated beyond laboratory settings to infer the internal states; however, eyeblink behavior has not been well investigated. Consequently, the relationship between eyeblink dynamics and internal states under natural behavior is still unclear. Eyeblink detection algorithms for head-worn eye trackers are susceptible to extreme eye angles and illumination changes; this limits the study of eyeblink in natural behavior. This issue is addressed by a supervised-machine-learning-aided data analysis pipeline that offers a reliable baseline for eyeblink detection, even when using datasets acquired under harsh conditions. Our pipeline consists of training a deep convolutional neural network model by a transfer learning approach, prediction using the model, and efficient performance monitoring incorporating eyeblink characteristics. In the case study, we analyze eye videos during formula car driving data by our proposed method and the default method of an off-the-shelf eyetracker. Our method predicts better than the present software in terms of the proportion of the predicted eyeblinks with the appropriate duration range as spontaneous eyeblinks and estimated precision. The results suggest that our approach could serve as a valuable tool for cognitive, psychological, and human factors research by enabling high-quality eyeblink dynamics data acquisition from various eyetracking data.

Keywords: Human-Computer Interaction, Augmented Cognition, Wearable Computing
Abstract: Navigating independently is a challenge for visually impaired vision due to the demand of obstacles avoiding, recognizing desired objects, and wayfinding in complicated environments. In this paper, we present an augmented wearable E-Glasses with a set of sensors, where an object detection neural network based on visual-auditory fusion method is employed to search desired targets, thus addressing navigation challenges and improving the mobility and independence of the visually impaired. We demonstrate advanced navigation capabilities: indoor wayfinding, recognizing and steering the users to desired goals, and a sequence of indoor challenges. The fusion network adopts a feature-level fusion strategy, which is capable to align two modalities automatically and effectively integrate visual features and audio features. Across all experiments, the developed fusion algorithm has a 94.67% success rate. The wearable E-Glasses supply a platform that helps to improve the mobility and quality of life of people with impaired vision.

Keywords: Assistive Technology, Human Factors, Kansei (sense/emotion) Engineering
Abstract: Some deaf and hard-of-hearing (DHH) individuals like to ``listen to" music. For them to access a wider range of music, both the preference for and the audibility of music are important. As a starting point for understanding music accessibility, we investigated the audibility and preference for 19 musical instruments using five pitches totaling 95 timbres. To examine how people listen to music daily, owing to the difficulty of having participants come to the laboratory due to the spread of COVID-19, we conducted the experiment online. Participants listened to short clips of 95 sounds and subjectively answered questions on their audibility and preferences. An analysis of their subjective answers revealed the audibility and preference differences among instruments, specifically that the recorder lacks the same audibility and preference as some of the other instruments at certain pitches. However, there was little correlation between audibility and timbre preferences. The study findings will be used in music classes in which DHH students participate and indicate that timbre audibility is necessary for building music recommendation systems for DHH individuals. We also discuss the limitations and possibilities of online experiments using sound.

Keywords: Enterprise Information Systems, Consumer and Industrial Applications, Infrastructure Systems and Services
Abstract: Profile-guided optimization (PGO) is an advanced technique used to improve the performance of Relational Databases (RDBs). However, the most common strategy is to perform profiling on the production environment, which can lead to instability and performance loss for the Database System. To address this issue, we propose an offline profiling strategy that uses a query reduction strategy to obtain a reduced query set from the production environment's log file. We then run this sample on an offline test environment to collect profile data and use it to conduct function reorder optimization.

To evaluate our approach, we compared the performance improvements achieved by running a reduced query set and a full query set on a MYSQL database. We generated function call graphs for both query sets and observed that the performance improvement achieved by the reduced query set was slightly lower than that of the full query set. These results demonstrate the effectiveness of our approach and highlight the potential benefits of offline profiling strategies for improving the performance of RDBs in production environments, while also avoiding performance losses and increasing stability.

Keywords: Decision Support Systems, Consumer and Industrial Applications
Abstract: As people become more aware of their food choices, food computation models have become increasingly popular in assisting people in maintaining healthy eating habits. For example, food recommendation systems analyze recipe instructions to assess nutritional contents and provide recipe recommendations. The recent and remarkable successes of generative AI methods, such as auto-regressive large language models, can lead to robust methods for a more comprehensive understanding of recipes for healthy food recommendations beyond surface-level nutrition content assessments. In this study, we explore the use of generative AI methods to extend current food computation models, primarily involving the analysis of nutrition and ingredients, to also incorporate cooking actions (e.g., add salt, fry the meat, boil the vegetables, etc.). Cooking actions are notoriously hard to model using statistical learning methods due to irregular data patterns - significantly varying natural language descriptions for the same action (e.g., marinate the meat vs. marinate the meat and leave overnight) and infrequently occurring patterns (e.g., add salt occurs far more frequently than marinating the meat). The prototypical approach to handling irregular data patterns is to increase the volume of data that the model ingests by orders of magnitude. Unfortunately, in the cooking domain, these problems are further compounded with larger data volumes presenting a unique challenge that is not easily handled by simply scaling up. In this work, we propose novel aggregation-based generative AI methods, Cook-Gen, that reliably generate cooking actions from recipes, despite difficulties with irregular data patterns, while also outperforming Large Language Models and other strong baselines.

Keywords: Human-Computer Interaction, Human-centered Learning
Abstract: Customer service is very important in many industrial fields, and the service quality is most essential. However, customer service practitioners are with a high turnover rate, and it usually takes months for a new customer service employee to be an experienced one. If the training of new employees is conducted by other experienced employees, there will be a high resource consumption. Therefore, intelligent training systems for customer service can be designed to replace the manual training. In this paper, we define the task of intelligent training for customer service and propose an architecture of intelligent training systems. Dialogue scripts are prepared offline, and a dialogue simulation module and a service evaluation module are separately intended for the online service training and the service quality evaluation. We evaluate state-of-the-art models with respect to the ability to provide service training, and the experimental results show that our proposed system is effective on this task.

Keywords: Human-Machine Interface, Human Factors, Virtual and Augmented Reality Systems
Abstract: As the communication between automated vehicles and human road users becomes increasingly important, this study aimed to investigate the effect of different communication modalities on traffic safety and efficiency. A VR study was conducted wherein 37 participants interacted as pedestrians with an automated vehicle to evaluate implicit communication via lateral offset, explicit communication via an external human-machine interface, and a combination of both. The results indicate that the combined communication approach was ranked the highest in subjective evaluations, while the external human-machine interface achieved the best efficiency results. In terms of traffic safety, no differences were found between the different communication modalities. The study concludes that a combined communication approach, consisting of both implicit and explicit signals, can enable safe and efficient cooperation between traffic participants in future interactions between automated vehicles and pedestrians. However, in situations involving further agents, such as manual vehicles, clearer communication may be necessary. Overall, this study contributes to a better understanding of communication strategies for automated vehicles and provides insights for future research.

Keywords: Human Performance Modeling, Human Factors
Abstract: One of the adjectives used to describe human behavior is ``Gracefulness''. Graceful motion can make a good impression on people.We have proposed an evaluation formula for ``Gracefulness'' by focusing on ``Graceful motion'' defined by William Hogarth and analyzing the trajectory of the arm.However, this evaluation formula is difficult to apply to whole-body movements.Therefore, in this report, in addition to the evaluation of the trajectory of the hand, we analyzed the periodicity and frequency components of movements focusing on the parts of the body other than the arm.The results of these analyses were compared with the impressions of gracefulness obtained from a questionnaire survey.

Keywords: Human-Collaborative Robotics, Systems Safety and Security,
Abstract: This work evaluates a Continuous Wave (CW) Time-of-Flight (ToF) camera, Stereoscopic camera, and LiDAR to determine if they are potential candidates for point-rich on-robot sensing in Speed and separation monitoring (SSM) applications. These experiments characterize the static and dynamic behaviors of the sensors while mounted on-robot. From these tests, it was found that ToF and Stereo cameras exhibit better performance to their more expensive LiDAR counterpart. Specifically, it was observed that the ToF camera demonstrated better depth accuracy while the Stereo camera generated better 3D reconstruction accuracy. Overall, ToF and Stereo Cameras demonstrate that with continued innovation and integration, these sensors could become the building blocks to point rich on-robot SSM.

Keywords: User Interface Design, Virtual and Augmented Reality Systems, Human-Collaborative Robotics
Abstract: The growing spread of robots for service and industrial purposes calls for versatile, intuitive and portable interaction approaches. In particular, in industrial environments, operators should be able to interact with robots in a fast, effective, and possibly effortless manner. To this end, reality enhancement techniques have been used to achieve efficient management and simplify interactions, in particular in manufacturing and logistics processes. Building upon this, in this paper we propose a system based on mixed reality that allows a ubiquitous interface for heterogeneous robotic systems in dynamic scenarios, where users are involved in different tasks and need to interact with different robots. By means of mixed reality, users can interact with a robot through manipulation of its virtual replica, which is always colocated with the user and is extracted when interaction is needed. The system has been tested in a simulated intralogistics setting, where different robots are present and require sporadic intervention by human operators, who are involved in other tasks. In our setting we consider the presence of drones and AGVs with different levels of autonomy, calling for different user interventions. The proposed approach has been validated in virtual reality, considering quantitative and qualitative assessment of performance and user's feedback.

Keywords: Human-Collaborative Robotics, Human-Machine Cooperation and Systems, Human-Machine Interaction
Abstract: Humans have spatiotemporal limitations. Robotic avatar technology enables the expansion of an individual’s innate physical characteristics, physical capabilities, and perceptual and cognitive abilities. Our research group has focused on scalability and co-creativity through co-operation and collaboration between humans, and has developed a system in which two operators are physically integrated into a single robotic arm. In particular, to enable more human-like tasks, such as receiving and delivering, we extended the system to a dual-arm robotic avatar and built a system in which multiple persons can integrate their bodies, with the expectation of further task efficiency and human extension. Specifically, we investigated the workability and operator cognition of the dual-arm robotic avatar when it was operated by one, two, or three individuals. The results showed that stability improved with the three-person operation. Furthermore, it was found that co-operation among multiple persons can overcome the original individual abilities for operation. In the three-person condition, all participants obtained a higher feeling of control than the actual amount of manipulation, which has potential applications in skill transfer. The three-person condition tended to be better, validating the usefulness of multi-person body integration.

Keywords: Human-Machine Interaction, Human-Collaborative Robotics, Human-Machine Interface
Abstract: In daily life, we realize various complex tasks with our two upper limbs based on the so-called bimanual coordination phenomenon. A fundamental feature of bimanual coordination is the natural tendency to synchronize the motion of two upper limbs, resulting in some preferred patterns of interlimb coordination. In this study, based on the coarse-to-fine human-robot collaboration framework, we investigate the possibility of realizing human-robot interaction and collaboration for accurate manipulation utilizing voluntary bimanual coordination. The practical motivation of utilizing the voluntary bimanual coordination, which can be perceived as an indirect way of implementing interlimb transmission of force feedback information, is to avoid the bad effect on force feedback presentation due to the counterforce if the human-robot collaboration was realized in an unimanual manner. For experimental studies, we firstly evaluated the synchronous protocols in terms of in-phase and anti-phase between two arms. Based on the studied protocol, moving target tracking was then demonstrated as an application scenario of the proposed human-robot collaboration utilizing voluntary bimanual coordination.

Keywords: Quantum Cybernetics, Deep Learning
Abstract: Reinforcement learning has been intensively applied to tackle complex quantum control problems owing to its adaptability in dynamic environments. However, commonly used reinforcement learning algorithms are restricted to selecting actions from a discrete action space, which may result in inaccurate control. In order to surpass this constraint, we propose an improved continuous reinforcement learning algorithm that can generate control policies in a continuous action space, enabling precise control of quantum systems. Moreover, a guided reward function design method is proposed to guide the learning process toward higher fidelity. Numerical results demonstrate that the proposed continuous reinforcement learning algorithm with a guided reward function can capably prepare states on one-qubit and two-qubit systems.

Keywords: Quantum Cybernetics, Quantum Machine Learning
Abstract: Abstract—Learning many-body quantum systems is of fundamental importance in quantum information processing, however, it is a challenging task which typically requires estimating quantum distributions of dimensionality exponentially scaling to the system size. As generative models have shown a great scalability to learn high-dimensional distributions and found wide applications in the domain of image and text, they can be a powerful tool to facilitate us to accomplish the challenging quantum tasks. In this work, we propose using normalizing flow (NF) models with fast sampling to learn discrete quantum distributions for quantum state tomography. Particularly, three NF models, including denoising flow, argmax flow, and tree flow, are first adapted to the task of explicit quantum probability density estimation. We then perform extensive experiments on a large scale of quantum systems, and our numerical results demonstrate that these discrete NFs admit an excellent sampling efficiency in the sense that they are insensitive to the system size to learn the high-dimensional quantum distributions, without compromising the learning performance. Finally, in comparison to the other generative models, such as autoregressive, the NFs avoid the problem of slow sequential sampling.

Keywords: Evolutionary Computation, Application of Artificial Intelligence, Computational Intelligence
Abstract: In recent years, artificial intelligence (AI) and machine learning have demonstrated remarkable potential in various application domains, including optimization. This study investigates the process of leveraging AI, particularly large language models (LLMs), to enhance the performance of metaheuristics, with a focus on the well-established Differential Evolution (DE) algorithm. We employ GPT, a state-of-the-art LLM, to propose an improved mutation strategy based on a dynamic switching mechanism, which is then integrated into the DE algorithm. Throughout the investigation, we also observe and analyze any errors or limitations the LLM might exhibit. We conduct extensive experiments on a comprehensive set of 30 benchmark functions, comparing the performance of the proposed AI-inspired strategy with the standard DE algorithm. The results suggest that the AI-driven dynamic switching mutation strategy provides a competitive edge in terms of solution quality, showcasing the potential of using AI to guide the development of improved optimization algorithms. This work not only highlights the effectiveness of the proposed strategy but also contributes to the understanding of the process of using LLMs for enhancing metaheuristics and the challenges involved therein.

Keywords: Agent-Based Modeling, Swarm Intelligence, Optimization and Self-Organization Approaches
Abstract: Biological inspiration from honeybees, insects, and other animals has been used to create interesting implementations of multi-robot swarms. When the robots in a swarm are completely distributed, that is they lack any form of centralized control, the swarm acts as an agent-based model (ABM) wherein each agent implements its own controller and collective behavior emerges from the interactions between agents. Differential equation and graph-based models of some types of swarms have been used to guarantee collective behavior, but guaranteeing or predicting outcomes for hub-based agent colonies with finite numbers of robots remains an open problem. This paper presents a case study of designing an agent-based, hub-based swarm that solves the best-of-N problem with predictable success rates and completion times. The key innovation is modifying a tripartite graph formulation (TGF) from previous work so that it acts as a graph schema which abstracts an ABM into a simplified four state model, which in turn leads to a large discrete time Markov chain (DTMC) that describes how the collective state evolves over time. The DTMC can be used to compute success rates and completion times, which act as predictions for the ABM. Deviations between observed ABM outcomes and DTMC predictions lead to modifications in the ABM so that the swarm becomes more predictable.

Keywords: Evolutionary Computation, Metaheuristic Algorithms, Application of Artificial Intelligence
Abstract: This paper presents an experimental study that compares the performance of four selected metaheuristic algorithms for optimizing a time delay system model. Time delay system models are complex and challenging to optimize due to their inherent characteristics, such as non-linearity, multi-modality, and constraints. The study includes an explanation of the choice and core functionality of the selected algorithms, which are both baseline and state-of-the-art variants of self-organizing migrating algorithm (SOMA), state-of-the-art variant from the Success-History-based Adaptive Differential Evolution family of algorithms, with emphasis on diverse search (DISH algorithm), and Covariance Matrix Adaptation Evolution Strategy (CMA-ES) algorithm. The hyperparameters of the metaheuristic algorithms were set using the iRace automatic algorithm configuration framework. The paper emphasizes the importance of metaheuristic algorithms in control engineering for time-delay systems to develop more effective and efficient control strategies and precise model identifications. The experimental results highlight the effectiveness of the state-of-the-art algorithms with specific adaptive mechanisms like population organization process, diverse search and adaptation mechanisms ensuring a gradual transition from exploration to exploitation. Overall, this study contributes to understanding the challenges and advantages of using metaheuristic algorithms in control engineering for time delay systems. The results provide valuable insights into the performance of modern metaheuristic algorithms and can help guide the selection of appropriate adaptive mechanisms of metaheuristics.

Keywords: Cyborgs,
Abstract: Engineering support for sports has the potential to improve training performance. Most research on sports training support has focused on support in areas that directly affect the point of force action. On the other hand, even if sports with hand-held tools, the importance of lower body motor function has been indicated. This suggests that supporting areas away from the point of force action can improve the performance of novice players. Therefore, in this study, we developed an assist suit that facilitates lower body twisting for table tennis beginners during forehand swing. Using this assistive suit, experiments were conducted for the experimental group under the following four conditions (1) "No wear (before)," (2) "Without assist," (3) "With assist," and (4) "No wear (after)" to clarify the effect of wearing, assist, and training. In addition, a control group in which participants hit the ball without the assist suit was set up to examine the training effect of repetition. The results showed that both the amount of waist rotation and racket velocity increased significantly in the experimental group compared to the control group, which indicates the effect of the assist suit on training.

Keywords: Haptic Systems, Human-Machine Interaction
Abstract: We experience haptic feedback during countless daily manual tasks which involve grasping, cutting, and manipulation of all kinds of objects and materials, either through direct contact with the hand and fingers, or mediated through handheld tools and instruments. Due to our experience and skill in these haptic tasks, it would be reasonable to surmise that teleoperated robotic minimally invasive surgery (RMIS) systems such as the well-known da Vinci from Intuitive would benefit from inclusion of haptic force feedback to the user. Yet despite extensive research and development in the areas of tool-tissue interaction force sensing and haptic feedback to teleoperators, the adoption of haptic feedack in RMIS systems remains limited. This lack of haptic feedback in most RMIS procedures may be due to many different reasons, which will be presented in detail.

To add haptic feedback to a teleoperated robotic system, the control console must incorporate some actuated haptic interface device to present instrument and tissue interaction forces to the surgeon operator. The robotic instruments must include a means of sensing of interaction forces, such as a multi-axis strain gauge sensor. In the case of RMIS, it is a technical challenge to provide useful, accurate, reliable and biocompatible force sensing at the end of surgical instruments. Furthermore, experienced surgeons are accustomed to the lack of haptic feedback in manual MIS when passing laparoscopic instruments through minimally invasive incisions and trocars, and may find added force feedback to be unnatural or a distraction.

Keywords: Brain-Computer Interfaces, Assistive Technology
Abstract: We present a brain-computer interface (BCI) system designed to enable individuals with severe motor disabilities to play Boccia, a Paralympic sport. Boccia is a precision sport in which the objective is to get a ball as close as possible to a target. In its most adapted form, Boccia allows for the use of a ramp to assist the user. The proposed system consists of a BCI-enabled ramp that can be controlled by the user's brain signals using a visual control paradigm (i.e., P300, or SSVEP). We developed a software interface using custom tools in Unity and Python for the front-end and back-end, respectively. To validate the software, we tested the system with five subjects who performed six pipelines (three with P300 and three with SSVEP) to simulate real-world use. Each pipeline consisted of 10 guided selections in the software. The classifiers used Riemannian geometry and shrinkage linear discriminant analysis (sLDA) for P300 and canonical correlation analysis (CCA) for SSVEP. The results showed that the P300 (93 +/- 3%, mean +/- SEM) paradigm had higher classification accuracy than the SSVEP (27 +/- 0.02%, mean +/- SEM) paradigm. Additionally, we designed and built a 3D CAD model and a hardware prototype of the ramp. The hardware prototype uses linear actuators to change the incline of the ramp and the height of the ball. Stepper motors allow for the rotation of the ramp and the release mechanism of the ball. Recommendations on improvements to the hardware and software components are made for future prototypes. The presented system opens new possibilities for sports applications that can improve the quality of life of people with severe motor disabilities.

Keywords: Active BMIs, Passive BMIs, Other Neurotechnology and Brain-Related Topics
Abstract: Brain-computer interfaces (BCIs) like e.g. different types of EEG-based BCI spellers (up to date the most common BCI applications) allow new methods of control and interaction with computers and machines. The impact of background music and noise on user’s performance (information transfer rate - ITR and accuracy) has already been investigated for different types of spellers, and usually led to a distractive, decreasing effect on the BCI performance, though there is always room for improving the subjective user experience. In this study, 11 participants used a cVEP-based BCI to perform two spelling tasks “BCI AND MUSIC” and “CONTRARY”, while listening via headphones to a standard instrumental song, a self-chosen instrumental song (“own music”), or to noise conditions: “no music” and “white noise”. Objective factors such as the blood pressure and heart rate were measured after each spelling task, and questionnaires were answered. The BCI accuracy was close to 100% in most cases, while the lowest accuracy was reached while listening to the individually selected, self-chosen music (“own music”). Similarly, the ITR was the highest in case of “no music” and lowest in case of “own music” (which, on the other hand, resulted in a better mood and higher excitement of the participants). Thus, while the general condition of “no music” could be recommended for further BCI experiments, additional research regarding individual factors like musicality or listening habits of BCI participants should be considered.

Keywords: Active BMIs, Passive BMIs, Other Neurotechnology and Brain-Related Topics
Abstract: The P300 speller is a widely used application in brain-computer interface research. It has been demonstrated that the P300 speller can serve as a neurofeedback training tool for attention enhancement by gradually increasing the difficulty of the spelling task. This adaptive approach makes it harder for users to spell words correctly, encouraging them to improve their attention to counteract the increasing difficulty. Therefore, the adaptive P300 speller has the potential to serve as a treatment option for children with ADHD, elderly patients with dementia, and as a cognitive enhancement tool for healthy adults. However, the training length, including setup time, needs to be quick to ensure user acceptability. This study investigates the effect of different electrode subsets on P300 speller performance, with and without the use of the xDAWN spatial filter. Results indicate that the xDAWN spatial filter can improve performance with many electrodes but can decrease results with fewer than eight electrodes. For scenarios where near-perfect performance is crucial and many electrodes are available, a set of 16 electrodes with the xDAWN spatial filter is recommended. For situations where cost and setup time are a concern and lower performances are acceptable, using six electrodes without the spatial filter were found to be sufficient.

Keywords: Active BMIs, Passive BMIs
Abstract: Children are an under-served population in the field of brain-computer interface (BCI) development. The high prevalence of lifelong disability coupled with the diversity and plasticity of children's brains make them ideal candidates for personalized BCI systems. Channel selection methods provide a tool for the in-session personalization of BCI systems. To evaluate the efficacy of channel selection for pediatric users, we tested four wrapper-based channel selection algorithms, sequential forward selection (SFS), sequential backward selection (SBS), sequential forward floating selection (SFFS), and sequential backward floating selection (SBFS) on offline motor imagery BCI data from three datasets involving typically developing children. The purpose was to assess the performance benefits and computational costs of each algorithm. All algorithms provided classification accuracy gains of 10-15 % with their optimal subsets. The time required to reach the optimal subsets varied between algorithms, but all took less than 80 s with mean completion times of 9.5 s and 35.8 s for the fastest (SFS) and slowest (SFFS), respectively. Adjusting the stopping criterion of the algorithm enables users to further reduce computation time with a disproportionately small effect on classification accuracy. All methods demonstrated an ability to prioritize expected physiological regions of interest and leave out channels detrimental to the classifier. Channel selection offers personalization of the BCI system for a specific user and a specific classifier. These findings emphasize the value of using personalized channel selection algorithms to improve motor imagery BCI systems for pediatric users.

Keywords: Brain-Computer Interfaces
Abstract: A new approach that includes the detection of Error Related Potentials (ErrP) for self-tuning wrong commands in MI-BMI, with the aim of improving the accuracy of a lower limb exoskeleton gait initiation system, is currently in its early stages of development. Due to the requirement of warning the subject before the command is executed, a different type of stimulus must be used to evoke the ErrP in cases where the exoskeleton is about to move against the subject's will. Therefore, it is essential to research a feedback type that better differentiates the ErrP from the correct cases, in order to achieve efficient performance of the BMI. As such, we have analyzed both Tactile (T) and VisuoTactile (VT) feedbacks to not only verify the realism of the designed protocol, but also to examine their effectiveness in eliciting ErrP.

Keywords: Machine Learning
Abstract: In this paper, Gaussian mixture models with tree structure and their variational inference methods are proposed for non-parametric Bayesian clustering. In this model, the tree structure is included as an unobservable random variable. The number of leaf nodes corresponds to the number of mixture components. This model is expected to capture not only the number of clusters but also tree structure from data.

Keywords: Machine Learning, Deep Learning, Neural Networks and their Applications
Abstract: The deep operator network (DeepONet) architecture is a promising approach for learning functional operators, that can represent dynamical systems described by ordinary or partial differential equations. However, it has two major limitations, namely its failures to account for initial conditions and to guarantee the temporal causality - a fundamental property of dynamical systems. This paper proposes a novel causal deep operator network (Causal-DeepONet) architecture for incorporating both the initial condition and the temporal causality into data-driven learning of dynamical systems, overcoming the limitations of the original DeepONet approach. This is achieved by adding an independent root network for the initial condition and independent branch networks conditioned, or switched on/off, by time-shifted step functions or sigmoid functions for expressing the temporal causality. The proposed architecture was evaluated and compared with two baseline deep neural network methods and the original DeepONet method on learning the thermal dynamics of a room in a building using real data. It was shown to not only achieve the best overall prediction accuracy but also enhance substantially the accuracy consistency in multistep predictions, which is crucial for predictive control.

Keywords: Machine Learning, Computational Intelligence
Abstract: Dynamic Ensemble Selection (DES) is a Multiple Classifier Systems (MCS) approach that aims to select an ensemble for each query sample during the selection phase. Even with the proposal of several DES approaches, no particular DES technique is the best choice for different problems. Thus, we hypothesize that selecting the best DES approach per query instance can lead to better accuracy. To evaluate this idea, we introduce the Post-Selection Dynamic Ensemble Selection (PS-DES) approach, a post-selection scheme that evaluates ensembles selected by several DES techniques using different metrics. Experimental results show that using accuracy as a metric to select the ensembles, PS-DES performs better than individual DES techniques.

Keywords: Machine Learning, Evolutionary Computation
Abstract: Traditional evolutionary multi-objective optimization (EMO) algorithm is to generate a set of non-dominated solutions on the Pareto front (PF). However, this technique falls short of delivering the outcomes for multi-objective optimization problems (MOPs) containing user preference. In this paper, we present a novel EMO algorithm that incorporates user preferences via a decision maker (DM). Our approach comprises three modules: consultation, preference elicitation and optimization. The DM undertakes the consultation and preference elicitation using Gaussian process (GP) to provide preference information. We employ the decomposition-based EMO algorithm (i.e., MOEA/D) for optimization. The experiment comprises two sessions. Firstly, we simulate the decision maker module with GP. Secondly, we simulate our proposed method and compare its performance with existing interactive optimization algorithms. Our research proposes a new preference-based EMO algorithm that addresses the shortcomings of traditional techniques and unlocks new possibilities for multi-objective optimization.

Keywords: Evolutionary Computation
Abstract: Hypervolume subset selection (HSS) is a hot topic in the evolutionary multi-objective optimization (EMO) community since hypervolume is the most widely-used performance indicator. In the literature, most HSS algorithms were designed for smallscale HSS (e.g., environmental selection: select N solutions from 2N solutions where N is the population size). Few researchers focus on large-scale HSS as a post-processing procedure in an unbounded external archive framework (i.e., subset selection from all examined solutions). In this paper, we propose a twostage lazy greedy inclusion HSS (TGI-HSS) algorithm for largescale HSS. In the first stage of TGI-HSS, a small solution set is selected from a large-scale candidate set using an efficient subset selection method (which is not based on exact hypervolume calculation). In the second stage, the final subset is selected from the small solution set using an existing efficient HSS algorithm. Experimental results show that the computational time can be significantly reduced by the proposed algorithm in comparison with other state-of-the-art HSS algorithms at the cost of only a small deterioration of the selected subset quality.

Keywords: Evolutionary Computation, Metaheuristic Algorithms, Machine Learning
Abstract: Data clustering has many applications in machine learning, data mining and image processing. K-means is the most popular clustering algorithm due to its efficiency and simplicity of implementation. However, K-means has limitations, such as large feature spaces, which may affect its effectiveness. To improve K-means accuracy, we adopt the Biogeography-Based Optimization (BBO) evolutionary technique to select the most relevant features of datasets. We conducted several experiments to compare our approach with other methods, such as PCA and Particle Swarm Optimization (PSO). The results demonstrate the effectiveness of BBO for feature selection.

Keywords: Evolutionary Computation
Abstract: Population initialization is always needed in evolutionary multi-objective optimization (EMO) algorithms. Intuitively, a well-designed initialization method can help facilitate the evolutionary process and improve the performance of EMO algorithms. However, very few studies have investigated the effects of initialization methods on the performance of EMO algorithms. Many existing EMO algorithms randomly generate an initial population to start the evolutionary process. To fill this research gap and attract more attention from EMO researchers to this important yet under-explored issue, in this paper, we examine the effects of various initialization methods that may become promising alternatives to the commonly-used random initialization method. Each initialization method is evaluated through computational experiments on test problems of various sizes with 5-1000 decision variables. Experimental results clearly demonstrate the advantage of well-designed initialization methods over the random initialization method. This study provides useful insights into EMO algorithm design and motivates further research on population initialization.

Keywords: Evolutionary Computation, Computational Intelligence, Swarm Intelligence
Abstract: Expensive optimization problems (EOPs) are becoming more and more ubiquitous nowadays. To effectively solve such problems, surrogate-assisted evolutionary algorithms (SAEAs) have been developed. Specifically, a SAEA usually maintains a surrogate model to simulate the real objective function of an EOP. Such a surrogate model is trained based on real-evaluated solutions. Then, it is utilized to evaluate the fitness of individuals in the EA instead of the real expensive fitness evaluation. Though many SAEAs have been designed, they mainly concentrate on dealing with low-dimensional EOPs with fewer than 300 dimensions. Their performance on large-scale EOPs with more than 300 dimensions is unknown. To fill this gap, this paper conducts a comparative study on two types of state-of-the-art SAEAs with a total of four algorithms on four classical EOPs. To make comprehensive comparisons, we range the dimension size from 50 to 1000. As far as we know, this is the first time to assess SAEAs on EOPs with such a wide range of dimension sizes and such high dimensionality. The comparison results show that the optimization performance of the compared four SAEAs on high-dimensional EOPs with more than 500 dimensions is not as satisfactory as their performance on low-dimensional EOPs because of their slow convergence. Therefore, research on large-scale SAEAs for high-dimensional EOPs still deserves intensive attention.

Keywords: Evolutionary Computation
Abstract: Recently, it has been pointed out in many studies that the performance of evolutionary multi-objective optimization (EMO) algorithms can be improved by selecting solutions from all examined solutions stored in an unbounded external archive. This is because in general the final population is not the best subset of the examined solutions. To obtain a good final solution set in such a solution selection framework, subset selection from a large candidate set (i.e., all examined solutions) has been studied. However, since good subsets cannot be obtained from poor candidate sets, a more important issue is how to find a good candidate set, which is the focus of this paper. In this paper, we first visually demonstrate that the entire Pareto front is not covered by the examined solutions through computational experiments using MOEA/D, NSGA-III and SMS-EMOA on DTLZ test problems. That is, the examined solution set stored in the unbounded archive has some large holes (i.e., some uncovered area of the Pareto front). Next, to evaluate the quality of the examined solution set (i.e., to measure the size of the largest hole), we propose the use of a variant of the inverted generational distance (IGD) indicator. Then, we propose a simple modification of EMO algorithms to improve the quality of the examined solution set. Finally, we demonstrate the effectiveness of the proposed modification through computational experiments.

Keywords: Human Factors, Multi-User Interaction
Abstract: This study examines the effects of self and other's intentions on their moving performance in crossing interactions with multiple agents. An experimental task paradigm was developed to verify self and other's effects simultaneously. Different intentions were assigned to self and other in the 2-person crossing situation and to the minority and majority in the 4-person crossing situation. Further, performance was assessed based on completion time, amount of operations, and interruptions. The results show that the effects of self-intention on self-performance were generally found, but the minority's intention does not affect its interruption. For completion time and operation, the effect of other's intention decreased in the order of the self in the 2-person crossing situation, minority, and majority in the 4-person crossing situation. Noteworthy, for the interruption, the other's intention affected the majority's interruption, but it did not affect the minority's interruption. These findings emphasize the significance of simultaneously considering self and other's intentions simultaneously when analyzing crossing interactions in shared space.

Keywords: Cognitive Computing, Networking and Decision-Making
Abstract: Offline reinforcement learning (RL) enables learning policies from fixed datasets, avoiding the potential safety risks and cost issues of online interaction with the environment. By collecting data from the real environment, offline reinforcement learning can also alleviate the cost of policy transfer in online learning, thus achieving higher learning efficiency and practicality. However, current offline reinforcement learning algorithms that use value functions to improve policies suffer from the problem of distributional shift, which makes it difficult to accurately evaluate state-action pairs within the dataset, and they pay little attention to the balance between reinforcement learning and imitation learning when improving policies. In this paper, we propose a novel offline learning algorithm ISAR that makes use of in-sample value function learning and advantage-regulated policy improvement. By learning the in-sample state value function, we avoid out-of-distribution action evaluation. And we introduce advantage-weighted behavioral cloning term during policy improvement to balance the relationship between reinforcement learning and behavior cloning. The experimental results show that the ISAR algorithm achieves results comparable to current state-of-the-art algorithms in various robot tasks without complex parameter tuning.

Keywords: Medical Informatics, Human-Computer Interaction, Human-Machine Interface
Abstract: Medical abdomen image segmentation is a challenging task owing to discernible characteristics of the tumour against other organs. As an effective image segmenter, Mask R-CNN has been employed in many medical imaging applications, e.g. for segmenting nucleus from cytoplasm for leukaemia diagnosis and skin lesion segmentation. Motivated by such existing studies, this research takes advantage of the strengths of Mask R-CNN in leveraging on pre-trained CNN architectures such as ResNet and proposes three variants of Mask R-CNN for multi-organ medical image segmentation. Specifically, we propose three variants of the Mask R-CNN transfer learning model successively, each with a set of configurations modified from the one preceding. To be specific, the three variants are (1) the traditional transfer learning with customized loss functions with comparatively more weightage on the segmentation performance, (2) transfer learning based on Mask R-CNN with deepened re-trained layers instead of only the last two/three layers as in traditional transfer learning, and (3) the fine-tuning of Mask R-CNN with expansion of the Region of Interest pooling sizes. Evaluating using Beyond-the-Cranial-Vault (BTCV) abdominal dataset, a well-established benchmark for multi-organ medical image segmentation, the three proposed variants of Mask R-CNN obtain promising performances. In particular, the empirical results indicate the effectiveness of the proposed adapted loss functions, the deepened transfer learning process, as well as the expansion of the RoI pooling sizes. Such variations account for the great efficiency of the proposed transfer learning variant schemes for undertaking multi-organ image segmentation tasks.

Keywords: Networking and Decision-Making
Abstract: Vehicular trajectory data can be widely used in applications such as traffic prediction and congestion control. However vehicular trajectory data is voluminous and requires significant storage and processing resources, which contradicts the resources-constraint of vehicular networks. Existing compression methods suffer either low compression effects or privacy leakage. A privacy-aware Trajectory Compression scheme based on Fuzzy markovian Prediction (TCFP) is proposed in this paper, which consists of two steps of fuzzy compression. The first step compression is achieved by converting the raw trajectory data into fuzzy information on the edge vehicle sides. Further compression is performed at edge RSUs through fuzzy multi-order Markovian prediction combined with new-devised fuzzy deviation filtering rules. Extensive experimental evaluation based on real-world data sets demonstrates the proposed TCFP scheme achieves desired QoS performance in terms of compression rate, compression time, and information loss.

Keywords: Manufacturing Automation and Systems
Abstract: In this study, we proposed a transfer-learning based variational autoencoder model for predicting the electrical characteristics in the parameter tuning process of a-IGZO TFT structure design. The result achieve a high R2 score of 0.9704 with a low-computing-power hardware-friendly method that reduced time consumption significantly compared to prior approaches. The findings have practical implications for mitigating the time-consuming nature of TCAD simulations, and the method can expand to various types of input data while ensuring high performance and generalization. We demonstrated significant improvement in generalization and accuracy through a k-fold validation.

Keywords: Smart Buildings, Smart Cities and Infrastructures
Abstract: We investigate a user-centered approach to trajectory search in large data sets. We assume that user draws a query trajectory and expects to obtain a number of similar trajectories. Unlike the research results reported elsewhere, we try to answer the following simple question: What if we only focus on user experience of trajectory search, and give less or no importance to computational efficiency, how would we design such a user-centered trajectory search? Our answer is an approach based on gates through which a trajectory has to pass in order to be considered similar to the query trajectory. User thus does not have to draw an accurate query trajectory what can require a considerable amount of time, effort and attention to detail, and can lead to user frustration. We compare our approach to existing approaches, Geodabs and Geohash. For experimental evaluation we apply a dense synthetic data set. Our approach finds all trajectories that pass through any required gates, whereas Geodabs and Geohash only find a subset of similar trajectories with respect to a similarity measure.

Keywords: Intelligent Power Grid, Decision Support Systems, Infrastructure Systems and Services
Abstract: Short-term photovoltaic (PV) energy production forecasting is critical for managing grid-connected systems and energy trading. Machine learning models are widely used for accurate prediction, and this study proposes using Time2Vec as an embedding for a transformer-based neural network architecture. Experiments on two PV power plants in India showed significant improvements comparing our proposed architecture to MLP, LSTM, and the persistence model, which is a standard baseline prediction in this type of forecasting, with over 20% improvements in some horizons. These findings demonstrate the effectiveness of the proposed approach for short-term PV forecasting using machine learning models.

Keywords: Decision Support Systems
Abstract: With unexpectedness as a component of serendipity, many previous studies on serendipity-oriented recommender systems have quantified the degree of unexpectedness of items for users as a score. A user's browsing and rating history is necessary for the score calculation. These studies either treated all histories as equal or used only the most recent histories. However, these calculation methods cannot cope with unexpectedness in the case of constant change owing to fluctuations in user preferences and public popularity. In this study, we propose a serendipity-oriented recommender system that sequentially calculates the unexpectedness score and predicts the score in the recommendation time. The proposed system consists of a reranking algorithm that reranks a recommendation list generated by accuracy-oriented recommender systems. It also introduces a parameter to adjust users' ability to accept unexpectedness and aims for serendipitous recommendations that are in line with user preferences. The experiment on two benchmark datasets showed two results: the first is the proposed system improved the serendipitous metric by 0.061 points compared with accuracy-oriented systems before reranking and by 0.045 points compared with them without using the acceptance parameter. The second is the error between the predicted and ground truth of the unexpectedness score was smaller for a large dataset.

Keywords: Robotic Systems, System Modeling and Control
Abstract: 农业采摘领域的主要问题之一 就是要实现全地形图的构建和自主化 导航。许多采摘场景的特点是许多 坑洼大，斜坡多，实际环境复杂。在 实际的自动拣选过程中，拣选机器人有 一定的穿越障碍的能力，并能通过 穿过一些坑洼，而传统 二维地图无法有效识别和判断 障碍，难以实现良好的最优 路径规划。基于这个问题，我们提出了一个基础 一种基于三维的三维地图提取与重建方法 点云数据。在目标区域，首先，无人机是 配备激光雷达，用于3D点云数据 采集，对采集到的点云数据进行去噪， 进行地面分割和提取以及点云 简化，重建3D点云图 为移动拣选提供自主导航 3D 地图 机器人。

Keywords: Consumer and Industrial Applications, Intelligent Green Production Systems
Abstract: With the rise in human activities, the trend of increasing carbon emissions is becoming more apparent. Enterprises are also facing serious challenges in the trend of low carbon and environmental protection, and the procurement process has a significant impact on carbon emissions. In this paper, we propose a procurement solution that integrates many aspects of procurement factors while focusing on reducing carbon emissions. Specifically, we first assess the cost of procurement, the carbon emissions involved, and the quality of the items. We then aggregate the evaluated data and combine them using the analytic hierarchy process to calculate the total qualification value. Also, we specified various constraints and used the E-CARGO model to formalize this problem. Based on the qualification values, we can use the IBM ILOG CPLEX Optimization (CPLEX) package to find the best sourcing solution according to the working hour allocation. In our experiments, our proposed solution can effectively derive the optimal procurement solution based on the purchaser's needs while focusing on reducing the carbon emissions involved in the procurement.

Keywords: Control of Uncertain Systems, Decision Support Systems, Discrete Event Systems
Abstract: Facial expression recognition(FER) is one of the important research in computer vision, which has been widely applied inhuman-computer interaction, education, healthcare, transportation, etc. However, the wide application of facial expression recognition technology also brings new challenges, where occlusion and pose variation are two of the worst factors that disturb facial expression recognition in the wild. We propose a facial expression recognition method based on local feature augmentation and multi-scale global correlation which can adaptively extract robust local features and global features from the feature level to suppress the disturbances of occlusion and pose variation on facial expression recognition. The experimental results show that our method performs well on the RAF-DB dataset and has stronger robustness compared with other algorithms.

Keywords: Adaptive Systems
Abstract: Although cloud manufacturing offers greater flexibility and diversity than traditional manufacturing, it also presents greater uncertainty and variability. How to select stable cloud providers for material procurement (SSCPFMP) has become a critical supply chain optimization issue in cloud manufacturing. By extending the Group Role Assignment (GRA) model, this paper formalizes the problem. Moreover, we propose a new method for evaluating cloud providers that incorporates stability as an important criterion. Additionally, in order to complete the stability assessment as quickly as possible, we propose using the KB4 logic instead of the commonly used KB5 to mine potential cooperative relationships between cloud providers. We prove by deduction that KB4 and KB5 are equivalent. Largescale simulation experiments indicate that the KB4 logic performs significantly better than the KB5 logic, which can be improved by up to 43.78%. By using this method, decision makers are able to find more stable cloud providers for material procurement within a shorter timeframe.

Keywords: Trust in Autonomous Systems, Cooperative Systems and Control, Robotic Systems
Abstract: Trust evaluation and trust establishment play crucial roles in the management of trust within a multi-agent system. When it comes to collaboration systems, trust becomes directly linked to the specific roles performed by agents. The Role-Based Collaboration (RBC) methodology serves as a framework for assigning roles that facilitate agent collaboration. Within this context, the behavior of an agent with respect to a role is referred to as a process role. This research paper introduces a role engine that incorporates a trust establishment algorithm aimed at identifying optimal and reliable process roles. In our study, we define trust as a continuous value ranging from 0 to 1. To optimize trustworthy process roles, we have developed a consensus-based Gaussian Process Factor Graph (GPFG) tool. Our simulations and experiments validate the feasibility and efficiency of our proposed approach with autonomous robots in unsignalized intersections and narrow hallways

Keywords: Virtual/Augmented/Mixed Reality, Virtual and Augmented Reality Systems, Systems Safety and Security
Abstract: The Metaverse is an emerging concept that aims to integrate the physical and virtual worlds, creating a shared 3D virtual world where users can interact and immerse in new experiences. With the rise of Metaverse-driven Cyber- Physical-Social Systems (CPSSs), integrating drones as a critical technology in the Metaverse has become increasingly important. CPSSs have become proliferating and integral to our daily lives. This paper proposes a conceptual framework for Metaverse- driven drone edge intelligence, which integrates drone-enabled sensing, communication, and computation to enable real-time decision-making in CPSSs. We present a detailed analysis of the challenges and opportunities for integrating drones in the Metaverse and discuss the potential impact of our framework on various application domains. Our work contributes to advancing the Metaverse and CPSSs by providing a novel approach for empowering real-time decision-making and enabling new user experiences through integrating drones and the Metaverse. The proposed framework has the potential to revolutionize the way we approach data-driven decision-making in various industries and applications, including precision agriculture, transportation, emergency response, smart cities, healthcare, manufacturing, and energy

Keywords: Discrete Event Systems, System Modeling and Control, Cyber-physical systems
Abstract: General anesthesia is used to induce a reversible loss of consciousness in a patient to ensur they are completely unaware and pain-free during a surgical procedure. This is achieved through the administration of a combination of drugs that depress the central nervous system and can include intravenous injections, inhaled gases, or a combination of both. During the anesthesia maintenance phase, the events in surgery are not continuous, but obey a profile similar to that endorsed by event-based control. The present study considers event-based control as a natural solution to the control challenge of general anesthesia. Two fractional order Proportional Integral (PI) controllers are tuned and implemented using a decentralized approach to control a part of general anesthesia based on hemodynamic variables. The results show that this approach can be successfully used to keep Cardiac Output and Mean Arterial Pressure stable, with a special focus on patient safety, through fast detection of surgical stimulus and robustness to inter/intra patient variability.

Keywords: System Modeling and Control
Abstract: In this paper, we formally analyse the security of Uptane 2.0 -- the latest framework for over-the-air, i.e., online, delivery of software to cars. We are doing so by using the threat model and security requirements found in standard document that accompanies Uptane 2.0, as well as a modulation of this threat model and requirements added by ourselves, for a deeper analysis. To undertake this verification, we use the well-known formal protocol-verifier and theorem prover called Tamarin. We discuss our responsible disclosure to and work with the Uptane Alliance.

Keywords: Cyber-physical systems, Technology Assessment, Service Systems and Organizations
Abstract: The implementation of Systems Engineering (SE) offers a solution for organizations facing challenges from digitalization and increasing product complexity and interconnectivity. However, the implementation of SE company-wide requires a holistic approach that considers human and organizational factors beyond just technology and technical processes. This paper presents a systematic literature review (SLR) that aimed to determine the key principles for the effective application of SE in industrial practice. Through open, axial, and selective coding, 12 key principles were identified. Finally, an application example is given to show how the principles can be used to improve the process of the offer phase in special purpose machinery.

Keywords: Expert and Knowledge-Based Systems
Abstract: Like many physiological systems, the various mathematical models describing the glucose-insulin system can only have non-negative inputs. Thus, the control method --- often model predictive control in artificial pancreas systems---must provide a non-negative control signal. Existing solutions include saturation and constrained optimization. In this paper, we propose a dynamic extension of the patient model as a way of ensuring the positivity of the control signal, a method previously applied in tumor growth control. We evaluate the controller in a closed-loop simulation and compare the results with a controller using saturation. Our simulations show that control performance with the extended model can reach or exceed the performance achieved with saturation.

Keywords: Expert and Knowledge-Based Systems
Abstract: Despite the increasing availability and reliability of artificial pancreas devices, many people with type 1 diabetes mellitus are still on multiple daily injections therapy consisting of a daily basal insulin injection and mealtime boluses. Use of an insulin bolus advisor may improve glycaemic control as well as reduce the burden of the disease on these patients. This paper investigates the application of an impulsive model predictive controller in a bolus advisor system. The bolus calculator is assessed in closed-loop simulations using different basal insulin scenarios. The simulations show that a model predictive controller can achieve good glycaemic control and may be suitable to give bolus recommendations to people with diabetes. Higher basal insulin levels can result in better times in target range, but also carry a higher risk of hypoglycaemia.

Keywords: Image Processing and Pattern Recognition, Application of Artificial Intelligence, Neural Networks and their Applications
Abstract: Brain tumor segmentation has been a widely researched topic for decades, and it intensified ten years ago as a consequence of the Brain Tumor Segmentation Challenges (BraTS), which provided and yearly updated a standard multi-spectral brain tumor MRI data set and a unified evaluation framework to the research community. This paper proposes a procedure for brain tumor segmentation, which uses a spatial histogram enhancement method to preprocess the data, and two identical cascaded U-net networks that work with 3D convolution. The first U-net accomplishes an intermediary segmentation of the brain volume, while the second one reevaluates the labels given to pixels based on the labels of neighbor pixels. The output of both U-nets are evaluated using statistical accuracy benchmarks. The proposed procedure achieved an average Dice score of 88.8% on the high-grade glioma records of the BraTS 2019 training data set. Post-processing increased the average Dice score by 1.1%, but in case of typical small high-grade tumor lesion it can achieve an improvement of up to 5%.

Keywords: Application of Artificial Intelligence
Abstract: Reinforcement learning (RL) has shown promise in controlling blood glucose levels in a personalized way in type 1 diabetic patients. In this study, we investigate the impact of different activation functions and layer numbers on RL performance in blood glucose control. We train RL agents with various combinations of activation functions and layer numbers on a virtual patient model. The RL agents are evaluated based on their ability to maintain blood glucose levels within a target range while minimizing the frequency and magnitude of hypoglycemia and hyperglycemia events. Our results show that the choice of activation function and layer number significantly affects the RL performance. Specifically, the agents with ReLU activation functions and two or three hidden layers outperform the other agents, achieving a higher percentage of time in the target range and fewer hypoglycemia and hyperglycemia events. These findings provide valuable insights for the development of RL-based blood glucose control systems in type 1 diabetic patients.

Keywords: Computational Life Science, Biometric Systems and Bioinformatics, Cybernetics for Informatics
Abstract: Scalp electroencephalography (EEG) provides a practical tool for the identification and characterization of various brain states, including healthy and diseased conditions. We measure brain dynamics on the scalp via a HydroCel Geodesic Sensor Net, 128 electrodes dense-array EEG. We compute the pragmatic information index (PI) by Hilbert analysis of the EEG signals of 20 healthy participants. We compare 6 task modalities, combining different audio-visual stimuli, leading to various mental states, predominantly relaxed versus engaged. We analyze PI values to classify different brain states. We show significant differences between the measured neural signatures depending on the task modalities, based on both qualitative and quantitative analysis. The results can help to develop tools for medical diagnostics of stress-related mental conditions. Keywords — EEG, Cognition, Intentional Action, Pragmatic Information, Meditation, Awareness, Meaning, Knowledge.

Keywords: Telepresence, Shared Control, Human-Computer Interaction
Abstract: Mobile Remote Presence (MRP) robots have emerged out of the need for telepresence in various settings such as the workplace and hospitals. As with face-to-face experiences, these robot mediated encounters have social aspects that current commercially available MRP robots lack the capabilities to incorporate. In previous work, we integrated a manipulator onto a commercial telerobotic platform to enable expressive gestures and demonstrated that the gesturing capabilities enhanced the social connection between remote and local users. However, we also found that controlling the robot for complex interactions, such as a handshake, diminishes the remote user’s social experience. This paper presents the discovery of models for handshakes in different social contexts, which can be used in a shared-control architecture to reduce the effort on the remote user. Using a haptic measurement glove, force and inertia data was collected for human-human handshakes in various social contexts. By applying a k-nearest neighbor algorithm in combination with dynamic time warping and a support vector machine algorithm, two classification models are derived that predict the social context and can be used in an intelligent shared-control robot architecture.

Keywords: Haptic Systems, Telepresence, Team Performance and Training Systems
Abstract: Encountered-type haptics (ETH) is an emerging research field that enables unencumbered physical haptic interaction in virtual reality (VR). In this paper, we propose Encountered-type haptics (ETH) as an interaction medium for immersive remote teleoperation, facilitating intuitive bare-hand interaction with visuo-haptic feedback. Our system allows a human operator to control a remote robot immersively through the visual rendering of the VR environment, while interacting with a haptic robot at the user site. The ETH feedback rendering and the teleoperation at the remote site are both implemented using a 7 degrees-of-freedom (DoFs) Franka Emika Panda robot under Cartesian-impedance control. The Cartesian goal poses for each robot are determined based on the pose of the operator's proxy hand in the VR environment and the operator's interaction intention, estimated through hand gestures and gaze direction. The impedances of both robots are updated at runtime to provide the operator with bilateral haptic interaction forces. Our system was evaluated through a user study involving a door-opening task under three teleoperation conditions: (1) without haptic feedback, (2) ETH feedback with constant impedance, and (3) ETH feedback with variable impedance. The results highlight the advantages of using ETH, including the ability to regulate forces at both the user and robot sites. ETH conditions demonstrate lower peak forces and reduced force jittering at the remote site. Furthermore, the variable impedance condition within ETH shows improved task execution times and reduced exerted force. This paper demonstrates that ETH is an effective medium for immersive bare-hand bilateral teleoperation.

Keywords: Visual Analytics/Communication
Abstract: Collaborative 3D object detection, with its improved interaction advantage among multiple agents, has been widely explored in autonomous driving. However, existing collaborative 3D object detectors in a fully supervised paradigm heavily rely on large-scale annotated 3D bounding boxes, which is laborintensive and time-consuming. To tackle this issue, we propose a sparsely supervised collaborative 3D object detection framework SSC3OD, which only requires each agent to randomly label one object in the scene. Specifically, this model consists of two novel components, i.e., the pillar-based masked autoencoder (PillarMAE) and the instance mining module. The Pillar-MAE module aims to reason over high-level semantics in a self-supervised manner, and the instance mining module generates high-quality pseudo labels for collaborative detectors online. By introducing these simple yet effective mechanisms, the proposed SSC3OD can alleviate the adverse impacts of incomplete annotations. We generate sparse labels based on collaborative perception datasets to evaluate our method. Extensive experiments on three largescale datasets reveal that our proposed SSC3OD can effectively improve the performance of sparsely supervised collaborative 3D object detectors.

Keywords: Visual Analytics/Communication
Abstract: Multi-agent collaborative perception depends on sharing sensory information to improve perception accuracy and robustness, as well as to extend coverage. However, most collaborative perception methods ignore the limitations of communication networks, such as limited bandwidth and the possibility of wireless conflicts. To fill this gap, this paper proposes BlindSpotEliminator, a conflict-free scheduler over the cellularV2X networks for supporting practical collaborative point cloud perception to eliminate blind spots. BlindSpotEliminator first identifies the blind spots for each vehicle, then lists the corresponding conflict relationships based on the distribution of the blind spots and communication conflicts, and finally designs an optimized point cloud data transmission strategy to eliminate the blind spots of each vehicle. Extensive experiments show that compared with greedy algorithm and random methods, BlindSpotEliminator achieves better efficiency, i.e., transmitting 20% more point cloud data.

Keywords: Visual Analytics/Communication
Abstract: In this paper, we propose a masked and sparse transformer to address the problem of point cloud registration with low overlap. The mask mechanism reduces the overall data, increasing the corresponding point ratio in the overlap region, while also reducing the computational cost to accelerate the algorithm’s execution speed. Moreover, we combine spatial position encoding and sparse self-attention to establish relationships within the source point cloud, as well as the relationships and attention scores between the source and target point clouds. This approach is specifically designed for the task of point cloud registration. Finally, we search for the maximum overlap area by matching the spatial consistency between points and calculate the 3D transformation matrix to complete the registration process. Our method achieves an improvement in the inlier ratio and performs well on the 3DMatch and 3DLoMatch datasets, demonstrating high registration efficiency.

Keywords: Telepresence, Virtual/Augmented/Mixed Reality, Ethics of AI and Pervasive Systems
Abstract: The digital transformation is ongoing and affects all aspects of our professional and personal life. This paper sketches three transformations and introduces the concept of Cyber-Physical Humans as an inevitable result of their merger. Cyber-Physical Humans connect the cyberspace (back) with the physical space and derive their value in their capability to interact directly, or indirectly, with the physical world. This means that we will encounter representations of people present in cyberspace in the real world, for instance as holographic projection or as robotic avatar. Emerging technologies allow us to be present in the real world, in a digital world, in a remote world, or in multiple worlds at the same time. It is no longer hypothetical to be in the real world and talk with one colleague who is present through the metaverse and a holographic projection and another who is present through a robotic avatar operated in telepresence. These digital transformations will blur the lines between the real, the virtual, and the remote world with potentially huge consequences for the way we live, work and communicate. This predictably comes with great opportunities and risks amongst others health risks (isolation, problematic use, cyber sickness), and security & privacy risks (deep-fakes, alternate presentations, spoofing, identity theft). It is important to address these risks to ensure that the benefits of Cyber-Physical Humans are maximized while minimizing the negative consequences.

Keywords: Haptic Systems, Telepresence
Abstract: Haptic perception is highly important for immersive teleoperation of robots, especially for accomplishing manipulation tasks. We propose a low-cost haptic sensing and rendering system, which is capable of detecting and displaying surface roughness. As the robot fingertip moves across a surface of interest, two microphones capture sound coupled directly through the fingertip and through the air, respectively. A learning-based detector system analyzes the data in real time and gives roughness estimates with both high temporal resolution and low latency. Finally, an audio-based vibrational actuator displays the result to the human operator. We demonstrate the effectiveness of our system through lab experiments and our winning entry in the ANA Avatar XPRIZE competition finals, where briefly trained judges solved a roughness-based selection task even without additional vision feedback. We publish our dataset used for training and evaluation together with our trained models to enable reproducibility of results.

Keywords: Human Performance Modeling, Virtual/Augmented/Mixed Reality, Human-Collaborative Robotics
Abstract: Virtual, Augmented, and Mixed Reality (VAMR) applications and trainings have gained traction across business sectors, including healthcare, all around the globe. Yet, while technology has made huge jumps towards displaying photo-realistic environments and applications, non-visual feedback (auditory, haptic, olfactory) is still evolving. In PR, humans are processing their environment multi-modally and base their actions on those multi-modal perceptions. Thus, it remains unclear if human action dynamics in information-reduced, vision-heavy VAMR of today resembles that in physical reality. We therefore present a pilot study in a simple human-robot co-action paradigm investigating dynamic performance differences between VAMR and PR during task performance. Results suggest that some aspects of task performance in VR may be the same as in the physical setting, however, users may utilize different strategies in these modalities because the consequences for failure are different, which may have consequences for VAMR related application and trainings.

Keywords: Virtual and Augmented Reality Systems
Abstract: In this presentation, we introduce the concept of the rehabilitation platform and computational approaches for cognitive modeling. Firstly, we discuss immersive VR/AR systems and assessment programs designed for higher brain dysfunction. Additionally, we explore the significance of the co-creative platform as a novel approach to rehabilitation spaces. We then explain the structured learning process, where each learning system serves as an interdependent subsystem, for modeling the perceiving-acting cycle in rehabilitation tasks. The architecture comprises four subsystems: the perceptual system, action system, attention system, and anticipation system. Topological mappings and recurrent neural networks are employed for spatiotemporal pattern modeling. Furthermore, we show several experimental examples conducted in clinical settings. Finally, we discuss the future directions of research concerning the co-creative rehabilitation platform.

Keywords: Transfer Learning, Machine Learning, Deep Learning
Abstract: As the central component of a significant power machine, the gas turbine’s fault diagnosis accuracy is critical to the equipment’s safety in service. However, the fault detection of the gas engine rotor system still faceschallenges due to the difficulty of acquiring sensitive features and the lack of labeled data. To address these issues, we propose an improved DQNbased Transfer Reinforcement Learning method (Transfer DQN) for robust gas turbine rotor fault diagnosis. The proposed method takes the collected one-dimensional raw vibration signal as input, with the fault sample set and fault category serving as the model environment and action. It uses a multi-scale one-dimensional wide convolutional neural network (M-WDCNN) with ϵ-greedy strategy for Q-network fitting and decision making. Additionally, to consider the computational efficiency and differences between fault classes, Transfer-DQN uses multiple fault sample data as the source domain and a single fault class as the target domain, while performing the source-to-target domain transfer learning based on generative adversarial. Extensive experiments on the bearing dataset of Western Reserve University and our gas turbine test bench demonstrate the superiority of Transfer-DQN, achieving accuracies of 98.95% and 96.91%, respectively. Compared with baseline approaches, our method breaks through the previous upper limit of 95% to meet the need for robust and efficient fault diagnosis.

Keywords: Transfer Learning, Computational Intelligence, Evolutionary Computation
Abstract: We present a Lifelong Multi-Tasking learning algorithm based on Genetic Programming referred to as``LLGP-Tasking". This paper extends previously published work on "GP-Tasking", an evolutionary drive optimisation approach for evolving a population of GP trees using a multifaceted strategy. In GP-Tasking, each individual is trained with different training sets and evaluated with multiple fitness functions (where each function represents one task). Empirical evidence demonstrated that the quality of evolved solutions is comparable to standard GP achieving significantly faster computational time while maintaining smaller evolved population sizes. In this work, we improved GP-Tasking and introduced a new crossover mechanism to transfer useful knowledge across different tasks. Further, we introduced new population initialisation approach to accumulate knowledge across different domains. The new LLGP-Tasking can solve multiple problems simultaneously and receive sequentially new batches of problems, Experimental results of the new LLGP-Tasking demonstrate superiority of evolved solutions over standard GP and it maintained same search speed produced by its predecessor (i.e., GP-Tasking).

Keywords: Neural Networks and their Applications, Machine Learning
Abstract: This paper presents two machine learning-based constraint management approaches based on Reference Governors (RGs). The first approach, termed NN-DTC, uses regression neural networks to approximate the distance to constraints. The second, termed NN-NL-RG, uses regression neural networks to approximate the input-output map of a nonlinear RG. Both approaches are shown to enforce constraints for a nonlinear second order system. NN-NL-RG requires a smaller dataset size as compared to NN-DTC for well-trained neural networks. For systems with multiple constraints, NN-NL-RG is also more computationally efficient than NN-DTC. Finally, promising results are reported by having both approaches implemented on a more complex spacecraft proximity maneuvering and docking application, through simulations.

Keywords: Neural Networks and their Applications, Machine Learning, Application of Artificial Intelligence
Abstract: This paper proposes a time-efficient model for solving the Weapon-target assignment (WTA) problem with actor-critic reinforcement learning. While typical heuristic algorithms and recently studied artificial neural network methodologies have shown good performance results, previous approaches has not been time-efficient in large-scale WTA problems. This paper utilizes the actor-critic framework to resolve the WTA problem, and this framework enables retrieving solutions 23 times faster than the previous deep Q-network approach. Additionally, we incorporate a recurrent neural network model of gated recurrent units (GRU) to allow agents to learn the latent state-space of the WTA problem. Our experiments demonstrate the solution quality and the time efficiency compared to traditional heuristic methods as well as recent DQN-based RL models

Keywords: Machine Learning
Abstract: The extension of health expectancy is getting a more important subject for developed countries with higher rate of elderly population. In order to sustain their societies, aged people are encouraged to participate in the social activities, while the social participation requires them to keep their locomotive function high enough to accomplish their missions. From this reason, the easy and frequent assessment of the locomotive function thought the people’s daily lives is strongly expected. One of the promising methods to evaluate locomotive function is the Timed Up and Go test (TUG test), which is a widely used method in the field of physical therapy for locomotive rehabilitation in medical facilities. Throughout our research activities, we have been investigating robotic approaches to assess cognitive and motor functions of aged people in the purpose of their health expectancy extension. In this article, we will propose a novel method of a gait measurement system that will be incorporated with health care mobile robots developed previously. We demonstrated experiments to measure the walking patterns of 23 subjects of aged people in order to evaluate their locomotive functions using the proposed gait measurement system.

Keywords: Adaptive Systems, Cyber-physical systems, System Architecture
Abstract: With higher education pushing toward larger class sizes, a large portion of current methodology focuses on one-size-fits-all approaches that can effectively educate a large class. However, when these approaches fail, students can be left behind and fail classes due to simple misunderstandings. Inspired by these issues, this paper proposes a modular reinforcement learning system that can be used in intelligent educational systems to inform personalized student support. Based on a similar method detailed in prior work, this paper proposes experience sharing with tutor agents as a computationally light approach to improve reinforcement learning training speed on the task of student support. We also provide preliminary results obtained from student simulations to demonstrate the effectiveness of the proposed method on reinforcement learning agent performance.

Keywords: Evolutionary Computation, Computational Intelligence, AI and Applications
Abstract: Multiple traveling salesmen problem (MTSP) is an extension of traditional traveling salesman problem (TSP). It involves both the city assignment optimization and the route optimization of each salesman. Genetic algorithms (GA) have been widely used to solve MTSP thanks to its easiness in implementation and good global search ability. To help GA effectively solve MTSP, researchers have developed various encoding schemes. However, there is no systematic and comparative study on the effectiveness of these encoding strategies. To fill this gap, this paper conducts investigations to compare four popular encoding strategies for MTSP, namely the one-chromosome encoding, the two-chromosome encoding, the two-part-chromosome encoding and the multi-chromosome encoding. Experimental results on different MTSP instances with different numbers of cities and salesmen show that the multi-chromosome encoding is far better than the other encoding strategies.

Keywords: Assistive Technology, Medical Informatics
Abstract: Postural monitoring in wheelchair users is a topic of growing interest. The detection of changes in the sitting patterns of these patients may serve to detect changes in their functional status and be able to adapt rehabilitation early. For this reason, this paper presents a methodology for the detection of specific postural anomalies that, unlike previous works, adopts unsupervised learning. The proposed methodology involves data dimensionality reduction using Principal Component Analysis, and the application of K-means clustering to group different normal posture states. The anomalies are detected using a threshold approach, where data points that fall outside a certain threshold are considered as anomalies. The results show that the methodology is effective in identifying anomalies with a high degree of accuracy (around 90%).

Keywords: Evolutionary Computation, Metaheuristic Algorithms, Computational Intelligence
Abstract: Mutation plays a crucial role in assisting differential evolution (DE) to effectively solve optimization problems. The key to mutation lies in the selection of parent individuals participating in the mutation. Along this road, this paper devises a binomial distribution-assisted individual selection strategy for DE. Spe-cifically, this paper takes advantage of the probability distribu-tion function of the binomial distribution to assign weights to individuals based on their fitness rankings. In this way, the selection of individuals focuses more on medium better individ-uals instead of the top best ones. Therefore, high mutation di-versity can be preserved and thus it is likely that falling into local regions can be effectively avoided. Embedding this selec-tion strategy into DE, a novel DE variant called binomial dis-tribution assisted DE (BDDE) is developed. Experiments con-ducted on the CEC2017 benchmark suite have verified the ef-fectiveness of BDDE in solving optimization problems. Particu-larly, BDDE gains much better performance against the well-known and representative mutation strategies.

Keywords: Evolutionary Computation, Metaheuristic Algorithms, Computational Intelligence
Abstract: Ant Colony Optimization (ACO) has shown very promising performance in solving Traveling Salesman Problem (TSP). However, most existing ACO algorithms utilize either the abso-lutely best ants or all ants to update the pheromone matrix. This leads to either serious diversity loss or slow convergence. To alleviate these predicaments, this paper designs a random pairwise competition based ant selection for pheromone updat-ing. Specifically, a number of ants are randomly selected from the ant colony and then are randomly paired together. Subse-quently the better one in each pair is selected to update the pheromone matrix. In this way, a good balance between search diversity and search convergence is potentially maintained. Integrating this selection strategy along with a local search scheme into the ACO framework, a new ACO algorithm called random pairwise competition based ACO (RPCACO) is devel-oped. Experiments conducted on 8 TSP instances from the TSPLIB benchmark set demonstrate that RPCACO is more effective and efficient than the five classical ACO algorithms in solving TSP.

Keywords: Human-Machine Cooperation and Systems, Human-Computer Interaction
Abstract: Expanding the shared control paradigm for autonomous vehicles to include the open-loop unstable region can improve safety in extreme vehicle conditions, such as tire saturation from encountering low friction or from emergency lane changes. Whereas the state-of-the-art sacrifices agility for stability by restricting the vehicle domain to operate in the open-loop stable regime, this paper expands existing shared control approaches to ensure safety outside the open-loop stable region. A Nonlinear Model Predictive Control framework balances a cost to follow the driver's commands and costs for safety, such that it does not disturb the driver in safe states and intervenes only when necessary in dangerous states seamlessly. Specifically, a novel cost function is formulated to incorporate driver intent, while ensuring safety through costs on the maximum phase recovery envelope and track bounds. Ensuring sufficient model fidelity for drifting is achieved through a nonlinear bicycle model that incorporates a nonlinear tire model and wheelspeed dynamics. Circular drifting experiments with a full-scale vehicle demonstrate the ability of the controller to follow driver commands in safe states, while augmenting the driver commands to avoid situations of track bound violation and spin-out when drifting a circle.

Keywords: Team Performance and Training Systems, Systems Safety and Security, Shared Control
Abstract: This paper studies how to renew and improve the expert rule and apply the rule update mechanism to optimize UAVs swarm adversarial strategy. A rule update approach is proposed, which uses the classifier subsystem to construct a training model based on expert experience, further trains the model through rule evaluation mechanisms and rule discovery subsystems to improve and enhance the rule base. A UAV swarm confrontation strategy model is further proposed based on the learning classifier system(LCS). Under the simulated aerial engagement environment of island capture between the red and blue sides, simulation experiments show that the model has robust combat effectiveness and offers significant practical utility for agent decision.

Keywords: Human-Computer Interaction, Shared Control, Virtual and Augmented Reality Systems
Abstract: We proposed a teleoperation method, illusory control (IC), that provides a comfortable operation experience using a seamless transition between real and pre-prepared virtual environments. Therefore, the mobile robot with IC can function solely in familiar environments. However, this study proposes a novel method, instant IC, that eliminates the requirement for a pre-prepared virtual environment. The proposed robot system can instantly generate a virtual environment using actual 360° images of the robot in motion, utilizing instant neural graphics primitives and neural radiance fields. The 360° images allow the entire surrounding environment to be virtualized without requiring specific camera orientations. In addition, by optimizing the density of neural radiance fields using depth estimation results beforehand, the reconstruction accuracy at unknown poses can be guaranteed. Furthermore, we proposed a depth scaling method based on the actual measurements obtained by LiDAR to increase the consistency of virtual and real environments. With this instant virtual environment, the proposed system enables teleoperation in unknown environments via the seamless transition between real and virtual environments. The experimental results exhibit consistent and smooth back-and-forth transitions between virtual and real space in mobile robot teleoperation.

Keywords: Human-Centered Transportation, Shared Control, Virtual/Augmented/Mixed Reality
Abstract: Driving through traffic often involves sequences of distinct maneuvers, for example changing lanes and overtaking slower vehicles. To assist drivers in these situations, a system for shared decision making and control (SDMC) is developed, taking inspiration from the lead-follow relationship in partner dancing. The SDMC system plans maneuvers through parallel nonlinear optimizations, infers the driver's intended maneuver, and shares control over steering, throttle, and braking actuators to jointly execute the maneuver. Experimental results in overtaking and lane changing scenarios demonstrate the driver's ability to guide the system through sequences of maneuvers and the system's support of the driver via shared lateral and longitudinal control.

Keywords: Human-Machine Cooperation and Systems, Shared Control, Design Methods
Abstract: In recent years, sustainable development goals (SDGs) have attracted attention. In Japan, ”Society 5.0” has been proposed and promoted by various organizations as a vision for a future society linked to the achievement of SDGs. In particular, ”i-Construction” is being promoted in the construction industry. For example, hydraulic excavators are being automated (or semiautomated). Nevertheless, it is important for operators to have a sense of accomplishment and motivation for their work through active operations. In this study, a control system that automatically adjusting an operational assist rate according to the person is proposed . It is difficult to model a person because they exhibit time-varying and non-linear characteristics. Therefore, the control system uses work data to evaluate differences from the desired characteristics, and adaptively adjusts the operational assist rate. The proposed method is implemented on a hydraulic excavator and its effectiveness is verified.

Keywords: Haptic Systems, Shared Control
Abstract: In this work, an optimization-based control scheme for a bilateral teleopered unmanned aerial vehicle (UAV) is proposed using control barrier functions (CBF) and passivity. We consider a human operator moving the position of an end effector of a haptic device as the velocity command input to the UAV. The CBF is applied as a constraint to maintain the collision-free motion of the UAV for safety. On the other hand, to preserve the stability of the systems, we consider passivity as another constraint together, but it leads to undesired behavior of an optimal solution for haptic feedback. To deal with the issue, the strategy called energy tank for passivity is included to replace the condition of strict output passivity. The aforementioned designs are formalized as a quadratically constrained quadratic program (QCQP) to solve numerically. Through numerical examples, we have a comprehensive discussion about the features of the proposed controller with better safety. The experimental results verify the effectiveness of our method in practice.

Keywords: Intelligent Transportation Systems, System Architecture, Smart Buildings, Smart Cities and Infrastructures
Abstract: Recently, various approaches have been developed for traffic analysis systems, ranging from Vehicle Detection Systems (VDS) to mobile detectors. However, stationary or mobile detectors have clear limitations. For example, in actual driving situations such as intersections or exits, each lane can face completely different traffic conditions, revealing the limitations of traffic information provided by conventional stationary detectors. In this paper, we propose a methodology to efficiently monitor the traffic condition and safety on the road by utilizing LiDAR sensors installed on vehicles to collect continuous traffic information about surrounding vehicles. To implement the Mobile Detection System (MDS), we collect point cloud data from LiDAR and detect the position and size of vehicles using a deep learning-based voxel-RCNN. We then convert the data into traffic information for analysis. Furthermore, we propose an efficient method for analyzing road hazards by introducing the MTTC method based on the TTC for hazard assessment. To evaluate the performance of the proposed method, we compare its reliability with that of conventional VDS and perform road hazards analysis using LiDAR-based probe vehicles with data collected directly from highways in Korea.

Keywords: Intelligent Transportation Systems, Autonomous Vehicle, Trust in Autonomous Systems
Abstract: With the growing demand for autonomous micromobility vehicles, developing robust and effective simulators for them becomes increasingly important. This research paper examines the essential requirements of a simulator for autonomous micromobility vehicles, focusing on aspects such as accurate sensor modeling, realistic pedestrian behavior, customizability, scenario and vehicle library, scalability, and user-friendliness. By analyzing these key features, we provide a comprehensive understanding of the necessary components for an effective simulation environment, aiming to enable researchers, developers, and other stakeholders to design, test, and evaluate autonomous micromobility vehicles in a safe and controlled manner. Addressing these requirements, simulators can significantly contribute to the advancement of autonomous micromobility technology, leading to safer and more efficient urban transportation systems in the future.

Keywords: Intelligent Transportation Systems, Infrastructure Systems and Services, Autonomous Vehicle
Abstract: This paper presents a novel approach for vehicle map mapping and estimating parking occupancy in parking lots using a single camera without parking lines. The system is capable of detecting around 10 vehicles even with just one camera, and is able to track vehicles with the same ID as they enter and pass through the parking lot. Furthermore, it proposes a method to map the locations of vehicles on real-world maps, allowing for accurate estimation of parking occupancy. The system achieved high accuracy in vehicle detection and tracking, with a 98.4% recall rate and 100% tracking accuracy. The accuracy of parking occupancy estimation was calculated using root mean square error, with an average error of 0.24m and a maximum error of 0.36m. The results demonstrate the feasibility of detecting and tracking vehicles, as well as estimating parking occupancy, in parking lots without parking lines using only one camera. Further testing on multiple sites, including nighttime and adverse weather conditions, is needed to increase the reliability of the system.

Keywords: Intelligent Transportation Systems, Smart Buildings, Smart Cities and Infrastructures, Infrastructure Systems and Services
Abstract: Smart-parking solutions use sensors, cameras, and data analysis to improve parking efficiency and reduce traffic congestion. Computer vision-based methods have been used extensively in recent years to tackle the problem of parking lot management, but most of the works assume that the parking spots are manually labeled, impacting the cost and feasibility of deployment. To fill this gap, this work presents an automatic parking space detection method, which receives a sequence of images of a parking lot and returns a list of coordinates identifying the detected parking spaces. The proposed method employs instance segmentation to identify cars and, using vehicle occurrence, generate a heat map of parking spaces. The results using twelve different subsets from the PKLot and CNRPark-EXT parking lot datasets show that the method achieved an AP25 score up to 95.60% and AP50 score up to 79.90%.

Keywords: Intelligent Transportation Systems, Cooperative Systems and Control, Autonomous Vehicle
Abstract: In crowdsourced delivery organizations, where individual vehicles with shared common goals may have conflicting individual interests, the preference is for collaboration over competition, provided it is less costly. However, achieving a balance between the efficiency of individual vehicles and the overall fleet poses a challenge. This paper introduces a novel Vehicle Routing Problem with Fair Profits and Time Windows (VRP-FPTW), which aims to meet customer demand and stringent time windows while maximizing the profit of the worst-off vehicle in the fleet. We propose a centralized and distributed vehicle routing model for this problem, both with quality of solution guarantees. The distributed approach is tailored for multi-agent systems relying on a coordination mechanism where each vehicle modeled as an individually rational agent finds its route autonomously in coordination with a fleet coordinator agent, without sharing its private information. The objective of a vehicle agent is to maximize its own profit while following the fleet's norms and regulations based on shared values. Simulation experiments provide compelling evidence of the robustness and scalability of the proposed distributed approach, showcasing significant enhancements in both solution quality and computational efficiency, particularly when dealing with larger vehicle fleets.

Keywords: Multimedia Systems, Design Methods, Virtual/Augmented/Mixed Reality
Abstract: The rise of AR/VR has led to an increased demand for 3D content. However, the traditional method of creating 3D content using Computer-Aided Design (CAD) is a labor-intensive and skill-demanding process, making it difficult to use for novice users. Sketch-based 3D modeling provides a promising solution by leveraging the intuitive nature of human-computer interaction. However, generating high-quality content that accurately reflects the creator's ideas can be challenging due to the sparsity and ambiguity of sketches. Furthermore, novice users often find it challenging to create accurate drawings from multiple perspectives or follow step-by-step instructions in existing methods. To address this, we introduce a groundbreaking end-to-end approach in our work, enabling 3D modeling from a single free-hand sketch, Deep3DSketch+backslash+. The issue of sparsity and ambiguity using single sketch is resolved in our approach by leveraging the symmetry prior and structural-aware shape discriminator. We conducted comprehensive experiments on diverse datasets, including both synthetic and real data, to validate the efficacy of our approach and demonstrate its state-of-the-art (SOTA) performance. Users are also more satisfied with results generated by our approach according to our user study. We believe our approach has the potential to revolutionize the process of 3D modeling by offering an intuitive and easy-to-use solution for novice users.

Keywords: User Interface Design, Human-Computer Interaction, Virtual/Augmented/Mixed Reality
Abstract: As the development of augmented reality (AR) and virtual reality (VR) applications is still limited to those with substantial amounts of technical knowledge, the prototyping and testing of user interface (UI) designs for AR and VR applications remotely proves difficult. Recent tools proposed for prototyping AR/VR applications focus on working toward increased fidelity of prototyping methods, but provide limited ability to easily collect objective quantitative data from user interactions with prototypes, especially in remote settings. In this paper, we present a remote usability study using Adobe XD rapid prototyping software integrated with Maze User Testing Software, that collect data for UI designs for an existing cross-platform software. The results were found by collecting task completion time, misclick rate, and user click-data using heatmaps. We discuss the results of our study, and show that objective quantitative data can be collected for AR/VR prototypes in remote testing settings to provide insightful usability feedback in early stages of interface design.

Keywords: Human-Machine Interaction, Human-Computer Interaction, Virtual/Augmented/Mixed Reality
Abstract: Literature highlights how virtual and augmented settings offer engaging solutions to improve one’s feeling of an artificial limb embodiment. In this paper, we explored the potential of a setting for Spatial Augmented Reality (SAR, where a display augments a surface without making the user wear any visor) in two conditions of a lower limb ownership training involving subjects without disabilities. In the first condition, the subject must contract the quadriceps of a leg for commanding (through electromyography, EMG) a virtual leg (a 3D model of the Hybrid Knee prosthesis) to kick a virtual wall: each collision corresponds to a vibratory feedback on the thigh (a position defined for upcoming tests with transfemural amputees). The second condition adds a social context to engage the user: the subject is asked to cooperate with another (fictional) player to kick on the same virtual wall. Subjective (through questionnaires) and objective (according to the number of kicks as a performance index, and the proprioceptive drift as an embodiment index) assessments have been performed before a rubber leg illusion test. Overall, we observed how the cooperative task can engage the subject to be more active. However, this condition can reduce the impact of the training on the embodiment itself, probably because the social task generates a distraction. Nevertheless, such findings suggest the possibility to alternate these two tasks in the same session to increase the duration of a prosthetic embodiment training.

Keywords: Human-Computer Interaction, Virtual/Augmented/Mixed Reality, Virtual and Augmented Reality Systems
Abstract: Virtual reality (VR) technology can be used for multiple purposes, for simulating or enhancing real-life situations. VR-based interfaces can provide new operational advantages and reduce hardware requirements, i.e., weight, by offering various enhanced input modalities that otherwise could not be used in a physical cockpit. This study presents a comparative analysis of three input modalities for a virtual keyboard in a fully immersive VR environment. Each input modality has two steps: 1) pointing to the desired button, and 2) the selection of the item. We compare the hand laser pointer with trigger selection, the head laser pointer with trigger selection, and the head laser pointer with the dwell time selection (no need to use the hands). In addition, we propose new types of visual feedback: for all the input modalities, we display in the field of view of the user the last 5 letters. For the modality with the dwell time, we add a circular progress bar around the field of view of the user to provide a constant awareness of the possible button selection. The proposed virtual keyboard input modalities were assessed by 32 participants. The results support the conclusion that the hand pointer with the trigger control provides the best speed compared to other modalities.

Keywords: Virtual/Augmented/Mixed Reality, Human-Computer Interaction, Biometrics and Applications,
Abstract: Upper Limb (UL) Rehabilitation in Multiple Sclerosis (MS) is an open research field due to the complex interplay between cognitive and physical dysfunctions. Virtual Reality (VR) can face such an issue by enriching physical training with engaging features, including biofeedback strategies to selfregulate autonomic functions according to the visualisation of indices like heart rate variability (HRV). In the present work, HRV biofeedback is introduced in a VR-based exergame (a game designed to promote exercising), tailored to rehabilitation of the dominant upper limb in Persons with MS (PwMS). The exergame is based on a dual-task paradigm, integrating a UL motor rehabilitative task with a breathing task. The aim is to investigate how the design developed for the HRV biofeedback affects engagement and performance during the exergame session. As a preliminary study, sixteen able-bodied subjects are tested in a within-subjects design, to assess the quality of the game features and design, before approaching MS patients. Two conditions are presented, with and without biofeedback. The proposed HRV biofeedback has two possible levels, depending on whether or not the desired respiratory rate of six breaths/min is successfully maintained. It is used to control game elements and change difficulty of the session. The main finding of this study is that biofeedback improves both user performance and experience in healthy subjects. These results underline the great potential of this technique to promote engagement. Thus, they point to fostering the rehabilitative effectiveness of repetitive motor tasks and encouraging adherence to the longterm training. Future studies will encompass fine tuning of the experimental setup and include PwMS to further adjust the game to patients’ needs and observe the setup compliance to rehabilitation settings.

Keywords: Assistive Technology, Virtual/Augmented/Mixed Reality, Human-Machine Interaction
Abstract: Amputees use prosthetic devices to perform activities of daily living. However, some users reject their devices due to the lack of usability or high cognitive workload. Although virtual reality has been studied in this domain for training purposes, there has not been any investigation on usability and cognitive workload of using virtual reality simulations for training of prosthetic devices. The objective of this study was to compare cognitive workload and usability of using virtual reality-based simulation of electromyography based prosthetic devices and physical devices. The findings suggested that using virtual reality simulations were helpful in reducing cognitive workload and increasing perceived usability of prosthetic devices.

Keywords: Virtual/Augmented/Mixed Reality, Human-Computer Interaction, Virtual and Augmented Reality Systems
Abstract: In the past decade, Extended Reality (XR), through rapid technological advancement, has managed to make modest changes in surgical training approaches. Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR)-based simulators have been adopted for training in various surgical domains such as brain, eye, laparoscopic, and orthopedic surgery among others. However, there is a lack of effort in exploring Human Extended Reality Interaction (HXRI)-based concepts to design effective training environments that corroborate the actual training procedure. In this paper, an HXRI-based multi-level approach has been proposed for the creation of XR-based training environments. The multi-level approach takes advantage of both Virtual and Mixed Reality to create a holistic training environment while eliminating the disadvantage arising from using stand-alone VR and MR-based training environments. The approach was based on the results of an exhaustive study in which medical personnel interacted with VR and MR environments to train in an orthopedic surgical procedure. The approach, design, and development of the XR environments, the study, and the results from the study which underscore the importance of such a multi-level approach are presented in the paper.

Keywords: Modeling of Autonomous Systems, Robotic Systems
Abstract: In this study, we developed a process of heuristically searching and selecting the optimal viewpoint for capturing a good composition of a group of moving subjects with an autonomous indoor drone camera. The subjects on the drone’s camera screen are represented using a Gaussian mixture model. The Kullback–Leibler divergence between the Gaussian mixture model and a user-defined reference composition is evaluated and defined as the composition evaluation value. The drone searches for a viewpoint in a 3D space to optimize this value using particle swarm optimization. To facilitate the search, a recursive Gaussian process is employed to update the prediction of the observation result. Through the proposed method, a sufficient optimal viewpoint can be obtained, even for moving subjects.

Keywords: Infrastructure Systems and Services, Smart Metering, Smart Buildings, Smart Cities and Infrastructures
Abstract: This study measured the depth of cracks on the ground surface by stereo matching aerial images of the cracks captured using a drone equipped with real time kinematics-global navigation satellite system (RTK-GNSS) and a high-resolution compact digital camera. Existing crack depth measurement methods are time-consuming, expensive, and cannot measure a wide area. In comparison, the proposed method can measure the depth of cracks in a short time and over a wide area by stereo matching only two aerial images. In addition, the drone's movement extends the baseline length, which increases the resolution in the height direction, resulting in precise depth measurements. To demonstrate the effectiveness of the proposed method, we took aerial images of cracks made of Styrofoam using shooting equipment made of aluminum frame and measured their depth. We realized stable millimeter-order crack depth measurements and achieved millimeter-order height resolution of 1.404 mm, and achieved 0.14% and 0.13% errors for 5 cm and 15 cm deep cracks, respectively.

Keywords: Intelligent Transportation Systems, Decision Support Systems
Abstract: Planning ship routes that take into account meteorological and oceanographic conditions is a salient problem for both commercial and Naval applications. The A* algorithm is a very common method for finding the minimum cost path in a graph. Finding appropriate heuristics for a cost function is critical to the computational efficiency and memory requirements of the A* algorithm. We propose heuristic evaluation functions (HEFs) based on the minimum, mean, median, and mode of the cost function values over the reachable nodes given time constraints. Only the HEF based on the minimum is admissible, but we show that the other heuristics are able to find near-optimal solutions in substantially shorter times than the Dijkstra's algorithm, which does not use a heuristic. We evaluate these heuristics over many scenarios and show that overall the HEF based on the mean value performs the best. This HEF can be used in time-critical applications where an occasional loss of optimality is sacrificed for faster run time, such as real-time planning and control, or as part of a multi-objective shortest path algorithm for planning and execution.

Keywords: Autonomous Vehicle, Intelligent Transportation Systems
Abstract: The effect of wind disturbance is one of the major factors that complicate heading control in ship control at low speeds. Approach maneuvering while decreasing speed and approaching the pier requires a control adopted to changing maneuverability and wind conditions. In this study, a control scheme using a deep neural network (DNN) was developed to output command values corresponding to wind conditions by training the DNN on maneuvering patterns of approach maneuvering under various wind conditions. The developed control scheme was validated through experiments using an actual ship.

Keywords: Machine Learning, Artificial Social Intelligence, Neural Networks and their Applications
Abstract: In total knee arthroplasty, nearly 100 types of surgical instruments are prepared in addition to the usual surgical instruments. Since there are many types of surgical instruments used and their shapes and sizes are similar, accidents due to incorrect selection of implants have occurred. In addition, there is concern that the shortage of nurses will accelerate around the world in the future. For these reasons, we developed a surgical instrument detection system for total knee arthroplasty to reduce the incidence of accidents and the burden on scrub nurses. The surgical instrument detection system in the previous study required a lot of labor and time to create training data. Also, when increasing the number of surgical instruments to be detected, it was necessary to retrain all the surgical instruments. To solve these problems, we developed an automatic annotation system using a turntable in this study. In addition, we develop learning and inference methods using standard objects. In the experiment, we verified the accuracy of the learning data created by the automatic annotation system and verified the object detection accuracy for five types of surgical instruments. As a result, all experiments showed the effectiveness of this system.

Keywords: Image Processing and Pattern Recognition, Machine Learning, Deep Learning
Abstract: Fetal Growth Restriction (FGR) slows or stops the fetal growth during pregnancy, resulting in low fetal weight at gestational age. Currently, FGR is diagnosed based on ultrasound weight estimation and blood test, but the estimated weight have an error and blood test cannot diagnose FGR correctly due to placental factor, so prenatal diagnosis of FGR is difficult. Therefore, the use of Blood oxygenation level-dependant magnetic resonance imaging (BOLD MRI) has been investigated as a method for prenatal diagnosis of FGR. This paper proposes a FGR estimation method using placenta region in time-series BOLD MRI. The proposed method extracted placenta region semi-automatically using Simple Linear Iterative Clustering (SLIC) on the time-series BOLD MRI, and input time-series radiomics features in the placental region into Long Short-Term Memory (LSTM) to estimate FGR/non-FGR status of patient. As a result, the semi-automatic extraction of the placental region achieved the highest dice coefficient of 0.774 at the final frame where the extraction accuracy would be decreased due to propagation. And the result of 4-fold cross validation test, FGR/non-FGR were predicted in F-measure of 0.773, 0.745, 0.794, and 0.800, respectively.

Keywords: Computational Life Science, Computational Intelligence, Cybernetics for Informatics
Abstract: Manual testing is one of the most common diagnostic methods for anterior cruciate ligament (ACL) injuries in the knee joint. In particular, the Lachman test is superior to others. In this study, a 9-axis inertial sensor and a force sensor are used to measure the examiner's manual technique. In this study, we experimented on the accuracy of the inertial sensor and an experiment to measure the Lachman test. The results of Experiment 1 were within ± 1.7 degrees of the theoretical values for each axis throughout the experiment. The results of Experiment 2 showed that Endpoint could be estimated from the acceleration component of the subject's tibia, the acceleration component of the examinee's hand, and the force of each finger and that the motion of the knee joint and the examinee's hand technique could be measured. The development of this system is expected to have training effects for unskilled examinees.

Keywords: Machine Learning, Image Processing and Pattern Recognition, Machine Vision
Abstract: Parkinson’s disease is a neurodegenerative disease characterized by slow progression and is never cured. Its symptoms include motor impairments such as bradykinesia, muscle stiffness, tremor at rest, and postural dysreflexia, as well as non-motor impairments such as depression and autonomic neuropathy. One measure to evaluate Parkinson’s disease is the Unified Parkinson’s Disease Rating Scale. One of the items in this scale that evaluates bradykinesia, one of the motor symptoms, is the finger tapping test. However, many of the expressions used there are subjective, and the evaluation criteria are ambiguous. Therefore, the purpose of this study was to develop a quantitative hand movement evaluation system using MediaPipe Hands, to analyze finger tapping test, and to evaluate the effectiveness of the system. With this developed system, hand movements can be analyzed by using a webcam. Hand motion was analyzed by calculating the length between hand joints, the speed of joint movement, and the angle between joints. To evaluate the accuracy of the system, we first measured the length between the joints of the fingers as the hand was moved away from and closer to the camera, and then examined the error. Then, the finger tapping test was conducted using the developed system to evaluate the bradykinesia. Accuracy evaluations have shown that MediaPipe Hands can detect hand movements with high accuracy. The finger tapping test also revealed that there was reproducibility in the magnitude of each finger movement, but not in the speed of the movement.

Keywords: Biometric Systems and Bioinformatics, Cyborgs,
Abstract: Artificial motion sensation can be used for virtual reality, rehabilitation, etc. It is known that joint motion illusion is induced by giving certain vibration stimulation on human muscles even though the joint is not actually moved. In this paper, a method to generate motion illusion of arbitrary hand reaching motion, which is one of the most important daily living movements, by giving multiple vibration stimulations on shoulder and elbow muscles is proposed. In order to generate hand reaching motion illusion, frequency of vibration stimulation is controlled on each muscle at the same time considering the spatial hand movement in the proposed method. The experimental of generating hand reaching motion illusion is carried out to evaluate the effectiveness of the proposed method.

Keywords: Deep Learning, Machine Learning, Neural Networks and their Applications
Abstract: We developed an automatic ensemble training method for fusing massive-training artificial neural network (MTANN) deep-learning modules in classification between polyps and non-polyps in CT colonography. We started from an initial MTANN module that had been trained with an initial set of polyps and non-polyps. We applied the trained initial module to polyps and non-polyps to analyze the weakness of the initial module. We arranged the output scores of the initial module to form a score scale in receiver-operating-characteristic (ROC) space, representing the “degree of difficulty” in distinction between polyps and non-polyps by the initial module. Based on the score-space, several sets of training polyps and non-polyps with different degrees of difficulties were determined. We trained several MTANN modules with the several sets of training samples so that each module became an expert at a certain level of difficulty. We then combined expert modules with a mixing module to form a “mixture of expert” MTANNs. Our database consisted of CT colonography datasets acquired from 100 patients, including 26 polyps. The mixture of expert MTANNs with the ensemble training method distinguished all polyps correctly from more than 50% of the non-polyps. We compared the effectiveness of the ensemble training with that of training with manually selected cases. The performance of the mixture of expert MTANNs with ensemble training method was superior to that of the “reference-standard” MTANNs trained with manually selected cases, which could reduce the cost of the manual selection.

Keywords: Machine Learning
Abstract: The incidence of falls in hospital facilities is high and can lead to a decrease in patients' quality of life and an increase in medical expenses. Therefore, the development of a system that can predict getting-up from a bed is necessary. This study proposes a get-up detecting sensor using 6-axis inertial sensor. This system can detect getting-up from a bed in real-time using machine learning with Edge AI. To evaluate the basic performance of the proposed system, a protocol was applied for four subjects, and data were collected. Alert accuracy rate and false alert rate were used as evaluation metrics, and a model was built using data from three of the subjects and evaluated with the remaining subject, which was repeated for all four subjects. For high-risk bed-leaving behavior, medium-risk pre-bed-leaving behavior, and low-risk get-up behavior, the alert accuracy rate (i.e., Recall) was 89.4%, 97.5%, and 86.3%, respectively, and the false alert rate (1-Precision) was 6.3%, 10.2%, and 0.0%, respectively. This confirmed the possibility of predicting rising behavior with high accuracy. Furthermore, as a proof of concept, a real-time get-up detection system was developed, and its practicality was demonstrated. Future challenges include reevaluating feature extraction and evaluating the performance of the proposed system with a diverse range of subjects of different ages, genders, and health statuses.

Keywords: Communications
Abstract: Developing advanced cyber defense techniques is essential for effectively detecting network anomalies that are becoming more challenging to identify. In this paper, we generate machine learning models based on real-time Internet and historical data and evaluate their classification performance. We introduce a network anomaly detection tool CyberDefense that integrates various stages of the anomaly detection process. It facilitates performance evaluation of machine learning algorithms and generation of new machine learning models. Its modular and scalable design enables incorporating new datasets and machine learning algorithms. The tool has been utilized to generate models and evaluate their classification performance using datasets collected during reported power outage and ransomware attacks.

Keywords: Communications
Abstract: Machine learning approaches for detecting anomalies in communication networks heavily depend on the properties of training data. We analyze the impact of data probability distributions on performance of machine learning models developed based on Border Gateway Protocol datasets collected during the worm and ransomware attacks. Feature selection is performed to determine the most important features and identify their best fitting distributions. Experimental results indicate that certain features follow heavy-tailed distributions. Traffic anomalies are then classified based on selected features using the gradient boosting decision tree models suitable for designing real-time and scalable intrusion detection systems.

Keywords: Decision Support Systems, Adaptive Systems, Trust in Autonomous Systems
Abstract: The Royal Canadian Navy (RCN) strategic vision Cyber Strategy 2020-2025 requires that the RCN workforce is trained, educated and aware of cyber risks and their role in cyber security and defence [1]. To support the RCN vision, Defence Research and Development Canada (DRDC) – Toronto Research Centre recently commenced an investigation to research advanced training strategies for maintaining operational resilience against cyber attacks onboard His Majesty’s Canada (HMC) platforms based on the identification of a corpus of human-noticeable aspects of cyber attacks. This paper presents training strategies that are based on emerging artificial intelligence (AI) technologies for teaching cyber security to RCN operators to support cyber damage control on future HMC platforms. We promote that AI systems can achieve state-of-the-art results for cyber security training.

We began our investigation by interviewing six Department of National Defence/Canadian Armed Forces subject matter experts (SMEs) with substantial experience in cyber damage control on how to identify potential cyber security risks, and mitigations strategies to detect, respond, and recover from cyber security incidents onboard HMC platforms [2]. Cyber awareness training for all RCN ranks was identified by the SMEs as the biggest mitigator to improve operator awareness of compromised platform systems such as the Integrated Platform Management System or the Combat Management System.

One means of future instruction for the RCN is the design and development of an Intelligent Tutoring System (ITS) [3, 4]. Traditional ITS uses static instructional training in cyber security education, which is not able to meet the evolving landscape of cyber threats across a wide student population. New ITS will need to employ adaptive learning for individuals with different backgrounds to improve the student learning experience [5, 6]. Combating cyber threats also requires trust in Information Technology (IT) as it relates to cyber damage control. As a potential additional component of an ITS, a Trust Management System (TMS) needs to be implemented to help build, maintain, and repair operator trust in IT. The TMS will identify when trust in the system is lost using psychoph

Keywords: Trust in Autonomous Systems, Communications
Abstract: Self-supervised speech pre-training has emerged as a useful tool to extract representations from speech that can be used across different tasks. While these models are starting to appear in commercial systems, their robustness to so-called adversarial attacks have yet to be fully characterized. This paper evaluates the vulnerability of three self-supervised speech representations (wav2vec 2.0, HuBERT and WavLM) to three white-box adversarial attacks under different signal-to-noise ratios (SNR). The study uses keyword spotting as a downstream task and shows that the models are very vulnerable to attacks, even at high SNRs. The paper also investigates the transferability of attacks between models and analyses the generated noise patterns in order to develop more effective defence mechanisms. The modulation spectrum shows to be a potential tool for detection of adversarial attacks to speech systems.

Keywords: Deep Learning, Machine Learning, Neural Networks and their Applications
Abstract: Data pruning is the method selecting a subset from an entire training dataset while maintaining the performance after training. Most pruning metrics measure the difficulty of each training sample for the target task and select the hardest samples first. The “hardest-first sampling” works well when the pruning ratio is low (the pruned dataset is large). However, while the pruning ratio increases, the performance decreases significantly and get worse than the random pruning. This is because the hardest samples are the special samples of the class, and a model trained on few hardest samples specializes in them and drops its performance. For solving this problem, we propose a new metric: AgreementLoss. This is computed from two classifier models trained on two datasets obtained by the disjoint decomposition of the original dataset. By the dataset decomposition, the predictions of the models are more different for a sample since either of them trains each sample in the original dataset. The metric is defined as the positive or negative loss value of the trained either model, depending on two predictions agree or disagree. This metric represents the difficulty of each sample, while extremely difficult “noisy” samples get lower scores by the negative sign. By selecting samples in descending order of the AgreementLoss scores, we can reduce the priority of the “noisy” samples. Experiments on various datasets show that our method gets stable performance over all pruning ratio and outperforms other metrics especially when the pruning ratio is high.

Keywords: Deep Learning, Machine Learning, Neural Networks and their Applications
Abstract: We propose a novel loss function for imbalanced classification. LDAM loss, which minimizes a margin-based generalization bound, is widely utilized for class-imbalanced image classification. Although, by using LDAM loss, it is possible to obtain large margins for the minority classes and small margins for the majority classes, the relevance to a large margin, which is included in the original softmax cross entropy loss, is not be clarified yet. In this study, we reconvert the formula of LDAM loss using the concept of the large margin softmax cross entropy loss based on the softplus function and confirm that LDAM loss includes a wider large margin than softmax cross entropy loss. Furthermore, we propose a novel Enlarged Large Margin (ELM) loss, which can further widen the large margin of LDAM loss. ELM loss utilizes the large margin for the maximum logit of the incorrect class in addition to the basic margin used in LDAM loss. Through experiments conducted on imbalanced CIFAR datasets and large-scale datasets with long-tailed distribution, we confirmed that classification accuracy was much improved compared with LDAM loss and conventional losses for imbalanced classification.

Keywords: Deep Learning
Abstract: Condition-based maintenance is becoming increasingly important in hydraulic systems. However, anomaly detection for these systems remains challenging, especially since that anomalous data is scarce and labeling such data is tedious and even dangerous. Therefore, it is advisable to make use of unsupervised or semi-supervised methods, especially for semi-supervised learning which utilizes unsupervised learning as a feature extraction mechanism to aid the supervised part when only a small number of labels are available. This study systematically compares semi-supervised learning methods applied for anomaly detection in hydraulic condition monitoring systems. Firstly, thorough data analysis and feature learning were carried out to understand the open-sourced hydraulic condition monitoring dataset. Then, various methods were implemented and evaluated including traditional stand-alone semi-supervised learning models (e.g., one-class SVM, Robust Covariance), ensemble models (e.g., Isolation Forest), and deep neural network based models (e.g., autoencoder, Hierarchical Extreme Learning Machine (HELM)). Typically, this study customized and implemented an extreme learning machine based semi-supervised HELM model and verified its superiority over other semi-supervised methods. Extensive experiments show that the customized HELM model obtained state-of-the-art performance with the highest accuracy (99.5%), the lowest false positive rate (0.015), and the best F1-score (0.985) beating other semi-supervised methods.

Keywords: Evolutionary Computation, Metaheuristic Algorithms, Optimization and Self-Organization Approaches
Abstract: Constrained multiobjective optimization problems (CMOPs) are commonly found in real-world applications. CMOP is a complex problem that needs to satisfy a set of equality or inequality constraints. This paper proposes a variant of the bidirectional coevolution algorithm (BiCo) with differential evolution (DE). The novelties in the model include the DE variation operator as the main search engine and a non-dominated sorting selection scheme. Experimental results on two benchmark test suites and eight real-world CMOPs suggested that the proposed model reached better overall performance than the original model.

Keywords: Cybernetics for Informatics, Computational Intelligence
Abstract: Chaotic resonance, resembling stochastic resonance, is induced by internal fluctuations (i.e., chaos). This phenomenon has been observed in numerous systems. The most representative form of chaotic resonance is the synchronization with respect to weak applied signals under chaos--chaos intermittency, where a chaotic orbit moves among multiple attractors. Chaotic resonance exhibits higher sensitivity than stochastic resonance, but its engineering applications are limited by concerns such as the requirement to adjust the state of chaos by internal system parameters to induce chaotic resonance rather than the external noise strength. However, achieving this adjustment is challenging, especially in biological systems. Therefore, to handle this limitation, we proposed a novel double-Gaussian-filtered reduced region of orbit (RRO) method (called the DG-RRO method) for obtaining perturbed feedback signals lower than the conventional RRO method. This DG-RRO feedback signal is determined by the inverse sign of the map function and double-Gaussian filters around the local maximum/minimum values of the map. Because of its fine local specification, the DG-RRO feedback signal induces a chaotic resonance by one-third the feedback strength of the conventional signal. This method may pave the way for using chaotic resonance in engineering applications.

Keywords: Expert and Knowledge-Based Systems, Deep Learning, Computational Intelligence
Abstract: One of the problems in imitation between humans and robotd is embodiment gap. Therefore, in this paper, we propose a system that realizes movement level imitation between humans and robots for embodiment gap. We target embodiment gap arising from link length and number of joints, and estimate embodiment gap by solving not only the inverse kinematics but also the forward kinematics using the Levenbarg-Marquart method. By doing this, human movement is separated into end-effector-oriented movement and joint angle-oriented movement according to the robot’s own embodiment. Then, based on the separated movement, we realize the imitation of movement using Behavior Cloning based Energy-Based Model. We applied the proposed method to a small humanoid robot and verified its effectiveness. As a result, we confirmed that the proposed method separates human movement dependent on the robot’s embodiment. In addition, we realized the imitation learning of movement corresponding to embodiment gap estimated from the separated human movement.

Keywords: Computational Intelligence in Information, AI and Applications, Expert and Knowledge-Based Systems
Abstract: The rapid development of the AI industry had led to the widespread utilization of numerous edge devices in the real world. However, AI services still heavily rely on high-performance computing systems, which means that these devices are not being effectively utilized. To address these issues, EdgeCPS technology has emerged. EdgeCPS is a technology that supports the seamless execution of various services, including AI applications, through the interconnection of edge devices and edge servers, as well as resource and function augmentation. In order to appropriately utilize edge devices in an EdgeCPS environment, the paper proposes an EdgeCPS-AI knowledge sharing model for supporting computing partition services. The proposed EdgeCPS-AI knowledge is represented as a graph, which systematically structures AI service-related information that arises on virtual and physical edge devices. This knowledge provides not only partitioned AI weighted models in order to support the characteristics of EdgeCPS, but also enables flexible provision of various information required for AI services. Thus, this approach can facilitate the reconstruction of AI services according to service requirements and maximize the utilization of edge devices. To achieve this, a microservice-based computing partition AI service is devised and experimentally proven by constructing a Kubernetes system using 8 heterogeneous edge devices and a graph DB system consisting of 2,762 nodes. Experimental results show that performing computing partition services on edge devices is significantly more efficient in terms of overall resource consumption, with up to 85.57% improvement, as compared to performing the same tasks on high-performance computing systems.

Keywords: Cybernetics for Informatics, Big Data Computing,, Cloud, IoT, and Robotics Integration
Abstract: Decentralized identity (DID) is an identity management framework aiming to return the ownership of an identity to its corresponding user. Recent studies propose to store the identifiers of DID issuers and implement identity management systems based on blockchain. However, existing systems cannot avoid identity tampering and verifiable credential abuse of decentralized identities, which makes the identity management opaque. In this paper, we propose TDID, a Transparent and efficient Decentralized IDentity management system with blockchain. The key insight behind TDID is to manage the registration and authentication of DIDs via smart contracts, and design Structured Merkle Patricia Tree (SMPT) as an underlying data structure to store identity data on blockchain. The smart contract based processes can improve transparency of decentralized identity management, while the SMPT data structure can realize efficient storage of DID data. We implement and evaluate TDID on different identity management operations, and the experimental results show that TDID can achieve about 3.1 times for write operation and 6.3 times for read operation while improving the transparency of DID management.

Keywords: Heuristic Algorithms, Complex Network, Optimization and Self-Organization Approaches
Abstract: Edge rectification is a widely used method to enhance network robustness. However, in some networked systems, edge rectification may be challenging or even infeasible to implement. An edge addition strategy is proposed as an alternative optimization method in this paper. Nested Ring Structure (NRS), whereby each node's edges connect its nearest neighbors along the backbone direction, have exhibited robust controllability against random attacks. Therefore, The Nested Edge Addition (NEA) strategy is proposed, which enhances network controllability by building NRS through edge addition to a given initial network. With a small number of added edges, NEA can rapidly enhance network controllability, allowing the network to be controlled using just one driver node. The more nested edges are added, the stronger the NRS in a network, thus exhibiting better controllability robustness. The effectiveness of NEA is verified by simulations on both synthetic and real-world networks. Extensive experimental results demonstrate that NEA is an efficient strategy for designing network topology and optimizing real-world networks.

Keywords: Soft Robotics, Mechatronics, Robotic Systems
Abstract: In this study, we propose a design methodology for anthropomimetic robotic wrists and forearms. Conventional robotic wrists and forearms have few examples of human mimicry, and they have been simplified. The ligaments and tendons of the robotic forearm proposed in this study were replicated using chain-stitched wires and arranged in a manner similar to the human anatomy. Motion capture and goniometer measurements were used to measure the range of motion (ROM) of the wrist and forearm, driven by 16 servomotors. The results of the experiment were compared with the human ROM reported in previous studies, and it was found that the developed robotic forearm could achieve a ROM similar to that of humans. The use of this robotic forearm in functional verification experiments will enhance our understanding of the human structure. The integration of mechanical mechanisms with structures created through evolution is expected to improve the functionality of future robots.

Keywords: Robotic Systems, Conflict Resolution
Abstract: Roadmap-based path planning is a well-established method for multi-robot systems, where the free space of the environment is represented as a roadmap graph. The Voronoi diagram is known for its efficiency in creating roadmaps for single-robot systems due to its ability to generate paths with high clearance from obstacles. However, the design of the Voronoi diagram does not allow for redundant paths, as only one path is obtained between each pair of obstacles. In contrast, multi-robot path planning requires multiple path options for efficient solutions. Therefore, we propose redundant Voronoi roadmap graph, which incorporates multiple paths computed from the Voronoi diagram. In this approach, we introduce imaginary obstacles to modify the costmap and obtain a roadmap with more paths. Our proposed roadmap retains the high clearance feature of the Voronoi diagram and is suitable for multi-robot systems due to the availability of alternative route options. We demonstrate that our method can generate roadmaps in various simulated environments with different levels of redundancy. Additionally, we verify the efficiency of our proposed roadmap graph through graph analysis and multi-robot path planning experiments. Comparative analysis shows that the use of the proposed roadmap increases the success rate and solution quality compared to the roadmap obtained directly from the conventional Voronoi diagram.

Keywords: Cooperative Systems and Control, System Modeling and Control, Autonomous Vehicle
Abstract: The increased use of commercial Unmanned Aerial Vehicles (UAVs) has generated a great interest in their potential to be used for transporting loads and other equipment. However, as the attraction of a UAV is its versatility and cost effectiveness, constraints are placed on the size and capability of a single UAV. Therefore, using multiple UAVs in flight formation has become an elegant solution to these limitations. The formation control of a team of load bearing UAVs is far from trivial. The UAV itself is an underactuated nonlinear system posing significant control challenges. Recent research on this topic has shown promising results and interesting modelling methods such as the Udwadia- Kalaba method has been proposed, to model the loaded system. This research will explore this problem using this method while seeking to bring in robust control techniques for low-level UAV stabilization by designing a sliding mode control system. The proposed low level controller will be combined with the formation controller and the stability demonstrated through simulations.

Keywords: Robotic Systems, System Modeling and Control, Mechatronics
Abstract: Ultrasound (US)-guided percutaneous nephrolithotomy is a surgical procedure for large kidney stone removal through an incision in the patient’s back. To gain kidney access, the surgeon steers a needle towards the kidney while simultaneously controlling the position and orientation of a US probe to keep the needle in the image plane. To successfully reach the kidney while avoiding delicate structures, a significant level of skill and precision is required. To alleviate the surgeon’s cognitive workload, robot-assisted needle tracking can be implemented to autonomously track the needle in the US images and adjust the US probe’s position and orientation such that the same portion of the needle is visible in the images.

This paper presents a US-guided visual servoing (VS) algorithm to track the translation and rotation of a needle in a plane. Image features representing the desired pose of the needle in the image are defined, through which an interaction matrix is devised to relate the rate of the change of the image features in US images to the required position and orientation of the US probe connected to a robotic manipulator. Experimental results in 4 experimental scenarios in a water tank demonstrate the capability of the proposed method in tracking the needle in real-time with an accuracy of 2.6 mm with a control rate of 20 Hz. Although VS has been used to track surgical targets in the past, this paper proposes the first implementation of VS for needle tracking in longitudinal US images subjected to 3-DOF motion in a plane without any prior knowledge of needle trajectory or additional position sensors.

Keywords: Biometric Systems and Bioinformatics, Machine Learning, AI and Applications
Abstract: Some unique characteristics of biological data are (1) that they are always High-Dimension and Low-Sample-Size (HDLSS) and (2) there are changes in the data distribution, such as an imbalance in classes, distribution and covariate shifts, etc. In this paper, we propose a Group Lasso with Checkpoints SElection (GL CSE) algorithm to tackle both issues. To address the first issue, we utilize a group Lasso regression model tailored for HDLSS data to perform feature selection on predefined groups of features, alleviating overfitting and being invariant under group-wise orthogonal reparameterizations. To address the second issue, we propose the checkpoint selection method to extract important model checkpoints while training on group Lasso via two proposed metrics, i.e., the average KL-divergence between training and validation features and the Frobenius error of the covariance matrices between training and validation features. Both metrics aim to select model checkpoints with minimal drifts between the training and validation features. The results of our experiments indicate that our proposed GL CSE algorithm achieves better performance compared to other baseline methods in terms of the MSE and R2 measurements. Specifically, on the biological age dataset, our GL CSE method achieves 0.8799 and 0.9883 for the MSE and R2 measurements, respectively. Additionally, we also show that our proposed checkpoint selection method performs better than regular K-fold cross-validation. Specifically, on the biological age dataset, GL CSE (Q2) achieves 0.9045 MSE and 0.9880 R2, respectively, which outperforms the regular K-fold cross-validation results, i.e., 1.0612 MSE and 0.9871 R2, respectively.

Keywords: Biometric Systems and Bioinformatics, Deep Learning, Image Processing and Pattern Recognition
Abstract: Face detectors are a subset of object detectors that output, at a minimum, a set of locations in an image if and where human faces are present. Face detection is challenging, in part, due to low variance in the structural content of frontal-view faces (i.e., most faces have two eyes, a nose and a mouth) and high variance in visual appearance. This aspect of the domain skews detectors to higher false positive rates as a consequence of many patches of imagery containing features spatially consistent with frontal-view faces. In this study we evaluate the performance of three state-of-the-art face detectors (BlazeFace, MTCNN, and SCRFD) on frontal-view face imagery in a novel human-labeled dataset of 64,104 images with reliable ground truth. We show evidence that modern CNN-based models rely heavily on low-level image features, in spite of their powerful capability to learn complex, discriminatory visual features and concepts. We do this by altering the spectral and color content of frontal-view face images. To gain a better understanding of detector failures, we apply the Deep Dream technique to enhance image features that lead models to false positives.

Keywords: Biometric Systems and Bioinformatics, Deep Learning, Image Processing and Pattern Recognition
Abstract: This work proposes a novel application of digital twins in the field of biometric security. Biometric systems have become widespread but their use carries risks of privacy invasion attacks due to the sensitive nature of biometric data. To address these concerns, we propose creating biometric clones for digital access and authentication systems. A user’s fingerprint can act as a virtual representation or cyberproxy, allowing users to exist in the digital world with a unique, changeable, and privacy- preserving identity. The digital twin or clone fingerprint is generated using deep neural networks combined with a user- specific token/key. This approach allows third parties to process and store the proxy biometrics without putting the user’s personal information at risk. We suggest that this approach could provide a safer and more secure alternative to traditional biometric security systems.

Keywords: Complex Network, Computational Intelligence, Evolutionary Computation
Abstract: Influence maximization (IM) is a problem of select- ing the most influential vertices with limited budget under a given propagation model. A significant challenge faced by many existing algorithms pertains to their inability to reconcile the competing goals of solution quality and computational efficiency, rendering them unsuitable in large-scale social networks. In this paper, we propose an adaptive community-based influence maximization algorithm, named AComA, to solve the IM problem effectively and efficiently. First, we introduce a community detection method to divide a large-scale network into several communities. An adaptive indicator is then defined to identify vertices with high propagation value in divided community networks. Based on community detection and the adaptive influence indicator, the number of candidate vertices is reduced, which helps significantly reduce the search space of the problem. Second, to select the final seed set from these candidate vertices, a genetic algorithm (GA) is introduced. Both the crossover and mutation operations are specifically modified in order to adapt to the IM problem. Taking into account both the local neighborhood information and the global community structure information, AComA manages to achieve a more accurate measurement of the influence spread for each vertex. The method proposed in this paper is tested on several real-world datasets. The experimental results show that AComA is promising.

Keywords: Complex Network, Deep Learning, Neural Networks and their Applications
Abstract: The connectivity and controllability of complex networks play an important role in ensuring the proper functioning of network systems. Robustness of connectivity and controllability is the ability of a network to maintain its basic functions against various malicious attacks. Convolutional neural network (CNN)-based approaches provide an efficient framework to approximate the network robustness, which significantly reduces computation time compared to attack simulations. In this paper, the performance of CNN-based prediction for connectivity and controllability robustness is investigated, when there are noises in the predicted network representation. Two CNN-based predictors are compared, 1) convolutional neural network-based robustness predictor (CNN-RP), and 2) spatial pyramid pooling-based convolutional neural network (CNN-SPP). Two aspects of network information noises are considered and investigated, 1) the random node information noises (RNIN), and 2) the random edge information noises (REIN). The following main conclusions are obtained from extensive experimental studies on synthetic networks: 1) CNN-RP is more tolerant than CNN-SPP to network noises, 2) The characteristics of small-world and scale-free networks make them have a favorable anti-noise ability, and 3) RNIN has less impact on CNN-based prediction performance than REIN, RNIN and REIN show opposite effects on prediction performance when the size of the predicted network is out of the training network size range.

Keywords: Application of Artificial Intelligence, Computational Intelligence, Machine Learning
Abstract: Selection of the optimal learning rate for training neural networks has often been a matter of concern for the machine learning community. The existing learning rates are dependent on multiple scaling factors. This paper proposes Cost-Responsive Learning (CoRL) which does not require manual hyper-parameter tuning. It maintains a linear relationship with the prediction error of the neural network. This is expected to offer the lowest learning rate at the global minima, and higher learning rates elsewhere. Hence, a number proportional to the prediction error is used as a learning rate, subject to the constraint that the number is within an acceptable range (here [0,1]). The derivation of an optimal learning rate from a given cost function is illustrated with the popular binary/categorical cross-entropy cost function(s). Experiments performed under multiple settings demonstrate that, with the CoRL optimizer needs no parameter tuning to obtain state-of-the-art results with significantly lower training time for equivalent performance.

Keywords: Application of Artificial Intelligence, Machine Learning, Neural Networks and their Applications
Abstract: Correctly classifying tumorous tissue and the resection margins proves to be a difficult task in endoscopic bladder cancer surgeries. As a tumor shows altered electrical properties, intraoperative impedance measurements can support the surgeon to classify the tissue. An important step in this process is the decision on how to pass the impedance information to the classification algorithm. This can be either in the form of a raw measurement vector or based on parameters extracted from the measurement. These parameters arise from curve characteristics of the Nyquist diagram, electrical tissue models, or indices defined as ratios between specific measurement points. In this work, different features proposed in the literature are reviewed and extracted from impedance measurements taken on bladder tissue. The most influential measurement frequencies are determined via a principle component analysis, and the most promising parameters are selected based on an analysis of variance. The different feature extraction approaches are compared based on the classification accuracy on unseen test data. The accuracy is very high for all different ways to define features, and it increases for every approach when only considering the selected features. Especially for the Nyquist parameters and indices, as well as the reduced measurement vector, the number of necessary frequencies is significantly reduced. This shortens the measurement, which is favorable for an intraoperative application.

Keywords: Application of Artificial Intelligence, Computational Intelligence, Transfer Learning
Abstract: Recent advances in language modelling has significantly decreased the need of labelled data in text classification tasks. Transformer-based models, pre-trained on unlabeled data, can outmatch the performance of models trained from scratch for each task. However, the amount of labelled data need to fine-tune such type of model is still considerably high for domains requiring expert-level annotators, like the legal domain. This paper investigates the best strategies for optimizing the use of a small labeled dataset and large amounts of unlabeled data and perform a classification task in the legal area with 50 predefined topics. More specifically, we use the records of demands to a Brazilian Public Prosecutor's Office aiming to assign the descriptions in one of the subjects, which currently demands deep legal knowledge for manual filling. The task of optimizing the performance of classifiers in this scenario is especially challenging, given the low amount of resources available regarding the Portuguese language, especially in the legal domain. Our results demonstrate that classic supervised models such as logistic regression and SVM and the ensembles random forest and gradient boosting achieve better performance along with embeddings extracted with word2vec when compared to BERT language model. The latter demonstrates superior performance in association with the architecture of the model itself as a classifier, having surpassed all previous models in that regard. The best result was obtained with Unsupervised Data Augmentation (UDA), which jointly uses BERT, data augmentation, and strategies of semi-supervised learning, with an accuracy of 80.7% in the aforementioned task.

Keywords: Application of Artificial Intelligence, Deep Learning, Machine Learning
Abstract: Cryptocurrency is particularly vulnerable to malicious activities due to its pseudo-anonymous nature. However, existing solutions that heavily rely on deep learning lack interpretability and are only available for retrospective analysis of specific malice types. To address these challenges, we propose Intention-Monitor for early malice detection in bitcoin. Our model utilizes a Decision-Tree based feature Selection and Complement (DT-SC) to build different feature sets for different malice types. Additionally, the Status Proposal Module (SPM) and a hierarchical self-attention predictor propose the global status and predict the address label in real-time. A survival module decides when to stop prediction and proposes the status sequence, namely intention. Our model can detect various malicious activities with strong interpretability, outperforming state-of-the-art methods according to extensive experiments on three real-world datasets. Additionally, our model can explain existing malicious patterns and find new suspicious characters through additional case studies.

Keywords: Application of Artificial Intelligence, Image Processing and Pattern Recognition, Machine Learning
Abstract: For a jar test in water purification, it is necessary to develop a video prediction method to assist in coagulant dosage decisions. However, such a method has not been established; video prediction methods in other fields require a large amount of training videos. This paper proposes a video prediction method based on multi-resolution echo state networks (ESN) for a jar test. Because the proposed method is constructed based on ESN, we succeed in long-term video prediction even if the amount of training videos is small. Specifically, we develop a new model that performs ESN while observing the input video at different resolutions and effectively fuses the prediction results. The multi-resolution approach enables us to acquire both rough shape changes and fine pattern features, resulting in accurate prediction. In fusion of the multiple prediction results, our method can calculate optimal weights by the least squares method. Experimental results for a jar test confirm that our method outperformed existing video prediction methods based on deep neural networks. In addition, the results on the sky timelapse dataset show the versatility of our method.

Keywords: AI and Applications, Fuzzy Systems and their applications
Abstract: In recent years, explainable AI (XAI) has been emphasized in various fields such as healthcare, finance, etc. Since the fuzzy rules in fuzzy inference models are structured as If-Then rules, the inference process of fuzzy inference models can be easily understood by humans. Among them, a deep TSK fuzzy inference model has fuzzy rules with low complexities due to select randomly features and map randomly them to fuzzy partitions. However, this randomness may lead to situations where important features or fuzzy partitions of them that have a significant impact on the results are not used in the inference. Therefore, this paper propose a deep modular fuzzy inference model with high accuracy and stability using small number of fuzzy rules with high interpretability.

Keywords: Cognitive Computing, Multi-User Interaction
Abstract: The occurrence of extreme events, which has increased the risk of major outages in the grid, makes the quick and efficient recovery of load in the distribution network become a key issue. The data-driven deep reinforcement learning method has great potential in providing fast decision-making. However, a large number of agents lead to the curse of dimensionality, making it inefficient to obtain effective control strategies. When tackling similar tasks in different power gird, the retraining of multiple agents will bring us great cost. To solve this problem, we propose a transferable multi-agent reinforcement learning framework that employs model reload and buffer reuse methods to transfer control strategies from small-scale simple scenes to large-scale complex scenes. It also utilizes attention mechanisms to aggregate observation features and handles the problem of variable observation dimensions. Finally, the distribution service restoration problem is modeled as a Markov decision process and solved using the QMIX algorithm. The performance of the proposed method has been verified in IEEE 34-node and IEEE 123-node distribution systems.

Keywords: Cognitive Computing, Human-Collaborative Robotics, Human-Machine Interface
Abstract: The perception of food texture in humans heavily relies on the way of biting and the condition and arrangement of teeth. Consequently, changes in dentures or biting speed can create an illusion of different textures for the same food. Likewise, in mastication robots, variations in control parameters and probe geometry can lead to differences in acquired data, resulting in texture misidentification. This study aims to bridge the gap in texture perception between humans and machines, and make their misperception of texture more similar. To achieve this, the possibility of replicating human texture perception by machines was investigated using a highly accurate texture discriminator called "Gel Biter", which comprises multiple soft materials. The findings indicate that misperception due to variations in mastication speed leads to misdiscrimination with a tendency similar to that in humans. Additionally, it was discovered that different contact between the tooth-shaped probes can cause a completely different texture perception. The potential contributions to food development using human dental techniques and food texture illusion are also discussed.

Keywords: Affective Computing, Cognitive Computing
Abstract: Previous studies on automatic personality perception have primarily focused on a limited number of personality traits. However, in real-world situations, humans exhibit a wide range of personality traits. To overcome this limitation, a new methodology for automatic personality perception is proposed in this paper. This revised approach can predict various personality traits (17 traits) with satisfactory performance by utilizing physiological signals. The underlying concept is to stimulate participants with different emotional stimuli to elicit physiological responses in a specific scenario. Biomarkers such as Electroencephalogram (EEG), Skin Conductance, Blood Volume Pulse, and Pupil Dilation reflect an individual's personality traits. Two experiments are conducted with different scenarios, including Image/Video Stimulation and Driving Simulation, to support this study. Based on the collection of data and validation of supervised learning models, Naive Bayes outperforms other classifiers explored in this research. EEG is the most effective signal for predicting personality, although combining other signals may produce similar results. Our method accurately predicts the 17 personality traits, demonstrating significant potential for clinical research.

Keywords: Affective Computing, Cognitive Computing, Human-Machine Interaction
Abstract: Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that affects social communication and interaction. Early diagnosis of ASD can mitigate the severity and help with ideal treatment direction. Computer vision-based methods with traditional machine learning and deep learning are employed in the literature for automatic diagnosis. Recently, deep learning with a facial image-based ASD classification has gained interest due to its ease of collection and non-invasiveness. We observed that the existing approaches utilized either local or global features of facial images to diagnose ASD. However, its important to consider both local and global features to obtain fine-grained details and larger contextual information for accurate detection and classification. This paper proposes a sequencer-based patch-wise Local Feature Extractor along with a Global Feature Extractor. Finally, the features from these modules are aggregated to obtain the final feature for the classification of ASD. Experiments on a publicly available Autism Facial Image Dataset demonstrate that our proposed framework achieves state-of-the-art performance. We achieved accuracy, precision, recall, and F1-score of 94.7%, 94.0%, 95.3%, and 94.6%, respectively.

Keywords: Human Performance Modeling, Cognitive Computing
Abstract: Visual illusions, as powerful tools for revealing the strategies and limitations of brain visual processing, have garnered attention in neural network research. This study aims to investigate the performance of various deep neural networks (DNNs) in handling the Müller-Lyer illusion task, thereby providing insights into the mechanisms of the brain's visual system in processing visual illusions. We adopted two types of visual stimuli and recruited 12 participants for a length-matching task. Based on the participants' perceptual data, we employed multiple classical neural network models to test their responses to each type of stimuli. To analyze the differences in model performance, we employed representational dissimilarity matrices (RDM) analysis and utilized Grad-CAM techniques to visualize neural network activation maps. The results revealed that some neural networks could simulate human visual illusion phenomena but exhibit differences in representational similarity and feature preferences. These findings provide a preliminary understanding of the computational principles and mechanisms of the brain's visual system and offer insights for future research in building more robust and efficient neural network models.

Keywords: Cognitive Computing, Brain-based Information Communications
Abstract: In recent years, with the recent remarkable development of large-scale language models in natural language processing research, there has been an increasing number of studies employing large-scale language models to investigate the information processing mechanisms of encoding and decoding in the brain. In this study, we developed a new pre-trained language model, BrainLM, which incorporates paired data of brain activity induced by text and stimuli, and verified the accuracy of estimating brain states from natural language in multiple NLP tasks. In essence, our research has achieved several noteworthy accomplishments. Firstly, we successfully developed a multimodal model that incorporates both brain and text. Subsequently, we conducted bi-directional experiments to validate the model and ensure the reliability of both brain encoding and decoding processes. Furthermore, we performed meticulous comparative experiments, wherein we introduced 20 state-of-the-art (SOTA) language models as a control group. Our findings reveal that our proposed model outperforms superior brain encoding ability compared to the control group. Lastly, we designed a discrete Autoencoder module that extracts brain features. This module can be utilized independently for the purpose of extracting brain features in a wider range of brain decoding studies beyond fMRI.

Keywords: Computational Intelligence, Deep Learning, Machine Learning
Abstract: The problem of unveiling the author of a given text document from multiple candidate authors is called authorship attribution. Manifold word-based stylistic markers have been successfully used in deep learning methods to deal with the intrinsic problem of authorship attribution. Unfortunately, the performance of word-based authorship attribution systems is limited by the vocabulary of the training corpus. Literature has recommended character-based stylistic markers as an alternative to overcome the hidden word problem. However, character-based methods often fail to capture the sequential relationship of words in texts which is a chasm for further improvement. The question addressed in this paper is whether it is possible to address the ambiguity of hidden words in text documents while preserving the sequential context of words. Consequently, a method based on bidirectional long short-term memory (BLSTM) with a 2-dimensional convolutional neural network (CNN) is proposed to capture sequential writing styles for authorship attribution. The BLSTM was used to obtain the sequential relationship among characteristics using subword information. The 2-dimensional CNN was applied to understand the local syntactical position of the style from unlabeled input text. The proposed method was experimentally evaluated against numerous state-of-the-art methods across the public corporal of CCAT50, IMDb62, Blog50, and Twitter50. Experimental results indicate accuracy improvement of 1.07%, and 0.96% on CCAT50 and Twitter, respectively, and produce comparable results on the remaining datasets.

Keywords: Computational Intelligence, Neural Networks and their Applications, AI and Applications
Abstract: Pursuit-Evasion (PE) games are regarded as a major platform for game theory. In this kind of game, an agent called an evader tries to escape from another agent called a pursuer. Active Target Defense (ATD) is a derivative of PE games, attracting attention recently. In an ATD game, the evader, often called an invader, strives to capture a moving target. The pursuer, called a defender, tries to intercept the invader. This paper implements the Spike-Timing-Dependent Plasticity (STDP) algorithm to train two Spiking Neural Networks (SNNs) to find a suitable solution for the ATD problem in decentralized situations. One of the SNNs is used to control the invader, while the other controls the defender. The performance is compared with the analytical solution for the pedestrian model. The results showed that an SNN can learn the optimal capture point only using relative velocities and line of sight.

Keywords: Computational Intelligence, Expert and Knowledge-Based Systems, Application of Artificial Intelligence
Abstract: Trust-building is of paramount importance for managing and improving consensus in group decision-making (GDM). This mechanism usually involves a trust propagation process for estimating the level of trust among decision-makers (DMs). However, this process is computationally expensive and hinders the speed of consensus reaching. To address this issue, this work proposes a novel trust-building mechanism that does not rely on the trust propagation process to quantify DMs' level of trust. Instead, it makes use of Blockchain technology to facilitate communication between the moderator and the group of DMs. This novel trust-building mechanism does not rely on trust propagation, which makes it computationally efficient for building trust among DMs while also providing a secure and efficient communication protocol to accelerate the consensus-reaching process. The proposed GDM model is illustrated through an example, and the sensitivity of the model to various assumptions is analyzed, demonstrating the practical applicability of this approach.

Keywords: Computational Intelligence, Expert and Knowledge-Based Systems, Big Data Computing,
Abstract: Deep neural networks show great advancement in recent decades in classifying medical images (such as CT-scans) with high precision to aid disease diagnosis. However, the training of deep neural networks requires significant sample sizes for learning enriched discriminative spatial features. Building a high quality dataset large enough to satisfy model training requirement is a challenging task due to limited disease sample cases, and various data privacy constraints. Therefore in this research, we perform medical image classification using transfer learning based on several well-known deep networks, i.e. GoogLeNet, Resnet and EfficientNet. To tackle data sparsity issues, a Wasserstein Generative Adversarial Network (WGAN) is used to generate new medical image samples to increase the numbers of training instances of the minority classes. The transfer learning process itself also allows the building of strong classifiers by transferring knowledge from the pre-trained image domain to a new medical domain using a small sample size. Moreover, the lottery ticket hypothesis is also used to prune each transfer learning network trained using the new target image data sets. Specifically, the L1 norm unstructured pruning technique is used for network reduction. Hyper-parameter fine-tuning is also performed to identify optimal settings of key network hyper-parameters such as learning rate, batch size and weight decay. A total of 20 trials are used for optimal hyper-parameter selection. Evaluated using multi-class lung X-ray images for pneumonia conditions and brain tumour CT-scans, the fine-tuned EfficientNet model obtains the best brain tumour classification accuracy rate of 96% and a fine-tuned GoogLeNet model with pruning has the highest pneumonia classification accuracy rate of 81.5%.

Keywords: Neural Networks and their Applications, Image Processing and Pattern Recognition, Deep Learning
Abstract: Facial expression recognition (FER) is an attractive research area with important applications, such as in human computer interaction, where FER can facilitate better collaboration between intelligent machines and humans. Facial Action Units (AUs) can describe facial changes in more detail, and Facial Action Unit Intensities (AUIs) reflect the intensity level of facial behavioral expression, and they can help enhance the performance of ambiguous expressions. In this paper, we propose a FER method based on AUs and use a simple and effective network called Action Unit-Enhanced Expressions (QUEEN). Specifically, first we define the center position of the region of interest of each AU through face landmarks. Second, we attach AUIs to facial expressions in the form of Gaussian heatmaps and crop AU regions of interest (RoI) based on each AU center point. Finally, we fuse the obtained AUI-enhanced expression features with the deep features of the AU RoI region via a convolutional neural network (CNN) for further expression recognition. Our proposed method is evaluated on the widely available FER databases RaFD and Oulu-CASIA. Experimental results and comparisons show that our model achieves better performance than several other popular methods.

Keywords: Neural Networks and their Applications, Deep Learning, Heuristic Algorithms
Abstract: The column generation (CG) algorithm is widely used in large-scale optimization problems. However, a large amount of columns in the restricted master problem (RMP) makes the computing process very time-consuming. This paper proposes a machine learning based column elimination strategy to accelerate the CG algorithm. Our approach represents the RMP by a bipartite graph and applies a learned Graph Neural Network model to predict redundant columns to be eliminated from the RMP, so as to reduce the time cost of solving the RMP and iterations required for convergence. Our approach is tested on cutting stock problem instances. Compared with the vanilla CG algorithm, the iterations and time required for convergence are reduced by up to 31% and 48%, respectively. Furthermore, our approach shows great generalization to cutting stock problem instances of different sizes.

Keywords: Neural Networks and their Applications, AI and Applications, Deep Learning
Abstract: Open-set recognition has gained significant attention in intrusion detection due to its ability to classify known attacks and identify novel attacks. However, current approaches based solely on discriminative features may fail to identify new data with non-discriminative feature differences and are prone to adversarial attacks. To overcome these limitations, this paper proposes an open-set intrusion detection framework that incorporates a cluster anomaly detection plugin to identify non-discriminative unknown classes by introducing primitive features and detecting adversarial examples through cluster design. Specifically, 1) we use the anomaly detection plugin trained with primitive features from the raw data to discover unseen data. Samples rejected by either deep discriminative classifiers or detection plugins are considered unknown. 2) We design the plugin as a cluster of one-class anomaly detectors. This approach effectively isolates adversarial examples carefully crafted to deceive the deep classifier. 3) We provide theoretical evidence of the proposed framework's ability to detect non-discriminative unknown classes and adversarial examples. Extensive experiments demonstrate the effectiveness of our approach when applied to open-set intrusion detection.

Keywords: Neural Networks and their Applications, Application of Artificial Intelligence, Knowledge Acquisition
Abstract: The primary aim of cognitive diagnosis is to predict students' performance and knowledge structures by analyzing their learning behavior and answering results, thereby enabling educators can provide personalized instruction. Scholars have proposed many cognitive diagnostic models. However, most of the models do not fully extract and utilize the relevant data and parameters of cognitive diagnosis. Moreover, some models only rely on artificially designed simple functions to analyze the cognitive process of students, which cannot fully capture the complex relationship between students and the exercises. To address these limitations, this paper proposes a neural cognitive diagnostic model named NCDFSA, which focuses on students' attention to knowledge concepts. The model utilizes neural networks to diagnose students' knowledge and considers students' implicit relationships with knowledge concepts. We introduce the concept of attention matrix(AM) and define the importance of knowledge concepts by the frequency of usage of knowledge concepts to improve the prediction effect. This paper compares NCDFSA with existing classical models on four real datasets and finds that the model has higher accuracy and rationality in predicting student performance.

Keywords: Neural Networks and their Applications, AI and Applications, Deep Learning
Abstract: Anomaly detection plays a critical role in condition monitors to support the trustworthiness of Cyber-Physical Systems (CPS). Detecting multivariate anomalous data in such systems is challenging due to the lack of a complete comprehension of anomalous behaviors and features. This paper proposes a framework to address time series multivariate anomaly detection problems by combining the Self-Organizing Map (SOM) with Deep Reinforcement Learning (DRL). By clustering the multivariate data, SOM creates an environment to enable the DRL agents interacting with the collected system operational data in terms of a tabular dataset. In this environment, Markov chains reveal the likely anomalous features to support the DRL agent exploring and exploiting the state-action space to maximize anomaly detection performance. We use a time series dataset, Skoltech Anomaly Benchmark (SKAB), to evaluate our framework. Compared with the best results by some currently applied methods, our framework improves the F1 score by 9% from 0.67 to 0.73.

Keywords: Neural Networks and their Applications, AI and Applications, Deep Learning
Abstract: Embedding&MLP paradigm based on deep learning has been widely used in click-through rate prediction tasks. Such approaches compress user features into a fixed-length vector, which causes the bottleneck of interest learning and makes it difficult to express diverse interests of users. To improve the accuracy and diversity of recommendation, more comprehensive user interest learning is required. We present Multi-Granularity Interest Learning(MGIL)—combining the memorization of user’s historical preferences with generalization of user’s interests. Our fine-grained interest module learns interests at the granularity of items, preserving the memory of users’ historical preferences as much as possible. The coarse-grained interest module, on the other hand, extracts multiple high-level interests from user behavior sequences to characterize multiple aspects of user interest, which are more generalized and abstract. The two modules complement each other from different granularities. We conduct extensive experiments on three real-world datasets, Movielens, Taobao and Amazon. Experimental results demonstrate that our method significantly improves the accuracy and diversity of recommendation compared to state-of-the-art models.