Search

Foundation models (e.g., ChatGPT, DALL-E, PengCheng Mind, PanGu-$\Sigma$) have demonstrated extraordinary performance in key technological areas, such as natural language processing and visual recognition, and have become the mainstream trend of artificial general intelligence. This has led more and more major technology giants to dedicate significant human and financial resources to actively develop their foundation model systems, which drives continuous growth of these models' parameters. As a result, the training and serving of these models have posed significant challenges, including substantial computing power, memory consumption, bandwidth demands, etc. Therefore, employing efficient training and serving strategies becomes particularly crucial. Many researchers have actively explored and proposed effective methods. So, a comprehensive survey of them is essential for system developers and researchers. This paper extensively explores the methods employed in training and serving foundation models from various perspectives. It provides a detailed categorization of these state-of-the-art methods, including finer aspects such as network, computing, and storage. Additionally, the paper summarizes the challenges and presents a perspective on the future development direction of foundation model systems. Through comprehensive discussion and analysis, it hopes to provide a solid theoretical basis and practical guidance for future research and applications, promoting continuous innovation and development in foundation model systems.

Virtual reality (VR) for training helps minimize risks and costs by allowing more frequent and varied use of experiential training experiences, leading to active and improved learning. However, creating VR training experiences is costly and time-consuming, requiring software development experts. Additionally, current authoring tools are desktop-oriented, which detaches the process of creating the immersive experience from experiencing it in a situated context. This paper presents the development of an immersive authoring tool designed to create immersive virtual environments that can be used to train operatives. The authoring tool can record and replay animations of each action the user performed that can later be used to instruct other users how the task should be performed. Participants were divided into two groups, and the proposed authoring tool was evaluated using usability, satisfaction, presence and cybersickness. Between groups, Independent T-tests revealed that there were no significant differences between expert and non-expert groups in any of the studied variables. Also, the results showed that the authoring tool had high usability and satisfaction, average presence, and low probability of cybersickness symptoms.

Accessibility and inclusion are fundamental human rights, enacted to ensure that everyone has equal access to digital content. However, in practice, a large number of interactive systems remain inaccessible, which prevents users from accessing essential content. One of the reasons why technology remains inaccessible is the limited training that technologists receive while following formal education. Therefore, to enhance accessibility teaching in higher education, we propose an empathy-driven intervention for college students. The contributions of this paper are multifold: first, we describe the multi-components of the accessibility intervention proposed, describing its materials, protocol for application, rubrics for scores and feedback. Then, we describe and discuss the results of its implementation in a classroom environment over the course of a semester as well as the impact of the intervention on the student’s personal and professional development regarding accessibility concepts.

Increasingly, the smart city space is requiring a reconceptualization of forms and factors of production, including factories and their place in the smart city space. Factories have always been a part of the city and many people spend a significant part of their lives there. Cities and factories share the same physical space and draw from the same resources, such as the energy grid, communication networks, public utilities, social connections, etc. Factories and cities should share the same IoT network in order to maximize their synergy level. In this view, as ICT-enhanced solutions are being implemented and so the concept of the smart city becomes a reality, it is mandatory that the connection between the smart city and the smart factory is examined. This paper represents the first step in this direction. We are presenting a new smart way to lighten the workload for employees (especially those involved in assembly, setup and maintenance) and increase factory efficiency. We have developed a brand-new smart solution for designing and presenting work instructions. The solution can be easily adapted to use in other fields like healthcare or smart-homes. This paper presents a comparison of different types of virtual/augmented and conventional assembly instructions. Today, we face the challenge of a lack of skilled employees and a high rate of employee turnover. Both result in huge time and production losses, because new employees have to be taught simple assembly tasks over and over again. In addition, as companies begin hiring many more foreign workers who do not understand the local language, the challenge of teaching becomes even more acute. Despite this, in modern production systems we can still find ineffective and complicated books and manuals with assembly, service and measurement instructions. We have prepared several variants for non-trivial multi-step assembly instructions: traditional “paper” instructions, video instructions, virtual instructions on screen (with/without in-situ projection and with/without a special controller). We have developed our own software system for working with and developing virtual assembly instructions. In this case the in-situ augmentation is a projection on to different parts of the workplace. 60 subjects were tested over two years in order to gather the learning curve for each of 5 types of instructions using virtual and augmented reality. We have proven that using any type other than “paper” will shorten the learning time by approximately half. Practitioner summary: We have prepared and tested variants for non-trivial multi-step assembly instructions. 60 subjects were tested over two years in order to gather the learning curve for all 5 types of instructions - traditional paper, video instructions, virtual instructions and two types of virtual instructions combined with augmented reality in-situ projection. We have proven that using any type other than “paper” will shorten the learning time by approximately half.

In this paper, we characterize the combined channel state information (CSI) and re-transmission feedback on wireless communication. Feedback, often assumed free, has traditionally been used to either learn the channel state, or to request the re-transmission of a failed reception. We propose here a generalized framework for feedback that captures the channel time-variation. We consider two-way communication model where all the resources needed for training, feedback (CSI and automatic repeat request), and data. In order to maximize the throughput and optimize resource allocations, we provide the expressions of outage probabilities for different rate/power adaptation and re-transmission protocols. The simulation results show that, besides signal-to-noise ratio (SNR) and Doppler, the optimal pilot overhead depends on whether we are using rate/power adaptation and the number of re-transmissions. We also prove that in slow-fading channels with high SNR regimes, varying the rate provides the best throughput. When dealing with fast time-varying channels and low SNR regimes, using constant power and rate gives the best throughput performance.

Advances in training technologies using Extended Reality (XR) offer dramatic improvements in terms of time and associated costs over traditional training methods. Industries that are reliant on human labor are starting to embrace XR devices such as Virtual and Augmented Reality (VR/AR) Head-Mounted Displays (HMDs) to capitalize on these advantages. Additionally, developers now have the capacity to create customized user experiences and reduce instructor workload by employing adaptive automation in XR training simulations. Adaptive systems analyze the behavior of a learner for a trigger and then deploy an adaptation to alter the system state. While examples of adaptive XR training systems in current literature show that they are feasible to develop, further research must be done to quantify their efficacy. Furthermore, recommendations for how to design triggers and adaptations for XR training simulations are currently non-existent, resulting in negative impacts to training if inappropriate adaptations are implemented. This paper provides novel, evidence-based recommendations for the design of future adaptive XR training systems by learning from expert XR instructors. To this end, semi-structured interviews were conducted with 11 XR trainers. Participants were asked to discuss their experiences dealing with learners who exhibited confusion during XR training. Interviews were analyzed for existing and emerging themes. Finally, these themes were applied to existing trigger and adaptation models and synthesized into design recommendations for XR training developers. The outcomes of this work will inform the future development of adaptive XR training platforms.

The self-encoder is a typical unsupervised deep learning algorithm. In the field of unsupervised learning, it is very popular with researchers. Therefore, in view of the shortage of labeled training samples, the convolution kernel of a typical convolutional neural network is set by experience, and the network structure is fixed and it is difficult to re-learn later. This paper combines the convolutional neural network and the automatic encoder, and proposes a multi-based the method of integrated network structure to extract the features of the image for recognition. First, the SAE pre-trained CNN model convolution kernel is used to pre-train based on the classic CNN structure. Secondly, input and process image data of different scales to extract image space and spectral features respectively. Then, construct multiple channels, and use different scale filters and sampling intervals for different channels. Finally, after one layer of down sampling, the feature maps obtained from multiple channels are input into the fully connected layer, and after a hidden layer, the features finally used for classification are obtained. Experimental results show that the proposed method uses sparse automatic coding for pre-training time efficiency increased by 50%, and can further improve the recognition accuracy, the highest recognition rate reached 0.985.

Massive multiple-input multiple-output (MIMO) systems with wireless backhaul networks have been considered as a candidate technology for 5G. Despite their potential, these systems still require improvement in areas such as resource allocation and interference management. In this paper, we derive a closed-form lower bound ergodic downlink rate expression for training-based multicell massive MIMO systems with full duplex wireless backhaul networks. We also propose a wireless backhaul link selection method, in which only certain access points will be activated and only certain users will use wireless backhaul links. The results show that the wireless backhaul with a link selection method can improve the downlink rate compared to other methods. The downlink rate of the minimum rate users can be significantly improved by prioritizing the wireless backhaul links for the minimum rate users. Power analysis shows that there is concave relationship between access point transmit power and median downlink rate. The results also show that the optimum access point transmit power is much lower than the standard transmit power from the users and base station. Additional investigation shows that users are more likely to use backhaul links when they are located farther from the base station. We also demonstrate that as the number of spatial multiplexed users increases, the performance improvement provided by wireless backhaul system decreases.

Point of care ultrasonography and the related focused assessment with sonography for trauma protocol, if performed by experienced physicians, is a highly sensitive examination, and specific for the detection of free fluids. Different systems and methods have been proposed for training, including simulation as one of the most efficient. This paper presents an ultrasound training system, specifically designed to be used during bedside high fidelity simulation scenarios, that could facilitate the learning process. The development of the proposed system exploited novel rapid prototyping electronic boards as a means to obtain good performances with a low cost. Moreover, the design of the data structure permits the construction of a library that caters for individual needs, with the possibility of adding emergency scenarios, collecting pictures or videos, as well as 3-D volumes. The device has been compared with currently commercial ultrasound simulators and its innovative aspects have been highlighted. Finally, it has been tested during a training session in order to evaluate features, such as realism and user-friendliness.

3-D motion tracking is a fundamental technology enabling virtual reality and mixed reality. Correct position and orientation detection are critical as both an input for 3-D interaction and for proper visual display. This paper presents a cross-platform solution for presenting data from multiple tracking systems as if it came from a single system. This frees simulations from having to account for multiple data sources, and allows tracked objects to transition seamlessly from an area covered by one tracking system to another. It also presents an application programming interface with a single data format, eliminating the need for hardware drivers. Finally, when self-contained tracking systems are used, they can themselves be tracked, allowing the trackable area to be adjusted in real time. This allows operators to leverage limited resources in more effective ways, improving simulation quality and opening new possibilities for simulations.

The use of computer-based and online education systems has made new data available that can describe the temporal and process-level progression of learning. To date, machine learning research has not considered the impacts of these properties on the machine learning prediction task in educational settings. Machine learning algorithms may have applications in supporting targeted intervention approaches. The goals of this paper are to: 1) determine the impact of process-level information on machine learning prediction results and 2) establish the effect of type of machine learning algorithm used on prediction results. Data were collected from a university level course in human factors engineering (n=35), which included both traditional classroom assessment and computer-based assessment methods. A set of common regression and classification algorithms were applied to the data to predict final course score. The overall prediction accuracy as well as the chronological progression of prediction accuracy was analyzed for each algorithm. Simple machine learning algorithms (linear regression, logistic regression) had comparable performance with more complex methods (support vector machines, artificial neural networks). Process-level information was not useful in post-hoc predictions, but contributed significantly to allowing for accurate predictions to be made earlier in the course. Process level information provides useful prediction features for development of targeted intervention techniques, as it allows more accurate predictions to be made earlier in the course. For small course data sets, the prediction accuracy and simplicity of linear regression and logistic regression make these methods preferable to more complex algorithms.