Search

This paper presents a systematic review on reinforcement learning approaches for combinatorial optimization problems based on real-world industrial applications. While this topic is increasing in popularity, explicit implementation details are not always available in the literature. The main objective of this paper is characterizing the agent–environment interactions, namely, the state space representation, action space mapping and reward design. Also, the main limitations for practical implementation and the needed future developments are identified. The literature selected covers a wide range of industrial combinatorial optimization problems, found in the IEEE Xplore, Scopus and Web of Science databases. A total of 715 unique papers were extracted from the query. Then, out-of-scope applications, reviews, surveys and papers with insufficient implementation details were removed. This resulted in a total of 298 papers that align with the focus of the review with sufficient implementation details. The state space representation shows the most variety, while the reward design is based on combinations of different modules. The presented studies use a large variety of features and strategies. However, one of the main limitations is that even with state-of-the-art complex models the scalability issues of increasing problem complexity cannot be fully solved. No methods were used to assess risk of biases or automatically synthesize the results.

This paper proposes a novel approach to the fuzzy newsvendor problem for inventory management applications. The main contributions of the paper are the following: a new credibility estimation is proposed, to explore the neighborhood around the most impactful demand scenarios; a simulation procedure was designed for the different demand scenarios, which allows comparison of the proposed approach with classical and fuzzy multi-item newsvendor problems; a modified genetic algorithm (GA) is introduced to ameliorate previous genetic algorithms in both the generation and evaluation of solutions. The new formulation of the fuzzy newsvendor problem, together with the modified GA, were shown to improve the average profit by up to 55% in problems with low-budget scenarios.

Robotic systems are increasingly present in dynamic environments. This paper proposes a hierarchical control structure wherein a behavior tree (BT) is used to improve the flexibility and adaptability of an omni-directional mobile robot for point stabilization. Flexibility and adaptability are crucial at each level of the sense–plan–act loop to implement robust and effective robotic solutions in dynamic environments. The proposed BT combines high-level decision making and continuous execution monitoring while applying non-linear model predictive control (NMPC) for the point stabilization of an omni-directional mobile robot. The proposed control architecture can guide the mobile robot to any configuration within the workspace while satisfying state constraints (e.g., obstacle avoidance) and input constraints (e.g., motor limits). The effectiveness of the controller was validated through a set of realistic simulation scenarios and experiments in a real environment, where an industrial omni-directional mobile robot performed a point stabilization task with obstacle avoidance in a workspace.

In the last few decades, there has been a growing necessity for systems that handle market changes and personalized customer needs with near mass production efficiency, defined as the new mass customization paradigm. The Industry 5.0 vision further enhances the human-centricity aspect, in the necessity for manufacturing systems to cooperate with workers, taking advantage of their problem-solving capabilities, creativity, and expertise of the manufacturing process. A solution is to develop a flexible manufacturing system capable of handling different customer requests and real-time decisions from operators. This paper tackles these aspects by proposing a digital twin of a robotic system for solution preparation capable of making real-time scheduling decisions and forecasts using a simulation model while allowing human interventions. A discrete event simulation model was used to forecast possible system improvements. The simulation handles real-time scheduling considering the possibility of adding identical parallel machines. Results show that processing multiple jobs simultaneously with more than one machine on critical processes, increasing the robot speed, and using heuristics that emphasize the shortest transportation time can reduce the overall completion time by 82%. The simulation model has an animated visualization window for a deeper understanding of the system.

In order to achieve a more efficient allocation of healthcare resources in the near future, it is crucial to understand the patterns and causes of excess mortality and hospitalizations. Neonatal mortality still poses a significant challenge, particularly in developed nations where the mortality rates are already low and healthcare resources are generally available to most of the population. Furthermore, the low mortality rates mean that the data available for modeling are often very limited, restricting the modeling methods that can be used. It is also important that the chosen methods allow for explainable, non-black-box models that can be interpreted by healthcare professionals. Considering these challenges, the work hereby presented thoroughly analyzed the time series of the neonatal mortality rates in Portugal between 2014 and 2019 in terms of trend and seasonal patterns. The applicability and performance of different data-based methods were also analyzed. Furthermore, the mortality rates were studied in terms of their relation to environmental variables, such as temperature and air pollution indicators, with the goal of establishing causal relations between such variables and excess mortality. The preliminary results show that ARMA, neural and fuzzzy models are able to forecast the studied mortality rates with good performance. In particular, neural models have the best predictive performance, while fuzzy models are well suited to obtain interpretable models with acceptable performance.

The simultaneous integration of information from sensors with business data and how to acquire valuable information can be challenging. This paper proposes the simultaneous integration of information from sensors and business data. The proposal is supported by an industrial implementation, which integrates intelligent sensors and real-time decision-making, using a combination of PLC and PC Platforms in a three-level architecture: cloud-fog-edge. Automatic identification intelligent sensors are used to improve the decision-making of a dynamic scheduling tool. The proposed platform is applied to an industrial use-case in analytical Quality Control (QC) laboratories. The regulatory complexity, the personalized production, and traceability requirements make QC laboratories an interesting use case. We use intelligent sensors for automatic identification to improve the decision-making of a dynamic scheduling tool. Results show how the integration of intelligent sensors can improve the online scheduling of tasks. Estimations from system processing times decreased by over 30%. The proposed solution can be extended to other applications such as predictive maintenance, chemical industry, and other industries where scheduling and rescheduling are critical factors for the production.

Left ventricular ejection fraction (LVEF) constitutes an important physiological parameter for the assessment of cardiac function, particularly in the settings of coronary artery disease and heart failure. This study explores the use of routinely and easily acquired variables in the intensive care unit (ICU) to predict severely depressed LVEF following ICU admission. A retrospective study was conducted. We extracted clinical physiological variables derived from ICU monitoring and available within the MIMIC II database and developed a fuzzy model using sequential feature selection and compared it with the conventional logistic regression (LR) model. Maximum predictive performance was observed using easily acquired ICU variables within 6 hours after admission and satisfactory predictive performance was achieved using variables acquired as early as one hour after admission. The fuzzy model is able to predict LVEF ≤ 25% with an AUC of 0.71 ± 0.07, outperforming the LR model, with an AUC of 0.67 ± 0.07. To the best of the authors’ knowledge, this is the first study predicting severely impaired LVEF using multivariate analysis of routinely collected data in the ICU. We recommend inclusion of these findings into triaged management plans that balance urgency with resources and clinical status, particularly for reducing the time of echocardiographic examination.

The growing environmental concerns and liberalization of energy markets have resulted in an increased competition between utilities and a strong focus on efficiency. To develop new energy efficiency measures and optimize operations, utilities seek new market-related insights and customer engagement strategies. This paper proposes a clustering-based methodology to define the segmentation of residential gas consumers. The segments of gas consumers are obtained through a detailed clustering analysis using smart metering data. Insights are derived from the segmentation, where the segments result from the clustering process and are characterized based on the consumption profiles, as well as according to information regarding consumers’ socio-economic and household key features. The study is based on a sample of approximately one thousand households over one year. The representative load profiles of consumers are essentially characterized by two evident consumption peaks, one in the morning and the other in the evening, and an off-peak consumption. Significant insights can be derived from this methodology regarding typical consumption curves of the different segments of consumers in the population. This knowledge can assist energy utilities and policy makers in the development of consumer engagement strategies, demand forecasting tools and in the design of more sophisticated tariff systems.