17,671 results
Search Results
2. Cellulose-Graphene Bifunctional Paper Conservation Materials: For Reinforcement and UV Aging Protection.
- Author
-
Tian, Peng, Shi, Meirong, Hou, Jingmin, and Fu, Peng
- Subjects
TENSILE tests ,CARBOXYMETHYLCELLULOSE ,REINFORCEMENT learning ,GRAPHENE oxide ,SERVICE life - Abstract
Paper artifacts have unique cultural and historical values. However, over time, many paper artifacts appear with disease characteristics such as embrittlement and photoaging, losing the most fundamental function of the literature archive. The reinforcement handling of degraded paper artifacts is, therefore, a necessary measure to extend their service life, the key to which lies in the reinforcement and prevention of photoaging. This paper intended to use graphene oxide (GO) as a UV protective agent, carboxymethyl cellulose (CMC) as a reinforcement, and polyethyleneimine (PEI) as a modifier. In this work, the amino-modified graphene oxide carboxymethyl cellulose composite (CMC-aGO) was prepared by chemical modification, which was used as bifunctional paper protection material with anti-ultraviolet and reinforcement. It showed excellent performance in both tensile strength testing and UV resistance testing. The CMC-aGO raw material is low cost, colorless, transparent, simple to synthesize, convenient to operate, and is an excellent conservation material with dual functions of UV aging protection and paper reinforcement. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF
3. XAI and Strategy Extraction via Reward Redistribution
- Author
-
Dinu, Marius-Constantin, Hofmarcher, Markus, Patil, Vihang P., Dorfer, Matthias, Blies, Patrick M., Brandstetter, Johannes, Arjona-Medina, Jose A., Hochreiter, Sepp, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Woeginger, Gerhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Holzinger, Andreas, editor, Goebel, Randy, editor, Fong, Ruth, editor, Moon, Taesup, editor, Müller, Klaus-Robert, editor, and Samek, Wojciech, editor
- Published
- 2022
- Full Text
- View/download PDF
4. Application of Reinforcement Learning in Production Planning and Control of Cyber Physical Production Systems
- Author
-
Kuhnle, Andreas, Lanza, Gisela, inIT - Institut für industrielle Informa, Beyerer, Jürgen, editor, Kühnert, Christian, editor, and Niggemann, Oliver, editor
- Published
- 2019
- Full Text
- View/download PDF
5. Continuous-Action Reinforcement Learning for Memory Allocation in Virtualized Servers
- Author
-
Garrido, Luis A., Nishtala, Rajiv, Carpenter, Paul, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Woeginger, Gerhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Weiland, Michèle, editor, Juckeland, Guido, editor, Alam, Sadaf, editor, and Jagode, Heike, editor
- Published
- 2019
- Full Text
- View/download PDF
6. Illustrated Technical Paper - On Modeling Network Slicing Communication Resources with SARSA Optimization
- Author
-
Xavier, Eduardo S., Agoulmine, Nazim, and Martins, Joberto S. B.
- Subjects
Optimization ,Communication Slice ,Network Slicing ,Network Slicing Modeling ,SARSA ,Reinforcement Learning ,Multidomain Slicing - Abstract
This paper illustrates and complements the article “On modeling network slicing communication resources with SARSA optimization.” The illustrated technical paper is intended to complement and subsidize content information extracting and reading and highlights explanatory text and additional or focused bibliographic references., {"references":["Shalitha Wijethilaka and Madhusanka Liyanage. Survey on Network Slicing for Internet of Things Realization in 5G Networks. IEEE Communications Surveys Tutorials, 23(2):957–994, 2021.","A. A. Barakabitze, A Ahmad, R Mijumbi, and A Hines. 5G Network Slicing Using SDN and NFV: A Survey of Taxonomy, Architectures and Future Challenges. Computer Networks, 167:106984, February 2020.","F. Yousaf, M. Gramaglia, V. Friderikos, B. Gajic, D. von Hugo, B. Sayadi, V. Sciancalepore, and M. Crippa. Network Slicing with Flexible Mobility and Qos/Qoe Support for 5g Networks. In IEEE International Conference on Communications, pages 1195–1201, May 2017.","P K Thiruvasagam, A Chakraborty, and C. Murthy. Resilient and Latency-Aware Orchestration of Network Slices Using Multi-Connectivity in MEC-Enabled 5G Networks. IEEE Transactions on Network and Service Management, 18(3):2502–2514, September 2021.","I Kovacevic, A Shafigh, S Glisic, B Lorenzo, and E Hossain. Multi-Domain Network Slicing With Latency Equalization. IEEE Transactions on Network and Service Management, 17(4):2182–2196, December 2020.","Jonathan Prados-Garzon and Tarik Taleb. Asynchronous Time-Sensitive Networking for 5G Backhauling. IEEE Network, 35(2):144–151, March 2021.","Y. Xiao and M. Krunz. Dynamic Network Slicing for Scalable Fog Computing Systems With Energy Harvesting. IEEE Journal on Selected Areas in Communications, 36(12):2640–2654, December 2018.","K. Samdanis, X. Costa-Perez, and V. Sciancalepore. From Network Sharing to Multi-Tenancy: The 5G Network Slice Broker. IEEE Communications Magazine, 54(7):32–39, July 2016.","Gilvan Durães, Rafael Reale, Romildo Bezerra, Alexandre Fontinele, André Soares, and Joberto S. B. Martins. Evaluating the Applicability of Bandwidth Allocation Models for EON Slot Allocation. In Proceedings of the IEEE International Conference on Advanced Networks and Telecommunications Systems (IEEEANTS), pages 1–6, India, December 2017.","Joberto S B Martins, Tereza C. Carvalho, Flavio Silva, and Rodrigo Moreira. SFI2 Network Slicing Reference Architecture. Technical Report TR03/2022, Universidade de São Paulo - USP, 2022.","G. N. Dias, J. F. Rezende, L N. Ciuffo, I. Machado, Flavio Silva, Tereza C. de Brito, F. Redigolo, Joberto S. B. Martins, Leobino Sampaio, and Antonio Abelem. SFI2 - Slicing Future Internet Infrastructures project. In Proceedings of the The Global Experimentation for Future Internet, pages 1–3, Coimbra, Portugal, November 2019.","S Clayman, A Neto, F Verdi, S Correa, S Sampaio, I Sakelariou, L Mamatas, R Pasquini, K Cardoso, F Tusa, C Rothenberg, and J Serrat. The NECOS Approach to End-to-End Cloud-Network Slicing as a Service. IEEE Communications Magazine, 59(3), March 2021.","J Nightingale, Qi Wang, J M. Alcaraz Calero, E Chirivella-Perez, M Ulbricht, J A. Alonso-López, R Preto, T Batista, T Teixeira, M J Barros, and C Reinsch. Qoe-driven, energy-aware video adaptation in 5g networks: The SELFNET self-optimisation use case. Int. J. Distributed Sens. Networks, 12(1), 2016.","P Gouvas, A Zafeiropoulos, C Vassilakis, E Fotopoulou, G Tsiolis, R Bruschi, R Bolla, and F Davoli. Design, Development and Orchestration of 5G-Ready Applications over Sliced Programmable Infrastructure. In 29th International Teletraffic Congress (ITC), volume 2, pages 13–18, September 2017.","IETF. Framework for IETF Network Slices. RFC- Request for Comments draft-ietf-teas-ietf-network-slice-framework-00, Internet Engineering Task Force, March 2021.","3GPP. 5G-Evolution-3GPP. Technical Report Release 16-17, 3GPP, 2020.","Telecommunication Standardization ITU-T. Framework of Network Virtualization for Future Networks. Technical Report ITU-T Y.3011, ITU-T, January 2012.","ETSI. Mobile Edge Computing A key technology towards 5G. Technical Report WP No. 11, European Telecommunications Standards Institute, September 2015.","ONF Open Networking Foundations. Applying SDN Architecture to 5G Slicing. Technical Report TR-526, ONF - Open Networking Foundations, 2016.","Shunliang Zhang. An Overview of Network Slicing for 5G. IEEE Wireless Communications, 26(3):111–117, June 2019.","X Foukas, G Patounas, A Elmokashfi, and M K Marina. Network Slicing in 5G: Survey and Challenges. IEEE Communications Magazine, 55(5):94–100, May 2017.","Alexandros Kaloxylos. A Survey and an Analysis of Network Slicing in 5G Networks. IEEE Communications Standards Magazine, 2(1):60–65, March 2018.","Eduardo S. Xavier, Nazim Agoulmine, and Joberto Martins. On Modeling Network Slicing Communication Resources with SARSA Optimization. In Proceedings of the International Workshop on ADVANCEs in ICT Infrastructures and Services (ADVANCE), pages 1–12, Fortaleza, Brazil, February 2023.","I Afolabi, T Taleb, K Samdanis, A Ksentini, and H Flinck. Network Slicing and Softwarization: A Survey on Principles, Enabling Technologies, and Solutions. IEEE Communications Surveys Tutorials, 20(3):2429–2453, 2018.","3GPP. 3rd Generation Partnership Project; Technical Specification Group Services and System Aspects; Management and Orchestration; Concepts, Use Cases and Requirements. Technical Specification 3GPP TS 28.530 V15.0, 3GPP, 2019.","Eliseu Torres, Rafael Reale, Leobino Sampaio, and Joberto S. B. Martins. A SDN/OpenFlow Framework for Dynamic Resource Allocation based on Bandwidth Allocation Model. IEEE Latin America Transactions, 18(5):853–860, April 2020."]}
- Published
- 2023
- Full Text
- View/download PDF
7. Illustrated Technical Paper - Reinforcement Learning Agent Design and Optimization with Bandwidth Allocation Model
- Author
-
Reale, Rafael F. and Martins, Joberto S. B.
- Subjects
Agent Design ,Agent Optimization ,RL Task Offloading ,Reinforcement Learning ,Bandwidth Allocation Model - Abstract
This ILLUSTRATED TECHNICAL PAPER presents the slides and related discussion describing the contents of the paper "Reinforcement Learning Agent Design and Optimization with Bandwidth Allocation Model". The talk was presented at the 9th Annual Conference on Computational Science and Computational Intelligence (CSCI'22), realized on 14 - 16 November 2022 in the USA. The ILLUSTRATED TECHNICAL PAPER format is intended to facilitate the reader's perception of the paper contents by complementing, enriching, and subsidizing the technical paper content and slides with complementary text, and additional and/or focused bibliographic references., {"references":["Joberto S. B. Martins, Romildo Bezerra, Rafael Reale, and Gilvan Durães. Uma Visão Tutorial dos Modelos de Alocação de Banda como Mecanismo de Provisionamento de Recursos em Redes IP/MPLS. Revista de Sistemas e Computação, 5(2):144–155, December 2015.","Rafael F. Reale and Joberto S. B. Martins. Reinforcement Learning Agent Design and Optimization with Bandwidth Allocation Model. In Proceedings of the International Conference on Computational Science and Computational Intelligence (CSCI), pages 1–7, US, December 2022.","Richard S. Sutton and Andrew G. Barto. Introduction to Reinforcement Learning. MIT Press, Cambridge, MA, USA, 1st edition, 1998.","Wen Wu, Conghao Zhou, Mushu Li, Huaqing Wu, Haibo Zhou, Ning Zhang, Xuemin Sherman Shen, and Weihua Zhuang. AI-Native Network Slicing for 6G Networks. IEEE Wireless Communications, 29(1):96–103, February 2022.","Rakesh Kumar, Mayank Swarnkar, Gaurav Singal, and Neeraj Kumar. IoT Network Traffic Classification Using Machine Learning Algorithms: An Experimental Analysis. IEEE Internet of Things Journal, 9(2):989–1008, January 2022.","Mahbuba Afrin, Jiong Jin, Akhlaqur Rahman, Ashfaqur Rahman, Jiafu Wan, and Ekram Hossain. Resource Allocation and Service Provisioning in Multi-Agent Cloud Robotics: A Comprehensive Survey. IEEE Communications Surveys Tutorials, 23(2):842–870, 2021. Conference Name: IEEE Communications Surveys Tutorials.","Baofeng Ji, Xueru Zhang, Shahid Mumtaz, Congzheng Han, Chunguo Li, Hong Wen, and Dan Wang. Survey on the Internet of Vehicles: Network Architectures and Applications. IEEE Communications Standards Magazine, 4(1):34–41, March 2020.","André L. C. Ottoni, Erivelton G. Nepomuceno, Marcos S. de Oliveira, and Daniela C. R. de Oliveira. Tuning of Reinforcement Learning Parameters Applied to SOP Using the Scott–Knott Method. Soft Computing, 24(6):4441–4453, March 2020.","Carlos E. Arruda, Pedro F. Moraes, Nazim Agoulmine, and Joberto S. B. Martins. Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning. Lecture Notes in Computer Science. Springer International Publishing, Cham, 2021.","Kaiqing Zhang, Zhuoran Yang, and Tamer Ba¸sar. Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms. In Kyriakos G. Vamvoudakis, Yan Wan, Frank L. Lewis, and Derya Cansever, editors, Handbook of Reinforcement Learning and Control, Studies in Systems, Decision and Control, pages 321–384. Springer International Publishing, Cham, 2021.","Shariq Iqbal and Fei Sha. Actor-Attention-Critic for Multi-Agent Reinforcement Learning. In Proceedings of the 36th International Conference on Machine Learning, pages 2961–2970. PMLR, May 2019. ISSN: 2640-3498.","Chen Zhang, Yu Xie, Hang Bai, Bin Yu, Weihong Li, and Yuan Gao. A Survey on Federated Learning. Knowledge-Based Systems, 216:106775, March 2021.","Wei Yang Bryan Lim, Nguyen Cong Luong, Dinh Thai Hoang, Yutao Jiao, Ying-Chang Liang, Qiang Yang, Dusit Niyato, and Chunyan Miao. Federated Learning in Mobile Edge Networks: A Comprehensive Survey. IEEE Communications Surveys & Tutorials, 22(3):2031–2063, 2020. Conference Name: IEEE Communications Surveys & Tutorials.","David Ha. Reinforcement Learning for Improving Agent Design. Artificial Life, 25(4):352–365, November 2019.","Thanh Thi Nguyen, Ngoc Duy Nguyen, Peter Vamplew, Saeid Nahavandi, Richard Dazeley, and Chee Peng Lim. A MultiObjective Deep Reinforcement Learning Framework. Engineering Applications of Artificial Intelligence, 96:103915, November 2020.","Douglas C. Montgomery. Design and Analysis of Experiments. Wiley, 9th edition edition, March 2017.","Lars Stahle and Svante Wold. Analysis of Variance (ANOVA). Chemometrics and Intelligent Laboratory Systems, 6(4):259–272, November 1989.","F L Faucher and W Lai. Maximum Allocation Bandwidth Constraints Model for DiffServ-aware MPLS Traffic Engineering. Request for Comments RFC 4125, Internet Engineering Task Force - IETF, June 2005.","Walter da Costa Pinto Neto and Joberto S. B. Martins. Adapt-RDM - A Bandwidth Management Algorithm Suitable for DiffServ Services Aware Traffic Engineering. In IEEE Network Operations and Management Symposium (NOMS), pages 975– 978, April 2008. ISSN: 2374-9709.","Rafael F. Reale, Walter da C. P. Neto, and Joberto S. B. Martins. AllocTC-sharing: A new bandwidth allocation model for DS-TE networks. In 2011 7th Latin American Network Operations and Management Symposium, pages 1–4, October 2011.","Rafael Freitas Reale, Romildo Martins Bezerra, and Joberto S. B. Martins. Analysis of Bandwidth Allocation Models Reconfiguration Impacts. In Proceedings of the III International Workshop on ICT Infrastructures and Services (ADVANCE), pages 67–76, Florida, US, 2014.","Gilvan Durães, Rafael Reale, Romildo Bezerra, Alexandre Fontinele, André Soares, and Joberto S. B. Martins. Evaluating the Applicability of Bandwidth Allocation Models for EON Slot Allocation. pages 1–6, December 2017.","Ahmed El-mekkawi, Xavier Hesselbach, and Jose Ramon Piney. Squatting and Kicking Model Evaluation for Prioritized Sliced Resource Management. Computer Networks, 167:107006, February 2020."]}
- Published
- 2022
- Full Text
- View/download PDF
8. Supplementary material for paper: Reinforcement Learning in Robotic Motion Planning by Combined Experience-based Planning and Self-Imitation Learning
- Author
-
Sha, Luo and Schomaker Lambert
- Subjects
robotics ,reinforcement learning ,obstacle avoidance ,motion planning ,self-imitation learning - Abstract
We addedextra experiments in simulation to evaluate the best-performing policy in environments with unseen obstacles. Here the pdf file describes the experiment designand shows the experimental settings and results in a figure and a table. A brief analysis of the results has been provided.We have also attached a video capturing part of the testing process in Gazebo.
- Published
- 2022
- Full Text
- View/download PDF
9. SDP-Based Bounds for the Quadratic Cycle Cover Problem via Cutting-Plane Augmented Lagrangian Methods and Reinforcement Learning: INFORMS Journal on Computing Meritorious Paper Awardee.
- Author
-
de Meijer, Frank and Sotirov, Renata
- Subjects
- *
REINFORCEMENT learning , *COMBINATORIAL optimization , *TRAVELING salesman problem , *ALGORITHMS , *SEMIDEFINITE programming , *MACHINE learning , *DIRECTED graphs - Abstract
We study the quadratic cycle cover problem (QCCP), which aims to find a node-disjoint cycle cover in a directed graph with minimum interaction cost between successive arcs. We derive several semidefinite programming (SDP) relaxations and use facial reduction to make these strictly feasible. We investigate a nontrivial relationship between the transformation matrix used in the reduction and the structure of the graph, which is exploited in an efficient algorithm that constructs this matrix for any instance of the problem. To solve our relaxations, we propose an algorithm that incorporates an augmented Lagrangian method into a cutting-plane framework by utilizing Dykstra's projection algorithm. Our algorithm is suitable for solving SDP relaxations with a large number of cutting-planes. Computational results show that our SDP bounds and efficient cutting-plane algorithm outperform other QCCP bounding approaches from the literature. Finally, we provide several SDP-based upper bounding techniques, among which is a sequential Q-learning method that exploits a solution of our SDP relaxation within a reinforcement learning environment. Summary of Contribution: The quadratic cycle cover problem (QCCP) is the problem of finding a set of node-disjoint cycles covering all the nodes in a graph such that the total interaction cost between successive arcs is minimized. The QCCP has applications in many fields, among which are robotics, transportation, energy distribution networks, and automatic inspection. Besides this, the problem has a high theoretical relevance because of its close connection to the quadratic traveling salesman problem (QTSP). The QTSP has several applications, for example, in bioinformatics, and is considered to be among the most difficult combinatorial optimization problems nowadays. After removing the subtour elimination constraints, the QTSP boils down to the QCCP. Hence, an in-depth study of the QCCP also contributes to the construction of strong bounds for the QTSP. In this paper, we study the application of semidefinite programming (SDP) to obtain strong bounds for the QCCP. Our strongest SDP relaxation is very hard to solve by any SDP solver because of the large number of involved cutting-planes. Because of that, we propose a new approach in which an augmented Lagrangian method is incorporated into a cutting-plane framework by utilizing Dykstra's projection algorithm. We emphasize an efficient implementation of the method and perform an extensive computational study. This study shows that our method is able to handle a large number of cuts and that the resulting bounds are currently the best QCCP bounds in the literature. We also introduce several upper bounding techniques, among which is a distributed reinforcement learning algorithm that exploits our SDP relaxations. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
10. Managerial decision-making: exploration strategies in dynamic environments
- Author
-
Wan, Claire K. and Chih, Mingchang
- Published
- 2024
- Full Text
- View/download PDF
11. How to Learn to Defeat Noisy Robot in Rock-Paper-Scissors Game: An Exploratory Study
- Author
-
Gregory V. Chernov
- Subjects
Fictitious play ,Computer science ,business.industry ,Best response ,Repeated game ,Reinforcement learning ,Robot ,Adaptive learning ,Noise (video) ,Artificial intelligence ,business ,General Economics, Econometrics and Finance ,Field (computer science) - Abstract
This paper studies learning in strategic environment using experimental data from the Rock-Paper-Scissors game. In a repeated game framework, we explore the response of human subjects to uncertain behavior of strategically sophisticated opponent. We model this opponent as a robot who played a stationary strategy with superimposed noise varying across four experimental treatments. Using experimental data from 85 subjects playing against such a stationary robot for 100 pe riods, we show that humans can decode their strategies, on average outperforming the random response to such a robot by 17%. Further, we show that human ability to recognize such strategies decreases with exogenous noise in the behavior of the robot. Further, we fit learning data to classical Reinforcement Learning (RL) and Fictitious Play (FP) models and show that the classic action-based approach to learning is inferior to the strategy-based one. Unlike the previous papers in this field,e.g. Ioannou, Romero (2014), we extend and adapt the strategy-based learning techniques to the 3×3 game. We also show, using a combination of experimental and expost survey data, that human participants are better at learning separate components of an opponent's strategy than in recognizing this strategy as a whole. This decomposition offers them a shorter and more intuitive way to figure out their own best response. We build a strategic extension of the classical learning models accounting for these behavioral phenomena.
- Published
- 2020
12. Illustrated Technical Paper - Enhanced Pub/Sub Network Communication
- Author
-
Arruda, Carlos E. Arruda, Moraes, Pedro F, Agoulmine, Nazim, and Martins, Joberto S. B.
- Subjects
IoT ,Publish/ Subscribe ,SARSA ,Massive IoT Traffic ,Reinforcement Learning ,Resource Allocation ,Network Communications - Abstract
This illustrated technical paper presents the slides describing the contents of the paper "Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning". The talk was presented at the 3rd International Conference on Machine Learning for Networking (MLN'2020), 24 - 26 November 2020 at Paris, France - http://www.adda-association.org/mln-2020. The illustrated technical paper format is intended to complement, enrich and subsidize the technical paper content and contains slides, complementary text and additional and/or focused bibliographic references., {"references":["Carlos E. Arruda, Pedro F. Moraes, Nazim Agoulmine, and Joberto S. B. Martins. Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning. In Proceedings of the 3rd International Conference on Machine Learning for Networking (MLN'2020), pages 1–20, Paris, France, November 2020.","Pedro Francesco Moraes, Rafael F. Reale, and Joberto S. B. Martins. A Publish/Subscribe QoS-aware Framework for Massive IoT Traffic Orchestration. In Proceedings of the 6th International Workshop on ADVANCEs in ICT Infrastructures and Services (ADVANCE), pages 1–14, Santiago, January 2018.","A. M. Farid, M. Alshareef, P. S. Badhesha, C. Boccaletti, N. A. A. Cacho, C.-I. Carlier, A. Corriveau, I. Khayal, B. Liner, J. S. B. Martins, F. Rahimi, R. Rossett,W. C. H. Schoonenberg, A. Stillwell, and Y.Wang. Smart City Drivers and Challenges in Urban- Mobility, Health-Care, and Interdependent Infrastructure Systems. IEEE Potentials, 40(1):11–16, January 2021. Conference Name: IEEE Potentials.","A. M. Farid, M. Alshareef, P. S. Badhesha, C. Boccaletti, N. A. A. Cacho, C.-I. Carlier, A. Corriveau, I. Khayal, B. Liner, J. S. B. Martins, F. Rahimi, R. Rossetti, W. C. H. Schoonenberg, A. Stillwell, and Y. Wang. Smart City Drivers and Challenges in Energy and Water Systems. IEEE Potentials, 40(1):6–10, January 2021. Conference Name: IEEE Potentials.","R. Martins da Silva Bezerra, Flavia Maristela, and Joberto Martins. On Computational Infraestruture Requirements to Smart and Autonomic Cities Framework. In IEEE International Smart Cities Conference - ISC2-2015, pages 1–6, Guadalajara, Mexico, January 2015. IEEE.","Eliseu Torres, Rafael F Reale, Leobino Sampaio, and Joberto S. B. Martins. A SDN/OpenFlow Framework for Dynamic Resource Allocation based on Bandwidth Allocation Model. IEEE Latin America Transactions, 18(5):853–860, April 2020.","Joberto S. B. Martins. Towards Smart City Innovation Under the Perspective of Software-Defined Networking, Artificial Intelligence and Big Data. Revista de Tecnologia da Informação e Comunicação, 8(2):1–7, October 2018.","José Santos, Tim Wauters, Bruno Volckaert, and Filip De Turck. Resource Provisioning in Fog Computing: From Theory to Practice. Sensors (Basel, Switzerland), 19(10), May 2019.","Xingzhou Zhang, Yifan Wang, Sidi Lu, Liangkai Liu, Lanyu Xu, and Weisong Shi. OpenEI: An Open Framework for Edge Intelligence. In 39th IEEE International Conference on Distributed Computing Systems (ICDCS), pages 1–12, Dallas, US, July 2019.","M. Mukherjee, L. Shu, and D. Wang. Survey of Fog Computing: Fundamental, Network Applications, and Research Challenges. IEEE Communications Surveys Tutorials, 20(3):1826–1857, thirdquarter 2018.","Raouf Boutaba, Mohammad A. Salahuddin, Noura Limam, Sara Ayoubi, Nashid Shahriar, Felipe Estrada-Solano, and Oscar M. Caicedo. A Comprehensive Survey on Machine Learning for Networking: Evolution, Applications and Research Opportunities. Journal of Internet Services and Applications, 9(1):16, June 2018.","Richard S. Sutton and Andrew G. Barto. Introduction to Reinforcement Learning. MIT Press, Cambridge, MA, USA, 1st edition, 1998.","Jaehoon Koo, Veena B. Mendiratta, Muntasir Raihan Rahman, and Anwar Walid. Deep Reinforcement Learning for Network Slicing with Heterogeneous Resource Requirements and Time Varying Traffic Dynamics. In 2019 15th International Conference on Network and Service Management, pages 1–5, October 2019.","Taha Alfakih, Mohammad Mehedi Hassan, Abdu Gumaei, Claudio Savaglio, and Giancarlo Fortino. Task Offloading and Resource Allocation for Mobile Edge Computing by Deep Reinforcement Learning Based on SARSA. IEEE Access, 8:54074– 54084, 2020.","D. Cote. Using Machine Learning in Communication Networks. IEEE/OSA Journal of Optical Communications and Networking, 10(10):D100–D109, October 2018.","Oscar Mauricio Caicedo Rendon, Felipe Estrada-Solano, Raouf Boutaba, Nashid Shahriar, Mohammad Salahuddin, Noura Liman, and Sara Ayoubi. Machine Learning for Cognitive Network Management. IEEE Communications Magazine, pages 1–9, 2018.","Boubakr Nour, Kashif Sharif, Fan Li, Song Yang, Hassine Moungla, and Yu Wang. ICN Publisher-Subscriber Models: Challenges and Group-based Communication. IEEE Network, 33(6):156–163, November 2019.","D. Happ and A. Wolisz. Limitations of the Pub/Sub Pattern for Cloud Based Iot and Their Implications. In Cloudification of the Internet of Things (CIoT), pages 1–6, Paris, November 2016. IEEE - Cloudification of the Internet of Things (CIoT).","Kyoungho An, Aniruddha Gokhale, Sumant Tambe, and Takayuki Kuroda. Wide Area Network-scale Discovery and Data Dissemination in Data-centric Publish/Subscribe Systems. pages 1–2. ACM Press, 2015.","M. Mohammadi and A. Al-Fuqaha. Enabling Cognitive Smart Cities Using Big Data and Machine Learning: Approaches and Challenges. IEEE Communications Magazine, 56(2):94–101, February 2018.","M. Mazhar Rathore, Awais Ahmad, Anand Paul, and Seungmin Rho. Urban Planning and Building Smart Cities Based on the Internet of Things Using Big Data Analytics. Computer Networks, 101:63–80, June 2016.","Joberto S. B. Martins. Towards Smart City Innovation Under the Perspective of Software-Defined Networking, Artificial Intelligence and Big Data. Revista de Tecnologia da Informação e Comunicação, 8(2):1–7, October 2018.","Diego Kreutz, Fernando M. V. Ramos, Paulo Verissimo, Christian Esteve Rothenberg, Siamak Azodolmolky, and Steve Uhlig. Software-Defined Networking: A Comprehensive Survey. Proceedings of the IEEE, 103(1):14–76, December 2014.","J. Xie, F. R. Yu, T. Huang, R. Xie, J. Liu, C. Wang, and Y. Liu. A Survey of Machine Learning Techniques Applied to Software Defined Networking (SDN): Research Issues and Challenges. IEEE Communications Surveys Tutorials, 21(1):393–430, Firstquarter 2019.","L. Zhao, J. Wang, J. Liu, and N. Kato. Routing for Crowd Management in Smart Cities: A Deep Reinforcement Learning Perspective. IEEE Communications Magazine, 57(4):88–93, April 2019.","M. Latah and L. Toker. Artificial intelligence enabled software-defined networking: A comprehensive overview. IET Networks, 8(2):79–99, 2019.","Thomas M. Moerland, Joost Broekens, and Catholijn M. Jonker. A Framework for Reinforcement Learning and Planning. PhD thesis, TU Delft, June 2020.","Xiaofei Liao, DequanWu, and YangWang. Dynamic Spectrum Access Based on Improved SARSA Algorithm. IOP Conference Series: Materials Science and Engineering, 768(7):072015, March 2020. Publisher: IOP Publishing.","Lauren S. R. Sampaio, Pedro H. A. Faustini, Anderson S. Silva, Lisandro Z. Granville, and Alberto Schaeffer-Filho. Using NFV and Reinforcement Learning for Anomalies Detection and Mitigation in SDN. In 2018 IEEE Symposium on Computers and Communications (ISCC), pages 00432–00437, June 2018.","J. V. Bibal Benifa and D. Dejey. RLPAS: Reinforcement Learning-based Proactive Auto-Scaler for Resource Provisioning in Cloud Environment. Mobile Networks and Applications, 24(4):1348–1363, August 2019.","Ali Asghari, Mohammad Karim Sohrabi, and Farzin Yaghmaee. Task Scheduling, Resource Provisioning, and Load Balancing on Scientific Workflows Using Parallel SARSA Reinforcement Learning Agents and Genetic Algorithm. The Journal of Supercomputing, July 2020.","Almuthanna Nassar and Yasin Yilmaz. Reinforcement Learning for Adaptive Resource Allocation in Fog RAN for IoT With Heterogeneous Latency Requirements. IEEE Access, 7:128014–128025, 2019.","Aaron Defazio and Thore Graepel. A Comparison of Learning Algorithms on the Arcade Learning Environment. arXiv:1410.8620 [cs], October 2014. arXiv: 1410.8620.","Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew Hausknecht, and Michael Bowling. Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents. Journal of Artificial Intelligence Research, 61:523–562, March 2018.","Jen-Hung Wang, Ping-En Lu, Cheng-Shang Chang, and Duan-Shin Lee. A Reinforcement Learning Approach for the Multichannel Rendezvous Problem. In 2019 IEEE Globecom Workshops (GC Wkshps), pages 1–5, December 2019.","Sridhar Mahadevan. Average Reward Reinforcement Learning: Foundations, Algorithms, and Empirical Results. Machine Learning, 22(1):159–195, March 1996.","Yue Wang and Shaofeng Zou. Finite-sample Analysis of Greedy-GQ with Linear Function Approximation under Markovian Noise. Proceedings of Machine Learning Research, 124(Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI)):1–26, 2020.","Kamil Ciosek, Quan Vuong, Robert Loftin, and Katja Hofmann. Better Exploration with Optimistic Actor Critic. pages 1787–1798, 2019.","Pedro F. Moraes and Joberto S. B. Martins. A Pub/Sub SDN-Integrated Framework for IoT Traffic Orchestration. In Proceedings of the 3rd International Conference on Future Networks and Distributed Systems, ICFNDS '19, pages 1–9, 2019. Paris, France."]}
- Published
- 2021
- Full Text
- View/download PDF
13. Illustrated Technical Paper: Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning
- Author
-
Arruda, Carlos E. Arruda, Moraes, Pedro F, Agoulmine, Nazim, and Martins, Joberto S. B.
- Subjects
IoT ,Publish/ Subscribe ,SARSA ,Massive IoT Traffic ,Reinforcement Learning ,Resource Allocation ,Network Communications - Abstract
This illustrated technical paperpresents the slides describing the contents of the paper "Enhanced Pub/Sub Communications for Massive IoT Traffic with SARSA Reinforcement Learning". The talk was presented at the 3rd International Conference on Machine Learning for Networking (MLN'2020), 24 - 26 November 2020 at Paris, France - http://www.adda-association.org/mln-2020. The illustrated technical paper formatis intended to complement, enrich and subsidize the technical paper content and contains slides, complementary text and additional and/or focused bibliographic references.
- Published
- 2021
- Full Text
- View/download PDF
14. Innovative Construction of Reinforcement Learning Model for Information Fusion in Music Education.
- Author
-
Zhang, Zehui
- Subjects
MUSIC education ,MUSIC in education ,INFORMATION modeling ,MACHINE learning ,LEARNING ,SUPERVISED learning ,REINFORCEMENT learning ,ELECTRONIC paper - Abstract
In order to improve the effect of modern music education, this paper applies the digital information technology of music resources to the construction of the music teaching system and derives two new types of semi-decision-making process reinforcement learning algorithms based on the Bellman optimality equation base on discrete time. Moreover, this paper uses the comparative research methods to obtain the Q-value learning curve of the incremental value iterative reinforcement learning algorithm based on the semi-Markov decision process and the incremental value iterative reinforcement learning algorithm based on the dichotomy to improve the fusion effect of music teaching resources. Finally, this paper combines the actual needs of modern music education to construct an intelligent music teaching model. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
15. Deep Reinforcement Learning for NFV-based Service Function Chaining in Multi-Service Networks : Invited Paper
- Author
-
Ning Wang, Rahim Tafazolli, and Zili Ning
- Subjects
Routing protocol ,Dynamic network analysis ,business.industry ,Computer science ,020206 networking & telecommunications ,0102 computer and information sciences ,02 engineering and technology ,Network topology ,01 natural sciences ,010201 computation theory & mathematics ,Chaining ,0202 electrical engineering, electronic engineering, information engineering ,Reinforcement learning ,The Internet ,Routing (electronic design automation) ,business ,Virtual network ,Computer network - Abstract
With the advent of Network Function Virtualization (NFV) techniques, a subset of the Internet traffic will be treated by a chain of virtual network functions (VNFs) during their journeys while the rest of the background traffic will still be carried based on traditional routing protocols. Under such a multi-service network environment, we consider the co-existence of heterogeneous traffic control mechanisms, including flexible, dynamic service function chaining (SFC) traffic control and static, dummy IP routing for the aforementioned two types of traffic that share common network resources. Depending on the traffic patterns of the background traffic which is statically routed through the traditional IP routing platform, we aim to perform dynamic service function chaining for the foreground traffic requiring VNF treatments, so that both the end-to-end SFC performance and the overall network resource utilization can be optimized. Towards this end, we propose a deep reinforcement learning based scheme to enable intelligent SFC routing decision-making in dynamic network conditions. The proposed scheme is ready to be deployed on both hybrid SDN/IP platforms and future advanced IP environments. Based on the real GEANT network topology and its one-week traffic traces, our experiments show that the proposed scheme is able to significantly improve from the traditional routing paradigm and achieve close-to-optimal performances very fast while satisfying the end-to-end SFC requirements.
- Published
- 2020
16. An intelligent decision system for virtual machine migration based on specific Q-learning.
- Author
-
Zhu, Xinying, Xia, Ran, Zhou, Hang, Zhou, Shuo, and Liu, Haoran
- Subjects
ARTIFICIAL intelligence ,MACHINE learning ,ROUGH sets ,REINFORCEMENT learning ,COMPUTER workstation clusters ,INTELLIGENT tutoring systems ,VIRTUAL machine systems - Abstract
Summary: Due to the convenience of virtualization, the live migration of virtual machines is widely used to fulfill optimization objectives in cloud/edge computing. However, live migration may lead to side effects and performance degradation when migration is overused or an unreasonable migration process is carried out. One pressing challenge is how to capture the best opportunity for virtual machine migration. Leveraging rough sets and AI, this paper provides an innovative strategy based on Q-learning that is designed for migration decisions. The highlight of our strategy is the harmonious mechanism for applying rough sets and Q-learning. For the ABDS (adaptive boundary decision system) strategy in this paper, the exploration space of Q learning is confined by the boundary region of rough sets, while the thresholds of the boundary region can be dynamically adjusted by the reaction results from the computing cluster. The structure and mechanism of the ABDS strategy are described in this paper. The corresponding experiments show a firm advantage for the cooperation of rough sets and reinforcement learning algorithms. Considering both the energy consumption and application performance, the ABDS strategy in this paper outperforms the benchmark strategies in comprehensive performance. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
17. Guest Editorial: Operational and structural resilience of power grids with high penetration of renewables.
- Author
-
Lei, Shunbo, Zhang, Yichen, Shahidehpour, Mohammad, Hou, Yunhe, Panteli, Mathaios, Chen, Xia, Aydin, Nazli Yonca, Liang, Liang, Wang, Cheng, Wang, Chong, and She, Buxin
- Subjects
MICROGRIDS ,ELECTRIC power distribution grids ,CYBER physical systems ,MIXED integer linear programming ,DEEP reinforcement learning ,ARTIFICIAL neural networks ,REINFORCEMENT learning ,ELECTRIC power - Published
- 2024
- Full Text
- View/download PDF
18. Autonomous exploration through deep reinforcement learning
- Author
-
Yan, Xiangda, Huang, Jie, He, Keyan, Hong, Huajie, and Xu, Dasheng
- Published
- 2023
- Full Text
- View/download PDF
19. Selected papers from WAFR2020.
- Author
-
Yu, Jingjin and Lin, Ming C.
- Subjects
- *
PARTIALLY observable Markov decision processes , *REINFORCEMENT learning , *HYBRID systems , *MOBILE robots , *STOCHASTIC learning models , *PETRI nets - Abstract
Leveraging Rubik Tables, the authors designed a constant-factor optimal algorithm for stack rearrangement and multi-robot motion planning problems under extreme robot density. A general algorithm and several specialized algorithms are then introduced to solve the problem, constructing a product between the event model and a specification automaton. Toward that, the authors analyze closed-loop dynamics to identify conditions that lead to deadlock and, using duality theory and KKT conditions, derive geometric properties of robot configurations in deadlock. [Extracted from the article]
- Published
- 2023
- Full Text
- View/download PDF
20. Improving the gossiping effectiveness with distributed strategic learning (Invited paper)
- Author
-
Christian Esposito, Massimo Ficco, Aniello Castiglione, Francesco Palmieri, Esposito, Christian, Castiglione, Aniello, Palmieri, Francesco, and Ficco, Massimo
- Subjects
Reliable multicasting ,Computer Networks and Communications ,Network packet ,Event (computing) ,Computer science ,Reliability (computer networking) ,Distributed computing ,ComputerSystemsOrganization_COMPUTER-COMMUNICATIONNETWORKS ,020206 networking & telecommunications ,02 engineering and technology ,Event-based communications ,Event-based communication ,Hardware and Architecture ,Gossip ,Middleware ,Reinforcement learning ,0202 electrical engineering, electronic engineering, information engineering ,020201 artificial intelligence & image processing ,Gossiping ,Game theory ,Strategic learning ,Software - Abstract
Gossiping is a widely known and successful approach to reliable communications, tolerating packet losses and link crashes. It has been extensively used in several middleware kinds, such as event notification services and application domains, like infrastructures for air traffic management, power grid control, health information exchange, just to cite some of them. Despite achieving a high loss-tolerance and scalability degrees, gossiping is affected by degraded performances and heavy traffic loads on the network. For this reason, it may be not optimal in applications where reliability must be provided jointly with timeliness and/or in congestion-prone networks. The crucial aspect for improving a gossiping scheme is deciding which nodes should receive a gossiping message, and our driving idea is to adopt a distributed strategic learning logic to determine such nodes in an efficient manner. This is able to resolve gossiping's weakness points and to achieve better performance and reduced traffic loads. This paper describes how to introduced strategic learning in a gossip scheme so as to determine the best set of nodes that can be used to send gossip messages and to optimize their utility. Such a solution has been experimentally assessed through a set of simulations demonstrating the effectiveness of the proposal.
- Published
- 2017
21. Intelligent obstacle avoidance path planning method for picking manipulator combined with artificial potential field method
- Author
-
Fang, Zheng and Liang, Xifeng
- Published
- 2022
- Full Text
- View/download PDF
22. Machine Learning for Networking: first international conference, MLN 2018, Paris, France, November 27-29, 2018, revised selected papers
- Author
-
Paul Muhlethaler, Selma Boumerdassi, Eric Renault, Réseaux, Systèmes, Services, Sécurité (R3S-SAMOVAR), Services répartis, Architectures, MOdélisation, Validation, Administration des Réseaux (SAMOVAR), Institut Mines-Télécom [Paris] (IMT)-Télécom SudParis (TSP)-Institut Mines-Télécom [Paris] (IMT)-Télécom SudParis (TSP), Département Réseaux et Services Multimédia Mobiles (RS2M), Institut Mines-Télécom [Paris] (IMT)-Télécom SudParis (TSP), Centre National de la Recherche Scientifique (CNRS), Wireless Networking for Evolving & Adaptive Applications (EVA), Inria de Paris, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria), CEDRIC. Réseaux et Objets Connectés (CEDRIC - ROC), Centre d'études et de recherche en informatique et communications (CEDRIC), Ecole Nationale Supérieure d'Informatique pour l'Industrie et l'Entreprise (ENSIIE)-Conservatoire National des Arts et Métiers [CNAM] (CNAM)-Ecole Nationale Supérieure d'Informatique pour l'Industrie et l'Entreprise (ENSIIE)-Conservatoire National des Arts et Métiers [CNAM] (CNAM), Département Réseaux et Services Multimédia Mobiles (TSP - RS2M), Ecole Nationale Supérieure d'Informatique pour l'Industrie et l'Entreprise (ENSIIE)-Conservatoire National des Arts et Métiers [CNAM] (CNAM), HESAM Université - Communauté d'universités et d'établissements Hautes écoles Sorbonne Arts et métiers université (HESAM)-HESAM Université - Communauté d'universités et d'établissements Hautes écoles Sorbonne Arts et métiers université (HESAM)-Ecole Nationale Supérieure d'Informatique pour l'Industrie et l'Entreprise (ENSIIE)-Conservatoire National des Arts et Métiers [CNAM] (CNAM), and HESAM Université - Communauté d'universités et d'établissements Hautes écoles Sorbonne Arts et métiers université (HESAM)-HESAM Université - Communauté d'universités et d'établissements Hautes écoles Sorbonne Arts et métiers université (HESAM)
- Subjects
User interfaces ,Sensor networks ,Ad hoc networks ,Artificial intelligence ,Human-Computer Interaction (HCI) ,Computer science ,02 engineering and technology ,Learning algorithms ,Machine learning ,computer.software_genre ,[INFO.INFO-NI]Computer Science [cs]/Networking and Internet Architecture [cs.NI] ,Bandwidth ,0203 mechanical engineering ,0202 electrical engineering, electronic engineering, information engineering ,Reinforcement learning ,Wireless telecommunication systems ,[INFO]Computer Science [cs] ,Data security ,Wireless networks ,Computer networks ,Internet ,Artificial neural network ,Routers ,business.industry ,Sensors ,Telecommunication networks ,020206 networking & telecommunications ,020302 automobile design & engineering ,Wireless sensor networks ,Telecommunication traffic ,Mobile telecommunication systems ,business ,computer ,Wireless sensor network ,Neural networks - Abstract
International audience; This book constitutes the thoroughly refereed proceedings of the First International Conference on Machine Learning for Networking, MLN 2018, held in Paris, France, in November 2018. The 22 revised full papers included in the volume were carefully reviewed and selected from 48 submissions. They present new trends in the following topics: Deep and reinforcement learning; Pattern recognition and classification for networks; Machine learning for network slicing optimization, 5G system, user behavior prediction, multimedia, IoT, security and protection; Optimization and new innovative machine learning methods; Performance analysis of machine learning algorithms; Experimental evaluations of machine learning; Data mining in heterogeneous networks; Distributed and decentralized machine learning algorithms; Intelligent cloud-support communications, resource allocation, energy-aware/green communications, software defined networks, cooperative networks, positioning and navigation systems, wireless communications, wireless sensor networks, underwater sensor networks.
- Published
- 2019
23. Editors' Introduction: Best Papers From the 2018 International Conference on Cognitive Modeling
- Author
-
Ion Juvina, Joseph W. Houpt, and Christopher W. Myers
- Subjects
Human-Computer Interaction ,Cognitive science ,Cognitive model ,Linguistics and Language ,Artificial Intelligence ,Computer science ,Cognitive Neuroscience ,Reinforcement learning ,Experimental and Cognitive Psychology ,Cognition ,Introductory Journal Article - Abstract
The International Conference on Cognitive Modeling brings together researchers from around the world whose main goal is to build computational systems that reflect the internal processes of the mind. In this issue, we present the four best representative papers on this work from our 18th meeting, ICCM 2020, which was also the first meeting to be held virtually. Two of these papers develop novel techniques for building larger and more complex models using Reinforcement Learning and Learning By Instruction, respectively. The other two show how cognitive models connect to neuroscience, drawing on details of the hippocampus and cerebellum to constrain and explain the cognitive processes involved in memory and conditioning.
- Published
- 2019
24. Deep reinforcement learning: Framework, applications, and embedded implementations: Invited paper
- Author
-
Yanzhi Wang, Tianshu Wei, Hongjia Li, Ao Ren, and Qi Zhu
- Subjects
Job shop scheduling ,Artificial neural network ,business.industry ,Computer science ,Cyber-physical system ,020302 automobile design & engineering ,Cloud computing ,02 engineering and technology ,Optimal control ,020202 computer hardware & architecture ,Smart grid ,0203 mechanical engineering ,Computer engineering ,0202 electrical engineering, electronic engineering, information engineering ,Resource allocation ,Reinforcement learning ,business - Abstract
The recent breakthroughs of deep reinforcement learning (DRL) technique in Alpha Go and playing Atari have set a good example in handling large state and actions spaces of complicated control problems. The DRL technique is comprised of (i) an offline deep neural network (DNN) construction phase, which derives the correlation between each state-action pair of the system and its value function, and (ii) an online deep Q-learning phase, which adaptively derives the optimal action and updates value estimates. In this paper, we first present the general DRL framework, which can be widely utilized in many applications with different optimization objectives. This is followed by the introduction of three specific applications: the cloud computing resource allocation problem, the residential smart grid task scheduling problem, and building HVAC system optimal control problem. The effectiveness of the DRL technique in these three cyber-physical applications have been validated. Finally, this paper investigates the stochastic computing-based hardware implementations of the DRL framework, which consumes a significant improvement in area efficiency and power consumption compared with binary-based implementation counterparts.
- Published
- 2017
25. Autonomous Agents and Multiagent Systems - AAMAS 2017 Workshops, Best Papers, São Paulo, Brazil, May 8-12, 2017, Revised Selected Papers
- Author
-
Sukthankar, Gita and Rodríguez-Aguilar, Juan Antonio
- Subjects
Artificial intelligence ,Information theory ,Intelligent agents ,Formal methods ,Multi-agent systems ,QoS ,Agents ,Robotics ,Semantics ,Graph theory ,Formal languages ,MAS ,Machine learning ,Reinforcement learning ,Cooperation and coordination - Abstract
This book features a selection of best papers from 13 workshops held at the International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2017, held in Sao Paulo, Brazil, in May 2017. The 17 full papers presented in this volume were carefully reviewed and selected for inclusion in this volume. They cover specific topics, both theoretical and applied, in the general area of autonomous agents and multiagent systems.
- Published
- 2017
26. Robot obstacle avoidance system using deep reinforcement learning
- Author
-
Zhu, Xiaojun, Liang, Yinghao, Sun, Hanxu, Wang, Xueqian, and Ren, Bin
- Published
- 2022
- Full Text
- View/download PDF
27. Optimization of news dissemination push mode by intelligent edge computing technology for deep learning.
- Author
-
DeGe, JiLe and Sang, Sina
- Subjects
DEEP reinforcement learning ,PATTERN recognition systems ,SOCIAL media ,NEWS websites ,RECOMMENDER systems ,DEEP learning ,REINFORCEMENT learning - Abstract
The Internet era is an era of information explosion. By 2022, the global Internet users have reached more than 4 billion, and the social media users have exceeded 3 billion. People face a lot of news content every day, and it is almost impossible to get interesting information by browsing all the news content. Under this background, personalized news recommendation technology has been widely used, but it still needs to be further optimized and improved. In order to better push the news content of interest to different readers, users' satisfaction with major news websites should be further improved. This study proposes a new recommendation algorithm based on deep learning and reinforcement learning. Firstly, the RL algorithm is introduced based on deep learning. Deep learning is excellent in processing large-scale data and complex pattern recognition, but it often faces the challenge of low sample efficiency when it comes to complex decision-making and sequential tasks. While reinforcement learning (RL) emphasizes learning optimization strategies through continuous trial and error through interactive learning with the environment. Compared with deep learning, RL is more suitable for scenes that need long-term decision-making and trial-and-error learning. By feeding back the reward signal of the action, the system can better adapt to the unknown environment and complex tasks, which makes up for the relative shortcomings of deep learning in these aspects. A scenario is applied to an action to solve the sequential decision problem in the news dissemination process. In order to enable the news recommendation system to consider the dynamic changes in users' interest in news content, the Deep Deterministic Policy Gradient algorithm is applied to the news recommendation scenario. Opposing learning complements and combines Deep Q-network with the strategic network. On the basis of fully summarizing and thinking, this paper puts forward the mode of intelligent news dissemination and push. The push process of news communication information based on edge computing technology is proposed. Finally, based on Area Under Curve a Q-Leaning Area Under Curve for RL models is proposed. This indicator can measure the strengths and weaknesses of RL models efficiently and facilitates comparing models and evaluating offline experiments. The results show that the DDPG algorithm improves the click-through rate by 2.586% compared with the conventional recommendation algorithm. It shows that the algorithm designed in this paper has more obvious advantages in accurate recommendation by users. This paper effectively improves the efficiency of news dissemination by optimizing the push mode of intelligent news dissemination. In addition, the paper also deeply studies the innovative application of intelligent edge technology in news communication, which brings new ideas and practices to promote the development of news communication methods. Optimizing the push mode of intelligent news dissemination not only improves the user experience, but also provides strong support for the application of intelligent edge technology in this field, which has important practical application prospects. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
28. Adaptive control for circulating cooling water system using deep reinforcement learning.
- Author
-
Xu, Jin, Li, Han, and Zhang, Qingxin
- Subjects
DEEP reinforcement learning ,ADAPTIVE control systems ,COOLING systems ,WATER use ,SMART structures ,REINFORCEMENT learning - Abstract
Due to the complex internal working process of circulating cooling water systems, most traditional control methods struggle to achieve stable and precise control. Therefore, this paper presents a novel adaptive control structure for the Twin Delayed Deep Deterministic Policy Gradient algorithm, which is based on a reference trajectory model (TD3-RTM). The structure is based on the Markov decision process of the recirculating cooling water system. Initially, the TD3 algorithm is employed to construct a deep reinforcement learning agent. Subsequently, a state space is selected, and a dense reward function is designed, considering the multivariable characteristics of the recirculating cooling water system. The agent updates its network based on different reward values obtained through interactions with the system, thereby gradually aligning the action values with the optimal policy. The TD3-RTM method introduces a reference trajectory model to accelerate the convergence speed of the agent and reduce oscillations and instability in the control system. Subsequently, simulation experiments were conducted in MATLAB/Simulink. The results show that compared to PID, fuzzy PID, DDPG and TD3, the TD3-RTM method improved the transient time in the flow loop by 6.09s, 5.29s, 0.57s, and 0.77s, respectively, and the Integral of Absolute Error(IAE) indexes decreased by 710.54, 335.1, 135.97, and 89.96, respectively, and the transient time in the temperature loop improved by 25.84s, 13.65s, 15.05s, and 0.81s, and the IAE metrics were reduced by 143.9, 59.13, 31.79, and 1.77, respectively. In addition, the overshooting of the TD3-RTM method in the flow loop was reduced by 17.64, 7.79, and 1.29 per cent, respectively, in comparison with the PID, the fuzzy PID, and the TD3. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
29. An FPGA-Accelerated CNN with Parallelized Sum Pooling for Onboard Realtime Routing in Dynamic Low-Orbit Satellite Networks.
- Author
-
Kim, Hyeonwoo, Park, Juhyeon, Lee, Heoncheol, Won, Dongshik, and Han, Myonghun
- Subjects
REINFORCEMENT learning ,DEEP reinforcement learning ,CONVOLUTIONAL neural networks ,ROUTING algorithms ,GATE array circuits ,ORBITS of artificial satellites - Abstract
This paper addresses the problem of real-time onboard routing for dynamic low earth orbit (LEO) satellite networks. It is difficult to apply general routing algorithms to dynamic LEO networks due to the frequent changes in satellite topology caused by the disconnection between moving satellites. Deep reinforcement learning (DRL) models trained by various dynamic networks can be considered. However, since the inference process with the DRL model requires too long a computation time due to multiple convolutional layer operations, it is not practical to apply to a real-time on-board computer (OBC) with limited computing resources. To solve the problem, this paper proposes a practical co-design method with heterogeneous processors to parallelize and accelerate a part of the multiple convolutional layer operations on a field-programmable gate array (FPGA). The proposed method was tested with a real heterogeneous processor-based OBC and showed that the proposed method was about 3.10 times faster than the conventional method while achieving the same routing results. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
30. Editorial: Decision-making and planning for multi-agent systems.
- Author
-
Tsiotras, Panagiotis, Gombolay, Matthew, Foerster, Jakob, and Tanner,, Herbert Glenn
- Subjects
MULTIAGENT systems ,DECISION making ,GAUSSIAN mixture models - Abstract
This article is an editorial that focuses on the topic of decision-making and planning for multi-agent systems. Multi-agent systems are used in various real-world applications, but there are challenges in coordinating the behaviors of multiple agents. The article discusses the difficulties in translating single-agent techniques to the multi-agent domain, including complex interaction dynamics, constrained communication, and heterogeneity. The objective of this research topic is to report on recent advances in multi-agent planning and decision-making and to address the existing challenges in this area. The article includes summaries of five papers that cover different aspects of multi-agent decision-making, such as emergent communication, motion planning, reinforcement learning, and cooperative navigation. These papers provide insights into the complexity of multi-agent systems and suggest potential areas for further research. [Extracted from the article]
- Published
- 2024
- Full Text
- View/download PDF
31. CONNA: Addressing Name Disambiguation on the Fly.
- Author
-
Chen, Bo, Zhang, Jing, Tang, Jie, Cai, Lingfan, Wang, Zhaoyu, Zhao, Shu, Chen, Hong, and Li, Cuiping
- Subjects
REINFORCEMENT learning ,SOCIAL systems ,ELECTRONIC information resource searching ,COMPUTER science ,SOCIAL networks ,IMAGE registration - Abstract
Name disambiguation is a key and also a very tough problem in many online systems such as social search and academic search. Despite considerable research, a critical issue that has not been systematically studied is disambiguation on the fly — to complete the disambiguation in the real-time. This is very challenging, as the disambiguation algorithm must be accurate, efficient, and error tolerance. In this paper, we propose a novel framework — CONNA — to train a matching component and a decision component jointly via reinforcement learning. The matching component is responsible for finding the top matched candidate for the given paper, and the decision component is responsible for deciding on assigning the top matched person or creating a new person. The two components are intertwined and can be bootstrapped via jointly training. Empirically, we evaluate CONNA on two name disambiguation datasets. Experimental results show that the proposed framework can achieve a 1.21-19.84 percent improvement on F1-score using joint training of the matching and the decision components. The proposed CONNA has been successfully deployed on AMiner — a large online academic search system. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF
32. Secure Knowledge Management and Cybersecurity in the Era of Artificial Intelligence.
- Author
-
Samtani, Sagar, Zhao, Ziming, and Krishnan, Ram
- Subjects
ARTIFICIAL intelligence ,DEEP learning ,KNOWLEDGE management ,INTERNET security ,REINFORCEMENT learning ,CONVOLUTIONAL neural networks - Abstract
In October 2021, the Secure Knowledge Management (SKM) Conference convened many of scholars and practitioners to discuss and share ideas about the use and development of AI for SKM and cybersecurity. Over the past half-decade, numerous federal funding agencies such as the National Science Foundation (NSF), National Academies of Science (NAS), and National Security and Technology Council (NSTC) have noted the significant role that Artificial Intelligence (AI)-enabled analytics techniques such as deep learning, machine learning, network science, generative models, reinforcement learning, text analytics, and other techniques could play for Secure Knowledge Management (SKM), and more broadly, cybersecurity. Papers in each theme had a component of AI that was based in one or more of the major themes of extant AI-enabled cybersecurity and SKM research summarized in the previous section. [Extracted from the article]
- Published
- 2023
- Full Text
- View/download PDF
33. Multiple peg-in-hole compliant assembly based on a learning-accelerated deep deterministic policy gradient strategy
- Author
-
Li, Xinwang, Xiao, Juliang, Zhao, Wei, Liu, Haitao, and Wang, Guodong
- Published
- 2022
- Full Text
- View/download PDF
34. Enabling intrusion detection systems with dueling double deep Q-learning
- Author
-
Badr, Youakim
- Published
- 2022
- Full Text
- View/download PDF
35. Multi-agent game operation of regional integrated energy system based on carbon emission flow.
- Author
-
Li Zhang, Dong Pan, Jianxiong Jia, Zhumeng Song, Xin Zhang, Nan Shang, and Jinshuo Su
- Subjects
CARBON emissions ,CARBON pricing ,SUPPLY & demand ,DIFFERENTIAL evolution ,REINFORCEMENT learning ,POWER resources ,ELECTRICITY pricing - Abstract
In the process of promoting energy green transformation, the optimization of regional integrated energy system faces many challenges such as cooperative management, energy saving and emission reduction, as well as uncertainty of new energy output. Therefore, this paper proposes a multi-agent game operation method of regional integrated energy system based on carbon emission flow. First, this paper establishes a carbon emission flow calculation model for each subject, and proposes a comprehensive tariff model based on the carbon emission flow, which discounts the carbon emissions from the power supply side to the power consumption side. Secondly, considering the interests of each subject, this paper establishes the decision- making model of each subject. And the new energy uncertainty, the cost of energy preference of prosumers, and the thermal inertia of buildings are considered in the decision model. Finally, the model is solved using differential evolution algorithm and solver. The case study verifies that the comprehensive electricity pricing model based on carbon emission flow developed in this paper can play a role in balancing economy and low carbon. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
36. Editorial for the Special Issue "Data Science and Big Data in Biology, Physical Science and Engineering".
- Author
-
Mahmoud, Mohammed
- Subjects
PHYSICAL sciences ,BIG data ,DEEP learning ,ARTIFICIAL neural networks ,DATA science ,MACHINE learning ,REINFORCEMENT learning - Abstract
This document is an editorial for a special issue of the journal "Technologies" focused on data science and big data in various fields such as biology, physical science, and engineering. The editorial highlights the importance of analyzing large amounts of data generated by digital technologies and the need for data scientists to use artificial intelligence and machine learning to extract valuable knowledge. The special issue includes 12 papers covering topics such as machine learning techniques for customer churn prediction, agile program management in the U.S. Navy, deep learning for cybersecurity in Industry 5.0, self-directed learning during the COVID-19 era, decision tree-based neural networks for data classification, data-driven governance in technology companies, and more. The papers explore different approaches, models, and tools in the context of data science and big data. [Extracted from the article]
- Published
- 2024
- Full Text
- View/download PDF
37. A robotic system with reinforcement learning for lower extremity hemiparesis rehabilitation
- Author
-
Xu, Jiajun, Xu, Linsen, Cheng, Gaoxin, Shi, Jia, Liu, Jinfu, Liang, Xingcan, and Fan, Shengyao
- Published
- 2021
- Full Text
- View/download PDF
38. Guest Editorial on "Computational intelligence in analysis and integration of complex systems".
- Author
-
Zhao, Bo, Zeng, Wenyi, Gao, Weinan, and Zhang, Qichao
- Subjects
COMPUTATIONAL intelligence ,SYSTEM integration ,MULTIAGENT systems ,DIFFERENTIAL evolution ,REINFORCEMENT learning ,MANIPULATORS (Machinery) ,CONVOLUTIONAL neural networks ,ARTIFICIAL neural networks - Abstract
In the third paper, I Deep transfer learning: a novel glucose prediction framework for new subjects with Type 2 diabetes i , the authors designed a novel cross-subject glucose prediction framework by integrating instance-based and network-based deep transfer learning via segmented continuous glucose monitoring time series. Control and optimization Six papers have devoted to the computational intelligence-based decision-making and analysis of complex systems. Complex systems, which are composed of many interconnected and interactive functional parts, widely exist in the nature and human society. [Extracted from the article]
- Published
- 2022
- Full Text
- View/download PDF
39. RESEARCH ON DATA MINING AND REINFORCEMENT LEARNING IN RECOMMENDATION SYSTEMS.
- Author
-
YUERAN ZHAO and HUIYAN ZHAO
- Subjects
RECOMMENDER systems ,INSTRUCTIONAL systems ,DATA mining ,DATABASES ,REINFORCEMENT learning ,MATHEMATICAL statistics - Abstract
This paper aims to help students better grasp the required professional knowledge and core concepts. This paper presents a design method for a multi-layer knowledge base based on XML. According to learners' identity characteristics and learning behaviour, using the mathematical statistics method, the feature expression for the learning system is constructed. Multivariable linear regression theory establishes convergence constraints for accurate and deep mining. The average detection results of the collected samples are used for high-quality deep mining of user portraits in the learning system. This project intends to study the method of solving accurate confidence intervals for user portrait data in the education system. Excel and Access are used to complete the data collection of the teaching object and the construction of the database. A multi-mode interactive editing and processing method of user portrait information for education systems is studied in cloud computing. Finally, a learning system based on mathematical loading mode is proposed, and an object-oriented learning recommendation system is designed. The developed teaching software can enable students to get more teaching guidance when they acquire the required knowledge to improve students' learning effect effectively. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
40. Deep reinforcement learning for adaptive frequency control of island microgrid considering control performance and economy.
- Author
-
Du, Wanlin, Huang, Xiangmin, Zhu, Yuanzhe, Wang, Ling, Deng, Wenyang, Yin, Linfei, and Saxena, Sahaj
- Subjects
DEEP reinforcement learning ,ADAPTIVE control systems ,MICROGRIDS ,REINFORCEMENT learning ,MAXIMUM entropy method ,INDEPENDENT system operators ,ADAPTIVE fuzzy control - Abstract
To achieve frequency stability and economic efficiency in isolated microgrids, grid operators face a trade-off between multiple performance indicators. This paper introduces a data-driven adaptive load frequency control (DD-ALFC) approach, where the load frequency controller is modeled as an agent that can balance different objectives autonomously. The paper also proposes a priority replay soft actor critic (PR-SAC) algorithm to implement the DD-ALFC method. The PR-SAC algorithm enhances the policy randomness by using entropy regularization and maximization, and improves the learning adaptability and generalization by using priority experience replay. The proposed DD-ALFC method based on the PR-SAC algorithm can achieve higher adaptability and robustness in complex microgrid environments with multiple performance indicators, and improve both the frequency control and the economic efficiency. The paper validates the effectiveness of the proposed method in the Zhuzhou Island microgrid. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
41. From Traditional Recommender Systems to GPT-Based Chatbots: A Survey of Recent Developments and Future Directions.
- Author
-
Al-Hasan, Tamim Mahmud, Sayed, Aya Nabil, Bensaali, Faycal, Himeur, Yassine, Varlamis, Iraklis, and Dimitrakopoulos, George
- Subjects
RECOMMENDER systems ,CHATBOTS ,NATURAL language processing ,GENERATIVE pre-trained transformers ,REINFORCEMENT learning ,DEEP learning - Abstract
Recommender systems are a key technology for many applications, such as e-commerce, streaming media, and social media. Traditional recommender systems rely on collaborative filtering or content-based filtering to make recommendations. However, these approaches have limitations, such as the cold start and the data sparsity problem. This survey paper presents an in-depth analysis of the paradigm shift from conventional recommender systems to generative pre-trained-transformers-(GPT)-based chatbots. We highlight recent developments that leverage the power of GPT to create interactive and personalized conversational agents. By exploring natural language processing (NLP) and deep learning techniques, we investigate how GPT models can better understand user preferences and provide context-aware recommendations. The paper further evaluates the advantages and limitations of GPT-based recommender systems, comparing their performance with traditional methods. Additionally, we discuss potential future directions, including the role of reinforcement learning in refining the personalization aspect of these systems. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
42. High-Frequency Quantitative Trading of Digital Currencies Based on Fusion of Deep Reinforcement Learning Models with Evolutionary Strategies.
- Author
-
Yijun He, Bo Xu, and Xinpu Su
- Subjects
DEEP reinforcement learning ,REINFORCEMENT learning ,ELECTRONIC money ,MACHINE learning ,CRYPTOCURRENCIES ,EVOLUTIONARY models - Abstract
High-frequency quantitative trading in the emerging digital currency market poses unique challenges due to the lack of established methods for extracting trading information. This paper proposes a deep evolutionary reinforcement learning (DERL) model that combines deep reinforcement learning with evolutionary strategies to address these challenges. Reinforcement learning is applied to data cleaning and factor extraction from a high-frequency, microscopic view-point to quantitatively explain the supply and demand imbalance and to create trading strategies. In order to determine whether the algorithm can successfully extract the significant hidden features in the factors when faced with large and complex high-frequency factors, this paper trains the agent in reinforcement learning using three different learning algorithms, including Q-learning, evolutionary strategies, and policy gradient. The experimental dataset, which contains data on sharp up, sharp down, and continuous oscillation situations, was chosen to test Bitcoin in January-February, September, and November of 2022. According to the experimental results, the evolutionary strategies algorithm achieved returns of 59.18%, 25.14%, and 22.72%, respectively. The results demonstrate that deep reinforcement learning based on the evolutionary strategies outperforms Q-learning and policy gradient concerning risk resistance and return capability. The proposed approach offers a robust and adaptive solution for high-frequency trading in the digital currency market, contributing to the development of effective quantitative trading strategies. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
43. Guest Editorial: Special issue on computational methods and artificial intelligence applications in low‐carbon energy systems.
- Author
-
Wang, Yishen, Zhou, Fei, Guerrero, Josep M., Baker, Kyri, Chen, Yize, Wang, Hao, Xu, Bolun, Xu, Qianwen, Zhu, Hong, and Agwan, Utkarsha
- Subjects
ARTIFICIAL intelligence ,ARTIFICIAL neural networks ,MACHINE learning ,REINFORCEMENT learning ,DEEP reinforcement learning ,DEEP learning - Abstract
This document is a guest editorial for a special issue on computational methods and artificial intelligence applications in low-carbon energy systems. The editorial highlights the urgent need for advanced computing and artificial intelligence in the clean energy transition to improve system reliability, economics, and sustainability. The special issue includes 19 original research articles covering topics such as energy forecasting, situational awareness, multi-energy system dispatch, and power system operation. The articles present state-of-the-art methods and techniques in these areas, including wind power forecasting, demand-side flexibility, fault diagnosis of photovoltaic strings, and energy management strategies. The authors express their gratitude to the participating authors and anonymous reviewers for their contributions to the special section. [Extracted from the article]
- Published
- 2024
- Full Text
- View/download PDF
44. Digital Twin and Reinforcement Learning-Based Resilient Production Control for Micro Smart Factory.
- Author
-
Park, Kyu Tae, Son, Yoo Ho, Ko, Sang Wook, Noh, Sang Do, Kyritsis, Dimitrios, Lu, Jinzhi, and Zheng, Xiaochen
- Subjects
PRODUCTION control ,REINFORCEMENT learning ,MANUFACTURING processes ,REWARD (Psychology) ,ELECTRONIC paper - Abstract
To achieve efficient personalized production at an affordable cost, a modular manufacturing system (MMS) can be utilized. MMS enables restructuring of its configuration to accommodate product changes and is thus an efficient solution to reduce the costs involved in personalized production. A micro smart factory (MSF) is an MMS with heterogeneous production processes to enable personalized production. Similar to MMS, MSF also enables the restructuring of production configuration; additionally, it comprises cyber-physical production systems (CPPSs) that help achieve resilience. However, MSFs need to overcome performance hurdles with respect to production control. Therefore, this paper proposes a digital twin (DT) and reinforcement learning (RL)-based production control method. This method replaces the existing dispatching rule in the type and instance phases of the MSF. In this method, the RL policy network is learned and evaluated by coordination between DT and RL. The DT provides virtual event logs that include states, actions, and rewards to support learning. These virtual event logs are returned based on vertical integration with the MSF. As a result, the proposed method provides a resilient solution to the CPPS architectural framework and achieves appropriate actions to the dynamic situation of MSF. Additionally, applying DT with RL helps decide what-next/where-next in the production cycle. Moreover, the proposed concept can be extended to various manufacturing domains because the priority rule concept is frequently applied. [ABSTRACT FROM AUTHOR]
- Published
- 2021
- Full Text
- View/download PDF
45. End-to-End Autonomous Driving Decision Method Based on Improved TD3 Algorithm in Complex Scenarios.
- Author
-
Xu, Tao, Meng, Zhiwei, Lu, Weike, and Tong, Zhongwen
- Subjects
DECISION making ,ALGORITHMS ,CRITICS ,CAMERAS ,SPEED - Abstract
The ability to make informed decisions in complex scenarios is crucial for intelligent automotive systems. Traditional expert rules and other methods often fall short in complex contexts. Recently, reinforcement learning has garnered significant attention due to its superior decision-making capabilities. However, there exists the phenomenon of inaccurate target network estimation, which limits its decision-making ability in complex scenarios. This paper mainly focuses on the study of the underestimation phenomenon, and proposes an end-to-end autonomous driving decision-making method based on an improved TD3 algorithm. This method employs a forward camera to capture data. By introducing a new critic network to form a triple-critic structure and combining it with the target maximization operation, the underestimation problem in the TD3 algorithm is solved. Subsequently, the multi-timestep averaging method is used to address the policy instability caused by the new single critic. In addition, this paper uses Carla platform to construct multi-vehicle unprotected left turn and congested lane-center driving scenarios and verifies the algorithm. The results demonstrate that our method surpasses baseline DDPG and TD3 algorithms in aspects such as convergence speed, estimation accuracy, and policy stability. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
46. Low-carbon economic dispatch strategy for integrated electrical and gas system with GCCP based on multi-agent deep reinforcement learning.
- Author
-
Feng, Wentao, Deng, Bingyan, Zhang, Ziwen, Jiang, He, Zheng, Yanxi, Peng, Xinran, Zhang, Le, Jing, Zhiyuan, Qing, Ke, Xi, Xianpeng, Zhang, Bin, and Li, Mingxuan
- Subjects
DEEP reinforcement learning ,REINFORCEMENT learning ,MACHINE learning ,CARBON emissions ,NATURAL gas ,DEEP learning - Abstract
With the growing concern for the environment, sustainable development centred on a low-carbon economy has become a unifying pursuit for the energy industry. Integrated energy systems (IES) that combine multiple energy sources such as electricity, heat and gas are essential to facilitate the consumption of renewable energy and the reduction of carbon emission. In this paper, gas turbine (GT), carbon capture and storage (CCS) and power-to-gas (P2G) device are introduced to construct a new carbon capture coupling device model, GT-CCS-P2G (GCCP), which is applied to the integrated electrical and gas system (IEGS). Multi-agent soft actor critic (MASAC) applies historical trajectory representations, parameter spatial techniques and deep densification frameworks to reinforcement learning for reducing the detrimental effects of time-series data on the decisional procedure. The energy scheduling problem of IEGS is redefined as a Markov game, which is addressed by adopting a low carbon economic control framework based on MASAC with minimum operating cost and minimum carbon emission as the optimization objectives. To validate the rationality and effectiveness of the proposed low-carbon economy scheduling model of IEGS based on MASAC, this paper simulates and analyses in integrated PJM-5 node system and seven nodes natural gas system. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
47. Inner External DQN LoRa SF Allocation Scheme for Complex Environments.
- Author
-
Pang, Shengli, Kong, Delin, Wang, Xute, Pan, Ruoyu, Wang, Honggang, Ye, Zhifan, and Liu, Di
- Subjects
REINFORCEMENT learning ,NETWORK performance ,TELECOMMUNICATION ,WIRELESS communications ,INTERNET of things - Abstract
In recent years, with the development of Internet of Things technology, the demand for low-power wireless communication technology has been growing, giving rise to LoRa technology. A LoRa network mainly consists of terminal nodes, gateways, and LoRa network servers. As LoRa networks often deploy many terminal node devices for environmental sensing, the limited resources of LoRa technology, the explosive growth in the number of nodes, and the ever-changing complex environment pose unprecedented challenges for the performance of the LoRa network. Although some research has already addressed the challenges by allocating channels to the LoRa network, the impact of complex and changing environmental factors on the LoRa network has yet to be considered. Reasonable channel allocation should be tailored to the situation and should face different environments and network distribution conditions through continuous adaptive learning to obtain the corresponding allocation strategy. Secondly, most of the current research only focuses on the channel adjustment of the LoRa node itself. Still, it does not consider the indirect impact of the node's allocation on the entire network. The Inner External DQN SF allocation method (IEDQN) proposed in this paper improves the packet reception rate of the whole system by using reinforcement learning methods for adaptive learning of the environment. It considers the impact on the entire network of the current node parameter configuration through nested reinforcement learning for further optimization to optimize the whole network's performance. Finally, this paper evaluates the performance of IEDQN through simulation. The experimental results show that the IEDQN method optimizes network performance. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
48. Computation Offloading with Privacy-Preserving in Multi-Access Edge Computing: A Multi-Agent Deep Reinforcement Learning Approach.
- Author
-
Dai, Xiang, Luo, Zhongqiang, and Zhang, Wei
- Subjects
DEEP reinforcement learning ,REINFORCEMENT learning ,EDGE computing ,REINFORCEMENT (Psychology) ,TELECOMMUNICATION ,QUALITY of service ,INTERNET of things - Abstract
The rapid development of mobile communication technologies and Internet of Things (IoT) devices has introduced new challenges for multi-access edge computing (MEC). A key issue is how to efficiently manage MEC resources and determine the optimal offloading strategy between edge servers and user devices, while also protecting user privacy and thereby improving the Quality of Service (QoS). To address this issue, this paper investigates a privacy-preserving computation offloading scheme, designed to maximize QoS by comprehensively considering privacy protection, delay, energy consumption, and the task discard rate of user devices. We first formalize the privacy issue by introducing the concept of privacy entropy. Then, based on quantified indicators, a multi-objective optimization problem is established. To find an optimal solution to this problem, this paper proposes a computation offloading algorithm based on the Twin delayed deep deterministic policy gradient (TD3-SN-PER), which integrates clipped double-Q learning, prioritized experience replay, and state normalization techniques. Finally, the proposed method is evaluated through simulation analysis. The experimental results demonstrate that our approach can effectively balance multiple performance metrics to achieve optimal QoS. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
49. RESEARCH ON THE APPLICATION OF ARTIFICIAL INTELLIGENCE TECHNOLOGY IN THE BANKING INTERNET FINANCE INDUSTRY.
- Author
-
TIANHAO ZHANG
- Subjects
ARTIFICIAL intelligence ,ONLINE banking ,FINANCIAL services industry ,MACHINE learning ,RECOMMENDER systems ,REINFORCEMENT learning - Abstract
This paper presents a collaborative filtering algorithm based on reinforcement learning theory. Then, the personalized bank financial recommendation system for users is constructed in the massive data environment. Tags mimic different types of user interest points to build a representative personalized data set. The collaborative screening of bank financial products is realized using the simulation results and users' historical access records. The ranking calculation of related financial products is added to the general bank financial product recommendation system. This method can more accurately express the query results for a specific user. It is found that the collaborative filtering algorithm based on enhanced learning theory can improve the efficiency of collaborative screening of bank financial products. The best results can be obtained by combining the two organically. This paper proposes that the recommendation algorithm of reinforcement learning bank financial products based on user preference and collaborative filtering is feasible. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
50. Data-driven active corrective control in power systems: an interpretable deep reinforcement learning approach.
- Author
-
Li, Beibei, Liu, Qian, Hong, Yue, He, Yuxiong, Zhang, Lihong, He, Zhihong, Feng, Xiaoze, Gao, Tianlu, Yang, Li, Yan, Ziming, and Zhang, Cong
- Subjects
DEEP reinforcement learning ,ARTIFICIAL intelligence ,REINFORCEMENT learning ,MARKOV processes ,DECISION making - Abstract
With the successful application of artificial intelligence technology in various fields, deep reinforcement learning (DRL) algorithms have applied in active corrective control in the power system to improve accuracy and efficiency. However, the "black-box" nature of deep reinforcement learning models reduces their reliability in practical applications, making it difficult for operators to comprehend the decision-making mechanism. process of these models, thus undermining their credibility. In this paper, a DRL model is constructed based on the Markov decision process (MDP) to effectively address active corrective control issues in a 36-bus system. Furthermore, a feature importance explainability method is proposed, validating that the proposed feature importance-based explainability method enhances the transparency and reliability of the DRL model for active corrective control. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.