Search

The NOTED (Network Optimised Transfer of Experimental Data) project has successfully demonstrated the ability to dynamically provision network links to increase the effective bandwidth available for FTS-driven transfers between endpoints, such as WLCG sites, by inspecting on-going data transfers and so identifying those that are bandwidth-limited for a long period of time. Recently, the architecture of NOTED has been improved and the software has been packaged for easy distribution. These improved capabilities and features of NOTED have been tested and demonstrated at various international conferences. For example, during demonstrations at Supercomputing 2022, independent instances of NOTED at CHCERN (Switzerland) and DE-KIT (Germany) monitored large data transfers generated by the ATLAS experiment between these sites and CA-TRIUMF (Canada). We report here on this and other events, highlighting how NOTED can predict link congestion or a notable increase in the network utilisation over an extended period of time and, where appropriate, automatically reconfigure network topology to introduce an additional or an alternative and betterperforming path by using dynamic circuit provisioning systems such as SENSE and AutoGOLE. [ABSTRACT FROM AUTHOR]

Infrastructures supporting distributed scientific collaborations must address competing goals in both providing high performance access to resources while simultaneously securing the infrastructure against security threats. The NetBASILISK project is attempting to improve the security of such infrastructures while not adversely impacting their performance. This paper will present our work to create a benchmark and monitoring infrastructure that allows us to test for any degradation in transferring data into a NetBASILISK protected site. [ABSTRACT FROM AUTHOR]

Network traffic optimisation is difficult as the load is by nature dynamic and seemingly unpredictable. However, the increased usage of file transfer services may help the detection of future loads and the prediction of their expected duration. The NOTED project seeks to do exactly this and to dynamically adapt network topology to deliver improved bandwidth for users of such services. This article introduces, and explains the features of, the two main components of NOTED, the Transfer Broker and the Network Intelligence component. The Transfer Broker analyses all queued and on-going FTS transfers, producing a traffic report which can be used by network controllers. Based on this report and its knowledge of the network topology and routing, the Network Intelligence (NI) component makes decisions as to when a network reconfiguration could be beneficial. Any Software Defined Network controller can then apply these decision to the network, so optimising transfer execution time and reducing operating costs. [ABSTRACT FROM AUTHOR]

The first production version of the CERN Tape Archive (CTA) software is planned to be released during 2019. CTA is designed to replace CASTOR as the CERN tape archive solution, to face the scalability and performance challenges arriving with LHC Run–3. In this paper, we describe the main commonalities and differences between CTA and CASTOR. We outline the functional enhancements and integration steps required to add the CTA tape back-end to an EOS disk storage system. We present and discuss the different deployment and migration scenarios for replacing the five CASTOR instances at CERN, including a description of how the File Transfer Service (FTS) will interface with EOS and CTA. [ABSTRACT FROM AUTHOR]

In the past, several scaling tests have been performed on the HTCondor batch system regarding its job scheduling capabilities. In this talk we report on a first set of scalability measurements of the file transfer capabilities of the HTCondor batch system. Motivated by the GlueX experiment needs we evaluate the limits and possible use of HTCondor as a solution to transport the output of jobs back to a submitter as a function of the number of concurrent jobs, the size of the output and the distance between a submitting node and computing node. We conclude that HTCondor transfer system satisfies the current GlueX computing demands. [ABSTRACT FROM AUTHOR]