Search

We study the relative impact of task aggregation, or wrapping, which is a technique meant for computational workflows that bundles jobs into a single submission to be sent to remote schedulers. Experiments inside the Earth Science community can be lengthy and compriseseveral steps with many dependencies. The community has traditionally focused in increasing the performance of the models, but the overall execution of the workflow, including the queue time, has received little interest. Aiming to reduce the time spent in queue, the developers of Autosubmit, a workflow manager developed for climate simulations, weather forecast simulations, and air quality simulations, came up with task aggregating, or wrapping. Our objective is to assess if this technique does indeed reduce the total queue time of the workflow. The complex interplay between the dynamic nature of the usage of the machine and the scheduler policy plays a central role in our analysis, which poses the main challenge of this work. Hence, we do an intricate study of the scheduling policy of the popular Slurm Workload Manager and a statistical characterization of the usage of both simulated machines: LUMI and cea-curie. With that, we perform a twofold experimentation: a simulation using dynamic workloads - where job arrival time plays a role - with a workflow composed of multiple jobs and a static workload - where all jobs in the workload are submitted at the same time - varying job and user factors that play a role into the scheduling. Results show that aggregation is beneficial in the majority of cases for the workflows that are vertically organized - that is, a chain of submissions where each job is dependent on the previous -, whilst for the horizontal arranged workflows - where jobs do not have dependencies - it might undermine the queue time depending on the user's past usage and the machine's current state.

Understanding mesopelagic carbon budgets is central to answering pressing ecological, climatic, and economic questions: from the oceans’ role in carbon sequestration to the transfer of primary production to higher trophic levels. However, biogeochemical models used to simulate and predict mesopelagic carbon budgets still suffer from large uncertainties, which arise in part from poorly constrained parameters. Global ocean biogeochemistry models are typically optimized using climatological fields (e.g., nutrients, oxygen, export fluxes) as observational targets, therefore tuning them to reproduce the long-term mean state of the ocean. Our working hypothesis is that model parameters can be further constrained using high-resolution profiles from biogeochemical (BGC-) Argo floats, using particulate backscattering as a proxy for particulate organic carbon (POC). To test this hypothesis, we selected single-float coherent annual time series in the Labrador Sea (subpolar North Atlantic) and matched them to 1D biogeochemical model simulations (NEMO4-PISCESv2_RC). We then identified the model parameters that control mesopelagic POC budgets to (i) run comprehensive sensitivity analyses via parameter perturbations, (ii) optimize the most sensitive parameters against BGC-Argo observations using a genetic algorithm, and (iii) quantify the impact of parameter optimization on tridimensional POC fluxes. Our approach sheds new light on elusive mesopelagic POC budgets and their modulation by intense events in productive high-latitude oceans

The Earth system model EC-Earth3 for contributions to CMIP6 is documented here, with its flexible coupling framework, major model configurations, a methodology for ensuring the simulations are comparable across different high-performance computing (HPC) systems, and with the physical performance of base configurations over the historical period. The variety of possible configurations and sub-models reflects the broad interests in the EC-Earth community. EC-Earth3 key performance metrics demonstrate physical behavior and biases well within the frame known from recent CMIP models. With improved physical and dynamic features, new Earth system model (ESM) components, community tools, and largely improved physical performance compared to the CMIP5 version, EC-Earth3 represents a clear step forward for the only European community ESM. We demonstrate here that EC-Earth3 is suited for a range of tasks in CMIP6 and beyond.

When working with Earth system models, a considerable challenge that arises is the need to establish the set of parameter values that ensure the optimal model performance in terms of how they reflect real-world observed data. Given that each additional parameter under investigation increases the dimensional space of the problem by one, simple brute-force sensitivity tests can quickly become too computationally strenuous. In addition, the complexity of the model and interactions between parameters mean that testing them on an individual basis has the potential to miss key information. In this work, we address these challenges by developing a biased random key genetic algorithm (BRKGA) able to estimate model parameters. This method is tested using the one-dimensional configuration of PISCES-v2_RC, the biogeochemical component of NEMO4 v4.0.1 (Nucleus for European Modelling of the Ocean version 4), a global ocean model. A test case of particulate organic carbon (POC) in the North Atlantic down to 1000 m depth is examined, using observed data obtained from autonomous biogeochemical Argo floats. In this case, two sets of tests are run, namely one where each of the model outputs are compared to the model outputs with default settings and another where they are compared with three sets of observed data from their respective regions, which is followed by a cross-reference of the results. The results of these analyses provide evidence that this approach is robust and consistent and also that it provides an indication of the sensitivity of parameters on variables of interest. Given the deviation in the optimal set of parameters from the default, further analyses using observed data in other locations are recommended to establish the validity of the results obtained., This research has been supported by the Fundación Bancaria Caixa d'Estalvis i Pensions de Barcelona (grant no. LCF/BQ/PI18/11630009) and the Ministerio de Ciencia e Innovación (grant nos. PID2019-107952GA-I00 and CGL2017-84493-R)., Peer Reviewed, Postprint (published version)

When working with Earth system models, a considerable challenge that arises is the need to establish the set of parameter values that ensure the optimal model performance in terms of how they reflect real-world observed data. Given that each additional parameter under investigation increases the dimensional space of the problem by one, simple brute-force sensitivity tests can quickly become too computationally strenuous. In addition, the complexity of the model and interactions between parameters mean that testing them on an individual basis has the potential to miss key information. In this work, we address these challenges by developing a biased random key genetic algorithm (BRKGA) able to estimate model parameters. This method is tested using the one-dimensional configuration of PISCES-v2_RC, the biogeochemical component of NEMO4 v4.0.1 (Nucleus for European Modelling of the Ocean version 4), a global ocean model. A test case of particulate organic carbon (POC) in the North Atlantic down to 1000 m depth is examined, using observed data obtained from autonomous biogeochemical Argo floats. In this case, two sets of tests are run, namely one where each of the model outputs are compared to the model outputs with default settings and another where they are compared with three sets of observed data from their respective regions, which is followed by a cross-reference of the results. The results of these analyses provide evidence that this approach is robust and consistent and also that it provides an indication of the sensitivity of parameters on variables of interest. Given the deviation in the optimal set of parameters from the default, further analyses using observed data in other locations are recommended to establish the validity of the results obtained

The Earth system model EC-Earth3 for contributions to CMIP6 is documented here, with its flexible coupling framework, major model configurations, a methodology for ensuring the simulations are comparable across different high-performance computing (HPC) systems, and with the physical performance of base configurations over the historical period. The variety of possible configurations and sub-models reflects the broad interests in the EC-Earth community. EC-Earth3 key performance metrics demonstrate physical behavior and biases well within the frame known from recent CMIP models. With improved physical and dynamic features, new Earth system model (ESM) components, community tools, and largely improved physical performance compared to the CMIP5 version, EC-Earth3 represents a clear step forward for the only European community ESM. We demonstrate here that EC-Earth3 is suited for a range of tasks in CMIP6 and beyond., The development of EC-Earth3 was supported by the European Union's Horizon 2020 research and innovation program under project IS-ENES3, the third phase of the distributed e-infrastructure of the European Network for Earth System Modelling (ENES) (grant agreement no. 824084, PRIMAVERA grant no. 641727, and CRESCENDO grant no. 641816). Etienne Tourigny and Raffaele Bernardello have received funding from the European Union’s Horizon 2020 research and innovation program under Marie Skłodowska-Curie grant agreement nos. 748750 (SPFireSD project) and 708063 (NeTNPPAO project). Ivana Cvijanovic was supported by Generalitat de Catalunya (Secretaria d'Universitats i Recerca del Departament d’Empresa i Coneixement) through the Beatriu de Pinós program. Yohan Ruprich-Robert was funded by the European Union's Horizon 2020 research and innovation program in the framework of Marie Skłodowska-Curie grant INADEC (grant agreement 800154). Paul A. Miller, Lars Nieradzik, David Wårlind, Roland Schrödner, and Benjamin Smith acknowledge financial support from the strategic research area “Modeling the Regional and Global Earth System” (MERGE) and the Lund University Centre for Studies of Carbon Cycle and Climate Interactions (LUCCI). Paul A. Miller, David Wårlind, and Benjamin Smith acknowledge financial support from the Swedish national strategic e-science research program eSSENCE. Paul A. Miller further acknowledges financial support from the Swedish Research Council (Vetenskapsrådet) under project no. 621-2013-5487. Shuting Yang acknowledges financial support from a Synergy Grant from the European Research Council under the European Community's Seventh Framework Programme (FP7/2007-2013)/ERC (grant agreement 610055) as part of the ice2ice project and the NordForsk-funded Nordic Centre of Excellence project (award 76654) ARCPATH. Marianne Sloth Madsen acknowledges financial support from the Danish National Center for Climate Research (NCKF). Andrea Alessandri and Peter Anthoni acknowle, Peer Reviewed, "Article signat per 61 autors/es: Ralf Döscher, Mario Acosta, Andrea Alessandri, Peter Anthoni, Thomas Arsouze, Tommi Bergman, Raffaele Bernardello, Souhail Boussetta, Louis-Philippe Caron, Glenn Carver, Miguel Castrillo, Franco Catalano, Ivana Cvijanovic, Paolo Davini, Evelien Dekker, Francisco J. Doblas-Reyes, David Docquier, Pablo Echevarria, Uwe Fladrich, Ramon Fuentes-Franco, Matthias Gröger, Jost v. Hardenberg, Jenny Hieronymus, M. Pasha Karami, Jukka-Pekka Keskinen, Torben Koenigk, Risto Makkonen, François Massonnet, Martin Ménégoz, Paul A. Miller, Eduardo Moreno-Chamarro, Lars Nieradzik, Twan van Noije, Paul Nolan, Declan O'Donnell, Pirkka Ollinaho11, Gijs van den Oord, Pablo Ortega, Oriol Tintó Prims, Arthur Ramos, Thomas Reerink, Clement Rousset, Yohan Ruprich-Robert, Philippe Le Sager, Torben Schmith, Roland Schrödner, Federico Serva, Valentina Sicardi, Marianne Sloth Madsen, Benjamin Smith, Tian Tian, Etienne Tourigny, Petteri Uotila, Martin Vancoppenolle, Shiyu Wang, David Wårlind, Ulrika Willén, Klaus Wyser, Shuting Yang, Xavier Yepes-Arbós, and Qiong Zhang", Postprint (author's final draft)

Autosubmit GUI is a front-end software developed using Javascript and ReactJS that aims to provide users with complex information from the workflow execution of scientific experiments (managed by Autosubmit) in any system, but mainly High-Performance Computing (HPC) platforms. Autosubmit (D. Manubens-Gil, 2016) is a Python-based workflow manager able to handle complex tasks involving different sub-steps (e.g., scientific computational experiments). These steps or jobs are executed in one or multiple computing systems (platforms), from High Performance Computers to small clusters or workstations. The workflow manager is able to orchestrate the jobs that constitute the workflow while respecting their dependencies and handling errors. This front-end software consumes information served by an API (Autosubmit API (Autosubmit API, 2020)) that collects data from the execution of experiment workflows. An experiment can be seen as a task that can be decomposed into different sub-steps with established interdependencies. The execution of an experiment and its jobs may generate a high amount of information that needs to be processed so users can visualize it. Autosubmit API summarizes this information and presents it as API requests. Autosubmit GUI consumes these API requests and shows the information available in a condensed, comprehensive, and dynamic way. Autosubmit GUI uses three highly popular and useful libraries: FancyTree (FancyTree, 2020), vis.js (Vis.js, 2020), and react-google-charts (React Google Charts, 2020); among other web resources that facilitate the visualization of information and information updates. The visual approach to experiment workflow management is not new in the High-Performance Computing scenario. As two widely adopted tools, we have Cylc (Cylc, 2020) and ecFlow (ecFlow, 2020). Autosubmit GUI attempts to present a development template for those willing to work in a web environment to develop tools for progress monitoring purposes, Peer Reviewed, Postprint (published version)

A dominant paradigm for mid-latitude air-sea interaction identifies the synoptic-scale atmospheric “noise” as the main driver for the observed ocean surface variability. While this conceptual model successfully holds over most of the mid-latitude ocean surface, its soundness over frontal zones (including western boundary currents; WBC) characterized by intense mesoscale activity, has been questioned in a number of studies suggesting a driving role for the small scale ocean dynamics (mesoscale oceanic eddies) in the modulation of air-sea interaction. In this context, climate models provide a powerful experimental device to inspect the emerging scale-dependent nature of mid-latitude air-sea interaction. This study assesses the impact of model resolution on the representation of air-sea interaction over the Gulf Stream region, in a multi-model ensemble of present-climate simulations performed using a common experimental design. Lead-lag correlation and covariance patterns between sea surface temperature (SST) and turbulent heat flux (THF) are diagnosed to identify the leading regimes of air-sea interaction in a region encompassing both the Gulf Stream system and the North Atlantic subtropical basin. Based on these statistical metrics it is found that coupled models based on “laminar” (eddy-parameterised) and eddy-permitting oceans are able to discriminate between an ocean-driven regime, dominating the region controlled by the Gulf Stream dynamics, and an atmosphere-driven regime, typical of the open ocean regions. However, the increase of model resolution leads to a better representation of SST and THF cross-covariance patterns and functional forms, and the major improvements can be largely ascribed to a refinement of the oceanic model component., The authors of this study wish to thank two reviewers for their many insightful comments. AB, PA, ES, RH, JG-S, DP, ESG, MJR, CR, JS, PV acknowledge PRIMAVERA funding received from the European Commission under Grant Agreement 641727 of the Horizon 2020 research programme. JG-S was additionally supported by the Spanish ‘Ramón y Cajal’ programme (RYC-2016-21181). The authors declare that they have no conflict of interest. The datasets used in this work are cited in this manuscript with appropriate doi’s in publicly available archives., Peer Reviewed, Postprint (published version)

In this paper, we present and evaluate the skill of an EC-Earth3.3 decadal prediction system contributing to the Decadal Climate Prediction Project – Component A (DCPP-A). This prediction system is capable of skilfully simulating past global mean surface temperature variations at interannual and decadal forecast times as well as the local surface temperature in regions such as the tropical Atlantic, the Indian Ocean and most of the continental areas, although most of the skill comes from the representation of the external radiative forcings. A benefit of initialization in the predictive skill is evident in some areas of the tropical Pacific and North Atlantic oceans in the first forecast years, an added value that is mostly confined to the south-east tropical Pacific and the eastern subpolar North Atlantic at the longest forecast times (6–10 years). The central subpolar North Atlantic shows poor predictive skill and a detrimental effect of initialization that leads to a quick collapse in Labrador Sea convection, followed by a weakening of the Atlantic Meridional Overturning Circulation (AMOC) and excessive local sea ice growth. The shutdown in Labrador Sea convection responds to a gradual increase in the local density stratification in the first years of the forecast, ultimately related to the different paces at which surface and subsurface temperature and salinity drift towards their preferred mean state. This transition happens rapidly at the surface and more slowly in the subsurface, where, by the 10th forecast year, the model is still far from the typical mean states in the corresponding ensemble of historical simulations with EC-Earth3. Thus, our study highlights the Labrador Sea as a region that can be sensitive to full-field initialization and hamper the final prediction skill, a problem that can be alleviated by improving the regional model biases through model development and by identifying more optimal initialization strategies., The work in this paper was supported by the European Commission H2020 projects EUCP (grant no. 776613), APPLICATE (grant no. 727862), INTAROS (grant no. 727890) and PRIMAVERA (grant no. 641727); a Spanish project funded by the Spanish Ministry of Economy, Industry and Competitiveness (CLINSA, grant no. CGL2017-85791-R); a FRS-FNRS/FWO-funded Belgian project (PARAMOUR, grant no. EOS-30454083); and an ESA contract (grant no. CMUG-CCI3-TECHPROP). The climate simulations analysed in the paper were performed using the internal computing resources available at MareNostrum and additional resources from PRACE (HiResNTCP, project 3: grant no. 2017174177) and the Red Española de Supercomputación (AECT-2019-2-0003 and AECT-2019-3-0006 projects) as well as technical support provided by the Barcelona Supercomputing Center. In addition, several co-authors have been supported by personal grants: Yohan Ruprich-Robert, Etienne Tourigny and Simon Wild received funding from the European Union Horizon 2020 research and innovation programme (grant agreement nos. 800154, 748750 and 754433 respectively); Ivana Cvijanovic was supported by Generalitat de Catalunya (Secretaria d'Universitats i Recerca del Departament d'Empresa i Coneixement) through the Beatriu de Pinós programme; Juan Acosta-Navarro was supported by the Spanish Ministry of Science, Innovation and Universities through a Juan de la Cierva personal grant (grant no. FJCI-2017-34027); Rubén Cruz-García was funded by the Spanish Ministry of Education, Culture and Sports with an FPU grant (grant no. FPU15/01511); and Markus Donat and Pablo Ortega were funded by the Spanish Ministry of Economy, Industry and Competitiveness through the Ramon y Cajal grants RYC-2017-22964 and RYC-2017-22772. We also want to thank Panos Athanasidis, Stephen Yeager and Gerald Meehl for their very helpful comments when reviewing the paper., Postprint (published version)