Matches in Nanopublications for { ?s <http://schema.org/description> ?o ?g. }
- 3925 description "" assertion.
- 3925 description "" assertion.
- 41097 description "Numerous methods have been developed to geolocate fish from data storage tags. Whereas demersal species have been tracked using tide-driven geolocation models, pelagic species which undertake extensive migrations have been mainly tracked using light-based models. Here, we present a new HMM-based model that infers pelagic fish positions from the sole use of high-resolution temperature and depth histories. A key contribution of our framework lies in model parameter inference (diffusion coefficient and noise parameters with respect to the reference geophysical fields—satellite SST and temperatures derived from the MARS3D hydrodynamic model), which improves model robustness. As a case study, we consider long time series of data storage tags (DSTs) deployed on European sea bass for which individual migration tracks are reconstructed for the first time. We performed a sensitivity analysis on synthetic and real data in order to assess the robustness of the reconstructed tracks with respect to model parameters, chosen reference geophysical fields and the knowledge of fish recapture position. Model assumptions and future directions are discussed. Finally, our model opens new avenues for the reconstruction and analysis of migratory patterns of many other pelagic species in relatively contrasted geophysical environments" assertion.
- 4ba5a56b-e756-499f-8efa-1c32d24fce4f description "# Description **Duration**: 6 months **Start date**: September 2024. **Partners**: Jean-Marc Delouis (CNRS/IFREMER), Tina Odaka (IFREMER), Anne Fouilloux (Simula), Quentin Mazouni (Simula) ## Context Biologing seeks to determine the most likely fish trajectory using tagging data that records only temperature and pressure. Understanding the routes of fish is crucial for various ecological reasons to enhance the protection of the species under study. While biologing has demonstrated effectiveness [de Pontual et al, 2023], there remains scope for further improvement. Specifically, in calculating the likeliest path, the pangeo-fish algorithm models the fish's movement capability using a normal distribution, indicating that there is no favoured direction. We suggest incorporating external data to refine our understanding of the fish's most likely movement, including bathymetric data, identification of the fish through behavioural biology insights (such as depth data at high frequency) or detailed sea surface temperature measurements, among others. ## Detailed work - Month 1 - Construct the input data sets: During the first month, efforts will focus on assembling an appropriate data set for research. The initial task involves analysing the available biological dataset from the existing 450 sets and applying unsupervised classification techniques (e.g., Scattering Transform) to categorise fish behaviours. In parallel, satellite data (Sea Surface Temperature) and ocean modelling systems such as Copernicus Marine must be identified and collected. - Months 2 to 4 - Estimate a new probable trajectory for fish: This phase involves the employing cutting-edge algorithms that use scale coupling statistics methods. This approach aims to achieve highly resolved (both spatially and temporally) data to predict the direction of fish movement. The feasibility of validating this novel method is supported by multiple observations of some fish within a few tracks. Thus, disregarding this existing data, we can assess the deviation of the fish paths predicted by these algorithms from observed locations. - Months 5 to 6 - Apply the algorithm to the 450 tags and evaluate the performances: in the final two months, apply the developed algorithm to the 450 tags and assess its efficacy. If the results are satisfactory, a scientific publication can be expected at the end of this period. ## Computing & storage infrastructure Cloud resources (storage & computing) are available for the entire duration of the project. These include 1) GFTS Jupyterhub (http://gfts.minrk.net) to conduct computation tasks like applying algorithms and constructing datasets. All the data, inclusive of tagging and satellite data, will be securely stored and managed via S3-compatible object storage to facilitate access and reuse during the project. Access to GFTS JupyterHub will be granted at the start of the project. 2) Destination Earth Services (https://platform.destine.eu/) to access environmental data like Ocean data (Sea Surface Temperature, Sea Water Potential Temperature on model levels, Surface Temperature and bathymetric data) from the Climate Adaptation Digital twin; Access to the Destination Earth Services require user registration at https://platform.destine.eu/ and is subject to their availability during the project (services are still under development and not available yet)." assertion.
- fsad087 description "Large-scale electronic tagging is a very powerful tool to study how fish movements and migrations shape the internal dynamics of populations. This knowledge, crucial for improving fishery management, was still limited for the European seabass, whose stocks in the Northeast Atlantic have declined drastically over the last decade. To better understand the species ecology and the spatio-temporal structure of the population, we tagged seabass in the North Sea, the English Channel, and the Bay of Biscay, from 2014 to 2016. Out of 1220 deployed DSTs, 482 have been recovered by November 2022. Approximately half of them included a period of potential spawning migration. Reconstructed trajectories confirmed seabass to be a partial migratory species, as individuals exhibited either long-distance migrations or residence. Most migrants exhibited fidelity to summer feeding areas and winter spawning areas. Our dataset enriches the knowledge of seabass biological traits (e.g. temperature and depth ranges, vulnerability to predation and fishing). Our results suggest a spatial structure of the Atlantic population that differs from the stock structure currently considered for assessment and management. The consequences should be explored at both the European level and by regional managers involved in conservation outcomes." assertion.
- 46796 description "Geo-referenced data plays a crucial role in understanding and conserving natural resources, particularly in studying biolog- ical processes such as fish migration. Biologging, which in- volves attaching small devices to animals to track their behav- ior and gather environmental data, is a valuable tool in this regard. However, tracking fish directly underwater remains challenging. To address this, models have been developed to geolocate fish by utilizing temperature and pressure data from biologging devices and comparing them with ocean tempera- ture and bathymetry models as reference data. When applying these models, the accuracy and resolution of the reference data has a significant impact on the quality of reconstructed fish trajectories. With recent advancements in earth observation technology and modeling techniques like digital twins, huge amounts of earth science datasets are avail- able. However, the large size and limited accessibility of these datasets, as well as the computational requirements for anal- ysis, present technical obstacles. The Pangeo ecosystem was created by a community of engineers and geoscientists specifically to address these challenges. Making use of these new tools and to leverage the advancements in biodiversity research, a software called pangeo-fish has been developed. This paper focuses on its development for studying the movement of the European sea bass (Dicentrarchus labrax)." assertion.
- 9b361a58-e5ba-4683-a004-08a489be9df4 description "**Use Case topic**: The goal of this use case is the development and implementation of the Global Fish Tracking System (GFTS) to enhance understanding and management of wild fish stocks **Scale of the Use Case (Global/Regional/National)**: Local to Global (various locations worldwide) **Policy addressed**: Fisheries Management Policy **Data Sources used**: Climate Change Adaptation (Climate DT: Routine and On-Demand for some higher resolution tracking), Sea Temperature observation (Satelite, in-situ) Copernicus Marine services (Sea temperature and associated value), Bathymetry (Gebco), biologging fish data **Github Repository**: [https://github.com/destination-earth/DestinE_ESA_GFTS.git](https://github.com/destination-earth/DestinE_ESA_GFTS.git)" assertion.
- 688347b8-63b1-4541-a870-f60b1c72763b description "This folder contains presentations or other kind of materials (such as training material) developed and presented during events." assertion.
- 6dffb82f-c859-4cc0-8e3b-262d36ad03c5 description "This folder contains project documents such as DMP, link to website and github repository, etc." assertion.
- 03712afd-4b2b-4f26-8bda-72c16bb20013 description "FAIR Implementation Profile (FIP) for the GFTS project." assertion.
- 1736b174-0fac-4ffa-8236-75bb42868af9 description "Poster presented at the 8th InternationalBio-logging Science Symposium by Tina Odaka, March 2024." assertion.
- 1997c5cc-9799-4c11-a43b-08ea1440d62f description "This pitcure shows Tina Odaka presenting the Global Fish Tracking System (GFTS) DestinE DESP Use Case at the 8th International Bio-logging Science Symposium, Tokyo, Japan (4-8 March 2024)." assertion.
- 1a3335ef-a237-452d-9a54-ca42a0ec37f2 description "The Software Verification and Validation Plan for the Global fish Tracking System DestinE Use Case." assertion.
- 1a5f7ba3-49a1-44c7-8bb8-27328c70798c description "These webpages are rendered from GitHub repository and contain all the information about the GFTS project. This includes internal description of the use case, technical documentation, progress, presentations, etc." assertion.
- 29cb6ae5-bcdb-4b21-9970-0e607acee782 description "Deliverable 5.5 corresponding to the Global Fish tracking System Use Case Promotion Package" assertion.
- 321909b6-a061-4665-870a-7086bf29c202 description "The Software Verification and Validation Report from the Global Fish Tracking System DestinE Use Case." assertion.
- 36b626d9-fd5a-481f-980e-5ec25294e32a description "This report corresponds to the Software Reuse File for the GFTS DestinE Platform Use Case. New version will be uploaded regularly." assertion.
- 51ecf2f0-7e2d-4f76-ad38-e9ddbe8bbef2 description "Deliverable 5.2 - Use Case Descriptor for the Global fish Tracking System Use Case on the DestinE Platform." assertion.
- 5fdbfc33-4a49-4535-a0ae-8e6aaa2885a1 description "The Software Requirement Specifications for the Global fish Tracking System DestinE Use Case." assertion.
- 7d5d343e-97a5-4d1f-9239-988df72cd0bd description "JupyterHub on Destination Earth Data Lake" assertion.
- 8ab9efc3-2528-4558-b531-8590c4e1ffc1 description "Link to the Pangeo JupyterHub we are using for developing Pangeo Fish. Only users from GFTS can register and authenticate to this JupyterHub" assertion.
- 97f93dd4-b71f-4781-a664-219aa1efa9d2 description "The Gobal Fish track system Use case Application on the DestinE Platform" assertion.
- a91b2c91-dcbb-44e3-bf5e-f70c642b4dfa description "Slides presented by Mathieu Woillez at the Roadshow Webinar: DestinE in action – meet the first DESP use cases (13 December 2023)" assertion.
- bb0050be-cb5c-4d55-95be-064e1ba2cc19 description "The Software Release Plan for the Global Fish Tracking System DestinE Use Case." assertion.
- bffd0f8c-9aad-4274-a60d-ba84b79c743a description "Project Management Plan for the Global fish Tracking System Use Case on the DestinE Platform." assertion.
- f13cbffd-ce00-4be1-829c-834750a2894c description "Link to the official GFTS DESP use case." assertion.
- notebook.ipynb description "Jupyter Notebook hosted by the Environmental Data Science Book" assertion.
- environment.yml description "Conda environment when user want to have the same libraries installed without concerns of package versions" assertion.
- conda-lock.yml description "Lock conda file of the Jupyter notebook hosted by the Environmental Data Science Book" assertion.
- icenet description "Contains input Codebase of the IceNet library used in the Jupyter notebook of Sea ice forecasting using the IceNet library" assertion.
- c9df67a3-0d77-4029-8556-e62fcc95a35b description "The research object refers to the Sea ice forecasting using the IceNet library notebook published in the Environmental Data Science book." assertion.
- s41467-021-25257-4 description "Related publication of the modelling presented in the Jupyter notebook" assertion.
- s41467-021-25257-4 description "Related publication of the modelling presented in the Jupyter notebook" assertion.
- s41467-021-25257-4 description "Related publication of the modelling presented in the Jupyter notebook" assertion.
- s41467-021-25257-4 description "Related publication of the modelling presented in the Jupyter notebook" assertion.
- s41467-021-25257-4 description "Related publication of the modelling presented in the Jupyter notebook" assertion.
- s41467-021-25257-4 description "Related publication of the modelling presented in the Jupyter notebook" assertion.
- s41467-021-25257-4 description "Related publication of the modelling presented in the Jupyter notebook" assertion.
- zenodo.12612126 description "Contains outputs, (results), generated in the Jupyter notebook of Sea ice forecasting using the IceNet library" assertion.
- notebook.html description "Rendered version of the Jupyter Notebook hosted by the Environmental Data Science Book" assertion.
- 9293f0f7-aad2-422f-ad6f-dfa4c87f5bd2 description "DDD" assertion.
- e2e015af-b02c-45c4-ae8a-f74271a813f0 description "In this presentation, we will share firsthand experiences and insights gained from navigating the EOSC (European Open Science Cloud), offering a glimpse into how EOSC influences our day-to-day work and how it has become an invaluable ally for our team. We belong to the Nordic e-Infrastructure Collaboration on Earth System Modeling Tools (NICEST), a small community composed of researchers, Research Software Engineers (RSEs), and engineers from Norway, Sweden, Finland, Denmark, and Estonia, working in different organisations such as national meteorological services, national compute/storage infrastructure providers and support services, universities and other research institutes either working directly on climate or supporting related activities. The NICEST community strengthens the Nordic position in climate modelling by addressing e-infra challenges, leveraging Earth System Models (ESMs) to understand climate processes, adapt to global change, and mitigate impacts. Our presentation extends beyond the technical aspects, offering a narrative of collaborative discovery that illustrates how EOSC has transformed into an indispensable companion, enabling our team to embody the principles of FAIR (Findable, Accessible, Interoperable, and Reusable) and open science. Throughout the session, we will highlight the operational intricacies of frameworks like EOSC, emphasising our nuanced approach to leveraging these frameworks for maximum impact. This personal narrative is not just about success stories; it explores the challenges we've faced and the lessons we've learned. We place a special emphasis on our evolving understanding of effectively exploiting specific EOSC services, transforming it into more than just infrastructure but a trusted friend in our professional lives. As we reflect on our collaborative journey, we'll share stories of triumphs, challenges, and the unique bond that has developed between our team and those contributing to EOSC's development. We'll explain how we've moved from being mere users to active contributors, contributing to the deployment of our own services to serve our community. Today, our aim is to actively participate in the construction of EOSC, demonstrating that through collaboration and co-design, we can significantly contribute to its ongoing evolution. This collaboration becomes truly effective when each actor recognizes the value of others, enabling us to pool efforts and enhance efficiency together." assertion.
- a0355564-8fc4-4be4-8e96-32d30e589b14 description "Oral presentation given at EGU DOI: [10.5194/egusphere-egu24-3441](https://doi.org/10.5194/egusphere-egu24-3441)" assertion.
- egusphere-egu24-3441 description "Abstract corresponding to the talk at EGU 2024." assertion.
- a57b6bcf-b4da-4dc3-b76e-5fed2bd180b5 description "This Research Object corresponds to the work done by Vanessa Stoeckl, and presented as a poster at EGU 2024, ESSI 2.9 "Seamless transitioning between HPC and cloud in support of Earth Observation, Earth Modeling and community-driven Geoscience approach PANGEO". - Abstract submitted and accepted at EGU: [https://doi.org/10.5194/egusphere-egu24-8343](https://doi.org/10.5194/egusphere-egu24-8343) - [Rendered Jupyter notebook](https://edsbook.org/notebooks/gallery/ac327c3a-5264-40a2-8c6e-1e8d7c4b37ef/notebook) - [Galaxy workflow showcasing the pipeline for forecasting sea ice](https://usegalaxy.eu/u/vstoeckl/w/icenet)" assertion.
- 0ad18b1b-594f-4b8a-937e-eff3460be9dd description "Poster EGU 2024 (pdf) Implementation of a reproducible pipeline for producing seasonal Arctic sea ice forecasts" assertion.
- 56186f37-5280-40bd-9fae-a5e7c64045a5 description "Galaxy workflow on the Galaxy Europe instance. To execute it, you would need first to get an account on Galaxy Europe (free of charge) and prepare the input dataset." assertion.
- 5695c875-13f3-40a5-8030-5f8679e5c9ce description "Galaxy is a mature, browser accessible workbench for scientific computing. It enables scientists to share, analyze and visualize their own data, with minimal technical impediments. A thriving global community continues to use, maintain and contribute to the project, with support from multiple national infrastructure providers that enable freely accessible analysis and training services. The Galaxy Training Network supports free, self-directed, virtual training with >230 integrated tutorials. Project engagement metrics have continued to grow over the last 2 years, including source code contributions, publications, software packages wrapped as tools, registered users and their daily analysis jobs, and new independent specialized servers. Key Galaxy technical developments include an improved user interface for launching large-scale analyses with many files, interactive tools for exploratory data analysis, and a complete suite of machine learning tools. Important scientific developments enabled by Galaxy include Vertebrate Genome Project (VGP) assembly workflows and global SARS-CoV-2 collaborations." assertion.
- 8105ba74-e8df-435b-ac88-b2fb762365a5 description "Sketch used in RoHub to illustrate the Research Object created for the poster at EGU 2024." assertion.
- f54b976b-60d3-408e-afcd-15e8a2b449a3 description "EGU abstract submitted." assertion.
- f5649f15-3910-4588-a57d-d7b5b156fa46 description "Research Object with the Jupyter Notebook showcasing Sea ice forecasting in the Environmental Data Science book [https://edsbook.org/welcome.html](https://edsbook.org/welcome.html)" assertion.
- UHI-create-movie.html description "This HTML page shows the HTML webpage of the rendered Jupyter notebook." assertion.
- d640e4aa-63bf-4aac-b579-eef79f1da470 description "The term "Urban Heat Island" (UHI) effect describes the phenomenon where urban environments exhibit higher air temperatures than their rural counterparts, a difference that is especially pronounced at night. This effect arises from the greater capacity of urban materials and man-made structures, such as buildings and pavements, to absorb, store, and then re-radiate heat compared to natural landscapes. First identified over two centuries ago, the UHI effect is subject of research to understand, measure, and mitigate its impacts on society, economic activities, and public health. Although traditionally the prerogative of specialists, the UHI is also attracting increasing interest among citizens. However, not all have the necessary technical expertise or infrastructure access to source relevant data (from in-situ measurements, satellite remote sensing, or numerical models), process it efficiently, synthesize it and interpret the changes over time or between different locations. The UHI-Stream tool was specifically developed to bridge this gap and quickly analyze temperature differences between two points anywhere on Earth's by leveraging EGI compute and storage resources (owned by CESNET) and ERA5-Land reanalysis data (available from 1950, as part of the Copernicus Climate Change Service). The corresponding hourly 2m air temperatures are streamed from S3 buckets, processed on-the-fly and visualized as annual heat-maps or animations spanning user-defined time-frames. Conveniently hosted on [RoHub][1] as a FAIR (Findable, Accessible, Interoperable, and Reusable) Executable Research Object, UHI-Stream is expected to be further converted into a Galaxy tool with a Graphical User Interface as part of the [EuroScienceGateway][2] project, potentially incorporating additional features to help users pinpoint representative urban and adjacent rural areas, or account for more grid cells. In summary, UHI-Stream is poised to become a valuable asset in urban climatology studies, enabling easier identification of UHI patterns and estimating climate impacts on a regional scale. The tool’s versatility in analyzing any two geographic points enhances its usefulness beyond the mere urban-rural context, allowing for comparative analyses of temperature changes across diverse locales, regardless of their relationship. [1]: http://www.rohub.org [2]: http://www.eurosciencegateway.eu" assertion.
- UHI-create-movie.ipynb description "Jupyter notebook to create a movie highlight the Urban heat island effect over a defined period of time." assertion.
- icenet description "Galaxy workflow on the Galaxy Europe instance. To execute it, you would need first to get an account on Galaxy Europe (free of charge) and prepare the input dataset." assertion.
- aab53e25-a351-46b0-bcfe-a0e0bf02f881 description "This Research Object corresponds to the work done by Vanessa Stoeckl, and presented as a poster at EGU 2024, ESSI 2.9 "Seamless transitioning between HPC and cloud in support of Earth Observation, Earth Modeling and community-driven Geoscience approach PANGEO". - Abstract submitted and accepted at EGU: [https://doi.org/10.5194/egusphere-egu24-8343](https://doi.org/10.5194/egusphere-egu24-8343) - [Rendered Jupyter notebook](https://edsbook.org/notebooks/gallery/ac327c3a-5264-40a2-8c6e-1e8d7c4b37ef/notebook) - [Galaxy workflow showcasing the pipeline for forecasting sea ice](https://usegalaxy.eu/u/vstoeckl/w/icenet)" assertion.
- 7e5193b9-8057-4afc-8dab-cfc472a3c4da description "Poster EGU 2024 (pdf) Implementation of a reproducible pipeline for producing seasonal Arctic sea ice forecasts" assertion.
- dd867f4d-c2e3-43a2-9e3c-5ba3d5148423 description "Sketch used in RoHub to illustrate the Research Object created for the poster at EGU 2024." assertion.
- ac327c3a-5264-40a2-8c6e-1e8d7c4b37ef description "Research Object with the Jupyter Notebook showcasing Sea ice forecasting in the Environmental Data Science book [https://edsbook.org/welcome.html](https://edsbook.org/welcome.html)" assertion.
- ac327c3a-5264-40a2-8c6e-1e8d7c4b37ef description "Research Object demonstrating sea ice forecasting using IceNet. The corresponding Jupyter Notebook has been published in the Environmental Data Science book." assertion.
- egusphere-egu24-8343 description "EGU abstract submitted." assertion.
- bf1e9074-2d18-42dc-9545-c016c4d0d1b4 description "This Research Object contains the developments made during the *B-Cubed Hackathon Project 7 - Interoperable eLTER Standard Observation variables for Biosphere*, including the data cubes and the code used to generate them." assertion.
- 083e5bf4-19f5-4e2a-8e27-f9b7f1f59abc description "This Research Object (RO) encapsulates the various steps leading to the creation of an "Urban Heat Island" (UHI) effect analysis tool tailored for the UseGalaxy.eu platform, providing an accessible resource for a broad user base, including urban planners, educators, and community members. The tool predominantly utilizes ERA5-Land meteorological reanalysis data to characterize and quantify the UHI effect. It will also offer a straightforward approach to examining the temperature contrasts between urban and surrounding rural areas either defined by the user or delineated with the aid of satellite data to ensure accurate urban-rural categorization. The interactive interface of the tool is meant to ensure that users can engage with the data without requiring specialized technical knowledge, promoting environmental awareness and facilitating informed decision-making for heat mitigation and urban planning. The objective of the integration with the UseGalaxy.eu platform is to democratize access to complex climate data, fostering community-driven solutions to combat the challenges posed by increased urban temperatures." assertion.
- eb4f3c60-ac13-4d3e-9a82-b42b9731ba76 description "This edition of Luke Howard’s CLIMATE OF LONDON has been produced by the International Association for Urban Climate (IAUC). While Howard is best known for his work on clouds1, he was also the first to recognise the effect that urban areas have on local climate. Much of his studies on climate, including his description, analysis and observations, are contained in CLIMATE , which can reasonably claim to be the first textbook on climatology. Despite its title, the majority of the book is concerned with weather and climate in general, rather than that of the London metropolis. Nevertheless, one hundred and thirty-two years after its publication, Chandler dedicated his study of the spatial and temporal character of metropolis’ climate to Luke Howard (1772-1864), whom he described as the pioneer of urban climatic studies." assertion.
- 2205.00319 description "There is a consensus that Urban Heat Island phenomenon - UHI occurs in every large city. This effect is characterized by higher air temperatures in cities than in the neighboring countryside at night. However, to date, there has been no systematic study on the Fortaleza case, the Brazil’s 5th largest city. By the comparison between screen-level air temperature measured by two automatic weather stations, one located in the city and the other in the neighboring region, this work shows the occurrence of the UHI in Fortaleza, even during the rainy season. In an attempt to find some effect of the UHI on precipitation (and vice-versa), historical series of air temperature and precipitation were analyzed from 1966 to 2020. Besides the considerable increase in air temperature over the years, a slight downward trend was observed in UHI and even more in precipitation between the hours of 15:00 and 21:00 (local time). However, one believes that these trends may be related to climate change at large scale rather than an urban scale." assertion.
- c9b5992d-96dd-4c62-b4d2-824a7023cd69 description "vre ro creation" assertion.
- 89b65fa5-bf53-47e3-9f5d-fd87fb575e67 description "Aims of the study to investigate the effect of the experimental synthesis route on the fundametal properties such as band gap, optical properties , surface area and catalytic activity. We synthesis the PCN nano composite having narrow band gap in the range 0f 1.6-2.0 eV to increase the solar to hydrogen effeciency. We will investigat its catalytic activty using densty funtional theory, to understand the mehcanism for the ehnacining the catalytic activity due to formation nanocomposite having typeII heterostrcuture." assertion.
- 3946 description "" assertion.
- 3946 description "" assertion.
- 3946 description "" assertion.
- 3946 description "" assertion.
- 707 description "" assertion.
- 3382037 description "An approach to reproducibility problems related to porting software across machines and compilers." assertion.
- 3491418.3530773 description "ABSTRACT The Slingshot interconnect designed by HPE/Cray is becoming more relevant in High-Performance Computing with its deployment on the upcoming exascale systems. In particular, it is the interconnect empowering the first exascale and highest-ranked supercomputer in the world, Frontier. It offers various features such as adaptive routing, congestion control, and isolated workloads. The deployment of newer interconnects raises questions about performance, scalability, and any potential bottlenecks as they are a critical element contributing to the scalability across nodes on these systems. In this paper, we will delve into the challenges the slingshot interconnect poses with current state-of-the-art MPI libraries. In particular, we look at the scalability performance when using slingshot across nodes. We present a comprehensive evaluation using various MPI and communication libraries including Cray MPICH, OpenMPI + UCX, RCCL, and MVAPICH2-GDR on GPUs on the Spock system, an early access cluster deployed with Slingshot and AMD MI100 GPUs, to emulate the Frontier system." assertion.
- 364-CHEP23_Paper_ApptainerWithoutSuid.pdf description "Abstract. Apptainer (formerly known as Singularity) since its beginning implemented many of its container features with the assistance of a setuid- root program. It still supports that mode, but as of version 1.1.0 it no longer uses setuid by default. This is feasible because it now can mount squashfs filesystems, ext3 filesystems, and overlay filesystems using unprivileged user namespaces and FUSE. It also now enables unprivileged users to build containers, even without requiring system administrators to configure /etc/subuid and /etc/subgid unlike other “rootless” container systems. As a result, all the unprivileged functions can be used nested inside of another container, even if the container runtime prevents any elevated privileges. As of version 1.2.0 Apptainer also supports completely unprivileged encryption of Singularity Image Format (SIF) container files. Performance with a particularly challenging HEP benchmark using the FUSE-based mounts both with and without encryption is essentially identical to the previous methods that required elevated privileges to use the Linux kernel-based counterparts." assertion.
- 8950983 description "Abstract: Linux Containers with the build-once run-anywhere principle have gained huge attention in the research community where portability and reproducibility are key concerns. Unlike virtual machines (VMs), containers run the underlying host OS kernel. The container filesystem can include all necessary non-default prerequisites to run the container application at unaltered performance. For this reason, containers are popular in HPC for use with parallel/MPI applications. Some use cases include also abstraction layers, e.g. MPI applications require matching of MPI version between the host and the container, and/or GPU applications require the underlying GPU drivers to be installed within the container filesystem. In short, containers can only abstract what is above the OS kernel, not below. Consequently, portability is not completely granted. This paper presents the experience of PRACE (Partnership for Advanced Computer in Europe) in supporting Singularity containers on HPC clusters and provides notes about possible approaches for deploying MPI applications using different use cases. Performance comparison between bare metal and container executions are also provided, showing a negligible overhead in the container execution." assertion.
- 18d3e6d6-e559-4e43-b464-55275bd14708 description "The modern form of containers, as popularly known through platforms like Docker, Singularity (Apptainer), Podman or Charliecloud, to cite only a few, began to take shape over a decade ago. However, the fundamentals behind containerization and actual benefits of containers in scientific computing remain largely unclear to a vast majority of users. In fact, there is a significant gap between simplistic “Hello world” examples found online and real scientific applications. Rumors suggest that achieving satisfactory performance on supercomputers across multiple nodes is impossible. This paper describes how to leverage the full potential of high-speed networks for inter-node communications with UCX on Fram and Betzy (from the Norwegian national e-infrastructure). It is then explained how to achieve near-native performance on Lumi (EuroHPC's flagship) despite its OFI "Slingshot-11" interconnect and proprietary library. Results obtained in the standard OSU Micro-benchmarks tests for latency and bandwidth and with a fully-fledged climate model show that containerized applications work just as well as "bare-metal", are portable and provide bit-for-bit reproducibility on different platforms. Containers are therefore highly recommended to minimize deployment and porting issues \emph{i)} for AaaS (Applications as a Service) coming with all the necessary software environment (rather than source code only); and \emph{ii)} so that HPC users do not have to rely on anybody to install what they need and can be operational within minutes whilst still getting top performance." assertion.
- 2a331f93-aaee-4f14-9bf3-c948f17984cc description "SLURM job scripts to submit jobs on HPCs." assertion.
- 4c848863-603c-4770-b1bf-6e114989e353 description "Container definition files for singularity/Apptainer." assertion.
- 729a11e7-acf6-4077-a795-5e0aeddb0cc3 description "Folder containing links to scientific papers and other articles used in the study." assertion.
- 7af73f42-db2d-4e99-aa71-724a6a24571e description "Source code for various tools used in this work" assertion.
- 99a2489a-1e9b-4dd3-b5d4-9c980646e3ff description "This folder contains output results." assertion.
- b23734da-8819-4c44-acf3-4fbb1fdfd153 description "Singularity image files to ensure reproducibility." assertion.
- 19c117d8-f89a-45f1-b806-1fff50ecb341 description "Slurm script to submit the OSU latency and get bandwidth tests with 2 CPUs on different nodes on Lumi using an OpenSUSE/Leap:15.3 container with MPIch4.1.2 and the OFI libraries from the host" assertion.
- 1eb356c5-3da5-4cdf-9741-fba770dcfcb8 description "This is a definition file to illustrate the use of an OpenSUSE/Leap:15.3 container with MPIch4.1.2 and support for UCX1.15.0 to run the OSU Micro-benchmarks tests version 7.3" assertion.
- 2e6ebb01-29ee-4f4f-a04e-56f83a599457 description "OSU Micro-Benchmarks 7.3 (10/30/23)" assertion.
- 3bdd74d8-8a9e-4e98-ba93-4706c0043767 description "OpenSUSE/Leap:15.3 container with MPIch4.1.2 and support for UCX1.15.0 to run the OSU Micro-benchmarks tests version 7.3" assertion.
- 6065fe78-4974-49bb-a6a2-4794b0298591 description "Slurm script to submit the OSU latency and get bandwidth tests with 2 CPUs on one node on Fram with the simple container" assertion.
- 657d591a-4b33-4376-aec9-b77704073916 description "Container with all the necessary packages to run CESM2.3 with the OFI-swap method" assertion.
- 75d28143-2135-440a-a508-c69f12d95a30 description "Slurm script to submit the OSU latency and get bandwidth tests with 2 CPUs on different nodes on Lumi using an OpenSUSE/Leap:15.3 container with MPIch4.1.2 and the OFI libraries from the host" assertion.
- 90b63535-65b3-4995-853a-ab66503a35ae description "OpenSUSE/Leap:15.3 container with MPIch4.1.2 and a "dummy" LibFabric1.14.0 to run the OSU Micro-benchmarks tests version 7.3" assertion.
- 9bab6133-709a-49e5-b65e-7139f18b83c1 description "Singularity Image file for a simple Ubuntu:22.04 container with MPIch4.1.2 to run the OSU Micro-benchmarks tests version 7.3" assertion.
- a1a3a1f8-9aa6-448e-8850-0319a40714a9 description "Slurm script to submit the a job on Lumi using the OpenSUSE/Leap:15.3 container with MPIch4.1.2 (like to prepare or run an ESM) and swapping the OFI for that on the host at runtime" assertion.
- b017e3bf-41c0-475e-b7cb-36a46365c3f0 description "This is a definition file to illustrate the use of an OpenSUSE/Leap:15.3 container with MPIch4.1.2 and support for UCX1.15.0 to run the OSU Micro-benchmarks tests version 7.3" assertion.
- b7d5827c-4355-4238-ad95-e290abaaeedc description "This is a definition file to illustrate the use of an OpenSUSE/Leap:15.3 container with MPIch4.1.2 and a "dummy" LibFabric1.14.0 to run CESM2.3 version alpha 17" assertion.
- b936509e-4dbc-47c1-bd80-079472aa4f70 description "This is a definition file to illustrate the use of a simple Ubuntu:22.04 container with MPIch4.1.2 to run the OSU Micro-benchmarks tests version 7.3" assertion.
- d54fca0c-130d-4615-8369-e8fe25c72bad description "This is a definition file to illustrate the use of an OpenSUSE/Leap:15.3 container with MPIch4.1.2 and a "dummy" LibFabric1.14.0 to run the OSU Micro-benchmarks tests version 7.3" assertion.
- f3125bda-90a3-47e9-98c1-c6a8169d71a6 description "Community Earth System Model version 2.3 Alpha 17" assertion.
- f5381fbc-627d-4ce3-8aaa-ff727850ec93 description "OpenSUSE/Leap:15.3 container with MPIch4.1.2 and a "dummy" LibFabric1.14.0 to run the OSU Micro-benchmarks tests version 7.3" assertion.
- fe49c3cd-6077-4f11-be47-7b0b946c0069 description "Slurm script to submit the OSU latency and get bandwidth tests with 2 CPUs on different nodes on Fram with the simple container" assertion.