The goal of this session is to discuss the architecture, workflows, and challenges faced in serving global climate research community, experienced by the Earth System Grid Federation (ESGF).
The session will cover a top-down view of the ESGF collaboration discussing:
- The WCRP/WGCM mission, goals, and overview
- The ESGF’s organization with a detailed discussion of the CMIP6 project and upcoming challenges
- An ESGF working group and that is building a platform for providing high-performance data replication and transfers
With about 20 minutes of discussion reserved at the end to hear from the audience.
ESGF is a coordinated multi-agency, international collaboration of institutions that continually develop, deploy, and maintain software that facilitates and empowers the study of climate change. Through ESGF, users access, analyze, and visualize data using a globally federated collection of networks, computers, and software.
ESGF is a leader in climate data discovery and knowledge integration. Its distributed and federated architecture is based on the exploiting multiple peer‐to‐peer nodes (or sites) that are geographically distributed around the world.
However, in addition to this federated climate data architecture, one of the main issues that became apparent during the CMIP5 project was the data interchange between the core archives for backups and enabling distributed data access to the community. To solve this issue, a handful of international research and education networks have been working with climate data centers in Australia, Germany, the Netherlands, the UK, and the US to develop and provide a replication workflow for climate data repositories and obtaining reasonable data transfer performance between each. This working group, the International Climate Network Working Group (ICNWG) within ESGF, works to develop and provide a workflow for climate data repositories to obtain reasonable data transfer performance. That is, the community is looking to achieve consistent site-to-site data transfer rates ranging from 1 petabyte a month to 1 petabyte a week by 2016.
The network connections made and/or improved through the ICNWG are helping climate and computational scientists manage and disseminate petabytes of modeling and observational data, which will traverse more than 13,000 miles of networks—the most cost-effective and reliable method of transporting data.
Contact person: Mary Hester
- Dr. V. Balaji, Princeton University and the National Oceanic and Atmospheric Administration, Geophysical Fluid Dynamics Laboratory (NOAA/GFDL) (Chair)
- Sébastien Denvil, Institut Pierre Simon Laplace (IPSL)
- Guido Aben, Australia’s Academic and Research Network (AARNet)
All slides for the sessions are attached below.