We’ve funded a range of projects to guide our roadmap for the future

Federated data access across the DRI

This project sees a survey of existing practice in direct shared (no-copy) data access between DRI facilities, assessment of gaps within this provision, and technical evaluation of a solution allowing direct shared data access between the DiRAC Memory Intensive HPC system (COSMA), the Durham Azimuth Cloud Facility (DACF) and the Somerville HPC system.

The work will touch all three pillars of the NetworkPlus, covering management of sensitive data in federation, identifying existing and emerging user stories, engaging with industrial partners, data protection, data sharing platforms using community portals, and organisational governance overseeing data management.

The no-copy data access will be key in reducing entry barriers to data sharing: access to the data will effectively be a “pull” operation, instigated by the destination, rather than a more traditional “push” operation where data are first copied from a source to destination (requiring dedicated storage at the destination to host it), before being operated on at the destination.

One key advantage is the case in which only parts of a dataset or file are required. Rather than having to transfer all of a large file across a network, only the parts of the file actually required will be pulled, depending on the technologies used.