A use case from the National data Service (NDS)

21 Oct 2014
Groups audience: 

I noticed that NDS has a sample use case and it seemed like it would be of interest to keep as part of our collection of use cases.  Note that it cites among other things the Virtual Observatory which was also discussed at our BoF session at P4.  Robert Pennington of NCAS may have more to say about this exampe since NDS is under him...

 

Using the NDS: a scenario

To illustrate how a scientist might use the NDS, consider this scenario for astronomy research which we think can be realized:

In 2021 the LIGO gravitational wave observatory detects a strong “transient” burst event with an unknown source; an alert is issued. Across the US, physicists and astronomers (who have never worked directly together) engage NDS discovery services to find relevant data from other instruments, leading them to detections from the IceCube neutrino observatory, further isolating the originating portion of the sky. NDS discovery services connect the researchers to the federated discovery tools of the Virtual Observatory to collect data by sky position from large surveys like DES and LSST to look for electromagnetic precursors. Through literature searches, they find publications describing characteristics of similar detections; recent publications and an arXiv preprint supporting NDS data linking lead them to the data underlying the analyses. They use NDS data transfer services to migrate previous detection data as well as simulation data held at the Blue Waters supercomputing system, containing previously unpublished neutrino emission predictions, to DataScope, a specialized computing platform to compare observations with theoretical models. From this analysis, a crucial insight suggests a new class of stellar object. Using NDS transfer tools, they pull together the LIGO data, corresponding IceCube detections, image cutouts from LSST, and analyses of simulation data into their private space in an NDS repository. NDS metadata generation tools help them organize a new collection. Soon, a paper is submitted to a journal, including identifiers for the new data collection. Once the paper is accepted, the NDS data collection is sent to a campus archive for longer-term curated management. With the new publication, readers have direct access to the underlying data, enabling them to verify and extend the results. Results and data are further available to educators, who bring the discovery to a broad audience by updating astronomy e-textbooks.

  • Keith Jeffery's picture

    Author: Keith Jeffery

    Date: 22 Oct, 2014

    Here’s one from EPOS
    [cid:***@***.***]
    Keith G Jeffery Consultants
    Prof Keith G Jeffery
    E: ***@***.***
    T: +44 7768 446088
    S: keithgjeffery
    Past President ERCIM www.ercim.eu (***@***.***)
    Past President euroCRIS www.eurocris.org
    Past Vice President VLDB www.vldb.org
    Fellow (CITP, CEng) BCS www.bcs.org
    Co-chair RDA MIG https://rd-alliance.org/internal-groups/metadata-ig.html
    Co-chair RDA MSDWG https://rd-alliance.org/working-groups/metadata-standards-directory-work...
    Co-chair RDA DICIG https://rd-alliance.org/internal-groups/data-context-ig.html
    ----------------------------------------------------------------------------------------------------------------------------------
    The contents of this email are sent in confidence for the use of the
    intended recipient only. If you are not one of the intended
    recipients do not take action on it or show it to anyone else, but
    return this email to the sender and delete your copy of it.
    ----------------------------------------------------------------------------------------------------------------------------------
    - Show quoted text -From: gbergcross=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of Gary
    Sent: 21 October 2014 22:21
    To: Data Fabric IG
    Subject: [rda-datafabric-ig] A use case from the National data Service (NDS)
    I noticed that NDS has a sample use case and it seemed like it would be of interest to keep as part of our collection of use cases. Note that it cites among other things the Virtual Observatory which was also discussed at our BoF session at P4. Robert Pennington of NCAS may have more to say about this exampe since NDS is under him...
    Using the NDS: a scenario
    To illustrate how a scientist might use the NDS, consider this scenario for astronomy research which we think can be realized:
    In 2021 the LIGO gravitational wave observatory detects a strong “transient” burst event with an unknown source; an alert is issued. Across the US, physicists and astronomers (who have never worked directly together) engage NDS discovery services to find relevant data from other instruments, leading them to detections from the IceCube neutrino observatory, further isolating the originating portion of the sky. NDS discovery services connect the researchers to the federated discovery tools of the Virtual Observatory to collect data by sky position from large surveys like DES and LSST to look for electromagnetic precursors. Through literature searches, they find publications describing characteristics of similar detections; recent publications and an arXiv preprint supporting NDS data linking lead them to the data underlying the analyses. They use NDS data transfer services to migrate previous detection data as well as simulation data held at the Blue Waters supercomputing system, containing previously unpublished neutrino emission predictions, to DataScope, a specialized computing platform to compare observations with theoretical models. From this analysis, a crucial insight suggests a new class of stellar object. Using NDS transfer tools, they pull together the LIGO data, corresponding IceCube detections, image cutouts from LSST, and analyses of simulation data into their private space in an NDS repository. NDS metadata generation tools help them organize a new collection. Soon, a paper is submitted to a journal, including identifiers for the new data collection. Once the paper is accepted, the NDS data collection is sent to a campus archive for longer-term curated management. With the new publication, readers have direct access to the underlying data, enabling them to verify and extend the results. Results and data are further available to educators, who bring the discovery to a broad audience by updating astronomy e-textbooks.
    --
    Full post: https://rd-alliance.org/group/data-fabric-ig/post/use-case-national-data...
    Manage my subscriptions: https://rd-alliance.org/mailinglist
    Stop emails for this post: https://rd-alliance.org/mailinglist/unsubscribe/45803

  • Rob Pennington's picture

    Author: Rob Pennington

    Date: 22 Oct, 2014

    This example taken from planned work in multi-messenger astronomy. There is a small pilot project booting up to test out concepts and tools as part of this. This would be a reasonable use case to consider that has been developed separately from RDA efforts. As such, it could provide an ‘external’ datapoint when considering the datafabric.
    Rob
    From: Gary <***@***.***>
    Reply-To: "***@***.***-groups.org" <***@***.***-groups.org>
    Date: Tuesday, October 21, 2014 at 11:21 PM
    To: Data Fabric IG <***@***.***-groups.org>
    Subject: [rda-datafabric-ig] A use case from the National data Service (NDS)
    I noticed that NDS has a sample use case and it seemed like it would be of interest to keep as part of our collection of use cases. Note that it cites among other things the Virtual Observatory which was also discussed at our BoF session at P4. Robert Pennington of NCAS may have more to say about this exampe since NDS is under him...
    Using the NDS: a scenario
    To illustrate how a scientist might use the NDS, consider this scenario for astronomy research which we think can be realized:
    In 2021 the LIGO gravitational wave observatory detects a strong “transient” burst event with an unknown source; an alert is issued. Across the US, physicists and astronomers (who have never worked directly together) engage NDS discovery services to find relevant data from other instruments, leading them to detections from the IceCube neutrino observatory, further isolating the originating portion of the sky. NDS discovery services connect the researchers to the federated discovery tools of the Virtual Observatory to collect data by sky position from large surveys like DES and LSST to look for electromagnetic precursors. Through literature searches, they find publications describing characteristics of similar detections; recent publications and an arXiv preprint supporting NDS data linking! lead them to the data underlying the analyses. They use NDS data transfer services to migrate previous detection data as well as simulation data held at the Blue Waters supercomputing system, containing previously unpublished neutrino emission predictions, to DataScope, a specialized computing platform to compare observations with theoretical models. From this analysis, a crucial insight suggests a new class of stellar object. Using NDS transfer tools, they pull together the LIGO data, corresponding IceCube detections, image cutouts from LSST, and analyses of simulation data into their private space in an NDS repository. NDS metadata generation tools help them organize a new collection. Soon, a paper is submitted to a journal, including identifiers for the new data collection. Once the paper is accepted, the NDS data collection is sent to a campus archive for longer-term curated management. With the new publication, readers have direct access to the underlying data, enabling them! to verify and extend the results. Results and data are further available to educators, who bring the discovery to a broad audience by updating astronomy e-textbooks.
    --
    Full post: https://rd-alliance.org/group/data-fabric-ig/post/use-case-national-data...
    Manage my subscriptions: https://rd-alliance.org/mailinglist
    Stop emails for this post: https://rd-alliance.org/mailinglist/unsubscribe/45803

  • Peter Wittenburg's picture

    Author: Peter Wittenburg

    Date: 22 Oct, 2014

    What we need for these use cases is kind of 2 page descriptions of the essentials with respect to our DF discussion.
    Best
    peter
    From: rlpennin=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of robpncsa
    Sent: Wednesday, October 22, 2014 12:05 PM
    To: ***@***.***-groups.org
    Cc: Pennington, Robert Lee
    Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
    This example taken from planned work in multi-messenger astronomy. There is a small pilot project booting up to test out concepts and tools as part of this. This would be a reasonable use case to consider that has been developed separately from RDA efforts. As such, it could provide an 'external' datapoint when considering the datafabric.
    Rob
    From: Gary <***@***.***>
    Reply-To: "***@***.***-groups.org" <***@***.***-groups.org>
    Date: Tuesday, October 21, 2014 at 11:21 PM
    To: Data Fabric IG <***@***.***-groups.org>
    Subject: [rda-datafabric-ig] A use case from the National data Service (NDS)
    I noticed that NDS has a sample use case and it seemed like it would be of interest to keep as part of our collection of use cases. Note that it cites among other things the Virtual Observatory which was also discussed at our BoF session at P4. Robert Pennington of NCAS may have more to say about this exampe since NDS is under him...
    Using the NDS: a scenario
    To illustrate how a scientist might use the NDS, consider this scenario for astronomy research which we think can be realized:
    In 2021 the LIGO gravitational wave observatory detects a strong "transient" burst event with an unknown source; an alert is issued. Across the US, physicists and astronomers (who have never worked directly together) engage NDS discovery services to find relevant data from other instruments, leading them to detections from the IceCube neutrino observatory, further isolating the originating portion of the sky. NDS discovery services connect the researchers to the federated discovery tools of the Virtual Observatory to collect data by sky position from large surveys like DES and LSST to look for electromagnetic precursors. Through literature searches, they find publications describing characteristics of similar detections; recent publications and an arXiv preprint supporting NDS data linking! lead them to the data underlying the analyses. They use NDS data transfer services to migrate previous detection data as well as simulation data held at the Blue Waters supercomputing system, containing previously unpublished neutrino emission predictions, to DataScope, a specialized computing platform to compare observations with theoretical models. From this analysis, a crucial insight suggests a new class of stellar object. Using NDS transfer tools, they pull together the LIGO data, corresponding IceCube detections, image cutouts from LSST, and analyses of simulation data into their private space in an NDS repository. NDS metadata generation tools help them organize a new collection. Soon, a paper is submitted to a journal, including identifiers for the new data collection. Once the paper is accepted, the NDS data collection is sent to a campus archive for longer-term curated management. With the new publication, readers have direct access to the underlying data, enabling them! to verify and extend the results. Results and data are further available to educators, who bring the discovery to a broad audience by updating astronomy e-textbooks.
    --
    Full post: https://rd-alliance.org/group/data-fabric-ig/post/use-case-national-data...
    Manage my subscriptions: https://rd-alliance.org/mailinglist
    Stop emails for this post: https://rd-alliance.org/mailinglist/unsubscribe/45803

  • Francoise Genova's picture

    Author: Francoise Genova

    Date: 22 Oct, 2014

    I can tell more about the virtual observatory (the disciplinary interoperability layer), and on how we astronomers deal with information found in published papers if useful.
    Cheers
    Francoise
    Envoyé depuis mon appareil Android.
    -----Original Message-----
    From: Peter Wittenburg
    <***@***.***>
    To: "***@***.***-groups.org" <***@***.***-groups.org>
    Sent: mer, 22 oct. 2014 14:06
    Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
    What we need for these use cases is kind of 2 page descriptions of the essentials with respect to our DF discussion.
    Best
    peter
    From: rlpennin=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of robpncsa
    Sent: Wednesday, October 22, 2014 12:05 PM
    To: ***@***.***-groups.org
    Cc: Pennington, Robert Lee
    Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
    This example taken from planned work in multi-messenger astronomy. There is a small pilot project booting up to test out concepts and tools as part of this. This would be a reasonable use case to consider that has been developed separately from RDA efforts. As such, it could provide an 'external' datapoint when considering the datafabric.
    Rob
    From: Gary <***@***.***>
    Reply-To: "***@***.***-groups.org" <***@***.***-groups.org>
    Date: Tuesday, October 21, 2014 at 11:21 PM
    To: Data Fabric IG <***@***.***-groups.org>
    Subject: [rda-datafabric-ig] A use case from the National data Service (NDS)
    I noticed that NDS has a sample use case and it seemed like it would be of interest to keep as part of our collection of use cases. Note that it cites among other things the Virtual Observatory which was also discussed at our BoF session at P4. Robert Pennington of NCAS may have more to say about this exampe since NDS is under him...
    Using the NDS: a scenario
    To illustrate how a scientist might use the NDS, consider this scenario for astronomy research which we think can be realized:
    In 2021 the LIGO gravitational wave observatory detects a strong "transient" burst event with an unknown source; an alert is issued. Across the US, physicists and astronomers (who have never worked directly together) engage NDS discovery services to find relevant data from other instruments, leading them to detections from the IceCube neutrino observatory, further isolating the originating portion of the sky. NDS discovery services connect the researchers to the federated discovery tools of the Virtual Observatory to collect data by sky position from large surveys like DES and LSST to look for electromagnetic precursors. Through literature searches, they find publications describing characteristics of similar detections; recent publications and an arXiv preprint supporting NDS data linking! lead them to the data underlying the analyses. They use NDS data transfer services to migrate previous detection data as well as simulation data held at the Blue Waters supercomputing system,
    containing previously unpublished neutrino emission predictions, to DataScope, a specialized computing platform to compare observations with theoretical models. From this analysis, a crucial insight suggests a new class of stellar object. Using NDS transfer tools, they pull together the LIGO data, corresponding IceCube detections, image cutouts from LSST, and analyses of simulation data into their private space in an NDS repository. NDS metadata generation tools help them organize a new collection. Soon, a paper is submitted to a journal, including identifiers for the new data collection. Once the paper is accepted, the NDS data collection is sent to a campus archive for longer-term curated management. With the new publication, readers have direct access to the underlying data, enabling them! to verify and extend the results. Results and data are further available to educators, who bring the discovery to a broad audience by updating astronomy e-textbooks.
    --
    Full post: https://rd-alliance.org/group/data-fabric-ig/post/use-case-national-data...
    Manage my subscriptions: https://rd-alliance.org/mailinglist
    Stop emails for this post: https://rd-alliance.org/mailinglist/unsubscribe/45803

  • Peter Wittenburg's picture

    Author: Peter Wittenburg

    Date: 22 Oct, 2014

    Yes would be very good to have short notes that help us how the "Data Fabrics" in your realm are currently implemented.
    Peter
    - Show quoted text -From: francoise.genova=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of Francoise Genova
    Sent: Wednesday, October 22, 2014 2:16 PM
    To: ***@***.***-groups.org
    Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
    I can tell more about the virtual observatory (the disciplinary interoperability layer), and on how we astronomers deal with information found in published papers if useful.
    Cheers
    Francoise
    Envoyé depuis mon appareil Android.
    -----Original Message-----
    From: Peter Wittenburg
    <***@***.***>
    To: "***@***.***-groups.org" <***@***.***-groups.org>
    Sent: mer, 22 oct. 2014 14:06
    Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
    What we need for these use cases is kind of 2 page descriptions of the essentials with respect to our DF discussion.
    Best
    peter
    From: rlpennin=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of robpncsa
    Sent: Wednesday, October 22, 2014 12:05 PM
    To: ***@***.***-groups.org
    Cc: Pennington, Robert Lee
    Subject: Re: [rda-datafabric-ig] A use case from the National data Service (NDS)
    This example taken from planned work in multi-messenger astronomy. There is a small pilot project booting up to test out concepts and tools as part of this. This would be a reasonable use case to consider that has been developed separately from RDA efforts. As such, it could provide an ‘external’ datapoint when considering the datafabric.
    Rob
    From: Gary <***@***.***>
    Reply-To: "***@***.***-groups.org" <***@***.***-groups.org>
    Date: Tuesday, October 21, 2014 at 11:21 PM
    To: Data Fabric IG <***@***.***-groups.org>
    Subject: [rda-datafabric-ig] A use case from the National data Service (NDS)
    I noticed that NDS has a sample use case and it seemed like it would be of interest to keep as part of our collection of use cases. Note that it cites among other things the Virtual Observatory which was also discussed at our BoF session at P4. Robert Pennington of NCAS may have more to say about this exampe since NDS is under him...
    Using the NDS: a scenario
    To illustrate how a scientist might use the NDS, consider this scenario for astronomy research which we think can be realized:
    In 2021 the LIGO gravitational wave observatory detects a strong “transient” burst event with an unknown source; an alert is issued. Across the US, physicists and astronomers (who have never worked directly together) engage NDS discovery services to find relevant data from other instruments, leading them to detections from the IceCube neutrino observatory, further isolating the originating portion of the sky. NDS discovery services connect the researchers to the federated discovery tools of the Virtual Observatory to collect data by sky position from large surveys like DES and LSST to look for electromagnetic precursors. Through literature searches, they find publications describing characteristics of similar detections; recent publications and an arXiv preprint supporting NDS data linking! lead them to the data underlying the analyses. They use NDS data transfer services to migrate previous detection data as well as simulation data held at the Blue Waters supercomputing system, containing previously unpublished neutrino emission predictions, to DataScope, a specialized computing platform to compare observations with theoretical models. From this analysis, a crucial insight suggests a new class of stellar object. Using NDS transfer tools, they pull together the LIGO data, corresponding IceCube detections, image cutouts from LSST, and analyses of simulation data into their private space in an NDS repository. NDS metadata generation tools help them organize a new collection. Soon, a paper is submitted to a journal, including identifiers for the new data collection. Once the paper is accepted, the NDS data collection is sent to a campus archive for longer-term curated management. With the new publication, readers have direct access to the underlying data, enabling them! to verify and extend the results. Results and data are further available to educators, who bring the discovery to a broad audience by updating astronomy e-textbooks.
    --
    Full post: https://rd-alliance.org/group/data-fabric-ig/post/use-case-national-data...
    Manage my subscriptions: https://rd-alliance.org/mailinglist
    Stop emails for this post: https://rd-alliance.org/mailinglist/unsubscribe/45803

submit a comment