Moving Data Fabric out of community review

20 Aug 2014

Dear Data Fabric people,
It is now the end of the community review period.
You have received some comments from Beth (reproduced for convenience
below).
Do you wish to update your charter based on these comments before
submission to TAB and Council?
Cheers,
Herman
I am a big fan the notion of data fabric into which services can plug
themselves, and am personally quite excited to see this come about.
Registries have well-known issues with staying current (e.g., failure of
UDDI and many other registry solutions that came about 5-8 years ago).
A different approach that doesn't suffer same centralized control
problems is to make the fabric more peer-to-peer like:* data fabric is
a minimalistic set of requirements by which services can plug into
(belong to) the fabric.*All services belonging to the Data Fabric must:
1. Be self-documenting -- a service contributes to the lifecycle of data
objects it handles and must keep track of the scientifically relevant
actions it performs on those data objects. The resulting log files are
periodically be sent to a provenance consolidator.
2. Track data objects through its service processing using one of the
well-known object identifier schemes
3. Identify itself as one type of service as drawn from an RDA- agreed
upon list of service types.
4. Implement an interface to a publish-subscribe system which serves as
the Data Fabric Control Plane. It is through the control plane that
the service broadcasts information about itself, and receives
information about the data fabric.
5. Periodically broadcast a heartbeat message. The message conveys the
timestamp, current status of the service, and the type of service that
the service provides. Service types are drawn from an agreed upon, and
minimal, list. The heartbeat message is conveyed on an
publish-subscribe system (such as AMQPhttp://www.amqp.org/) that every
service supports as minimal barrier to entry to belong to the fabric
Rules 3-5 constitute a very minimal Data Fabric Control Plane. The
Control Plane is a fibrous connectivity used only minimally to know what
is plugged into a data fabric and alive and what is not. It's not used
to move data or for services/users to interact. It's purely utilitarian
like the power grid: the power grid knows which homes are on the grid
and which are not; which currently have power and which do not. Why a
control plane? It does not suffer the currency problems known with
registries.
Finally, a minimal data fabric control plane has the nice feature of
allowing RDA to say things like*"The European Data Fabric went dark for
an hour today because of the volcanic eruption"*, and "*The
Atmospheric-Hydrology Data Fabric grew by 50% over the last 6 months we
speculate because of softening stances on climate change."*
--
Dr. ir. Herman Stehouwer
Rechenzentrum Garching @ Max Planck for Plasmaphysics
RDA Secretariat
***@***.*** 0031-619258815

  • Larry Lannom's picture

    Author: Larry Lannom

    Date: 20 Aug, 2014

    This is an excellent contribution that, in my view, does not require an update of the Case Statement. It is a good example of where the IG discussions could go, in some cases resulting in a WG to pursue a specific approach. I would characterize this as the Control Plane approach. I hope/believe Beth will be able to join the BOF at P4 and describe her ideas there.
    Larry

  • Keith Jeffery's picture

    Author: Keith Jeffery

    Date: 20 Aug, 2014

    Larry, all -
    I agree the case stmt does not need changing. Data fabric is a 'broad church' with many aspects. Beth proposes an interesting lightweight approach; of course there are many different approaches to provide the necessary integration and monitoring/control. I believe the discussions will be interesting, illuminating, exciting.
    Best
    Keith
    --------------------------------------------------------------------------------------------------------
    Keith G Jeffery Consultants
    Prof Keith G Jeffery
    E: ***@***.***
    T: +44 7768 446088
    S: keithgjeffery
    Past President ERCIM www.ercim.eu (***@***.***)
    Past President euroCRIS www.eurocris.org
    Past Vice President VLDB www.vldb.org
    Fellow (CITP, CEng) BCS www.bcs.org
    Co-chair RDA MIG https://rd-alliance.org/internal-groups/metadata-ig.html
    Co-chair RDA MSDWG https://rd-alliance.org/working-groups/metadata-standards-directory-work...
    Co-chair RDA DICIG https://rd-alliance.org/internal-groups/data-context-ig.html
    ----------------------------------------------------------------------------------------------------------------------------------
    The contents of this email are sent in confidence for the use of the
    intended recipient only. If you are not one of the intended
    recipients do not take action on it or show it to anyone else, but
    return this email to the sender and delete your copy of it.
    ----------------------------------------------------------------------------------------------------------------------------------
    -----Original Message-----
    From: llannom=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of llannom
    Sent: 20 August 2014 18:46
    To: ***@***.***-groups.org
    Cc: ***@***.***-alliance.org
    Subject: Re: [rda-datafabric-ig] Moving Data Fabric out of community review
    This is an excellent contribution that, in my view, does not require an update of the Case Statement. It is a good example of where the IG discussions could go, in some cases resulting in a WG to pursue a specific approach. I would characterize this as the Control Plane approach. I hope/believe Beth will be able to join the BOF at P4 and describe her ideas there.
    Larry

submit a comment