Architecture or Not?

Hi DFIG,

Sorry for the long post! If I had the time I would have crafted this as a whitepaper...

<tl;dr version>Let's get on the same page for future DFIG discussions</tl;dr version>

I agree with what Jane said about components, and indeed I would go even further and suggest we might benefit from adopting existing standardized terminology in our discussion. Let me start by sketching out the big picture as I see it, using terms that are widely adopted outside of RDA:

We are discussing how to build a system, [1] which is a collection of interactng elements organized to achieve RDA's purpose of "building social and technical solutions that help people share data across technologies, disciplines, and countries to address the grand challenges of society." [2]  Those interacting elements in RDA's case include technology/software components, data, people, governments, organizations like RDA, EUDAT, Google, etc. Crucially, systems are not just technology -- they have a human component.

The best way technical people have found to collaborate in large groups to build systems is to get together and disucss and argue about the system's architecture. An architecture is the fundamental organization of a system embodied in its components and their relationships to each other (i.e. interfaces, protocols, etc). [1] Over the decades some best practices have emerged and the art of building large, complex systems has grown into a mature field of engineering called systems engineering. A very important subset of that field is software systems engineering. At an even higher level people have agreed on bigger structures like a "system of systems" and recognized fields like Enterprise Architecture, but in our case in think RDA is dealing with a system so we would benefit from systems engineering.

Think of an architecture as a communication tool to help us reduce confusion and collaborate coherently as we conduct the business of RDA. It should not be normative or prescriptive and this is not about telling people what to do. Just because a home builder advertises a blueprint for a house doesn't mean people are required to use it, but many families hoping to build a house might find it really helpful.

Perhaps the most important elements in an architecture are its components. [1] Following the home building analogy, the components are things like lumber, nails, drywall, roof shingles, windows, doors, sinks, refrigerators, etc. In the RDA context, components are things like data type registries. Another key concept is the idea of a framework, a reusable design that can be refined (specialized) and extended to provide some
portion of the overall functionality of many applications. [1]  For home building this is like when the builder decides to use 2x4 and 2x6 wood boards, and 4x8 sheets of plywood, or 110 volt 60 Hz AC electrical wiring and lights and refrigerators that use 110 volt 60 Hz power. This framework would be useful in the United States, but Europeans might use a different framework even if they were building a house with the same architecture as their American colleagues.

So, some important questions for RDA:

1. RDA has decided it is not a standards development organization (SDO). Does it see itself as an architect, as a home builder, as a hardware store like Home Depot, or as a loose collection of groups like lumber companies and refrigerator makers (or a combination of the above)?
2. The TAB has been tasked with managing the WGs and maintaining the RDA strategic roadmap. Does this role include deciding which components are missing from the big picture and identifying needs? Does it include the difficult technical work of harmonizing the interfaces between components so we don't build a refrigerator that runs on 220 volt 50 Hz power but install outlets that provide 110 volts at 60 Hz?

When I first heard about DFIG I was excited because I hoped it would step up to be the organizing influence that I feel RDA has been lacking so far. Don't take that as a criticsm of the TAB; the job is huge and I don't think the TAB can't do it alone. If the DFIG can define the big picture (i.e. the architecture), the TAB can use that detailed technical definition to make sound decisions about which WGs should be recognized and endorsed. At this point, however, I am not even sure what DFIG is discussing exactly, partly because of the Tower of Babel caused by different people using the same words in very different ways.

What I said about systems engineering at the DFIG P4 kickoff meeting in Amsterdam was that RDA/DFIG can benefit from the enormous body of expertise that the systems engineering community has built up over the decades. Just as an example, hundreds of person-years of work have gone into developing standard terminology for systems engineering and software engineering. And then even more work went into rationalizing the two fields, resulting in a three-way international standard (ISO + IEC + IEEE). And that's just terminology -- not counting procedures, methodologies, diagramming tools, etc! If we all learn just a tiny amount of this jargon I think our conversations will go more smoothly, and as a bonus the rest of the world will understand what we are saying.

By the way, I am not a systems engineer and I have no special expertise here. I am just a physicist cross-dressing as a data person at the moment, so don't take what I say too seriously :)

--John Henry

REFERENCES

from Systems and Software Engineering -- Vocabulary, ISO/IEC/IEEE 24765-2010(E), also see the online term database at http://pascal.computer.org/sev_display/index.action

[1]

3.2970
system
1. combination of interacting elements organized to achieve one or more stated purposes.

3.150
architecture
1. fundamental organization of a system embodied in its components, their relationships to each other, and to
the environment, and the principles guiding its design and evolution.

3.503
component
1. an entity with discrete structure, such as an assembly or software module, within a system considered at a
particular level of analysis

3.1197
framework
1. a reusable design (models and/or code) that can be refined (specialized) and extended to provide some
portion of the overall functionality of many applications.

[2] RDA main web page

John Henry –
A really useful post. I agree with your concept, your analogy (house-building architecture, fabrics by continent) and also I look to DF to provide an architecture in the ‘blueprint’ sense i.e. a reference model not necessarily wholly or partially implemented.
Other engineering disciplines (think automobiles, aircraft, construction, electrical, plumbing…) use standard components (plugged together like LEGO) and standard interfaces. This is because of cost reduction and reliability improvement. Hopefully the Computer Science / IT community will stop ‘hand-knitting’ and take on these concepts more widely.
Personally I believe RDA should move in the area of recommendations (not standards) for components…
Best
Keith
Keith G Jeffery Consultants
Prof Keith G Jeffery
E: ***@***.***
T: +44 7768 446088
S: keithgjeffery
Past President ERCIM www.ercim.eu (***@***.***)
Past President euroCRIS www.eurocris.org
Past Vice President VLDB www.vldb.org
Fellow (CITP, CEng) BCS www.bcs.org
Co-chair RDA MIG https://rd-alliance.org/internal-groups/metadata-ig.html
Co-chair RDA MSDWG https://rd-alliance.org/working-groups/metadata-standards-directory-work...
Co-chair RDA DICIG https://rd-alliance.org/internal-groups/data-context-ig.html
----------------------------------------------------------------------------------------------------------------------------------
The contents of this email are sent in confidence for the use of the
intended recipient only. If you are not one of the intended
recipients do not take action on it or show it to anyone else, but
return this email to the sender and delete your copy of it.
----------------------------------------------------------------------------------------------------------------------------------
- Show quoted text -From: johnhenry.scott=***@***.***-groups.org [mailto:***@***.***-groups.org] On Behalf Of js0b
Sent: 29 October 2014 14:26
To: Data Fabric IG
Subject: Re: [rda-datafabric-ig] Architecture or Not?
Hi DFIG,
Sorry for the long post! If I had the time I would have crafted this as a whitepaper...
Let's get on the same page for future DFIG discussions
I agree with what Jane said about components, and indeed I would go even further and suggest we might benefit from adopting existing standardized terminology in our discussion. Let me start by sketching out the big picture as I see it, using terms that are widely adopted outside of RDA:
We are discussing how to build a system, [1] which is a collection of interactng elements organized to achieve RDA's purpose of "building social and technical solutions that help people share data across technologies, disciplines, and countries to address the grand challenges of society." [2] Those interacting elements in RDA's case include technology/software components, data, people, governments, organizations like RDA, EUDAT, Google, etc. Crucially, systems are not just technology -- they have a human component.
The best way technical people have found to collaborate in large groups to build systems is to get together and disucss and argue about the system's architecture. An architecture is the fundamental organization of a system embodied in its components and their relationships to each other (i.e. interfaces, protocols, etc). [1] Over the decades some best practices have emerged and the art of building large, complex systems has grown into a mature field of engineering called systems engineering. A very important subset of that field is software systems engineering. At an even higher level people have agreed on bigger structures like a "system of systems" and recognized fields like Enterprise Architecture, but in our case in think RDA is dealing with a system so we would benefit from systems engineering.
Think of an architecture as a communication tool to help us reduce confusion and collaborate coherently as we conduct the business of RDA. It should not be normative or prescriptive and this is not about telling people what to do. Just because a home builder advertises a blueprint for a house doesn't mean people are required to use it, but many families hoping to build a house might find it really helpful.
Perhaps the most important elements in an architecture are its components. [1] Following the home building analogy, the components are things like lumber, nails, drywall, roof shingles, windows, doors, sinks, refrigerators, etc. In the RDA context, components are things like data type registries. Another key concept is the idea of a framework, a reusable design that can be refined (specialized) and extended to provide some
portion of the overall functionality of many applications. [1] For home building this is like when the builder decides to use 2x4 and 2x6 wood boards, and 4x8 sheets of plywood, or 110 volt 60 Hz AC electrical wiring and lights and refrigerators that use 110 volt 60 Hz power. This framework would be useful in the United States, but Europeans might use a different framework even if they were building a house with the same architecture as their American colleagues.
So, some important questions for RDA:
1. RDA has decided it is not a standards development organization (SDO). Does it see itself as an architect, as a home builder, as a hardware store like Home Depot, or as a loose collection of groups like lumber companies and refrigerator makers (or a combination of the above)?
2. The TAB has been tasked with managing the WGs and maintaining the RDA strategic roadmap. Does this role include deciding which components are missing from the big picture and identifying needs? Does it include the difficult technical work of harmonizing the interfaces between components so we don't build a refrigerator that runs on 220 volt 50 Hz power but install outlets that provide 110 volts at 60 Hz?
When I first heard about DFIG I was excited because I hoped it would step up to be the organizing influence that I feel RDA has been lacking so far. Don't take that as a criticsm of the TAB; the job is huge and I don't think the TAB can't do it alone. If the DFIG can define the big picture (i.e. the architecture), the TAB can use that detailed technical definition to make sound decisions about which WGs should be recognized and endorsed. At this point, however, I am not even sure what DFIG is discussing exactly, partly because of the Tower of Babel caused by different people using the same words in very different ways.
What I said about systems engineering at the DFIG P4 kickoff meeting in Amsterdam was that RDA/DFIG can benefit from the enormous body of expertise that the systems engineering community has built up over the decades. Just as an example, hundreds of person-years of work have gone into developing standard terminology for systems engineering and software engineering. And then even more work went into rationalizing the two fields, resulting in a three-way international standard (ISO + IEC + IEEE). And that's just terminology -- not counting procedures, methodologies, diagramming tools, etc! If we all learn just a tiny amount of this jargon I think our conversations will go more smoothly, and as a bonus the rest of the world will understand what we are saying.
By the way, I am not a systems engineer and I have no special expertise here. I am just a physicist cross-dressing as a data person at the moment, so don't take what I say too seriously :)
--John Henry
REFERENCES
from Systems and Software Engineering -- Vocabulary, ISO/IEC/IEEE 24765-2010(E), also see the online term database at http://pascal.computer.org/sev_display/index.action
[1]
3.2970
system
1. combination of interacting elements organized to achieve one or more stated purposes.
3.150
architecture
1. fundamental organization of a system embodied in its components, their relationships to each other, and to
the environment, and the principles guiding its design and evolution.
3.503
component
1. an entity with discrete structure, such as an assembly or software module, within a system considered at a
particular level of analysis
3.1197
framework
1. a reusable design (models and/or code) that can be refined (specialized) and extended to provide some
portion of the overall functionality of many applications.
[2] RDA main web page
--
Full post: https://www.rd-alliance.org/group/data-fabric-ig/post/architecture-or-no...
Manage my subscriptions: https://www.rd-alliance.org/mailinglist
Stop emails for this post: https://www.rd-alliance.org/mailinglist/unsubscribe/46202

John
Long post aside, I think the systems approach you’re suggesting is an excellent way to think about data fabric questions and issues. I also think the questions you’ve raised for RDA and TAB are quite good, but I would hope that these questions will be addressed by the TAB or RDA management rather than having a parallel discussion going on in the Data Fabric Interest Group.
Alan

Hi Alan,
I understand what you are saying, and I take your post as a valid opinion about how RDA should proceed. Myself, I'm neutral on that question. If RDA decides to leave these questions to the TAB, I would not object, but in that case I think they need to get on with it and publish a skeleton architecture we can work with. Also (in that case) DFIG needs to define its role within the TAB's architecture and articulate much more clearly what RDA means by a "data fabric" using language we all understand.
--John Henry
- Show quoted text -From: arblatecky=***@***.***-groups.org <***@***.***-groups.org> on behalf of Alan Blatecky <***@***.***>
Sent: Wednesday, October 29, 2014 10:38 AM
To: ***@***.***-groups.org
Subject: Re: [rda-datafabric-ig] Architecture or Not?
John
Long post aside, I think the systems approach you’re suggesting is an excellent way to think about data fabric questions and issues. I also think the questions you’ve raised for RDA and TAB are quite good, but I would hope that these questions will be addressed by the TAB or RDA management rather than having a parallel discussion going on in the Data Fabric Interest Group.
Alan
On Oct 29, 2014, at 10:26 AM, js0b <***@***.***> wrote:
Hi DFIG,
Sorry for the long post! If I had the time I would have crafted this as a whitepaper...
Let's get on the same page for future DFIG discussions
I agree with what Jane said about components, and indeed I would go even further and suggest we might benefit from adopting existing standardized terminology in our discussion. Let me start by sketching out the big picture as I see it, using terms that are widely adopted outside of RDA:
We are discussing how to build a system, [1] which is a collection of interactng elements organized to achieve RDA's purpose of "building social and technical solutions that help people share data across technologies, disciplines, and countries to address the grand challenges of society." [2] Those interacting elements in RDA's case include technology/software components, data, people, governments, organizations like RDA, EUDAT, Google, etc. Crucially, systems are not just technology -- they have a human component.
The best way technical people have found to collaborate in large groups to build systems is to get together and disucss and argue about the system's architecture. An architecture is the fundamental organization of a system embodied in its components and their relationships to each other (i.e. interfaces, protocols, etc). [1] Over the decades some best practices have emerged and the art of building large, complex systems has grown into a mature field of engineering called systems engineering. A very important subset of that field is software systems engineering. At an even higher level people have agreed on bigger structures like a "system of systems" and recognized fields like Enterprise Architecture, but in our case in think RDA is dealing with a system so we would benefit from systems engineering.
Think of an architecture as a communication tool to help us reduce confusion and collaborate coherently as we conduct the business of RDA. It should not be normative or prescriptive and this is not about telling people what to do. Just because a home builder advertises a blueprint for a house doesn't mean people are required to use it, but many families hoping to build a house might find it really helpful.
Perhaps the most important elements in an architecture are its components. [1] Following the home building analogy, the components are things like lumber, nails, drywall, roof shingles, windows, doors, sinks, refrigerators, etc. In the RDA context, components are things like data type registries. Another key concept is the idea of a framework, a reusable design that can be refined (specialized) and extended to provide some
portion of the overall functionality of many applications. [1] For home building this is like when the builder decides to use 2x4 and 2x6 wood boards, and 4x8 sheets of plywood, or 110 volt 60 Hz AC electrical wiring and lights and refrigerators that use 110 volt 60 Hz power. This framework would be useful in the United States, but Europeans might use a different framework even if they were building a house with the same architecture as their American colleagues.
So, some important questions for RDA:
1. RDA has decided it is not a standards development organization (SDO). Does it see itself as an architect, as a home builder, as a hardware store like Home Depot, or as a loose collection of groups like lumber companies and refrigerator makers (or a combination of the above)?
2. The TAB has been tasked with managing the WGs and maintaining the RDA strategic roadmap. Does this role include deciding which components are missing from the big picture and identifying needs? Does it include the difficult technical work of harmonizing the interfaces between components so we don't build a refrigerator that runs on 220 volt 50 Hz power but install outlets that provide 110 volts at 60 Hz?
When I first heard about DFIG I was excited because I hoped it would step up to be the organizing influence that I feel RDA has been lacking so far. Don't take that as a criticsm of the TAB; the job is huge and I don't think the TAB can't do it alone. If the DFIG can define the big picture (i.e. the architecture), the TAB can use that detailed technical definition to make sound decisions about which WGs should be recognized and endorsed. At this point, however, I am not even sure what DFIG is discussing exactly, partly because of the Tower of Babel caused by different people using the same words in very different ways.
What I said about systems engineering at the DFIG P4 kickoff meeting in Amsterdam was that RDA/DFIG can benefit from the enormous body of expertise that the systems engineering community has built up over the decades. Just as an example, hundreds of person-years of work have gone into developing standard terminology for systems engineering and software engineering. And then even more work went into rationalizing the two fields, resulting in a three-way international standard (ISO + IEC + IEEE). And that's just terminology -- not counting procedures, methodologies, diagramming tools, etc! If we all learn just a tiny amount of this jargon I think our conversations will go more smoothly, and as a bonus the rest of the world will understand what we are saying.
By the way, I am not a systems engineer and I have no special expertise here. I am just a physicist cross-dressing as a data person at the moment, so don't take what I say too seriously :)
--John Henry
REFERENCES
from Systems and Software Engineering -- Vocabulary, ISO/IEC/IEEE 24765-2010(E), also see the online term database at http://pascal.computer.org/sev_display/index.action
[1]
3.2970
system
1. combination of interacting elements organized to achieve one or more stated purposes.
3.150
architecture
1. fundamental organization of a system embodied in its components, their relationships to each other, and to
the environment, and the principles guiding its design and evolution.
3.503
component
1. an entity with discrete structure, such as an assembly or software module, within a system considered at a
particular level of analysis
3.1197
framework
1. a reusable design (models and/or code) that can be refined (specialized) and extended to provide some
portion of the overall functionality of many applications.
[2] RDA main web page
--
Full post: https://www.rd-alliance.org/group/data-fabric-ig/post/architecture-or-no...
Manage my subscriptions: https://www.rd-alliance.org/mailinglist
Stop emails for this post: https://www.rd-alliance.org/mailinglist/unsubscribe/46202

John
I think we’re more in agreement than it may appear. I would hope (and expect) the TAB (or RDA) to be able to respond fairly quickly to the questions, especially since a couple of members of TAB are already on the Data Fabric list. I would also hope that DF members weigh in as well — I just want to minimize coming up with different answers :)
Alan

All,
Multiple TAB members plus representative from seven RDA WGs plus Mark Parsons plus John Henry (?) and other NIST folk will be at the upcoming WG Collaboration Meeting at NIST (Nov 13-14) and I’ve put Data Fabric on the agenda for the second day, so I am hoping for some discussion there, although this is mainly a WG Collaboration meeting focused on metadata, not a DF meeting. See
https://rd-alliance.org/groups/2nd-working-group-collaboration-meeting.html
for more information.
On a substantive note — architecture is a loaded term for many of us so it would be useful to find some terms that don’t get folks concerned that one IG is trying to tell the rest of RDA what to do. One notion I have had is that the DFIG could spawn WGs that addressed more specific ‘architectures.’ Recommendations coming out of the IG (assuming IGs are permitted to create recommendations) should be fairly high level but more than motherhood and apple pie.
Larry

Hallo John,

great that you kicked this discussion off, I was waiting on your written input, since for me as non-native E-speaker it is more optimal to read carefully what is being said. Let me respond to your long, but good note (and of course I saw what the other said and also looked at Beth slides). And excellent that you had a discussion about this at the recent meeting. Hope that you will be around when we have the next discussion in Washington as Larry pointed out.

Let me first clarify in which role I will answer, since I was re-elected for the TAB as well. One part of what you stated addresses RDA etc., the other part addresses DF issues and there are certainly relationships. I will argue here as a bottom-up person (my preferred role) being interested to make data work more efficient in future, so all I am saying is just my personla view :)

TAB Issue

This relates mainly to your second question. RDA needs to remain primarily a bottom-up driven organization, so if we two would decide to start a new initiative, then it should be hard for boards etc to stop us if we can present a well-thought through idea and fullfill the formal requirements. The role of TAB is to synch, harmonize etc. where possible and of course check adherence to requirements. Beyond that as you note the role of TAB is to describe and structure the overall "landscape" of issues that RDA should be dealing about. To me it is obvious that at this moment it is too early to describe and structure this landscape in too detailed form, since we still need to learn more. Therefore DF will have an important role in driving this discussion about the "ladnscape" if we do not make big errors such as narrowing down the scope too early etc. For me there is no doubt that the crowd who is organized in the IGs and WGs is the most important driving force, the TAB, if it makes use of its role in a smart way, will help to consolidate, bring people together, etc.

Important for me is also to note that RDA covers much more than DF people, but this is already part of the answer to your DF points. When you say that system engineering has already though about a system where for example also humans are components, then I would answer that also psychology, law etc. has worked out elaborative models of "systems" that include humans. For issues such as Legal Interoperability I would rather much more look to models about human behavior coming from those disciplines than from system engineering. Since RDA covers IGs/WGs that look from different views to "humans" in the data game, "systems engineering" will have one important view, but it will not be the only one. I am repeating very frequently that RDA needs to define its own culture, if it will not fail, and part of this culture is terminology.

DF Issue

Probably I am a bit along Larry's view as so often. It is very good to know that systems engineering defined a variety of terms relevant for us and I am happy to make use of it. But let us take the term "architecture" as an example. In our software design & developing experience the term "architecture" is very much related to design and then implement a system (A house) which then will do some function for us. In EUDAT we had a 2 year long discussion when to start discussing about an overall architecture. We as people who wanted to get specific services did not want to get in endless discussions about an overall architecture too early. I think that this comes down into a never-ending debate between those who like to start with a systematic and structured approach and those who like to start with tests, demos, etc. My attitude is very clear: when you operate in a "landscape" which you need to explore better start with tests, demos etc. But I know for example that my colleagues from ENVRI followed another path, used ODP to specify their infrastructure based on the different views and created an enormous amount of specification notes etc. Probably both ways are important, but in a bottom-up initiative such as RDA where many people address the data issues from a large number of different aspects only the test/demo approach will work. This may change at a certain moment when we understand the landscape. This will probably also be the moment when industry will drop in.

So I could live with the term "architecture" as defined by systems engineering, but then we need to define for our community how we want to understand it. The way you describe it in your text would not be satisfying for me, since different interpretations are possible. A blueprint of a house already specifies where you have doors, windows, how big they are, etc. I think that in DF we want to remain at the level that we state that we will need doors and windows for certain functional reasons and what their requirements are etc. (some doors must be bigger than others etc.). I think that Keith also argued in this direction.

So if we do this kind of small exercise and say what we mean with architecture in the realm of RDA I would be ok with using the term as defined by SE. At least at this moment the definition (3.150) seems to be abstract enough. I think we as data practitioners simply do not want to get in these endless architecture discussions some of our IT people like so much. Similar holds with the other terms you are mentioning. So perhaps we should have augmented term definitions by making use of what we can get from SE. More important to me is to define now components/services (there is a dualism between the two) which we need within the DF so that working with data gets much more efficient, effective, self-explanatory, etc.

AFter having made my points let me also come back on your second questions to Beth. does DFIG have an overall organizing influence on the big picture? Yes if it understands that DF is not the whole RDA and restricts itself. Yes if it helps exploring the data landscape we are operating in, in particular by pointing to components/services that are required to make progress and making terms such as "architecture"much more concrete by doing.

Again - thanks a lot for kicking this off and sorting out the terms. Perhaps we are not so far away from each other. At least for writing the White Paper this is excellent material and discussion.

Peter