RDA Brokering Governance Case Statement

03 Jun 2014

RDA Brokering Governance Case Statement

To address the Governance of the brokering framework middleware and interconnect existing international e-infrastructures. The Working Group will address the following:
1. Brokering configuration and strategies;
2. Brokering governance and agreements;
3. Publications and transparency;
4. Community adoption and sustainability.
Value Proposition
Middleware significantly simplifies distributed system construction, as well as providing a much more efficient means of integrating legacy systems with new technology. Brokering middleware provides mediation and transformation services to simplify data discovery, evaluation, access, and use. Brokering was conceived to work as a third-party tier in a three-tier architecture (extending the Client-Server paradigm). This introduces the clear need to govern and manage the new brokering tier. This is particularly important when brokering middleware is used to interconnect existing large e-infrastructures in a way that is sustainable. Given the brokering middleware’s importance in the integration of disparate information and data systems, continued access to and availability to these middleware components is vital to supporting long-term development and continued use of integrated data systems. It is important also as brokering should be transparent to most users, thus lacking high-level visibility of its impacts.
Effective middleware governance has the potential to support longer-term development under a variety of funding models, to simplify and standardize access models, and establish a basis for the continued value of brokered systems. It is not, however, clear what the best practices for this governance are, and how those practices shift in response to different funding and property models, under different architectures or as standards change. To ensure sustainable, stable development and effectiveness in an operational environment of brokering systems reliant on middleware service architectures, an effective model for the governance and reuse of that middleware must be agreed upon.
We propose to consider and recommend a set of best practices for governing and managing brokering middleware. These practices will work to ensure future interoperability, access, and use to brokering middleware independent or in light of various development and funding models to support long-term planning of brokered, integrated systems. These will be of value not only to interoperability architects and to developers (who can plan integrated systems assuming the continued use and support of brokering middleware) but also to system managers and end users. The potential for scaling and expansion of integrated data resources and systems in brokering middleware is of value to increasingly interdisciplinary research work as well as in managing growing big data sets.
The expected outcomes of the Brokering Governance WG will be:

  • A Position Paper including guidelines, best practices and recommendations for the management and support of an international brokering capability to interconnect existing and future multidisciplinary data systems and infrastructures. This will include a consensus recommendation of a path for adoption of this capability at the international level.
  • A set of use cases to assess options and best practices for governance recommendations in three diverse areas:

o Global Changes: GEO-BON;
o Environmental sciences: European Commission Danube SDI;
o International repositories: ICSU WDS.
Engagement with existing work in the area
The working group will engage with significant international programmes, including:

  • NSF BCUBE project;
  • GEO GEOSS and in particular its Infrastructure Implementation Board (IIB);
  • European Commission GEOWOW project;
  • Transatlantic ODIP project;

These projects are part of the brokering technology development, which has been going on for the last decade. They have addressed the areas of: discovery, access, semantic harmonization, and use for facilitating multidisciplinary and organizational science research (see the Useful References section).
An operational capability for discovery and access brokering: the GEO DAB (Discovery and Access Brokering framework) has been implemented by GEO (Group on Earth Observation) in the GEOSS (Global Earth Observation System of Systems). Presently, GEO DAB interconnects more than 20 diverse research infrastructures sharing more than 65 Millions of resources. This experience showed the technical scalability of brokering, as far as resources and service performances.
Operational capability in GEOSS works on a GEO unique governance model of country contributions. While this works, the growth and sustainability of international broker service needs to address a broader and open governance modality, which offers a strategic and practical process to sustainability and operations.


Download full document


TAB Review: https://rd-alliance.org/groups/rda-technical-advisory-board-tab/wiki/bro...

  • Mark Parsons's picture

    Author: Mark Parsons

    Date: 02 Jul, 2014


    I would like to comment briefly on this, not as Secretary General, but simply as someone who was heavily involved in brokering in the past. I also participated in a panel discussion on this topic last week at the EarthCube all-hands meeting.

    First off, I think this is a very important topic and I like where the group is going. It is great to see they are already thinking about sustainability.

    My first comment is that I think the broader community is still unclear on what brokering really is. This case statement describes some of that and points to references, but I think it could be fleshed out a bit more. I think there are some misconceptions on what this group considers brokering, so that should be made as clear as possible. Probably excerpting some from the EarthCube roadmap would help. The white paper that some of us wrote a couple years ago on "Why EarthCube must Broker" might also be useful.

    Second, I can anticipate some of the critique from TAB and Council--the deliverables need to be more specific and implementable. Use cases and a position paper are a great start, but they are insufficient. I think it would be great (and achievable) if the group could build from their recommendations and create an actual governance model or set of processes. This model should then be adopted by three or more organisations or projects, to actualy demonstrate governance in action.

    If this last concern is addressed, I think this could be a really strong case statement and the results of the WG could have significant ramifications even beyond brokering. I look forward to this going forward.



  • Peter Wittenburg's picture

    Author: Peter Wittenburg

    Date: 07 Jul, 2014

    Overcoming or mastering the heterogeneity of data organizations and data infrastructures requires without any doubt new ways of thinking. “Brokering” between different solutions with the help of automatically supported ways is one promising way to overcome the complexity. It seems that in some projects in particular in the earth sciences this approach is being tested and seems to lead to the development of middleware as well. Two solutions can often be seen: (1) Traditionally Golden Standards were defined to which all solutions should be mapped to. (2) Brokering tries to find optimal solutions between two selected solutions and thus be more optimal for that specific mapping case. In practice such as in many large metadata aggregation initiatives the experts prefer a approach with mappings guided by pragmatic mapping rules which hardly be called ontologies.

    The question of how to deal with large heterogeneity in almost all aspects of data infrastructure building is a core topic for RDA, therefore any progress with new ideas such as with brokering would be very important. Therefore I believe this initiative is very important for RDA.

    With respect to the current Brokering Governance WG suggestion I have the following comments/questions:

    • In the case statement there is no reference to the results of the Brokering IG which would help me for example to understand the orientation and motivation for exactly this "Management" topic. The proposal refers to a number of publications, but they cannot replace a short concept note or so which would require much reading. Without this background the suggestion to work on “Brokering Governance” is surprising to me - I would need a bit orientation which may have taken place in the plenary sessions, but I could not participate in them.
    • The WG proposal is speaking about “middleware governance”. For me it is not obvious what is meant by this. In my naive view I would assume that in RDA we speak specifications of open frameworks which could help to register brokering solutions etc. What does “governance” in this context mean?
    • As the initiators indicate Brokering can be applied to many areas where we are faced with heterogeneity. I could imagine that the solutions for metadata mediating will be different from those when mediating between repository access protocols. Given the 18 months time limit it would seem to be useful to focus on a few specific areas.
    • Yet Brokering is not very much used in daily practice as far as I know. Or is it that people call the solutions we are currently using to bridge between heterogeneous service providers brokering solutions - new name for existing practice. To me it is not clear what types of solutions are meant with brokering and thus what the hurdles are which need to be overcome.
    • It seems that the initiators widely come from Geo-science. It would be good to have other people on board, since finally all communities are facing this heterogeneity. Is there a special reason for this restriction? ODIP and WDS motivation to participate is not clear.

    I would be hally if the initiators could provide some more information to understand which problems they will tackle.



  • Siri Jodha Khalsa's picture

    Author: Siri Jodha Khalsa

    Date: 08 Jul, 2014

    The case statement needs clarification regarding what activities would be carried within the context of the use cases and international programes, and how these activities would inform the creation of a govenance model that would offer "a strategic and practical process to sustainability and operations."  For example, would these programs be experimenting with large-scale community support of brokering middleware? Just establishing additional instances of brokering services, without a plan for experimenting with modes of sustainability and evolution, will be of limited value in mapping possiible solutions for middleware governance.

  • Beth Plale's picture

    Author: Beth Plale

    Date: 08 Jul, 2014

    Speaking as an individual who was involved in discussions of the proposed WG at P3, the notion of governance in a broker based environment emerged in the discussions as reasonable and important.  While the case statement doesn't get at the kinds of governance (but should), governance could take the form of rules for replication of data across brokers, commitments of accurate representation of the data it brokers, commitments to longevity/refresh of data a broker brokers.  In an international setting, the commitment could respect country agreements.    But this is only to suggest a level of detail that would help clarify current uncertainty expressed by commenters.  I further agree with sjskhalsa's comment wondering how context of use cases and international setting will inform the creation of a governance model. 

  • Simon Cox's picture

    Author: Simon Cox

    Date: 10 Jul, 2014

    I am encouraged to see the issue of governance brought up, but also somewhat concerned that it has taken so long. If our infrastructures come to rely on services provided by 3rd parties to support interoperability, then sustainability becomes critical. The broker is the key domain-specific element of the infrastructure in this architecture. Infrastructure is notoriously difficult to attract funding for - everyone needs it, so everyone assumes it is ('surely') someone else's job! And everyone notices when it stops working. 

    The other issue is scalability. I'm not convinced that succesfully brokering 20 infrastructures is sufficient evidence that brokering is scalable. True brokering would be O(N^2) (from each possible provider to each possible consumer). Would be interesting to see some metrics on whether adding the 21st and 22nd involve progressively more or less work. And what about 100? 

    I see brokering as a pragmatic way to bootstrap standardization, but not necessarily as long term infrastructure. 

  • Mark Parsons's picture

    Author: Mark Parsons

    Date: 21 Jul, 2014

    Note: the community comment period for this Case Statement has ended, The group plans to revise the CS before TAB reviews it.

  • Mark Parsons's picture

    Author: Mark Parsons

    Date: 12 Aug, 2014

    The group has revised the document in light of public comments. The revised document is attached above.

submit a comment