June 13, 2025
Infrastructure for Non-Traditional Research Outputs and Processes (Infra4NTROs) Statement of Work
Files
Name
Date Added
Comments
You must be logged in or join the group to leave a comment.
What are you looking for?
July 2, 2025 at 1:26 pm
Andrew S. Hoffman says:
...but I think your comment suggests that it's not as clear as we had hoped, so I'll take this up with Jenny and Robin and try to do a better job drawing this out (assuming they agree with any of what I've said below 😉
July 2, 2025 at 1:26 pm
Andrew S. Hoffman says:
Thanks for your constructive feedback, Beth, and for picking the thread up already, Jenny.
@Beth: I think you’ve touched on a really important element of this conversation in terms of where ‘data’ ends and ‘output’ begins (or vice versa) – which has implications in a whole bunch of areas: for researchers’ own practices, for the role that different data/scholcomm infrastructures can play here, and I think it even raises questions about which professional groups have ‘jurisdiction’ when it comes to supporting researchers working with these ‘non-traditional’ modalities and processes.
A bit of a proviso for what follows: I am but one of three (prospective) co-Chairs of this IG, and so I speak only for myself here. Most of my conversations in this space are with folks in the multimodal/artistic/practice-oriented research space, and my personal interest (following theirs) is with *non-textual* data/outputs, which are not exhausted by the more general ‘non-traditional’ qualifier, so…
If we imagine that research objects sit on some spectrum, with tabular formats on one end, non-textual modalities at the opposite end, and a whole bunch of other stuff somewhere in the middle (interview transcripts are an example that most immediately comes to mind), then I think the closer we get to the non-textual end of the spectrum, the blurrier the boundary becomes between ‘data’ and ‘output’ – both for ‘us’ (the RDA people) but also for the researchers. And it’s not just a matter of semantics, but of relationality.
An example: Say a project produces several artifacts/objects, two of which are a film and a peer-reviewed article. The film is shown as part of a curated festival, while the paper is published in a journal. Both the film and the paper make certain claims about the world, albeit using different modalities, possibly targeting different audiences. Now, in one way, the film and the paper are self-standing research outputs – finished products that have undergone some amount of ‘peer review’ and then circulate through their respective audience-communities; in another way, the paper may make arguments that are drawn from the contents of the film, which may include interviews with folks, or provide some other evidence for the claims made in the paper. Thinking with Sabina Leonelli’s ‘relational’ approach to data, the point is not that there is some essential difference between the film and the paper in terms of which is ‘data’ and which not; but, in this example, the film can *become* data when it is mobilized elsewhere (e.g., in the paper) to make some (other) claim.
And here’s Sabina in her own words: ‘I define ‘data’ as a relational category applied to research outputs that are taken, at specific moments of inquiry, to provide evidence for knowledge claims of interest to the researchers involved. Data thus consist of a specific way of expressing and presenting information, which is produced and/or incorporated in research practices so as to be available as a source of evidence, and whose scientific significance depends on the situation in which it is used. In this view, data do not have truth-value in and of themselves, nor can they be seen as straightforward representations of given phenomena. Rather, data are essentially fungible objects, which are defined by their portability and their prospective usefulness as evidence.’ (see p. 811, https://doi.org/10.1086/684083)
I do not think that it is, or should be, the remit of the Infra4NTROs Interest Group to legislate the question of what are ‘data’ and what are ‘outputs’, nor do I think it would serve us well to only focus only on one or the other (which would also require us to police some non-existent boundary between the two.)
Rather, as I hope our SoW makes clear, our objective is to create this shared space – ideally one inhabited not just by us, the RDA nerds, but also (at least sometimes) researchers themselves, who have active-but-often-unmet storage/archiving/publishing needs – where we can collectively explore how to evolve whichever piece of infrastructure we identify such that it can better meet those needs beyond its current capabilities. In the end, I think we’re still asking the same questions about knowledge representation, persistence, storage/archiving, connectivity/referencing, reuse, etc. regardless of whether you call a film (or whatever) ‘data’ or ‘output.’
July 10, 2025 at 1:26 pm
Beth Knazook says:
I love your response Andrew. It definitely hits on the nuance that I hope is somewhat reflected in my suggested edit to Jenny just now. Sabina's quote is also fantastic! I think it's extremely valuable to assert that all these outputs can also be data, on a continuum perhaps, able to reach potential audiences unanticipated by their primary publication or presentation method. It'll be really interesting if you can also develop this conversation with practitioners who also actively choose not to see their outputs that way, but who might benefit from practices associated with good data management and data sharing created for their use cases and materials. You might crossover with communities focused on self-archiving as well, as the end result of a lot of creative practice is ideally an well-managed artist's archive.
July 2, 2025 at 1:26 pm
Nick says:
Part of the issue with NTROs is how to assign academic credit to them. We wrote anarticle about this for linguistics: Nick Thieberger, Anna Margetts, Stephen Morey, Simon Musgrave. 2016. Assessing annotated corpora as research output. Australian Journal of Linguistics. Vol 36: 1, 1-21
Available here: https://nthieberger.net/ALSAssessingCollectionsArticle.pdf
July 2, 2025 at 1:26 pm
Jenny Evans says:
Hi Nick,
Yes contributorship is a huge part of this work - thanks for the reference - we can add that to the resources. We have referenced this further down - however take the point that this should be highlighted further up in the introduction section.
Thanks,
Jenny
July 1, 2025 at 1:26 pm
Jenny Evans says:
Hi Beth,
Thanks for making the time to comment - it is much appreciated!
We definitely see this to include 'data' - so perhaps we need to make this clearer? I do think that there is work to do to make the 'research data' space more inclusive as well as simply expecting communities creating e.g. performance to see these as part of 'data' practice? I am also very mindful of us needing to bring in discipline representatives so that we are not speaking 'about' them rather than engaging with them. I know we don't necessarily talk about 'data' when we speak to researchers in these disciplines.
I've added some suggested text below in italics - do you think that this works - or does it need a clearer 1-2 sentences on its own?
Non-Traditional Research Outputs (often referred to as NTROs) are those which do not take the form of published books, book chapters, journal articles or conference publications[1] – examples include film, photography, sound recordings, performances, gallery exhibitions, and other composite multimedia outputs which can also be usefully viewed through the lens of research data and its associated practices. Often the primary means of communication in the creative, visual and performing arts, NTROs are created across a wide range of other disciplines as well (including in the social sciences, humanities, natural sciences, and engineering), which bring their own languages for talking about and describing these materials, and their own cultures of storing, sharing, and reusing them. NTROs and their associated processes, narratives, contexts; and the relationships between their components, are not well described by existing standards based open infrastructures, which reflect a research rewards and recognition landscape that assume a sole, often textual based output and possibly an underlying dataset.
And is there an argument to call the group 'Infrastructure for Non-Traditional Research Outputs, Processes and Data'? Which is getting a bit unwieldy - but worth thinking about. Or is making sure the definition is right enough?
Many thanks,
Jenny
July 10, 2025 at 1:26 pm
Beth Knazook says:
Apologies for the delayed response Jenny!
I think your addition is good one but I would push it a little further and say that not only can these outputs be viewed as similar to research data, but that they can be the actual data in some circumstances and would benefit from a robust community of practice around that potential. (I also think your sentence is stronger if you focus on what the outputs are versus what they aren't, but I understand if you would prefer to leave that in.)
Non-Traditional Research Outputs (often referred to as NTROs) may include film, photography, sound recordings, performances, gallery exhibitions, and other composite multimedia outputs which can be usefully viewed as reusable research data, in specific contexts, and benefit from its associated practices.
I have to admit that the addition of Processes and Data to the title is really attractive! Processes particularly, if only because I think it draws attention to an under-researched benefit and indicator of the impact of open science actually opening the way we do research, not just the results we get from it, and the creative fields have a lot to teach us about reflection on process. NTROPD is a terribly acronym though. I won't insist you saddle yourselves with that.
July 11, 2025 at 1:26 pm
Jenny Evans says:
Hi Beth,
Sorry - I've only just seen this - thanks this is also really helpful. We will have a think about how we can re-work the title and definition - as this is all really important. Tbh I really dislike starting with the 'non' - what this isn't - it's more that that seems to have enabled engagement - but I am sure there is a way to re-work this so that it's not too unwieldy and acronym and it also makes sure process and data is clearly referenced.
Thanks,
Jenny
June 26, 2025 at 1:26 pm
Beth Knazook says:
Really nice to see this group taking shape! I think the Statement of Work is very comprehensive and clearly carves out a niche not currently served by RDA despite strong ties to other Interest and Working Groups addressing related data types or the complexity of citation, recognition, and characterization of non-textual outputs.
I empathize with the challenge of choosing language that does not unfairly 'other' this work through the group title, 'Non-Traditional Research Outputs.' It makes sense why this is the approach for now, and there's potential to maybe evolve into another name over time. But if I could make a small request for further clarification on the description of what these non-traditional objects are, I'd like to better understand if you are viewing this group as exclusively about outputs as analysed (or interpreted) end products (equivalents to publications, commentary, etc.) or if you're also looking at these through the lens of data. It's often a challenge in the Humanities to get researchers to see some of the products you reference, such as performances and sound recordings, as part of a data practice and potential the raw materials for secondary analyses or creative works. I think it would be helpful to that discussion to situate the work on NTROs as inclusive of data outputs.