Feedback Analysis

Back to 21st Plenary

International Data Week 2023 (IDW 2023) including the Research Data Alliance 21st Plenary Meeting (RDA P21) Feedback Analysis

This page is available for download in PDF format.

This detailed analysis has been drafted by the RDA Secretariat to capture all the feedback on International Data Week 2023 (IDW 2023) and RDA 21st Plenary Meeting. The document includes considerations and actions, where appropriate, to ensure that the investment of the event participants and the community in providing valuable feedback is acted upon, where feasible.

Feedback was collected by the RDA from four main sources: the official IDW 2023 participant survey feedback, the RDA community members (via email and verbally), the RDA Technical Advisory Board (TAB), and the RDA Secretariat self-analysis.  The feedback will be used by the RDA Plenary / IDW organisers (including PC – programme committee and OC – organising committee) to identify successes, as well as decisions, considerations, and actions to improve future events. 

1. Participants Feedback Analysis

IDW 2023 was attended by 702 in-person and 132 online participants. The IDW 2023 survey was opened on 26/10/2023 and promoted to all participants. After 21 days the survey was closed and a total of 167 attendees responded. Approximately 20% of the total attendees responded, indicating that the majority of respondents were first-time, in-person IDW 2023 attendees.

2. “Why attend IDW 2023?” webinar

The reason for not attending “Why attend IDW2023?” webinar  held on 11 October?

• 52% were not interested
• 46% were not available at scheduled time 
• 7% watched the recording after

Relevant written feedback: 

• I expected more details on the IDW2023 logistics, how it will be structured, not how to register
• Webinar did not cover topics regarding the presentations, instead very common info. 

Consideration (s): The webinar’s target and focus were unclear for some attendees. The agenda included topics that targeted registered and not registered attendees and content should have been clearer.
IDW PC Action (s): Consider scheduling multiple webinars to propose different information during the planning stages to target the right audiences. Focus more on the content, not logistics (e.g., registration).

 

How did you find the virtual attendee engagement possibilities?

3.1. In-person attendees’ interaction with Whova: 

• Did not find Whova useful as networked with in-person attendees
• Interacting with remote attendees was impossible, difficult for in-person during sessions
• Adding in a virtual element is distracting from being in person
• There is so much to do in-person that it is easy to choose to only engage with in-person people

Consideration (s): In-person attendees did not interact with remote attendees through Whova due to focusing on face-to-face networking at the venue. 

3.2. Positive feedback on interaction through Whova:

• Whova was actually pretty useful
• Attending through Whova was a seamless experience
• Great platform for making new networks and reconnecting
• It’s great that there were meet-ups in Whova for virtual attendees.
• Having a single space for everyone to interact made the remote attendee experience more interactive and real-time.
• Engagement through Whova in our breakout session was terrific. We had a dedicated chair for the virtual attendees, and the platform supported their participation really well.

Consideration (s): overall positive experience with Whova interaction functionalities and opportunity to socialise for remote audiences.

3.3. Not-so-positive feedback on interaction through Whova:

• Would be nice to have more thoughts on how networking sessions could work.
• Whova is overly busy.  Whova also sends notifications too late to be useful.
• Impossible to keep up with all that happened on Whova. 
• Virtual attendees should be invited in the discussion groups
• Please do not allow virtual presenters. Technology worked and the presenters tried really hard, but it just doesn’t work. 
• In the sessions, online participants were not visible, so it was like they weren’t even there.
• The networking sessions were a bit too short
• Whova app is a disaster of UX. It was hard to find out about sessions let alone anything else.
• The inclusion of remote participants was suboptimal – they were more viewers than participants.
• There was no place to engage online after talks 

Consideration (s): 

• Not enough networking sessions, Whova is not easy to follow for some, poor opportunities to network for virtual attendees during breakouts.
• Networking sessions are organised but consideration should be given to the fact that online participants are joining from different time zones (40% of online participants joined from non-friendly CET time zones), and are present sporadically. 

RDA Secretariat Action: find ways on better attendees engagement, send announcement-reminders to encourage attendees to engage between each other. 

Implication: Note that this requires dedicated staff effort.

3.4. Technical practicalities in Whova feedback:

• Provide a camera view for remote attendees so they get a better sense of who is in the physical room (+3 more similar feedback)
• Whova very useful especially for contacting organisers when problems occurred
• Interface colour design  was terrible as a visually impaired person. Especially the lack of contrast between scroll bars and page panel background. 
• Would be nice to have two large screens, one for the slides in the room and one for the zoom-screen
• The Live Stream integration was extraordinarily good.

Consideration (s):

• Adding extra AV equipment requires additional very high cost. 
• Providing functionalities for visually impaired people must be considered for future events.

RDA Secretariat Action: implement RDA Accessibility checklist for documents and slide decks.

Implication: Note that this requires extra financial resources.

3.5. Agenda overview feedback:

• Presenter names were nowhere to be found and even session descriptions were very difficult to navigate (+9 similar comments)
• Way too many notifications. Seeing a description of a specific session required 3 clicks. Session titles were displayed partially and a group wasn’t visible.
• Would have been nicer, if the session content had directly been available in Whova, not on external websites.
• The social programme was hard to find in the app

Consideration (s): 

• Not having all programme elements in Whova’s agenda was very confusing. Having no descriptions under each session led to use of links to external websites which was complex. 

Rationale: There are 3 different components in the IDW event. RDA & SciDataCon use their own web platforms to collect the session applications & content. In the RDA case, the breakout session content is intrinsically linked to the RDA community work (groups).

IDW PC Action: Simplify and make agenda use a priority and ensure that all overarching programme components are in the Whova.

3.6. Whova’s notification functionality feedback:

• App tries to be cool with the leader board, with a very high number of unwanted notifications. Give me a basic app. 
• Too many emails
• It would be nice to have more moderation on posts (cats/dogs pictures are cute but useless)
• It is definitely useful but there are too many notifications
• Couldn’t “subscribe” or “unsubscribe” from any topic, so I wasn’t able to make sure I could follow a particular conversation or not follow a conversation that is irrelevant.

Consideration (s): There were a few comments raised on the amount of notifications Whova automatically sends that IDW 2023 platform admins can’t control.

RDA Secretariat Action: to explore what can be done to reduce the automatic notifications. Organisers not to set up or promote a non IDW-relevant themes e.g “Pet corner” for future events. 

IDW PC Action: to draft IDW ‘Code of Conduct’ for future events, to be included in Whova’s social groups/meet-ups for attendees to adhere to. 

3.7. Other feedback: 

• Organisers were very attentive and considerate of online attendees. 
• Go back and check one of the IASSIST sessions. It was easier to coordinate with remote attendees.
• I have used whova at IASSIST and this time it was not nearly as easy to navigate.  Consider pairing the conference presentations with the matching zooms.
• Hybrid is a critical aspect of IDW and RDA plenaries. Whova is a great platform. 
• If I ever attend an IDW again I will print out all the info I need beforehand & not register for the painfully awful app.

Consideration (s): better ways of using Whova’s functionality, building more user-friendly agenda, attendees communication was used for other events. IASSIST is quite social, tight-knit organisation that has a simpler programme agenda. IDW includes three conferences in one and much more complex to navigate the schedule in the first place. 
RDA Secretariat Action: follow up with IASSIST to identify ways to improve Whova set-up.

3.8. Feedback on Whova performance / functionalities:

• Hybrid session is very important as it provides flexibility for those who are not able to attend on site.
• Keep Whova. Easy to use!
• Complete interactive hybrid setting well done.
• I like the daily summaries pushed through the platform.

Overall Whova Consideration (RDA): Use of an event app will continue, currently RDA finds Whova to be valuable, cost-effective and includes the majority of the features and functionalities for both fully virtual and hybrid events.

4. Poster session and recordings

4.1. Poster session: 

• ’’Hybrid sessions should involve a hybrid poster session’’
• ‘’Poster session was very crammed which didn’t allow for good conversation.’’ 

Consideration (s): by only holding a poster session on-site with no virtual copies duplicated elsewhere, we separate the virtual audience. Should ‘hybrid’ be clarified?

RDA Secretariat Action: Similar feedback was received after RDA’s 19th Plenary where it was concluded that poster sessions are an effective way to present work, network & connect with an audience only during in-person events. Virtual poster sessions require extra resourcing and a high amount of planning with very small benefits as virtual poster sessions are poorly attended. 

IDW PC Action: IDW to go ahead and facilitate in-person poster sessions. This is dependent on the venue if it can be held in a suitable location, ideally near catering. 

RDA Secretariat Action: Clarify in clear terms for the hybrid component what the poster session offers, i.e. not interactive for online participants, and make it clear during the registration. 

4.2. Recordings availability within 24h:

IDW 2023 organisers promised to upload recordings within 24h but not delivered.

Consideration (s): It’s time consuming to process recordings (download, edit, upload to software, link to Whova). This is normally done by one person on one laptop on top of other tasks. Occasionally, there are issues with the recording and more time is needed.

RDA Secretariat Action: RDA to extend the timing to “within a maximum of 48 hours”. Try to get local technicians to assist. For IDW each component should state their own timeframe.

5. Comments or suggestions on IDW2023 or future IDW, including SciDataCon and RDA Plenaries

5.1 Catering:

• Provide enough, cater for dietary restrictions 
• Have a formal dinner instead of bar-hopping.
• The worst catering I have ever seen at a conference 
• Food service was completely inadequate for such a large international conference. 
• Queueing for food was ridiculous 

Consideration (s): Organising bodies should clarify with the local organisers about what is being provided on their end.

RDA Secretariat Action: If we ask about dietary restrictions during registration we must ensure that they are respected and a clear understanding of what and how dietary requests will be respected. 

5.2. IDW improvements:

• IDWs are a great idea but need to be logistically run better.
• I’m not sure it really makes sense to combine SciDatCon and RDA. They are very different style conferences and the context switching was awkward.
• Get rid of the distinction between RDA and SciDataCon altogether. It created confusion
• Be more explicit on the programme between what is RDA and what is SciDataCon.
• IDW conference was my first and will be my last. It will be remembered and talked about for many a year as the worst ever, a joke (poor organisation, high price, no food, inclusivity & accessibility dire). 
• Please consider accessibility & inclusivity more carefully both for those with visible & invisible disabilities.
• The organization of the IDW2023 was superb, from the call for papers to the reminders to the ‘Why attend the IDW2023’ webinar to end of day emails sharing highlights of the day.
• Be very clear on the venue accessibility issues. For example, please give more details on what they are and where. People have no idea what Bar hopping or the music event  meant; e.g. are there chairs, is there a lift?
• For the virtual: please upload the Plenary session slides beforehand so that people in Zoom can read as it is hard to read presentations from the Zoom. 

Consideration(s): Neurodiverse/neurodivergence needs to be considered for accessibility and inclusivity. Sharing presentation slides before the sessions is problematic due to various reasons (last-minute changes in slides, etc.)

RDA Secretariat Actions:

• Generate a document / checklist on neurodiversity at events. 
• Include ‘Best practices for presenters’, so they can create an appropriate presentation.
• Could IDW consider having a room for neurodivergent people – calm space needed and then we’ll get a wider net of people to the in-person conference. 

5.3. Venue:

• Have enough charging points for laptops and smartphones! 
• High tables are really bad to stand for so long. And wheelchair users
• Poster sessions – too cramped – next time please hold it in a much larger space. Impossible to network with any accessibility issues. 

Consideration (s): Despite this being a requirement of the application terms and checking with the local organisers in advance, the lack of charging points, accessibility options were surprising. 

IDW OC Actions: ensure to check the venue accessibility and what’s included, in what quantities all rooms and in networking sections and make sure it is confirmed in writing.

5.4. Registration fees:

• The fee for the event was exorbitantly high, especially for remote participants. 

Consideration (s): costs are based on the venue and local and local cost of living. Organisers aim to make fees as cost effective as possible. None of these events make revenue / profit.

IDW OC Action: ensure that registration fees are well explained, what is included and that they are cost effective. Particularly important for the on-line attendance. Try at all stages to ensure the registration fee is not too high and prohibitive.

5.5. Session titles

• Difficult to understand what the session title meant and when reading the description it didn’t match my understanding.

Consideration (s): some session titles couldn’t fit to Whova, which brought extra steps for attendees. Shorter, punchier titles are more attractive to attendees.

IDW PC Action: Generate a guide on session titles (including a number of characters a title can include) to be followed by all session applicants.

5.6. Presentation time slots:

• Session chairs either did not do their job properly or were not given the power to properly control the length of presentations

Consideration (s): the Secretariat provides suggestions/guidance materials for session chairs, presenters to follow with clear focus on how to run a session. Session chairs are strongly encouraged to familiarise themselves in advance and follow them.

Secretariat Action: continue to distribute the chairs & moderators guidelines to all and encourage all to follow them. Include a checklist for session chairs on tips for moderation.

5.7. Programme:

• Starting the program at 9:00 and finishing at 20:30 makes for a long day. 
• Program was very messy, would be nice to have metadata for each session, presentors, topics, content, activity. 
• Stick to 9-18 at the most. Could do with less plenary sessions and cut out the joint WDS and CoData session.
• Avoid holding sessions at 5:45pm!

Consideration (s): conference days were too long and the programme packed. The length of the event is a direct result of the combination of two different events: RDA & SciDataCon. RDA already sacrifices a number of breakout sessions during the IDW events.

IDW PC Action: 

• Generate an IDW pathway that runs across both events (CODATA, RDA) and assists the attendees in navigating the sessions. 
• Explore means of reducing the length of the days while maintaining networking sessions and not sacrificing the core components of the event, i.e. SciDataCon & RDA.

5.8. Venue signage

• All venue rooms should have clear signs of the name and title of the next session 

IDW OC Action: Ensure all meeting rooms have paper or electronic door signs that are changed for all the programme meetings.

5.09. Cloak room

• Charging for cloak room was hilarious.
• Inappropriate for them to charge for coat check.

Consideration (s): this is something IDW founding organisations were not aware of, although had consulted the LOC whether anything requires attention.

IDW OC Action: Check that cloak room costs are not being charged. Also check what on-site services are being offered and are being charged for future events.

5.10. Other feedback:

• WGs that are in the maintenance mode should be included in the program, because they sustain what RDA is proud of!

Consideration (s): there were less session slots available to RDA and guidelines did not includ specific parameters for Maintenance groups. 

Secretariat Action: RDA must define clear parameters and quotas for sessions, e.g. X Interest Groups, X BoFs, X WGs, X CoPs, X Joint Meetings & X Maintenance. Publish & state those parameters when the call for session is opened. 

For all groups but particularly for BoFs there must be a minimum of two session chairs / applicants present during the session. (Note: the three cancelled RDA breakout sessions at IDW2023 were due to unavailability / absence of the one session coordinator, so there was no contingency or possibility to run the session).

• It was a pity that co-organising the event meant that many of the RDA sessions couldn’t go ahead so I felt that my community suffered as a result.

Consideration (s): there were less session slots available to RDA.

Secretariat Action: Assess and analyse the value of IDW to the RDA community. 
 

6. RDA P21 Framework – P21 Session chairs feedback on how to improve Plenary planning 

6.1. How did your session go? 

• Minor issues with keeping track of remote participants’ comments, raised hands, wasn’t straightforward. There wasn’t anyone to monitor Zoom chat. Virtual attendees were ignored or noticed at the end of session. 
• Those who came were very active! 
• Tech set up was confusing as technicians seemed to struggle with PC and Mac computers.

Consideration(s): 

• IDW/RDA organisers clearly marked that each session organiser is responsible for managing audience feedback / Q&A, both in-person and online in distributed Guidance / Handbooks for session chairs. 
• An external professional tech company was hired to support hybridization. It should be noted that there was a complex chain of tech procedures/settings involved, which may have led to a few delays and unexpected situations. 

Secretariat Action(s): Include a clear, visible 10 top points / reminders in the session chairs handbook.

6.2. Were you satisfied with the information you received before RDA P21 from us by email?

• Yes from all 10 responses. 
• It was quite comprehensive and distributed well in advance. / It was quite detailed.

Consideration(s): Ten emails were sent out to P21 session chairs with planning updates within 4 months of IDW2023. The emails’ content was found very detailed and useful.

Secretariat Action(s): 

• Continue communication with approved session chairs on a regular basis.
• Duplicate correspondence with session chairs, (e.g. emails we sent them) and publish as PDFs on a Plenary webpage to refer to “Tips & Tricks for session organisers”.

6.3. Where can we improve next time?

• Avoiding overlap with sessions on similar topics. Have an “emergency contact” link to the local organisers posted under each session in Whova, to be used by remote participants who cannot access the Zoom room.
• It would be better not to schedule working meetings in the large plenary hall – it is not conducive to discussion and creates an awkward atmosphere for a small group. 
• Would be great for session speakers and chairs to have control over the zoom room (at least as co-hosts).
• It is important that there is one drive where speakers can put their talks. The media centre complicated things unnecessarily. Give the option to upload PPTs or PDFs as not everybody uses slide decks. 

Consideration(s): 

• RDA Secretariat relied on session applicants’ data and based on that, manually creates a programme. Some overlaps are unavoidable, but a couple of weeks are always given to session chairs to submit any changes they wish to make before a final programme is ready. 
• Hybridisation means that meetings assigned rooms cannot be changed on-site during the venue given the technical setup for on-line participants.
• Physical rooms allocation is a complex process, and it is difficult to find a Plenary venue with physical rooms varying in sizes. It will always be very difficult to predict attendees’ interest for each session or collect data in advance. Hence, some room issues arise. 
• The plenary session room is the most well equipped in terms of technology, the cost paid is for the duration of the conference and therefore to be cost effective it is often used for breakouts. 
• Zoom co-chair rights are normally granted to session organisers/speakers prior to the session start time. This allows them to facilitate Zoom functionality. 
• One file was shared with all session chairs to upload their presentation files upfront. 

Secretariat Action: Once there is a draft programme version, the RDA Secretariat should ask TAB to check eventual programme clashes that may not have been indicated during application.

7. Technical Advisory Board (TAB) Feedback

7.1. Benefits to the RDA community of co-locating the RDA Plenary meeting within the framework of International Data Week:

• Participants can join two events at the same time, instead of managing 2 trips. Possibility to present contributions to RDA and IDW.
• IDW brings in a different audience, it is a good way to onboard new RDA members. IDW, RDA complement each other well, the SciDataCon sessions are a familiar style of conference and RDA creates a lot of opportunities to engage with IGs and WGs.
• IDW could be an event for promoting RDA outputs.
• Alignment with broader issues outside of the RDA community and understanding how they differ or align. Acknowledgement of RDA role in the broader context and what part it plays in community engagement.

Consideration(s): benefits for networking and presentations at the merged event; opportunities for RDA promotion and understanding external methods of work.

Secretariat / IDW PC Action: identify possible means to synchronise the RDA Plenary and SciDataCon programmes.

7.2. Challenges for RDA in co-locating the RDA Plenary meeting within the framework of International Data Week:

• The main challenge is the reduced space/time for RDA sessions, which forces them to be more selective. Parallel sessions are too packed.
• Unconference sessions from previous RDA plenary were good to talk about new topics or find peers who are interested in new topics – but this requires space and time.
• There was too much going on at the same time. We could not accept all session proposals but I would not plan for a longer conference. Could give preference for joint sessions.
• Suggest preparations with co-chairs prior to IDW to indicate there is less session time,  and to consider prioritising RDA outputs, recommendations, and key issues, needs. Less focus on maintenance areas unless there are pressing needs. Early planning for joint sessions.

Consideration(s): Merged programme is too packed and reduces number of RDA sessions.

Secretariat Action / IDW PC: Identify possible means to synchronise the RDA Plenary and SciDataCon programmes.

7.3. Elevate RDA visibility within IDW?

• In case of no co-located events, agree on an “RDA session” in IDW program, in which the main RDA outputs can be highlighted.
• Contrast the different styles and intentions of the CODATA/WDS, RDA sessions in the high-level marketing, in addition to the titles but can’t see how to do so in a concise way.
• Have one event after each other (or have one day break in between), instead of having parallel sessions.This way, each could advertise their own event as their own.
• RDA was more visible than WDS or CODATA at IDW. Could RDA contributions be flagged, e.g. RDA logo in the joint IDW programme.

Consideration(s): the RDA Adoption and Outputs plenary session was included in the IDW2023 programme. As the programme is already long (at 4 full days), running the sessions back-to-back rather than concurrently is not feasible.

Secretariat Action / IDW PC Action: 

• Identify possible means to synchronise the RDA Plenary and SciDataCon programmes. 
• Include branding / logos in the overarching programme to assist participants in identifying the programme components.

7.4. New but feasible components that can or should be incorporated to RDA plenaries (fully virtual or hybrid) going forward:

• Set up a survey for RDA members to ask the same questions as above.
• One virtual, one hybrid works well.
• Whova is very well received and used by participants. Could a central place to upload pictures taken by participants be available? Have an extra room available for spontaneous meetings?
• The most difficult part for hybrid meetings are posters. Partly due to expenses, there is no solution to get virtual and on-site participants together. Having a poster session in a virtual plenary as an option or plenaries without posters.
• Whova is nice, having it used more than one plenary lowered barriers for participants. Pre-plenary review of Whova was not very helpful. Some encouragement to use features like using the Messages feature. Nice thing with Whova is that it also supports continued interaction, community Building aspects. Having a TAB corner/TAB table would be helpful. Poster session was OK.

Consideration(s): Whova is a platform that RDA has used for 4 plenary meetings (since P18) and the community are familiar with it. It has an excellent price / quality ratio and offers a series of valuable features and functionalities (e.g. networking, Stripe finance integration). 

Secretariat Action / IDW PC Actions: Whova should be published at least 10 working days before the event takes place to allow the participants to take full advantage of the features, create their own programme, get familiar with the App, etc.

---

RDA’s 21st Plenary Highlights are the summary of the four-day event in figures and images. Check it here.

---

♦ Ari Asmi is the owner of the pictures on this page. We appreciate that he let us use them. ♦