Array Database assessment Working Group meeting at Plenary 8

This page documents the inaugural meeting of the Array Database assessment Working Group at Plenary 8 in Denver, USA, 2016-09-16.

Agenda

In order to maximise focus in getting hands-on, just two introductory presetnations were given by the chairs, then leading into a discussion on how to proceed concretely.

• Array Database Assessment Working Group: Introduction (Peter Baumann)
• tbd (Kwo-Sen Kuo)

Discussion

• Q: what is the focus of this WG?
  A: Focus is strictly on arrays (grids, rasters) to be able to achieve meaningful results
• Q: Are only array databases in focus?
  A: Anything that offers a flexible query interface is of interest, and could be called a "database system" in the widest sense. However, further array-related technology should be inspected as well to obtain comparisons (e.g., MapReduce-based systems like SciHadoop, GeoTrellis, MrGeo) - but this depends on the contributions; in other words: come and champion your favourite tool!
• Q: what is a realistically large deployment?
  A: anything larger than server RAM qualifies (otherwise, you just load data into RAM without any need for a database system); as a rule of thumb, going beyond 10 TB should be fine.
• Q: what kind of contributions do you seek?
  A: three types of contributors are needed in order to have comprehensive expertise on board:
  • use case providers: they bring in data and challenges (aka queries)
  • service providers: they provide storage and compute space for setting up realistically large deployments of systems
  • technology (software) providers: they provide their tools for installation and operation, as well as support.
• In an elicitation of the interests of those present the following was articulated:
  • ETL: 2
  • SQL/MDA: 4
  • formats (HDF, ...): 5
  • query power (Fourier Transform etc.): 3
  • OPeNDAP: 2
  • model-to-model communication via database: 2

Outcome

The following concrete initial groups shaped:

• use case providers:
  • Mohan: Earth science
  • Schaaff: Space science
• data center: none
• software providers:
  • Peter Baumann: rasdaman
  • Ted Habermann: HDF
  • Mohan: HDF, OPeNDAP
  • Doug Lindholm: tbd

Action items

• chairs: convene group, grow it from membership via email - until beginning of Oct
• data center group: establish infrastructure for installation - tbd
• software providers: stand up installations, provide tutorials - once installation points are defined