Point of contact
Jean-Daniel Fekete, Université Paris-Saclay & Inria, Jean-Daniel.Fekete@inria.fr
Summary
We live in an era where data is abundant and growing rapidly; databases storing big data sprawl past memory and computation limits, and across distributed systems. New hardware and software systems have been built to sustain this growth in terms of storage management and predictive computation. However, these infrastructures, while good for data at scale, do not well support exploratory data analysis (EDA) as, for instance, commonly used in Visual Analytics. EDA allows human users to make sense of data with little or no known model on this data and is essential in many application domains, from network security and fraud detection to epidemiology and preventive medicine. Data exploration is done through an iterative loop where analysts interact with data through computations that return results, usually shown with visualizations, which in turn are interacted with by the analyst again. Due to human cognitive constraints, exploration needs highly responsive system response times: at 500ms, users change their querying behavior; past five or ten seconds, users abandon tasks or lose attention. As datasets grow and computations become more complex, response time suffers. To address this problem, a new computation paradigm has emerged in the last decade under several names: online aggregation in the database community; progressive, incremental, or iterative visualization in other communities. It consists of splitting long computations into a series of approximate results improving with time; in this process, partial or approximate results are then rapidly returned to the user and can be interacted with in a fluent and iterative fashion. With the increasing growth in data, such progressive data analysis approaches will become one of the leading paradigms for data exploration systems, but it also will require major changes in the algorithms, data structures, and visualization tools.
Keywords
exploratory data analysis, visualization, visual analytics, database, approximate query processing, progressive data analysis.
Researchers involved or interested
- Jean-Daniel Fekete, Université Paris-Saclay & Inria, Jean-Daniel.Fekete@inria.fr
- Anastasia Bezerianos, LRI, Université Paris-Saclay, Anastasia.Bezerianos@lri.fr
A few references
- Jean-Daniel Fekete, Danyel Fisher, Arnab Nandi, Michael Sedlmair. Progressive Data Analysis and Visualization. Wadern, Germany. Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik. 2018.