2022 I Harpreet Sarren

Prof. Harpreet Sareen Assistant Professor, Interaction and Media Design Parsons School of Design, The New School

Prof. Sareen will be working on a projecte around alternative and “calm” visualizations of data using plants or other biological organisms. This work will be included in the recently accepted Equipex+ Continuum grant (successor of Digiscope) and enable a fusion between natural and digital visualization tools. The project fits nicely into one of the goals of Continuum, providing a digital infrastructure to artist to explore alternative data visualizations. Prof. Sareen has a
perfect background in computer science and design to be able to leverage the provided infrastructure. Prof. Sareen would be able to provide a symbiotic effect between both projects, Labex DigiCosme and Equipex+ Continuum.

Dates

3 months: April 1st – July 31st

Axe scientifique DigiCosme IID

Work Program

1. Motivation, Problem and Proposal

With the rise of ubiquitous computing and IoT, the amount of data generated for the user is currently growing at an extremely high rate (e.g. smart home data, health data). To be able to manage and understand this data, researchers have explored a variety of ways to visualize and present this information to the user often focusing around performance metrics such as efficiency, effectiveness and understandability. For data visualizations or displays, designers often borrow metaphors from the nonliving parts of the natural world. Such artificial displays are all too common. However, a new scope of information visualization is emerging where combining living organisms with electronic components can serve as interactive displays as well. Research around technology usage indicates that with the rise of data and information processing comes a set of psychological implications for well-being (e.g. anxiety, depression, information overload). In this research project, we aim to explore how the current amount of digital data can be presented to the user in alternative ways, with the goal of preserving certain performance metrics but mainly optimizing for well-being. There is value in exploring the use of living material and biomimetic forms in displays, and in using lightweight robotics to deliver simple rewards. We aim to investigate concept “calm visualization” or “calm data”, by leveraging non-traditional displays (e.g. bio displays / natural displays) to be able to present certain data to the user in a subtle and calming but still understandable form. The project will be positioned within the research axis IID-4 of DigiCosme, working on alternative forms of interacting and visualizing data streams.

2. Solution and Method

Our approach builds upon the background of the invited Professor Harpreet Sareen, who has a history of instrumenting and augmenting plants and nature and developing novel forms of display technologies. Some of this research includes ‘growable electronics – electronics [1] growing inside xylem of a plant ’, ‘nanobionics [2] – nanosensors that glow from within plants to inform of water quality’ and directed growth of plants to visualize data. These projects have previously been published in international media (Discovery, CNN, BBC etc.) and showcased at international scholarly/artistic venues around the world.
This project aims to explore how plants (e.g. house plants) and nature can be symbiotically included and leveraged to visualize certain data to the users in a calm and more natural form. Consider posters/billboards or interactive displays for showcase of information in a natural environment. Luminous displays themselves stand out when placed in natural scenery which makes them difficult to integrate in cities and even in indoor environments. Unlike a ‘Times Square’ and constant onslaught of information’, we want to research displays that can keep our cities and urban spaces more warm/hospitable. We propose ‘green media’ that has both qualities of greening and presenting information. By leveraging the attributes of plants, and computationally and gently controlling their behavior, we aim to develop a novel information representation that can blend in with the natural environment. Such information displays could present novel and exciting ways of showcasing
information in the cities without necessarily standing out like interactive displays. In our research project, we aim to explore slow moving data (e.g. movement of a glacier — suitable for a slow changing bio-display made by controlling the growth of a slow-moving moss – Racomitrium canescens [3]). The surface of open Racomitrium canescens shifts as time elapses. In this way, the opened and closed Racomitrium canescens create a gradation pattern that shifts with time. We will
develop the hardware to control the appearance of the Racomitrium canescens, or another grass species [4]. By arranging several units, we simultaneously open and close the alternate columns of living display to develop into a large living display wall. Such methods of using plants and nature as a bio display is also applicable to indoor environments. The aliveness of plants could contribute to a compelling and engaging emotional quality in designs.

3. Implications

Our design is an example of lightweight robotics in a public setting, using simple sensors, processors, and
actuators to deliver small yet effective visualizations. If people’s affinity for other living things were to
translate into a receptivity to living things used as displays, we would expect such displays to be engaging
and compelling. The conducted research project has the goal of exploring a new type of visualization known as “calm
visualization” which has the goal of presenting data to the user not with the aim of optimizing
performance metrics but with the aim of optimizing for well-being. The addition of living materials may
introduce an emotional quality absent from interactions with purely electromechanical systems We aim
to characterize public reactions to such biodisplays and disseminate our research via two forms: (a)
submitting the resulting prototype as an art installation / art exhibition and (b) submit the concept and
insights of a study to a fitting top-tier HCI conferences such as CHI, UIST, DIS or IEEE VIS.

References:
[1]: Sareen, Harpreet, Jiefu Zheng, and Pattie Maes. « Cyborg botany: augmented plants as sensors, displays and
actuators. » Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 2019.
[2] Sareen, Harpreet, and Pattie Maes. « Cyborg Botany: Exploring In-Planta Cybernetic Systems for Interaction. »
Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 2019.
[3] https://plants.usda.gov/core/profile?symbol=RACA11
[4] Ackroyd, Heather, and Dan Harvey. « Chlorophyll Apparitions. » Signs Of Life (2007): 199.

Research

Post faculty discussions and considering the recent grant Equipex+ Continuum (successor of Digiscope), Prof. Sareen’s focus was on self-assembly processes in biology for purposes of data visualization. He studied several polymers, colloids and macroscopic particles that could potentially be organized into functional structures or patterns because of local interactions among the components themselves. In such materials, computation may be offloaded by utilizing their natural mechanisms of actuation/sensing, where new interaction loops become possible (a spectrum of full/partial/zero computation enabled through the material).
Through further research, and in context of discussions of zero-silicon fabrication, Prof. Sareen studied water and associated bubble nucleation interactions. Interaction researchers have previously utilized water and its elements of foam, bubbles, droplets, flow, condensation for functional or novel interactions as well as aesthetic means. These devices have associated electronics that require constant energy consumption. Prior bubble displays are also hard to scale because of linked electronic costs. In this period of research, Prof. Sareen focused on possibility of chemically manipulating surfaces to engineer controlled bubble nucleation.
On such surfaces, bubbles from a liquid may nucleate at specific points, starting small and eventually stabilizing when they reach a pre-defined size. This research and its associated interactions is currently in progress, and will be helpful as a new “data material” to the visualization community at Telecom Paris and to the larger academic community.

Teaching

1. Lecture and student discussions in Atelier de cartographie des futures Workshop on visualization of the futures, co-teaching workshop with Prof. Samuel Huron
(April 25-29, Mon to Fri full day workshops – 35 hours)

In lieu of Prof. Sareen’s research focus and overlaps with the Recherche en design of ENS X Telecom Paris, he was invited for student discussion/mentoring, lecture and critique in the class. The students started with their introductions through visualizations of their past leading to the present. Prof. Sareen shared feedback on some of his design perspectives from Parsons where the agenda of their program (Design and Technology) is to look at technologies critically. He shared some of the observational techniques employed by activist practitioners, emerging trends in cultural lookback for digital fabrication, dark patterns in technology markets and critical ways of looking at technology

 

Prof. Sareen also gave a talk on alternative materials in computation, where natural world and computation coincide.

The students were prompted to think of various scenarios where animal brains controlled robots, organic materials led to grown structures instead of them artificially designed and various strategies of placing computation in conjunction with plant capabilities.

2. IGR 205 [11 April - 27 June, 2 hrs/week * 10 + 5 hours for last two weeks = 30 hours]

This course is group based, where a team develops a set of modules linked to a specific area of research. The project topics are proposed by the entire teaching staff in the IGR/IGD Master program and the invitation to teach was also extended to Prof. Sareen, following which he proposed three potential topics for students:


• [Potential Topic 1] Exploring radiation visualization in augmented reality
• [Potential Topic 2] Creating Novel Display Using Bubble Nucleation
• [Potential Topic 3] Controlling locomotion of multiple ground robot units in a physical space


The students opted for Topic #3 and were advised by Prof. Sareen throughout the duration of his visit. The goal of this project was to develop an application that can control multiple robot units in a physical environment.


This class began with an introduction to robot locomotion and electronics. Students received robots and modified each robot with a location tracker beacon. The research components included interaction systems in various human-robot augmented reality publication. The goal of the project was to develop:
(1) NodeJS application: Receive current positions of robots in space,
(2) Central command application (developed in Unity or NodeJS etc.): Process real-time positions of robots, Set initial position of robots, Trigger robots for circular locomotion path in space, Receive data from location trackers for feedback control, and
(3) Robot programming (Arduino): Locomotion functions and communication with central command for position feedback control.


The project involved a lot of components, managed by complex communication and
networking protocols, where Prof. Sareen was in constant conversation with the students throughout the project. The students successfully completed the class with a project and presented this to the peers and faculty at IGR 205 finals held on June 27.

External Talks / Visits

1. Prof. Sareen delivered a talk to the wider human computer interaction community, hosted at Telecom Paris. The talk titled “Interaction Design and Non-human Behaviors” was held on Wednesday May 11th in a hybrid (online+offline) format and attended by various practioners/researchers.

The talk focused on how fields of design have been fundamentally shaped by human behavior and revisiting fundamentals that shaped this human-centric context can also reshape our technological transformation from an anthropomorphic to a nature centric

Prof. Sareen showed various instances of how technology could be applied to our ecology today and shared opportunities to flatten hierarchies in the field of design. By incorporating non-human animals and organisms.

2. In the spirit of fostering academic exchange between institutions, and sharing design strategies of Biodesign, Prof. Sareen was also hosted for a short visit (May 19th) to the labs at Da Vinci Innovation Center by Resilient Futures PI Marc Teyssier. At the center, Prof. Sareen helped students develop strategies for software simulation for fabrication of inflatable objects.

3. Parsons School of Design, rated as top design university in the United States, also has a branch at Paris with teaching centers near the Louvre and undergrad senior fabrication studios in 18th Arrondissement. Prof. Sareen shared his experiences of visiting Telecom Paris with students of Parsons at their final graduation show on 14th May and invited the students to collaborate with students at Telecom ParisTech.

News/Updates

– “Biohacked cyborg plants may help prevent environmental disaster” Popular Science Magazine featured the work of Prof. Sareen on Jun 7, 2022.
– Prof. Sareen’s work was awarded IFDesign Prize for “Argus: Water Monitoring through Nanosensors inside Living Plants” on 25 May, 2022.
– “Algaphon: Transducing Human Input to Photosynthetic Radiation Parameters in Algae Timescale” published at the International Symposium of Electronic Art, 2022.
– (Work in Progress for publication) Self-assembly of materials for data visualization

Prospective collaborations

• Biomaterials discussions and student advising with Prof. Samuel Huron / Marc Teyssier,
• Non-human organisations partner search for Eva Sirven’s (Design student in Sam’s workshop) animal-centered projects
• Nour Boulahcen, student IGD Master, to develop software for fabrication workflow of pixelization for bubble nucleation (currently in progress)
• Attending art-science institutional conversations with faculty at Telecom Paris; and
facilitating organizational/university partner institutions or model examples in New York, California and Tokyo.
• Developing an exchange program centered on critical design with Parsons School of
Design in New York.
• Discussions of plant bioelectricity with botanists of LaBotanique (Botanical Garden of Nantes)