This dataset contains all the raw data of fNIRS, salivary oxytocin levels, behavioral measures and self-reported questionnaire responses collected for the multimodal HRI 1 study.
This dataset is licensed under a Creative Commons Attribution 4.0 International License.
You should have received a copy of the license along with this work. If not, see https://creativecommons.org/licenses/by/4.0/.
This dataset contains the following files:
1)ARTANOVA folder
This folder includes all R scripts and Excel worksheets applied for Aligned Rank Transform (ART) ANOVA (Wobbrock et al., 2011) results for fNIRS, oxytocin, state/trait questionnaires and behavioral measures. The corresponding R scripts for each of these measures are also included in the folder (usable in R v4.4.2 (https://www.r-project.org/) and RStudio v2024.12.1 (https://posit.co/download/rstudio-desktop/)).
In order to run the scripts, you must have the following packages installed in RStudio:
readxl, ARTool, rcompanion, writexl, rstudioapi
2)fNIRS.zip for fNIRS data folder
All raw light intensity data for 50 participants (M100-M149) are included in the fNIRS folder with a separate subfolder for each participant. Each participant folder has two sub-folders: experiment parts 1 and 2 recorded together in one folder, and part 3 in the second folder. Only participants M113, M117, M137, M139, M143, M144, and M149 have three folders, one folder for each experiment part. Only participant M101 had technical issues that prevented recording and was excluded. Parts 1 and 2 were recorded during interaction with the Congruent robot, while Part 3 was recorded during interaction with the Erroneous robot. Each sub-folder (one recording session) has five files with the same file-names and different file-extensions indicating the different content from the same session:
1a).log file contains meta-data and supplementary notes from the recording session. 1b).nir file contains the raw light intensity data that were recorded by COBI Studio Modern v1.2 (Ayaz et al., 2011). This file is a tab-separated text format and can be opened with any text editor (including Excel, Matlab, etc.) The header contains date/time and additional information, followed by the data matrix of rows and columns. Each row is another sample with the first column showing the relative time in seconds with respect to the recording start time (origin) as listed in the header. Every three columns after the first column represent the light intensity values for one optode in the order of 730nm light, Ambient light, and 850nm light (for Optode 1, it’s columns 2-4, for Optode 2, it’s columns 5-7…).
1c)*.oxy file contains the raw hemoglobin concentration data calculated by modified Beer-Lambert law (MBLL) from the nir file using the global baseline. Same as the nir file, this is a tab-separated text-formatted file and can be opened with any text editor (including Excel, Matlab, etc) The header contains date/time and additional information, followed by the data matrix of rows and columns. Each row is another sample with the first column showing the relative time in seconds with respect to the recording start time (origin) as listed in the header. Every two columns after the first column represent the hemoglobin concentration changes for one optode in the order of deoxygenated-hemoglobin and oxygenated-hemoglobin values (for Optode 1, it’s columns 2-3, for Optode 2, it’s columns 4-5…).
1d and 1e) *_Mark1.mrk and *_Mark2.mrk files contain the marker (event time synchronization) information. The marker files are also tab-separated text format and contain the markers for the beginning and end of task condition epochs. Marker2 file contains automated markers sent from the robot control panel to the data collection computer, whereas Mark1 files contain supplementary manual keypress logs from the keyboard.
In the Mark2.mrk file, the markers that signify the start and end times of the decision-making blocks are as follows:
77 and 80: start and end (respectively) of the Island Survival Task (for participants who interacted with the Animate robot)
86 and 48: start and end (respectively)of Save the Art Task (for participants who interacted with the Animate robot)
109 and 112: start and end (respectively) of Island Survival Task (for participants that interacted with the Stationary robot)
118 and 50: start and end (respectively) of Save the Art Task (for participants who interacted with the Stationary robot)
The folder includes fNIRSoft scripts (usable in fNIRSoft v4.11 (Ayaz, 2010)) that apply the preprocessing to raw fNIRS data for the subjects with two recordings (fNIRS script for 2 PART SUBJECTS.fss) and 3 recordings (fNIRS script for 3 PART SUBJECTS.fss)
3)BehavioralData.xlsx
This spreadsheet contains all participants’ behavioral data (time of talking and eye contact) and robot performance measures (compliance and influence) recorded during decision-making tasks for 50 participants (M110-M149). For behavioral measures, 4 participants were excluded due to poor video quality.
Task conditions and assigned index values for us in this file are listed below:
Expressiveness: 1=Animated, 2=Stationary
Performance: 1=Congruent, 2=Erroneous
Part: 1=Before Interaction, 2=After Interacting with Congruent Robot, 3=After Interacting with Erroneous Robot
Task: 1=Island Survival, 2=Save the Art
Agree: 0=Participant did not agree with Robot’s suggestion, 1=Participant agreed with Robot’s suggestion.
Change: 0=Participant did not change initial choice according to Robot’s suggestion, 1=Participant changed initial choice according to Robot’s suggestion.
Influe: (Influence) 0=Participant did not agree with Robot’s suggestion (hence did not change), 0.5=Participant agreed with Robot’s suggestion, but did not change initial decision, 1=Participant agreed and changed initial choice according to Robot’s suggestion.
4)ELISASalivaryOxytocin.xlsx
This spreadsheet contains all ELISA test values for salivary oxytocin concentration for 40 participants (M110-M149).
Task conditions and assigned index values for us in this file are listed below:
Expressiveness: 1=Animated, 2=Stationary
Performance: 1= Before Interaction, 2= After Interacting with Congruent Robot, 3= After Interacting with Erroneous Robot
TestAttempt: Every sample was tested twice.
Concentration(pg/ml): Oxytocin concentration measured in the sample.
Mean_concentration(pg/ml): Average of the 2 test attempts of the same sample.
CV: Coefficient of variation.
5)ModeratedMediationAnalysis.sav and ModeratedMediationAnalysis.xlsx:
This spreadsheet contains all the information used in the mediation analysis for 40 participants (M110-M149).
Task conditions and assigned index values for us in this file are listed below:
Express: Expressiveness condition. 0=Stationary, 1=Animated
Prfmnce: Performance condition. 0=Erroneous, 1=Congruent
Part: 1=Before Interaction, 2=After Interacting with Congruent Robot, 3=After Interacting with Erroneous Robot
Task: 1=Island Survival, 2=Save the Art
Optode1-16: Average HbO concentration changes for the specified task block.
LDLPFC: Left dorsolateral PFC activity, calculated by the average of Optodes 1-4.
LMPFC: Left medial PFC activity, calculated by the average of Optodes 5-8.
RMPFC: Right medial PFC activity, calculated by the average of Optodes 9-12.
RDLPFC: Right dorsolateral PFC activity, calculated by the average of Optodes 13-16.
OTLevel: Oxytocin levels(pg/ml).
OTDiff: Oxytocin difference between the collection time points (pg/ml).
Agree: 0=Participant did not agree with Robot’s suggestion, 1=Participant agreed with Robot’s suggestion.
Change: 0=Participant did not change initial choice according to Robot’s suggestion, 1=Participant changed initial choice according to Robot’s suggestion.
Influe: (Influence) 0=Participant did not agree with Robot’s suggestion (hence did not change), 0.5=Participant agreed with Robot’s suggestion, but did not change initial decision, 1=Participant agreed and changed initial choice according to Robot’s suggestion.
Rapport: Rapport Questionnaire total score.
Anxiety: STAI-6 total score.
Anger: STAXI-S total score.
Trust: MDMT total score.
EyeGaze: The time of eye contact in seconds (s).
Talking: The time of talking in seconds (s).
The SPSS script that also applies moderated mediation results in the paper is included (moderated mediation analysis script.sps) and works with .sav file. (Usable in SPSS v29.0 (https://www.ibm.com/spss), before running the script PROCESS macro v4.2 (Hayes, 2018) needs to be installed)
6)QuestionnaireState.xlsx:
This spreadsheet contains the information and scales for State Questionnaires:
STAI-6 (State-Trait Anxiety Inventory – 6 Item Scale),
STAXI-S (State-Trait Anger Expression Inventory – State section),
MDMT (Multi-Dimensional Measure of Trust),
RQ (Rapport Questionnaire)
for the 50 participants (M100-M149).
MDMT was collected only after interactions with the Congruent and Erroneous robots.
Task conditions and assigned index values for us in this file are listed below:
Expressiveness: 1=Animated, 2=Stationary
Performance: 1=Before Interaction, 2=After Interacting with Congruent Robot, 3=After Interacting with Erroneous Robot.
7)QuestionnaireTrait.xlsx:
This spreadsheet contains the information and scales for Trait Questionnaires:
IRI (Interpersonal Reactivity index),
PANAS (Positive And Negative Affect Schedule),
STAI-T (State-Trait Anxiety Inventory – Trait Section),
STAXI-T (State-Trait Anger Expression Inventory – Trait Section),
NEO-FFI (NEO Five-Factor Inventory: Neuroticism, Extraversion, Openness to Experience, Agreeableness, and Conscientiousness),
CFQ (Cambridge Friendship Questionnaire)
for the 50 participants (M100-M149).
The last two questions in CFQ were excluded from scoring due to being missing for 27 participants because of technical issues.
References:
Wobbrock, J. O., Findlater, L., Gergle, D. & Higgins, J. J. in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 143–146 (Association for Computing Machinery, Vancouver, BC, Canada, 2011).
Ayaz, H., Shewokis, P. A., Curtin, A., Izzetoglu, M., Izzetoglu, K., & Onaral, B. (2011). Using MazeSuite and Functional Near Infrared Spectroscopy to Study Learning in Spatial Navigation. J Vis Exp(56), e3443. https://doi.org/10.3791/3443
Ayaz, H. Functional near infrared spectroscopy based brain computer interface, Drexel University (2010).
Hayes, A. F. Partial, conditional, and moderated moderated mediation: Quantification, inference, and interpretation. Communication Monographs 85, 4-40 (2018). https://doi.org/10.1080/03637751.2017.1352100