Hi-Audio — User Evaluation Results

Preliminary user evaluation of the Hi-Audio online platform for collaborative audio recording. Results from 22 participants including task-based performance, SUS, and NASA-TLX metrics.

View the Project on GitHub idsinge/hiaudio_user_evaluation_results

Hi-Audio User Evaluation — Per-Participant Task Performance and Comments

Anonymized record of per-participant task outcomes and feedback from the Hi-Audio user evaluation. Participants are identified by anonymous labels P01–P22. All email addresses have been replaced with JotForm Submission IDs. Comments are labeled by source: [Survey] (post-task questionnaire) or [Email] (follow-up correspondence). Evaluator notes (observations from recording review) are shown in blockquotes.

See SUPPLEMENTARY.md for aggregate tables (SUS, NASA-TLX, latency).


Participant 01

Task Description Result
T1 Record singing on an existing collaboration ✗ NO[2] (did not record voice, recorded instrument)
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✗ NO[3] (did change the title but in the template copy)
T4 Clone right template Happy Birthday ✗ NO[4] (used template for instrument but recorded voice)
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES[4] (used template for instrument but recorded voice)
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (guide track)
T10 Invite collaborator ✗ NO

Footnotes:

Feedback and comments:

[Survey]


Participant 02

Task Description Result
T1 Record singing on an existing collaboration ✗ No
T2 Manual annotation track language ✗ No
T3 Change title composition country/language ✗ No[3] (change title in the template copy)
T4 Clone right template Happy Birthday ✓ Yes
T5 Screenshot latency test ✓ Yes
T6 Record on Happy Birthday copy ✓ Yes
T7 Create a collection ✓ Yes
T8 Create new composition inside collection ✓ Yes
T9 Use metronome/use guide track ✓ Yes (metronome)
T10 Invite collaborator ✓ Yes

Footnotes:

Feedback and comments:

[Email]

[Survey]


Participant 03

Task Description Result
T1 Record singing on an existing collaboration ✓ YES[5] (created a new separated version, collaboration expired)
T2 Manual annotation track language ✗ NO[15]
T3 Change title composition country/language ✓ YES[6] (changed in the new separated version)
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (metronome)
T10 Invite collaborator ✗ NO

Footnotes:

Feedback and comments:

[Survey]


Participant 04

Task Description Result
T1 Record singing on an existing collaboration ✗ NO[2][5] (collaboration expired, but did not sing only played piano)
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✓ YES[6] (changed in the new separated version)
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (metronome)
T10 Invite collaborator ✓ YES[10]

Footnotes:

Feedback and comments:

[Survey]


Participant 05

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✗ NO
T4 Clone right template Happy Birthday ✗ NO
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✗ NO
T7 Create a collection ✗ NO
T8 Create new composition inside collection ✗ NO[7] (composition without collection)
T9 Use metronome/use guide track ✓ YES (guide track)
T10 Invite collaborator ✗ NO

Footnotes:

Evaluator notes:

(Tracks not well sync, latency might be wrong estimated or simply bad performed)

Feedback and comments:

[Email]

[Survey]


Participant 06

Task Description Result
T1 Record singing on an existing collaboration ✓ YES[8] (he started the tasks before having the collaboration set)
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✓ YES[6]
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (guide track)
T10 Invite collaborator ✗ NO

Footnotes:

Evaluator notes:

Conclusion: he confirms he added the email to invite “hiaudioparis@gmail.com” but eventually no collaborator is present which means the process of invitation was not clear enough and he did not confirm in the pop-up dialog,

Feedback and comments:

[Survey]


Participant 07

Task Description Result
T1 Record singing on an existing collaboration ✗ NO
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✗ NO
T4 Clone right template Happy Birthday ✗ NO[4] (recorded voice on instruments template)
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✗ NO[9] (Au clair de la Lune)
T8 Create new composition inside collection ✓ YES[9]
T9 Use metronome/use guide track ✓ YES[9] (Au clair de la Lune template)
T10 Invite collaborator ✓ YES[10] (invited everywhere not only first composition)

Footnotes:

Feedback and comments:

[Survey]


Participant 08

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✓ YES
T3 Change title composition country/language ✗ NO
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (guide track)
T10 Invite collaborator ✓ YES

Feedback and comments:

[Survey]


Participant 09

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✓ YES
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES[12]
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (metronome, guide track)
T10 Invite collaborator ✓ YES

Footnotes:

Feedback and comments:

[Survey]


Participant 10

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✗ NO[3]
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✗ NO
T8 Create new composition inside collection ✗ NO[7] (composition without collection)
T9 Use metronome/use guide track ✓ YES (guide track)
T10 Invite collaborator ✗ NO

Footnotes:

Feedback and comments:

[Survey]


Participant 11

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✓ YES[13] (on the title of the track)
T3 Change title composition country/language ✗ NO[3] (changed on the clone)
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (guide track)
T10 Invite collaborator ✗ NO

Footnotes:

Feedback and comments:

[Survey]


Participant 12

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✓ YES
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✗ NO[7] (composition without collection attached)
T9 Use metronome/use guide track ✓ YES (guide track)
T10 Invite collaborator ✗ NO

Footnotes:

Evaluator notes:

(No headphones he says)

Feedback and comments:

[Survey]


Participant 13

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✓ YES
T3 Change title composition country/language ✓ YES
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (metronome, guide track)
T10 Invite collaborator ✓ YES

Feedback and comments:

[Survey]


Participant 14

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✓ YES
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✗ NO
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✗ NO[9] (record Au Clair de la Lune)
T8 Create new composition inside collection ✓ YES[9] (Au clair de la Lune)
T9 Use metronome/use guide track ✓ YES[9] (Au clair de la Lune)
T10 Invite collaborator ✓ YES

Footnotes:

Evaluator notes:

No latency value provided, from the recordings I can tell there’s delay and no earphones were used so there’s feedback (echo) in the recording

Feedback and comments:

[Survey]


Participant 15

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✓ YES
T3 Change title composition country/language ✓ YES
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES
T10 Invite collaborator ✓ YES

Feedback and comments:

[Survey]


Participant 16

Task Description Result
T1 Record singing on an existing collaboration ✗ NO
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✓ YES
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (metronome)
T10 Invite collaborator ✗ NO

Evaluator notes:

He managed to change the title in collaboration but there’s no audio recorded, maybe the track was deleted before being sync with the server

Feedback and comments:

[Survey]


Participant 17

Task Description Result
T1 Record singing on an existing collaboration ✗ NO[2] (did not sing)
T2 Manual annotation track language ✓ YES
T3 Change title composition country/language ✓ YES
T4 Clone right template Happy Birthday ✗ NO[14] (cloned language and played trumpet)
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES[14] (using wrong template)
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (metronome)
T10 Invite collaborator ✓ YES[10]

Footnotes:

Evaluator notes:

(On Android phone Pixel, probably bluetooth headset: Marshall Major IV) He did record Au clair de la Lune but he created a new collection and composition himself for it

Feedback and comments:

[Survey]


Participant 18

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✓ YES
T3 Change title composition country/language ✓ YES
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (metronome)
T10 Invite collaborator ✗ NO

Feedback and comments:

[Survey]


Participant 19

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✓ YES
T3 Change title composition country/language ✓ YES
T4 Clone right template Happy Birthday ✗ NO[4] (cloned template for instrument and recorded voice)
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES[4] (used template for instrument but recorded voice)
T7 Create a collection ✗ NO
T8 Create new composition inside collection ✗ NO[7] (composition outside collection)
T9 Use metronome/use guide track ✓ YES (metronome)
T10 Invite collaborator ✗ NO

Footnotes:

Evaluator notes:

He used “Bluetooth buds” He recorded Au clair de la lune outside the main collection

Feedback and comments:

[Email]

[Survey]


Participant 20

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✓ YES
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✗ NO
T8 Create new composition inside collection ✗ NO[7] (composition outside collection)
T9 Use metronome/use guide track ✓ YES (metronome, guide track)
T10 Invite collaborator ✗ NO

Evaluator notes:

She recorded Au clair de la lune outside the main collection

Feedback and comments:

[Survey]


Participant 21

Task Description Result
T1 Record singing on an existing collaboration ✓ YES
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✗ NO[3]
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✗ NO
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES(metronome)
T10 Invite collaborator ✓ YES

Footnotes:

Evaluator notes:

Nice estimation of the latency (metronome recorded) The person cloned both templates for Happy Birthday (language and instruments but did not record anything inside any of them)

Feedback and comments:

[Survey]


Participant 22

Task Description Result
T1 Record singing on an existing collaboration ✗ NO
T2 Manual annotation track language ✗ NO
T3 Change title composition country/language ✗ NO
T4 Clone right template Happy Birthday ✓ YES
T5 Screenshot latency test ✓ YES
T6 Record on Happy Birthday copy ✓ YES
T7 Create a collection ✓ YES
T8 Create new composition inside collection ✓ YES
T9 Use metronome/use guide track ✓ YES (metronome)
T10 Invite collaborator ✗ NO

Feedback and comments:

[Survey]