Preliminary user evaluation of the Hi-Audio online platform for collaborative audio recording. Results from 22 participants including task-based performance, SUS, and NASA-TLX metrics.
View the Project on GitHub idsinge/hiaudio_user_evaluation_results
Anonymized record of per-participant task outcomes and feedback from the Hi-Audio user evaluation. Participants are identified by anonymous labels P01–P22. All email addresses have been replaced with JotForm Submission IDs. Comments are labeled by source: [Survey] (post-task questionnaire) or [Email] (follow-up correspondence). Evaluator notes (observations from recording review) are shown in blockquotes.
See SUPPLEMENTARY.md for aggregate tables (SUS, NASA-TLX, latency).
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✗ NO[2] (did not record voice, recorded instrument) |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✗ NO[3] (did change the title but in the template copy) |
| T4 | Clone right template Happy Birthday | ✗ NO[4] (used template for instrument but recorded voice) |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES[4] (used template for instrument but recorded voice) |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (guide track) |
| T10 | Invite collaborator | ✗ NO |
Footnotes:
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✗ No |
| T2 | Manual annotation track language | ✗ No |
| T3 | Change title composition country/language | ✗ No[3] (change title in the template copy) |
| T4 | Clone right template Happy Birthday | ✓ Yes |
| T5 | Screenshot latency test | ✓ Yes |
| T6 | Record on Happy Birthday copy | ✓ Yes |
| T7 | Create a collection | ✓ Yes |
| T8 | Create new composition inside collection | ✓ Yes |
| T9 | Use metronome/use guide track | ✓ Yes (metronome) |
| T10 | Invite collaborator | ✓ Yes |
Footnotes:
Feedback and comments:
[Email]
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES[5] (created a new separated version, collaboration expired) |
| T2 | Manual annotation track language | ✗ NO[15] |
| T3 | Change title composition country/language | ✓ YES[6] (changed in the new separated version) |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (metronome) |
| T10 | Invite collaborator | ✗ NO |
Footnotes:
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✗ NO[2][5] (collaboration expired, but did not sing only played piano) |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✓ YES[6] (changed in the new separated version) |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (metronome) |
| T10 | Invite collaborator | ✓ YES[10] |
Footnotes:
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✗ NO |
| T4 | Clone right template Happy Birthday | ✗ NO |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✗ NO |
| T7 | Create a collection | ✗ NO |
| T8 | Create new composition inside collection | ✗ NO[7] (composition without collection) |
| T9 | Use metronome/use guide track | ✓ YES (guide track) |
| T10 | Invite collaborator | ✗ NO |
Footnotes:
Evaluator notes:
(Tracks not well sync, latency might be wrong estimated or simply bad performed)
Feedback and comments:
[Email]
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES[8] (he started the tasks before having the collaboration set) |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✓ YES[6] |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (guide track) |
| T10 | Invite collaborator | ✗ NO |
Footnotes:
Evaluator notes:
Conclusion: he confirms he added the email to invite “hiaudioparis@gmail.com” but eventually no collaborator is present which means the process of invitation was not clear enough and he did not confirm in the pop-up dialog,
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✗ NO |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✗ NO |
| T4 | Clone right template Happy Birthday | ✗ NO[4] (recorded voice on instruments template) |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✗ NO[9] (Au clair de la Lune) |
| T8 | Create new composition inside collection | ✓ YES[9] |
| T9 | Use metronome/use guide track | ✓ YES[9] (Au clair de la Lune template) |
| T10 | Invite collaborator | ✓ YES[10] (invited everywhere not only first composition) |
Footnotes:
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✓ YES |
| T3 | Change title composition country/language | ✗ NO |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (guide track) |
| T10 | Invite collaborator | ✓ YES |
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✓ YES |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES[12] |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (metronome, guide track) |
| T10 | Invite collaborator | ✓ YES |
Footnotes:
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✗ NO[3] |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✗ NO |
| T8 | Create new composition inside collection | ✗ NO[7] (composition without collection) |
| T9 | Use metronome/use guide track | ✓ YES (guide track) |
| T10 | Invite collaborator | ✗ NO |
Footnotes:
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✓ YES[13] (on the title of the track) |
| T3 | Change title composition country/language | ✗ NO[3] (changed on the clone) |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (guide track) |
| T10 | Invite collaborator | ✗ NO |
Footnotes:
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✓ YES |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✗ NO[7] (composition without collection attached) |
| T9 | Use metronome/use guide track | ✓ YES (guide track) |
| T10 | Invite collaborator | ✗ NO |
Footnotes:
Evaluator notes:
(No headphones he says)
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✓ YES |
| T3 | Change title composition country/language | ✓ YES |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (metronome, guide track) |
| T10 | Invite collaborator | ✓ YES |
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✓ YES |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✗ NO |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✗ NO[9] (record Au Clair de la Lune) |
| T8 | Create new composition inside collection | ✓ YES[9] (Au clair de la Lune) |
| T9 | Use metronome/use guide track | ✓ YES[9] (Au clair de la Lune) |
| T10 | Invite collaborator | ✓ YES |
Footnotes:
Evaluator notes:
No latency value provided, from the recordings I can tell there’s delay and no earphones were used so there’s feedback (echo) in the recording
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✓ YES |
| T3 | Change title composition country/language | ✓ YES |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES |
| T10 | Invite collaborator | ✓ YES |
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✗ NO |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✓ YES |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (metronome) |
| T10 | Invite collaborator | ✗ NO |
Evaluator notes:
He managed to change the title in collaboration but there’s no audio recorded, maybe the track was deleted before being sync with the server
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✗ NO[2] (did not sing) |
| T2 | Manual annotation track language | ✓ YES |
| T3 | Change title composition country/language | ✓ YES |
| T4 | Clone right template Happy Birthday | ✗ NO[14] (cloned language and played trumpet) |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES[14] (using wrong template) |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (metronome) |
| T10 | Invite collaborator | ✓ YES[10] |
Footnotes:
Evaluator notes:
(On Android phone Pixel, probably bluetooth headset: Marshall Major IV) He did record Au clair de la Lune but he created a new collection and composition himself for it
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✓ YES |
| T3 | Change title composition country/language | ✓ YES |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (metronome) |
| T10 | Invite collaborator | ✗ NO |
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✓ YES |
| T3 | Change title composition country/language | ✓ YES |
| T4 | Clone right template Happy Birthday | ✗ NO[4] (cloned template for instrument and recorded voice) |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES[4] (used template for instrument but recorded voice) |
| T7 | Create a collection | ✗ NO |
| T8 | Create new composition inside collection | ✗ NO[7] (composition outside collection) |
| T9 | Use metronome/use guide track | ✓ YES (metronome) |
| T10 | Invite collaborator | ✗ NO |
Footnotes:
Evaluator notes:
He used “Bluetooth buds” He recorded Au clair de la lune outside the main collection
Feedback and comments:
[Email]
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✓ YES |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✗ NO |
| T8 | Create new composition inside collection | ✗ NO[7] (composition outside collection) |
| T9 | Use metronome/use guide track | ✓ YES (metronome, guide track) |
| T10 | Invite collaborator | ✗ NO |
Evaluator notes:
She recorded Au clair de la lune outside the main collection
Feedback and comments:
[Survey]
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✓ YES |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✗ NO[3] |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✗ NO |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES(metronome) |
| T10 | Invite collaborator | ✓ YES |
Footnotes:
Evaluator notes:
Nice estimation of the latency (metronome recorded) The person cloned both templates for Happy Birthday (language and instruments but did not record anything inside any of them)
Feedback and comments:
[Survey]
I don’t find how modifie the description. The place where are the buttons is not instinctive and natural. For exemple the clone’s button in the top right hand corner. (Then I don’t find the colour of buttons always instinctive)
| Task | Description | Result |
|---|---|---|
| T1 | Record singing on an existing collaboration | ✗ NO |
| T2 | Manual annotation track language | ✗ NO |
| T3 | Change title composition country/language | ✗ NO |
| T4 | Clone right template Happy Birthday | ✓ YES |
| T5 | Screenshot latency test | ✓ YES |
| T6 | Record on Happy Birthday copy | ✓ YES |
| T7 | Create a collection | ✓ YES |
| T8 | Create new composition inside collection | ✓ YES |
| T9 | Use metronome/use guide track | ✓ YES (metronome) |
| T10 | Invite collaborator | ✗ NO |
Feedback and comments:
[Survey]