Perception of timbre features by cochlear implant users

Abstract

Timbre perception is fundamental to music enjoyment. Various cochlear implant (CI) studies have investigated the identification of musical instruments and the perception of timbre, finding that the ability of CI users to perceive timbre is poor compared to normal hearing (NH) listeners. To better understand timbre perception for CI users, the limitations that cause poor timbre perception in CI users should be investigated when performing timbre discrimination tasks. Therefore, the present study investigated the perception of the timbre by measuring discrimination abilities using just-noticeable differences (jnds) for a set of representative acoustic features that underlie timbre perception.

Two spectral features, brightness and irregularity (referred to as brightness and IRR), and a temporal feature, logarithmic rise-time (LRT), were identified in the literature as salient timbre features. The timbre features were used in a synthesis model to create a set of nine synthetic instrument tones. The latter allows for independent variation of the timbre features. Synthetic tones were used in a two-alternative forced-choice (2AFC) experiment to measure jnds for NH individuals and CI users for each of the timbre features. The data showed that the jnds of CI users were larger than those of NH individuals. The findings suggested that CI users had difficulty to attend to the timbre feature when performing the discrimination tasks.

To investigate whether or not CI users had access to the timbre features, electrodograms were used to analyse the jnds. Electrical stimulation pulse trains of the original instrument sound were generated and compared with the electrical stimulation pulse trains generated at jnd for each of the CI users. Difference metrics were calculated to determine whether CI users had access to the timbre feature or only to the difference between the reference and probe electrical stimulation signal. Spatial and temporal differences between reference and probe stimulation signals showed that CI users did not have access to the timbre feature, but rather to the differences in electrical pulse trains.

The extent to which CI users received the timbre features was investigated using feature information transmission analysis (FITA). This estimated the percentage of available information of the timbre features that CI users received. Confusion matrices were predicted from the jnds of CI users to perform the FITA. The results showed that the information received by CI users is user-dependent and that the information received for each of the features is mostly the same within users. These findings support the notion that CI users probably did not attend to the timbre features and conceivably did not have adequate access to these.

The representation of the spectral harmonics of the musical instrument tones by the electrical stimuli was investigated. The spectral representation in the electrical stimulus pattern was found to be a distorted version relative to that of the acoustic sound.

The study aimed to answer the question To which extent is musical timbre perceived by CI users and what underlies this? A core objective was to understand what constraints underlie timbre perception. It was concluded that CI users do not attend to the timbre features when performing timbre discrimination tasks, and that the electrical stimuli representing the instrument have a distorted spectrum relative to that of the acoustic sound.