The retrieval of soccer highlights is a suitable technique for video indexation, required by the multimedia database management or by the development of television on demand. For these purposes, it should be interesting to have an automatic annotation of events happening in soccer games. On of the solutions consists in analyzing the audio soundtrack associated with the soccer video and to detect and index the interesting frames. In this paper we use the adaptive time-frequency decomposition of the soundtrack as a feature extraction procedure. This decomposition is based on the Matching Pursuit concept and a dictionary composed by Gabor function. [4] The parameters provided by these transformations constitute the input of the classification stage. The results provided for real soccer video will prove the efficiency of the adaptive time-frequency representation as a feature extraction stage.