Paradoxically, one of the hardest to measure characteristics of a sound reproduction device such as a loudspeaker is its sound quality. The perception of this subjective character is linked to numerous parameters (stimulus type, listening environment, etc.) that must be drastically controlled to lead to reliable and repeatable judgments. Industrial and academic researchers are still focusing on the design of standard assessment procedures. The conditions under which a sound reproduction system is assessed in laboratory tests is often very far from those under which it is designed to be used. As a result, the assessment task might appear unnatural to test subjects, which could possibly bias the test results. The aim of this study is to compare, on the basis of sound quality ratings, three different test procedures based on paired comparison and exhibiting procedural differences. One of the procedures consisted in comparing loudspeakers by listening to short music excerpts (5 s) at a preset level, which was assumed to be a very controllable method. In the two other procedures, the listener could compare the systems by listening to long music excerpts (30 s), which was assumed to be more natural for loudspeaker assessment. The level was either preset by expert listeners or set by the subject himself in the two latter procedures. This paper shows that the test results were very stable over the different assessment procedures, but that some of them enabled, under certain conditions, to separate between systems obtaining very close quality ratings.