The Pictorial Usability Inventory (PUI) is an image-based instrument that was designed for the assessment of perceived usability of smartphone apps. The aim was to provide users with an easy and intuitive access to the evaluation of perceived usability. PUI was developed in several iterations using a user-centred approach. The pictorial items were based on selected items of various standardized usability questionnaires. A pilot study with 83 participants was conducted online to test the psychometric properties of the newly developed instrument. The factor structure, reliability, convergent, divergent and criterion-related validity, and sensitivity of the instrument were evaluated. Furthermore, respondent motivation, questionnaire preference, and questionnaire completion time were assessed. Participants interacted first with a prototype of the app in which the product-inherent usability was manipulated on two levels. The manipulation was implemented using a between-subjects design. Afterwards, participants completed the PUI and further verbal instruments to evaluate usability, as well as further questionnaires to determine the different forms of validity. First results indicate good psychometric qualities of the scale and an increased motivation among participants when completing the questionnaire. Nevertheless, 60% of the participants preferred the verbal scales. In addition, questionnaire completion time was longer for verbal scales. Some ideas for the further development of the PUI are discussed.