Supporting evaluation in design

Design problem-solving requires designers to be creative and to express evaluative judgments. Designers propose successive partial solutions and evaluate these solutions with respect to various criteria and constraints. Evaluation plays a major role in design because each successive evaluation step guides the course of design activity. However, evaluation of design solutions is difficult for both experienced and inexperienced designers because: (1) in complex domains, no single person can know all the relevant criteria and constraints, and (2) design solutions must be evaluated from multiple, and sometimes conflicting, perspectives. Domain-oriented design environments have been proposed as computational tools supporting designers to construct and evaluate design solutions. Critiquing systems embedded in these environments support evaluation activities by analysing design solutions for compliance with criteria and constraints encoded in the systems' knowledge-base. To investigate the impact of such systems, we have designed, built, and evaluated a domain-oriented design environment for a specific area: phone-based interface design. Professional designers were observed using the design environment to solve a complex design task. Analyses of these design sessions enabled us to identify reactions common to all designers, as well as reactions depending on the designers' level of domain experience.