Category Archives: Prototyping

Prototyping – a short summary

The following post is a summary of goals and kinds of prototyping as well as a short analysis of best practices. This was an assignment for my MSc course Requirements Engineering.

Goals of prototyping

The goals of prototyping depend on the phase of the development process where it is applied. Examples for the use of prototyping can be found for requirements elicitation, requirements validation, design and testing [14]. As some goals apply to more than one phase, no distinction between the phases is made in the following list.

  • improve the planning and succes of software projects [754]
  • facilitates user participation and getting feedback by ”making it easier for people to imagine what it might be like to use the real product to do work” [105]
  • provides a communication basis for discussions between users, developers, and other stakeholders [7]
  • to elicit requirements from users [1214] by helping to find the requirements before the product is built [10]
  • increases quality of requirements by helping correcting misunderstandings and ambiguities of requirements specifications [1213414]
  • exploration and testing the product [13]
  • identify shortcomings and design inconsistencies [13514]
  • prevention from requirements- and scope-creep [13]

Kinds of prototyping

In the literature many different classifications of prototyping can be found. However, they all fall into one of these categories: prototyping processes, prototyping products [7], and relations between prototyping processes and products [10]. To classify the different kinds of prototyping we distinguish between these categories.

Prototyping processes

In [127], the following types of prototyping processes are described:

  1. Exploratory prototyping
  2. Experimental prototyping
  3. Evolutionary prototyping

Exploratory prototyping explicitly involves user participation. [1213] describe the following participation types: prototyping by demonstration, prototyping by testing, cooperative prototyping and user-led prototyping.

Prototyping products

Two main distinctions can be found in the literature. The first divides into horizontal (aka breadth) prototypes and vertical (aka depth) prototypes [12714]. Horizontal prototypes include only specific layers while vertical prototypes implement a part of the system completely.
The second one uses the fidelity of prototypes as distinction (where fidelity is the level of detail in the representation of the prototype [13]). They are classified into low-fidelity prototypes (e.g. paper prototypes), high-fidelity prototypes (e.g. interactive prototypes which nearly equal the system [10], and mixed-fidelity prototypes (a mixture of the two other types [15]).

Related to processes and products

Another distinction is the relation between prototypes as product and the prototyping process (also described in [10]):

  • Throwaway prototypes are used for specification purposes only, they are not used as actual building blocks [7]. They are mostly used in exploratory and experimental prototyping.
  • Evolutionary prototypes provide a basis for the real system which is evolved out of the prototype, they are used in evolutionary prototyping but can also be found in experimental prototyping (if it shows that the prototype provides a good basis for a system) [7].

Robertson et al. also describe the prototyping techniques Storyboards and Object Life History (aka state model) [10], which are not further discussed here.

Where is each kind of prototyping well suited for and where less (related to the goals of prototyping)

  • Exploratory Prototyping – User participation with prototyping by demonstration is not suited for identifying shortcomings and design inconsistencies because the prototype is not exercised by the user [13];
  • Experimental Prototyping – Not much can be found in the literature about experimental prototyping. One of the reasons might be that this actually is common practice without recognizing that this practice is experimental prototyping. Brooks describes in [2] that one has to plan to throw one away, meaning that the first version of program is mainly for experimenting and learning how to do it and then building a second, better version based on the experience made. Actually this practice is experimental prototyping.
  • Evolutionary Prototyping – Hekmatpour shows in [4] that evolutionary prototyping is also applicable for large software projects.
  • Horizontal Prototypes – Useful for eliciting requirements from the user if user interfaces (with forms and menus) are used [7]. It also can be used to explore specific behaviours of a system and letting users if the system based on the prototype is usable for them [14].
  • Vertical Prototypes – help in identifying shortcomings and design inconsistencies while exploring different implementation and functionality options [714].
  • Low-fidelity Prototypes – Can be used if prototyping needs to be cheap and fast [1314]. They enable stakeholders to give more feedback when used early in the development cycle, as the stakeholders then are less fixated on design or appearance issues and more interested in functionality and overall structure [10]. Many non-functional requirements–security, maintainability, and so on–can’t be easily elicitated with low-fidelity prototypes [10]. Low-fidelity prototypes are especially suitable for requirements elicitation, as they facilitate rapid iteration [14].
  • High-fidelity Prototypes – Help in finding more usability problems than low-fidelity prototypes [1310]. They also help in exploring exceptions and alternative paths [10].
  • Mixed-fidelity Prototypes – With mixed-fidelity prototypes it is possible to cover different aspects and properties of a future system appropriately [159], which helps in achieving many of the earlier described goals. Lim et al. also describe the effectiveness of this kind of prototypes [8].
  • Throwaway Prototypes – They are well suited if a quick implementation is needed and the requirements are vague, uncertain, ambigue, or incomplete [14]. They also can be used for experimental prototyping [14].
  • Evolutionary Prototypes – Vertical prototypes are often evolved into final systems, e.g. the architecture prototype in RUP [6].

What can you do wrong using the different kinds of protoyping (what’s the difference between doing it good and doing it bad)

  • Exploratory Prototyping – The effort on the end-users side if often ignored or underestimated [7]. Verhamme concludes in [13] that in order to identify UI change requests the prototype should not bet demonstrated to the user, but the users should perform tasks using the prototype themselves. Another important thing mentioned in [114] is the proper recording of feedback and reactions from users while they explore the prototype, as this flood of information is valuable.
  • Experimental Prototyping – The important part in experimental prototyping is to clearly define the purpose and how to measure the achievement of the purpose [10]
  • Evolutionary Prototyping – Hekmatpour gives in [4] a great summary of lessons learned in applying evolutionary prototyping in large software projects. The most important ones are a disciplinary approach, keeping the design clean and structured, and having a good prototyping support environment.
  • Horizontal Prototypes – If UI-prototypes are only used to gather functional requirements and not for validating the user interfaces, this should be explicitly made clear to the users. Otherwise there’s a chance that the users expect that same look and feel (and performance) in the final product [514]. Wiegers mentions that one should use actual data to enhance the validity of a prototype as model of the real system [14].
  • Vertical Prototypes – When building vertical prototypes the same construction tools should be used as in the a production-like operating environment [14].
  • Low-fidelity Prototypes – These are not suitable for giving the users the ”feel” of the future system [15].
  • High-fidelity Prototypes – If these are used to early in the development cycle, the stakeholders will concentrate on the appearance and less on making improvement or requirement suggestions [10].
  • Mixed-fidelity Prototypes – The fidelity used for aspects of the system has to match the goals of prototyping of these system parts [159].
  • Throwaway Prototypes – Brooks states in [2] that one has to plan to throw one away, essentially describing an exploratory throwaway prototyping approach. Lichter et al. state in [7] that this is problematic as traditional project planning and control techniques often don’t take prototyping approaches into account. The focus on milestones and deliverables in software processes rarely fits together with prototyping cycles [7]. So, one has to make the decision in advance if the prototype is really thrown away or elaborated into the final product [14].
  • Evolutionary Prototypes – ”They must be designed for easy growth and frequent enhancement, so developers must emphasize software architecture and solid design principles.” [14].

In general prototyping can be made more convinient and accurate if a single use case (business or product) is involved [10]. The advantage of this is that a use case has a known, measurable, testable outcome, which helps in examining the prototype [10]. Prototyping should only be used for requirements which aren’t already understood [14].

How good do the kinds of prototyping work according to literature

  • Exploratory Prototyping – This was succesfully used for the design of a new operation system and in other projects described in [5].
  • Experimental Prototyping – No explicit data found, but as this approach is closely related to low-fidelity prototypes and horizontal prototypes it can be assumed that it is succesfully applicable.
  • Evolutionary Prototyping – An evolutionary approach, where in the beginning UI-prototypes are used and later expanded with functionality with the goal to deliver the final prototype as the required system is not recommended uniformly by Lichter et al.[7]. They state that it first has to be made clear which questions the prototype should answer before choosing the methodological approach.
  • Horizontal Prototypes – Lichter et al. say that horizontal prototyping (in the form of user interfaces) is the best solution for the area of database systems, as the functionality of these systems is largely prescribed and experience with similar problems is available [7]. Houde and Hill offer examples of succesful applications of horizontal prototypes [5].
  • Vertical Prototypes – The Rational Unified Process uses as one of the key practices a vertical architecture prototype in the elaboration phase of a project [6].
  • Low-fidelity Prototypes – In [5] project are described which used low-fidelity prototypes succesfully. Sefelin et al. show that it makes no difference if low-fidelity prototyping is performed paper-based or computer-based [11].
  • High-fidelity Prototypes – Houde and Hill describe in [5] many projects where high-fidelity prototypes were succesfully used.
  • Mixed-fidelity Prototypes – In [1589] succesful projects using mixed-fidelity prototypes are described.
  • Throwaway Prototypes – In [5] a few succesful examples of throwaway prototypes can be found.
  • Evolutionary Prototypes – Again, if it is explicitly included in the software process than evolutionary prototypes can be used succesful. This is applied with the architecture prototype in RUP [6]. Wiegers reports about using evolutionary prototypes for internet development projects [14].

Why prototyping doesn’t necessarily lead to succes

The following is a list of reasons why prototyping could fail due to human causes. The first three of these are discussed in more detail in the next section.

  • Implicit knowledge and experience
  • A throwaway prototype is not thrown away [3].
  • The terms prototype and prototyping as well as the concepts behind their use are only vaguely defined [7514]. This can lead to misinterpretation of the goals and usage of prototyping.
  • More details in prototypes (as in high-fidelity prototypes) restrict the creativity of users if it comes to new ideas and requirements [7]. If the representation of a prototype includes many details and much functionality, the shared knowledge of the users is biased by the prototype [12].
  • Not giving users high responsibility in the development of a prototype will decrease the quality of it [12].
  • No cooperation between developers and users leads to lower quality of acquisition and exploitation of knowledge in a prototyping process [12].
  • Gradient of resistance: Timmermann states that ”the more effort a developer puts in the development of a prototype, the more a prototype limits the acquisition and exploitation of users’ knowledge” [12].
  • The proactiveness of a person affects knowledge sharing [12].
  • End user involvement in the creation of a prototype is important as the lack of it can lead to acceptance problems of the system [7]. It is also of crucial importance in clarifying the requirements of a system [710].
  • User representatives should have a clear enough idea what to expect from the system, he/she should know what the real users need [714].
  • The drivers from responsible stakeholders behind the decision if and how to use prototyping play an important role in the succesful application of it [7]. This is especially true if it is decided to not let end users participate in the prototyping process (because of e.g. staff rationalization or the believe that intermediate management knows the needs of end users better than the end users themselves [7].
  • Not thinking about the kind of prototype used and which aspects are covered by it–and which are not–might lead to problems when building a prototype [7105].
  • If no measures for the succes of a prototype are defined, it’s not possible to determine if the intended purpose has been achieved [10].

Discussion of three reasons and known countermeasures from other areas

Implicit knowledge and experience

All stakeholders involved in prototyping bring their own work-related baggage to the prototyping exercise [10], which implies different perceptions of work. Handling this requires tact on part of the requirements
elicitor. It is important to know the metaphors used by the stakeholders for their work and how they envision themselves while doing their work [10]. So one of the human problems is to turn away from ones experience and comfort and accept a new probable new, as-yet-untried way of doing their work [10].

Throwaway prototype not thrown away

One of the reasons that ”quick-and-dirty” throwaway prototypes evolve into the final system is that this
is enforced by the expectations of the clients [7143]. As the purpose of throwaway prototypes is e.g. demonstrating feasibility, and therefore no focus is put on the implementation of this prototype, experience shows if these prototypes eventually evolve into the final system they turn out to be difficult to maintain and develop [514]. Also temporary user interfaces are often not redesigned properly before the final release of the system [5].
Foote and Yoder describe this in the pattern Throwaway Code (aka permanent prototype) [3]. One of the
countermeasures they suggest in [3] is to write the prototype in a language or tool couldn’t possibly be used for a production version of the product.

Vague definitions of goals and usage of prototyping

The terms prototype and prototyping as well as the concepts behind their use are only vaguely defined [7]. This can lead to misinterpretation of the goals and usage of prototyping. One of the extremes is that ”anything goes” [7], meaning that practices are classified as prototyping without further definitions or
that organizations have their own ”prototyping cultures” [5]. Another consequence of these vague definitions is that prototyping as a method can not be defined exactly and therefore cannot be used for professional software development [7].

Robertson states that prototyping is not for designing the product, but for capturing all things it might possibly do to help the stakeholders do their work [10].

Which known countermeasures can be found in prototyping and are these sufficient

In the section on how to do prototyping good, a couple of countermeasures already are described. Most prototyping techniques don’t explicitly include countermeasures.
However, a few approaches can be found to minimize some of the known problems. Houde and Hill offer a model for classifying questions which should be answered by a prototype and which therefore determine the prototyping approach [5]. These classes of questions are about Implementation, Role, and Look and Feel.
Wiegers states that one of the key factors for succesful prototyping is expectation management [14]. The goals of prototyping have to be clear for every participant.

References


[1]   
Ian Alexander and Richard Stevens. Writing Better Requirements. Addison-Wesley Professional,
August 2002.

A small book with a collection of techniques and practices which can be applied while eliciting and documenting requirements.


[2]   
Frederick P. Brooks, Jr. The mythical man-month (anniversary ed.). Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1995.


[3]   
Brian Foote and Joseph Yoder. Big ball of mud. In Pattern Languages of Program Design, pages 653–692. Addison-Wesley, 1997.

A classic paper about patterns in software development. These patterns don’t describe best practices, but the reality of software construction and evolution. They help understanding why things go wrong and therefore also can help to improve practices in software construction and maintenance and as a consequence also of software processes.


[4]   
S Hekmatpour. Experience with evolutionary prototyping in a large software project. SIGSOFT Softw. Eng. Notes, 12:38–41, January 1987.

This paper shows that evolutionary prototyping can be succesfully applied in large software projects. They also summarize a number of lessons learned.


[5]   
Stephanie Houde and Charles Hill. What do prototypes prototype? Handbook of Human-Computer Interaction, 2008.

Houde and Hill offer a model which helps designers to think about prototypes differently. Their model focuses on questions about the design of the artifact itself: What role will it play in a users life? How should it look and feel? How should it be implemented? They offer a lot of examples of different kinds of prototyping and use their model to demonstrate the added value of it.


[6]   
Philippe Kruchten. The Rational Unified Process: An Introduction. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 3 edition, 2003.


[7]   
H. Lichter, M. Schneider-Hufschmidt, and H. Zullighoven. Prototyping in industrial software projects-bridging the gap between theory and practice. In Software Engineering, 1993. Proceedings., 15th International Conference on, pages 221–229. IEEE, 2002.


[8]   
Youn-Kyung Lim, Erik Stolterman, and Josh Tenenberg. The anatomy of prototypes: Prototypes as filters, prototypes as manifestations of design ideas. ACM Trans. Comput.-Hum. Interact., 15:7:1–7:27, July 2008.

Lim et al. describe a taxonomy for prototypes and how this helps in understanding qualities of prototypes in a more conceptually structured and pragamtically useful way.


[9]   
Michael McCurdy, Christopher Connors, Guy Pyrzak, Bob Kanefsky, and Alonso Vera. Breaking the fidelity barrier: an examination of our current characterization of prototypes and an example of a mixed-fidelity success. In Proceedings of the SIGCHI conference on Human Factors in computing systems, CHI ’06, pages 1233–1242, New York, NY, USA, 2006. ACM.


[10]   
Suzanne Robertson and James Robertson. Mastering the Requirements Process (2nd Edition). Addison-Wesley Professional, 2006.


[11]   
R. Sefelin, M. Tscheligi, and V. Giller. Paper prototyping-what is it good for?: a comparison of paper- and computer-based low-fidelity prototyping. In CHI’03 extended abstracts on Human factors in computing systems, pages 778–779. ACM, 2003.


[12]   
Arne Timmermann. Prototyping – a stepping-stone to more succesful prototyping. Master’s thesis, University of Amsterdam, 2008.

Timmermann proposes a theoretical framework for addressing the question of how prototyping can be applied best in order to increase the succes of software. In an experiment it is shown that cooperative prototyping and the use of low-fidelity paper prototypes lead to more, and more valuable, ideas articulated by the experiment participants. The amount of (valuable) ideas is correlated to the sharing of knowledge. more sharing of knowledge increases the quality of the software requirements, where also the quality of the software depends on.


[13]   
Dennis Verhamme. Effective prototyping in embedded systems – comparison of high- and low-fidelity prototyping. Master’s thesis, University of Amsterdam, 2009.

This thesis shows that high-fidelity prototypes (simulations) help in finding more usability problems than low-fidelity, paper-based prototypes.


[14]   
Karl Eugene Wiegers. Software Requirements. Microsoft Press, Redmond, WA, USA, 2 edition, 2003.


[15]   
Ansar-Ul-Haque Yasar. Enhancing experience prototyping by the help of mixed-fidelity prototypes. In Proceedings of the 4th international conference on mobile technology, applications, and systems and the 1st international symposium on Computer human interaction in mobile technology, Mobility ’07, pages 468–473, New York, NY, USA, 2007.
ACM.

Yasar explains some shortcomings from low- and high-fidelity prototypes using a concrete example. Then he shows how mixed-fidelity prototypes can help to reduce these shortcomings using an example from the NASA.

Advertisements