The question of prototypes

having worked for now more than a decade in the design and innovation field the question of prototypes keeps coming up again and again.
of course there are many ways of doing it but it is their purpose that I have been trying to nail. Is it a simulation for checking the principle? is it a method to check it's value with the users? is it for demonstration? etc etc.
Within the technology space where we are constantly looking for the next big wow how does one deal with prototyping. This is a question often asked mostly by people who are constantly watching the dollars drain in the spread sheets. Some time ago a manager who had been working on creating a working demo for his technology concept caught up with me and asked me if there were any shortcuts to achieve this final objective. In this case for a long time they were just trying to work out something that could be used so that they could test it out with people and basically validate the experience. Now looking back if that was the end in mind one could have easily mocked the experience with a technique usually known as the wizard of oz.
Now the question is that is there anything that was achieved by actually making a working demo that is of more value than the wizard of oz.
Then the question is why do we prototype. what are all those things that should be considered before prototyping. Are there any guides to this decision making process that can help innovation groups decide on the kind of prototyping that should be employed in their context.
So I'm going to just try to make a list of all the kinds of things that could be done and let's see where this all leads:

• Wizard of oz simulation (just fake the entire user experience)
  
• Working principle model (works but looks like something out of a garage)
• paper prototypes (wire frames on paper, good for software UIs)
• HTML prototype (actual pages that can be navigated through)
• Styrene and clay models
• Detailed model (looks like the real thing but doesn't work)
• Concept sketches (just some doodles on paper)

OK now that it's all out lets see if we can dig within the basic act of representation and see what is the true spirit of this activity. So what do I do when say I get an idea, I draw it on paper. This act of drawing or sketching itself seems like an act of representation of the thought. Digging deeper it feels like the individual seems convinced about the thought. Once on paper the thoughts enters a very different phase. Within the brain the ideas were generated due to a creative process. This was followed by analysis within the brain the output of which was an excitement that triggers the want to see the idea on paper(basically represented.. Once on paper the mind is focussed around the core concept and seeks to refine it. One needs to know what to refine in order to refine. Things get interesting once we start looking at almost everything around us as a product of ideation. It would be interesting to know of the tools for instance a programmer would use in order to get a feel for his/her idea. The obvious one in their case is to just go ahead and try to code it. things will get tricky once there are more ideas. So what does one do then. because everything cannot be coded. There has to be some sort of a selection and prioritization that will help select the contextual focus.