In the '90's I used to be a regular at E3 where I would report as Press for various computer magazines on the latest in "VideoPlace" technology, of which MicroSoft's Kinect now appears to be positioned to take the lead.

"VideoPlace," generically, is any technology that allows the user to directly control a computer system via his video image.

See Patent Title: Real Time Perception Of And Response To The Actions Of An Unencumbered Participant/user, Application Number: 06/850,770, Patent Filed Date: April 11, 1986 by Myron W Krueger.

It appears Myron's original patent has run out.  Now the battle begins.  Myron, the real pioneer of VR, demonstrated VideoPlace in the '70's running off university mainframes.  He did thousands of tests and studies of real interactions and the parameters, which were summarized in his "Artificial Reality."

In 1986, a small company in Toronto released a full-fledged totally programmable version of VideoPlace called "Mandala" for the Amiga computer. It was a little jerky, as the Amiga system could handle about 14 frames/second capture and processing, but it was sufficiently usable that many people bought it for personal use or for museum virtual walk-throughs, rock concerts, etc. It was used with disabled as well, as in a paralyzed kid playing virtual drums.  Carel Stuycken ("Lurch") also designed virtual reality systems using CyberScape for retraining people who had serious disabilities due to injuries.

http://ecat.dcccafe.com/corpprofile.php?vendor_id=5002187

http://en.wikipedia.org/wiki/Carel_Struycken

Several lessor-known systems were also developed independently in the late '80's and early '90's, most depending on the Amiga as the only platform with the horsepower to do the job for under $2500.  The competing PC systems ran in the tens of $thousands.  In the Netherlands, for example, Phil Burk who later was one of the lead designers of the 3DO game system, designed his own version of VideoPlace on the Amiga that read facial expressions, for the express purpose of allowing a band of quadraplegics to perform music with their faces on TV.

http://www.softsynth.com/burkresume.html

Meanwhile, for the past quarter century, patent disputes have apparently kept VideoPlace largely off the market.  And when Sony finally introduced the EyeToy, they did their usual proprietary thing and restricted developer access.

Note that both the original "Mandala," released in '86, as well as its closest competitor, "CyberScape," provided a complete programming platform for user VR development. 

With both Mandala and CyberScape, various famous artists, such as Dean Friedman working with Nickelodean, created complex VideoPlace environs.  Both systems allowed total control over anything the computer could do, and Mandala included, in addition to the Mouse controlled GUI, frame accurate control over laser disk videos.  Thus, a background or an object could be quickly moved into a running VideoPlace.  Note that the Vivid Group, the creators of Mandala, released a series of demo videos, showing around a hundred different sample applications that they had built in-house to provide starting points for developers.

http://en.wikipedia.org/wiki/Dean_Friedman

I recall following a performance art crew as a journalist for several years while they developed Mandala apps for various show venues, usually in the Hollywood area.  One of the most bizarre involved virtual sex for the DragonFly club, owned partially by "Prince."  I helped do the graphics for the piece, which featured clothes that flew off the virtual models of men and women when you touched them with your on screen-hand.  (Further detail would entail altering the rating for this article...)

Meanwhile, in sharp contrast, Sony, with its usual "let them eat cake" attitude (see the Onion take on Sony for an objective and accurate portrayal), apparently released a Developers Kit for the EyeToy and then quickly cancelled it, thereby losing yet another opportunity to really take over the gaming market.  (How could Sony have missed the "Wii" aspect to gaming, which should have been a snap to implement on the PS?  Wait!  And what about CD/I?  Anyone reading this remember that gaming system, that Sony put $billions into in the late '80's, when every knowledgeable person in that market was screaming at Sony about how they had completely botched it?  How can a company as rich in money and talent as Sony keep screwing up so incredibly?)

An easy example of what could just as easilly have been - one of millions - suffices.  Suppose you create a videoplace for your parrot.  Parrots are highly intelligent and extremely sociable birds who get lonely if left alone too much, become depressed and hostile and pull their feathers out.  Passive entertainment (TV) just doesn't work (tried it).  But suppose you had an interactive gaming system for your parrot, in which the parrot would see its image in a real jungle or game environment with other parrots and humans.  If you had the bandwidth, you could make it even more real by linking up over the net to other parrots with the same system, so they could videochat.  They did this BTW - with humans (and virtual birds) - on the Mandala system circu 1989, linking conference members in Toronto and Paris in a seamless virtual whole.

(There was actually a project along these lines at MIT several years ago, but it used relatively clumsy buttons that the parrots had to be trained to press.  I attempted to call them to discuss using Videoplace in place of the buttons, but the project's interface person could not allow it, on grounds of intellectual property issues.  The ironic thing was that MIT's famous Media Lab had been showing up at virtually every SIGGRAPH with some kind of home-grown VideoPlace, generally reinventing the wheels that Myron had already tested and documented a couple decades prior.  So, they already had the tech in-house but were unable to discuss using it.)

Why NOT give the EyeToy users free reign to build applications to connect their pets to other pets around the world, to teach your pet where it may and must not go, to watch your infant while you're in the kitchen and sound an alarm if there is activity outside the perimeter of a crib?

Why shouldn't a submarine trainer be able to create an environment for the EyeToy - or the Kinect - just as easilly as a power point presentation, assigning logic to the images of buttons or other controls, allowing the trainee to enter and literally hands-on deal with all sorts of routine and emergency scenarios, 1,000 miles from deep water and all without building any physical LINK trainers?  It was fully doable in 1986 on an Amiga.  Why did Sony shut that option - and lose 99% of the potential EyeToy market? 

What will MicroSoft do?  Will they stupidly follow Sony's blunders and lock the system down from home, artistic, business, military - any kind of development not directly controlled by MS?  Will you have to become an official MicroSoft Developer to create new apps for Kinect?  Or, will their heritage of programming platforms - BASIC, Visual BASIC, etc. - lead them in the direction of providing the kind of user developer platform we had 24 years ago?

I don't know.  I have many, many outstanding criticisms of MicroSoft, who I think has cost the world at least several $trillion in lost productivity.  However, they sometimes do the right thing.  (They make good mice, at least.)

My criticisms, however, include a loathing for Visual BASIC, which is clearly designed to once again restrict access to those who have jumped the various hoops, such as spending at least a year just learning how to do anything with the software.  BASIC?  As in, simple, easy to use?  Give me a break!

There is a strategy, which is about to be seriously challenged by both "Cloud Computing" and also on the Smartphones, in which you defeat piracy by making software so difficult to use that the amateurs  and home users will barely be able to use it, while the professionals will have to keep coming back with license in hand to get updates, patches, bug fixes, malware security, etc.  Essentially, you forget about the average user and simply focus on the people who utterly depend upon having the very latest versions of Illustrator, etc. and/or 3D-Max to fork up the hundreds or thousands of dollars to stay abreast with the curve.

Let the amateurs pirate all they want.  You can bust them at will, with ENORMOUS fines, if they step out of line.  So long as the pros pay up, you're happy.  The home users would never pay for it anyway, so you haven't really lost anything by ignoring their piracy.  If they go pro at some point, the incentives are there to force them to fork up.  If your software is easy to use by anyone, however, this strategy doesn't work so well. 

Note that according to Steven Levy in his classic "Hackers," it was Bill Gates who initiated the whole marketing trend away from user-control and Do-It-Yourself software and towards the shrink-wrapped, non-modifiable packaged product when he began sueing people who used his original product, "MicroSoft BASIC," without paying for it.  The result of this policy is what we have today: bloatware filled with workarounds to avoid patent disputes and with obvious missing features that are missing because of intellectual property issues, plus long wait times for bug fixes, plus high prices, and the actual microserfs who write the actual code get a miniscule fraction of the profits as salary, while the marketing and management gurus who foist this junk off on us get filthy rich.

I should point out that there ARE alternate strategies.  As I suggested in the '80's, one could charge by usage - as may be the ultimate model for Cloud Computing.  Software authors could establish on-line libraries of custom functions available for micropayments for each actual running use.  Other authors could then modify, customize and improve that function and add a small charge of their own for people who used the updated library.  Everything would be available to everyone all the time, so long as you debited your account to pay the actual authors for the use of their product.  Creators of actual software packages would actually WANT you to know the source code, as that would enable other authors to improve it and increase or preserve the market share of the package.

In such a system, freed of the constraints of patent bondage, there would be every incentive to make things EASY to use.  Meta-packages - HTML and DreamWeaver as working models - would empower home users to write their own packages for their own custom applications.  Every additional minute of use would mean more money to the creators. 

I.e., much like the model for smartphone apps.  With largely free competition, prices for personal apps are kept in the range of the cost of a candy bar instead of a new laptop.  And, if you need 3D Studio Max for an hour or a day, why should you pay $3,000, the same as someone who is going to be using it continuously for the next two years and then upgrading?  Maybe Adobe will even make Illustrator usable without a six-month learning curve. (Dream on)... 

(I use COREL Draw at work, even though I have a legit copy of Illustrator, simply because there isn't time in the world to learn Illustrator, when 90% of my job is web design anyway, whereas anyone could start doing the simple stuff in general with COREL on day one without even a manual.  Similarly, we bought InDesign because our printer wouldn't accept PageMaker files for our magazine ads and flyers, but 99% of our manuals are still done in PageMaker 7, because it is much faster to use and anyone in the office or in the manufacturing plant in Taiwan can learn PageMaker in less than an hour.)

Getting back to Kinect, it is easy to foresee that this kind of interface has potentially very wide application.  Eventually the SmartPhones will be linked to lightweight HMDs, and our daily reality will be overlaid with all manner of objects, drop-downs, info-bubbles, acoustic enhancements, interpretations and commentaries, styles, overlaid fashions, color shifts and other visual enhancements, suggestions, orders, advice, and more, and our kids will be just as facile within that new world as the current generation is with a laptop or current SmartPhone.  Imagine being able to essentially see through someone else's eyes anywhere on the planet.  Something like Kinect will be necessary to put us in that overlaid reality so that our movements, and gestures directly interact with it.

(After covering various VideoPlace applications and systems for various publications in the late '80's and early '90's, I wrote about the more general subject of overlaid or "augmented" reality in 1993, in a published article in "Amiga User International" magazine, although the piece was attributed to my friend Michael Hanish by error at the magazine.  See also Vernor Vinge's HUGO Award winning "Rainbows End," for a great story and a highly detailed rendition of a future 25 years hence in which everyone "wears.")