Back when the web was only a limited affair and the most exciting thing on it in the realm of imagery was the ‘coffee pot webcam’ I came up with a way to make a real live video stream that worked in a browser. A few months earlier I’d made a very basic version of this using ftp upload of images to a server and then a cgi script to get them out to the browser by refreshing the page but this was horribly slow, on the order of one frame every 10 seconds on a very good day, with an annoying flash in between the frames. You really couldn’t call it video, no matter how you stretched the definition.
The idea of video over the internet definitely wasn’t novel, but the browser as the delivery medium for the stream definitely was. The computer I was using at the time, a very early model SGI Indy had something only very few computers at the time had: a video in port, and moreover a little camera called an indycam that connected to a special socket at the back of the computer.
The way in which that very first streaming webcam worked (this was in march 1995 just after moving in to our new spot in Hoofddorp, the Netherlands) was that each image was encoded to a single frame in a GIF animation. The gif was served via a tcp port and new clients connecting to the cam were given the last frame and then periodic updates as they became available from the video device. Depending on how fast the clients could deal with the images they got more frames per second or fewer frames per second. The program was really pretty trivial, a few hundred lines at best, and most of those boilerplate code to set up the sockets, control the video in port and compress the video frames. The magic (if you could call it that) was in the glue code between those parts and a highly optimized GIF compressor.
Of course, March ‘95 wasn’t exactly the best time to launch a live video product, there were hardly any computer connected cameras (the Connectix Quick Cam already existed but worked only with Macs afair) and the bandwidth really wasn’t there. 14.400 baud was hot stuff back then.
But it did work! Plenty of people that landed on that first webcam page sent me email along the lines that they thought that it looked just as if the image was live :) And within the week we had our first paying customer (only they wanted a colour version, so there was some work to be done).
For quite a few years we happily created products descended from that initial - and frankly quite silly, if I look at the code now I’m horrified - SGI specific program. We caused the sale of many Indy’s. A windows version was developed (Thanks Rembert!), camarades.com (now ww.com) was launched and we grew to a fairly large (for my taste) number of employees. Gray scale gave way to colour and GIFs were replaced by JPEGs, I wrote a companion program to process credit card payments to unlock access to the video streams later generalized that code to become the basis for ‘webpay’, a payment system for access to websites, audio support was added and we came up with a way to scale the output to 10’s of thousands of viewers using a network of cascading slave video servers (which came in handy during events, such as sporting events, space exploration and other moments where a single mid 90’s internet pipe (1 Mbit if you were very lucky, more likely a bonded ISDN line for a whopping 128Kbit of data) would not be able to satisfy the simultaneous requests). Most, if not all of these developments were driven by customer requests, all I did was make them happy and they kept buying more software (which wasn’t exactly cheap).
Then in 2002 or so a company called Acacia contacted our Toronto offices to discuss a license deal so that they wouldn’t have to sue us for violation of their patent filed in 1992 detailing digital video transmission over telephone lines.
I don’t think I’ve ever been as angry as on that particular day. After regaining my normal sunny compsure I went through the patent, read all the claims and realized they didn’t have a leg to stand on. In fact, I thought that their patent was a typical case of one that should have never been awarded in the first place. I most definitely wasn’t aware of the patent until they contacted me and I had a pretty good log of the way the original program was developed, complete with date-and-timestamps.
But that ‘92 date was quite nasty because if the patent ended up being rated as ‘valid’ I might end up having to pay a license fee to a company that never developed a product for the product that I had developed myself. Madness! When you’re out of your depth, consult an expert…
I talked it over with our lawyers in Toronto and the Netherlands and they jointly suggested we play the international card to our advantage to forestall a protracted lawsuit that we might even win (because the patent was absolutely ridiculous) but that would cost a small fortune and would take a long time. You see, the Toronto offices were technically in the jurisdiction of a court that was possibly amendable to such idiocy. But the Toronto offices didn’t have any rights, didn’t receive any royalties and didn’t do any software development, in fact it didn’t even use the software. All that stuff had remained in the Netherlands with the parent company (TrueTech BV, which had bought out the rights to the software from my one-man company at some point in the past). The Toronto offices basically did technical support for the website, server maintenance and some local administration, it didn’t make any money other than what it invoiced the Dutch company, the sum of the salaries, the rent and so on.
And Dutch courts at the time were very very tough on patents, especially those that read like ‘X’ but now digital, or even ‘X’ but now on the internet. That plus the home-court advantage would likely give us a win over this parasite (but whenever you go the courts you roll the dice, even with a good case).
And so, the next time Acacia called I told them in very polite terms to fuck off and that the Dutch parent company was the party they should sue if they still intended to do so.
Guess what, they never did. They probably made the same calculation that we had made and had come to the conclusion that it wouldn’t do to be seen to sue overseas in an unfriendly jurisdiction and then to lose because other parties might come to the conclusion that their patents were not as strong as they seemed. Besides that, you can’t really pluck too many feathers from a frog so to them it was all loss and very little upside.
Eventually, in September 2009(!!) a judge in the United States finally invalidated the patents.
The streaming media world breathed a collective sigh of relief and went on with their irreverent consumption of bandwidth, case closed.
And that’s where the story really should have ended.
And it did, until early 2010 when a new party started suing people left, right and center for violation of - you guessed it - streaming media patents, this time in combination with credit card payments and video chat services.
The previous round of research had prepared me (very) well for the request that landed at my doorstep on behalf of one of the defendants in these suits. If I could please conclusively prove that streaming video webcam chat services in combination with credit card payments existed before March 27th 1996. This time the patent troll was a company called Joao Controls.
I’m pretty anal when it comes to data retention, I can go back 25+ years when it comes to software that I have developed. And that came in pretty handy. I provided the counsel for the defense through the defendant (which I had had prior dealings with, they were an early licensee of the software) with all the relevant data and made myself available as an expert witness. It never came to that.
If anybody out there finds themselves sued because of violation of patents regarding streaming video and payment systems then here is a standing offer to give you a full packet of data with a detailed timeline documenting the first use of streaming video on the world-wide-web, sample images from public broadcasts using this technology from well published events (such as World Media Live using the software to broadcast the ‘96 Yves St. Laurent fashion show but there were plenty of others before that, it took a while to gain traction), the first use of pay-per-view and subscription based access to streaming video and audio and many more such milestones. This offer is free of charge, if it should come to any form of testimony then I’m more than happy to go where ever it is needed if you pay my way (airfare + basic accomodation is fine) or to give remote testimony via - rich irony here - video link, in which case it is free.
It is high time that this matter is laid to rest once and for all, Joao Controls is clearly abusing the legal system in the full knowledge that their patents are of 0 value.
I’ve never harped on public recognition for coming up with a way to stream live video to the browser (as far as I know it’s not mentioned in places like wikipedia) because as far as I’m concerned it was a rather trivial affair software wise. Some of the stuff I have built was several orders of magnitude more work and vastly more complex than this. To register a patent for something like that to me is abuse of the patent system. There is no substantial research to invest in, it is just basic plumbing, a re-use or composition if you want of existing elements in a new but obvious form. So, in order to frustrate any further attempts at shake-down of hardworking people by parasitic patent trolls I hereby stake a public claim on the creation of the first streaming webcam software, in March of 1995, to the claim of having the first combination of streaming video and credit card payments for pay-per-view and subscription based access in May of 1995 for a customer that became very well known with the product, and for plug-in free streaming video and audio combined in July of 1996. And I’m willing to stand by that claim and defend it with documentation if challenged. The software was first demonstrated to the outside world a day after it was first created in March of 1995 on http://www.mattheij.com/webcam/ and was on display there for several years. (The website has long since disappeared, the domain has lapsed once and passed to a relative, but there are still some vestiges of it in the archive, the oldest copy I can find is from Dec. ‘96, by that time we were selling commercial licenses in volume to anybody that could afford it, and I still have each and every one of those licenses. This was the version with the integrated credit card payment system hooks (webpay) and audio support). A full archive of source code as well as a whole raft of publicly documented instances of use of the software are available as well as the names and in some cases current contact information for a very large number of witnesses to the original development and use of the product at various stages of the development.
That should hopefully help to put the final nail in the coffin of this particular case of abuse of the patent system. 1 Down, a much too large number to go.
http://web.archive.org/web/19961229205008/http://mattheij.nl/
http://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=64601