>> Saturday, December 2, 2006
One month to go and it's 2007; which means it will be ten years ago the
Philips Interactive Media Centre was transformed into a new company with
the same name. The P for Philips only changed into Professional, so the
testing facility is now called Proffesional Interactive Media Centre. The
new PIMC went independant and settled in the Science Park in Diepenbeek.
The old PIMC was the testing facility for Philips Media and tested CD-i
discs as well as other formats like CD-ROMS and VCD's. In the years between
1997 and 2007 the company has built up more and more experience with
testing multimedia products, still concentrated on Interactive Media on CD,
DVD and flash tools (cell phones). Now the PIMC received a ISO 9001
certificate and it became a certified lab for USB2.0.
Still, the Philips link wasn't completely lost, and they both investigated in projects for consumer network appliances.
A member from our CD-i forum worked at the PIMC and shared a nice story about his time with Philips (unaltered, uncensored text): Testing was done at customer's request. We offered a couple of standard packages, like compatibility on a few players and cartridge/player combinations, input devices etc, or the full suite of all players etc... Sometimes customers wanted only a limited test, like "test those and those PAL players and pay only attention to sound". I remember the 220/40 having a bad rep with sound issues, swicthed channels or something...
Once a test came in, a coordinator was assigned and he checked basic functionality first - making sure we didn't schedule time and set up a team etc. only to discover the disc wouldn't make it past the bumper. Next, depending on the test, a test sheet was made. It stated what tests needed to be done (agreed with customer), a list of players, cartridges, input devices etc that got ticked off as each was tested.
Some basic tests were done on each player. For instance, if you could interrupt the bumper, did the disc continue if the bumper was interrupted or if you let it play (you'd be surprised how many discs actually had issues here, especially in the beginning), is a copyright message present, is it in the right language and is it the right message - Philips had strict rules on these messages, they had them prewritten. Studio's not dependant on Philips of course didn't have the same rules, and one had to pay attention for whom you were testing. Next input devices were all tested, making sure they were supported and on both inputs. Nasty bit, some input devices simple never worked on the back inputs, there were 2 types of mouse, and one wouldn't work on some players, and that sort of thing. Constantly needed to remember which combinations were not bugs but problems with devices or players. Gamepads even needed to be tested on all speed settings... ugh... Some cartridges always gave green flashes on some players. 905 and 205, some others, depended on the motherboard versions (sorry, no longer remember the tech details here) so those also had to be taken into account.
Then you had basic tech tests. We used an old 180 for some of them - had to put in an OS9 floppy and type some commands to load data from the disc then play it. Don't remember all the tests but they were for instance about audio warning messages - those things you get to hear when you popped in the disc in an audio CD player, that warn you to turn down volume or the next tracks will damage your speakers. These messages sometimes were incomplete, not clear, cut off, or not there at all.
Compatibility testing was all in all really routine and a bit of a drag - just playing the title on all players for a few hours, making sure all hotspots could be reached (sometimes the cursor wouldn't move over the entire screen, usually PAL/NTSC compatibility issue), checking if audio channels were not switched, making sure all types of input devices were covered, on both inputs (back and front), all cartridges, do the 605 test (various memory settings and a few other parameters in the 605 menu were changed, have to check on my 605 which, don't recall just now), do a safety area test for PAL and NTSC.
Now that last bit was really a bore Laughing You had to go through the entire disc and make sure no hotspots or crucial graphics or text were outside the safety area on the screen. We used an old Amiga 500 with a genlock for that - it overlaid the safety area boundaries on the screen. Calibrated of course with the attached 605.
Etc. etc...sorry if all this is kinda unstructured but I'm racking my brains to remember all this as I write it. We used to have a couple of sheets were all this stuff was listed and you ticked off boxes to indicate it was tested. A coordinator made sure all bases were covered and steered a team of testers, 1 to 10-15 people depending on the project.
Now full functionality test, that was fun. Especially when we were called in at the start of a title. I remember we were involved from the start with Chaos Control (another game I can still play in my sleep Smile ) . We had 10-12 people on that at the beginning. Playing it and trying to break it Laughing We had competition then from a test team in the USA, and we were trying to get more bugs in as they did. Kinda funny, we reported 20-30 a day in the beginning, they just a few. No contest really. Poor guys there were just students, not pro testers, so they missed a lot.
With these functionality tests all bugs were free game, with compatibiliy tests, if you found a small crack in the audio or some sync loss, no biggie, title was finished and only real bad issues were still accepted. The test only was supposed to cover compatibility after all. Which could yield some nasty surprises if a major incompatibility was found and fixing it would mean rewriting part of the code that could break other parts etc. Heads would shake at PIMC and mutter "serves you right.. we told you so, get testing team in from the start..." and that sort of thing.
We actually worked months on Chaos Control. Each time they thought they were final, we tickled out another bug. Often new ones, that didn't appear in previous versions . Was a big event when it was finally finished. Customer was happy even though it took so long. Send us a leather jacket with Chaos Control logo, which we had a lottery for amongst testers. I still think it was rigged and they wanted my old worn jacket thrown in the dustbin Laughing Still have it though, tough jacket
Things went downhill from there really, I think that was our last major - I mean really big - project. It quickly became clear studios had much smaller budgets, and demanded tests became smaller and cheaper. Less fun too. We saw the rain coming, so we started testing on PCs too (we already did testing on VCDs, as those for Philips/Polygram were actually made - menu's and encoding - at PIMC). Guess what. Had to test all those CDi games that made it to PC like Burn Cycle and Chaos Control AGAIN Rolling Eyes Since I now was the resident "senior expert" on those programs, most other testers had changed jobs...
Hmm, that really did become a novel, didn't it. As for your other questions, what bugs were most prevalent, I'm not sure. It's a bit too long ago, and I'd be going on my memories of my experiences, not at all a good overview of the entire test coverage.
Lots of detail issues: Philips insisted on particular texts and audio warnings, and those were often not OK. Issues with the bumper too. They changed the bumper at a certain point and forbade the use of the old one. Sometimes some controllers would not work because the title had no proper support for it. Lost of safety area issues (text out of bounds mostly, or hostpost out of bounds). Plenty of peculiar compatibility issues that sometimes took developers a bit to figure out. We sometimes had to lend out players to them (some portables have been around the world) so they could debug.
Plenty of audio cracks, hiss, noise, graphical glitches... quality issues really. Like a character in Secret Mission walking into a hut and seeing part of him through the hut graphics, that kind o stuff. Text errors... don't start me on those. If we were asked to test those we got scripts and had to locate each text and check it. If not specifically asked, we only reported them if we bumped in to them. And we bumped, a lot.
Just remembered, we also tested Dirty Disc messages. Language, correct message, but also that it worked i.e. that if something happened, you got a DD message and didn't just fell back to the shell. Plenty problems here too, weird non-standard messages, wrong languages, no message at all,...
Usability issues too. Philips wanted all titles to be playable with a remote. Ever try Chaos Control or The Apprentice or Flashback with one of those pressure sensitive remotes? We actually had to finish games with these, in order to prove it could be done, and report difficulties. Not that developers changed things much anyway Laughing Like those nasty controls in Flashback, having to be at a pixel-accurate location in order to make a jump ...I can still remember a very frustrated week between Xmas and New Year testing that @*&%@#$%
As for your last question, sorry. I'm sure I saw a few of those but don't remember them. We didn't really look what was in stores, that was old news for us. If you have a list somewhere, I can look if any title rings a bell or not.