What to Do when it's Really New?

Test the requirements, not the thing.

Dragged it out of the lab with no requirements? - write some.

1. What does it do.

2. What should it NOT do.

3. Quantify how well it should do each thing.

4. Quantify how it prevents NOT doing something.

5. Test it's ability to meet those expectations, now test what happens when asked to exceed them.

The key here is to distill the thing into quanitfiable aspects, then measure them.

You have different types of testing:

-type tests

-routine tests

Type testing is to validate/qualify that a product fulfills what it has been designed to do. Routine testing is when you have type tested your product and are now in production: you want to validate that your product is OK for being operational.

In your case, it looks like you're after type testing guidance. Here, you have two perspectives: you're either the designer or you're the Customer.

If you're the Customer who specified the product, you usually don't want to know about the internal structure of the product: you expect the product to fulfill some functions and you want to test them. In addition, you usually have specified performances criteria, so you also want to test the product meets those -the very basic ones should be endurance test and repeatability test. Last, if this product has some critical role, you may also want to specify 'negative' tests. Negative tests are for testing the product under not-normal conditions: what happens if your product looses its power supply, a sub-function e.g. data acquisition, ... or if you've done something wrong with the configuration of the product. The hard point with negative tests is that you are not going to be comprehensive if you don't know the bits and pieces which the product is made of.

If you have hardware and software in your product, you should normally have your software 100% validated before starting the hardware tests. Hardware tests may also cover EMC, mechanical, environmental/climatic tests depending on the target environment requirements, at least to cover CE/UL or similar approvals.

If you are the designer of the product, you have the same objectives as the Customer but with the constraints of detecting, diagnosing and fixing the possible issues. So you need to have a structured approach which allows you to view your product as a set of smaller boxes which each fulfills one or several functions. And for every 'smaller box', you may also have to break down it into even smaller boxes depending on the complexity to test it. In fact this breaking down should have been made during the design stage.

So you should start to thoroughly test the 'cannot-be-smaller-boxes', and then associate/integrate a few of them together and so on till you can test you entire product as a kind of black-box.

All the tests should be documented, the integration plan/schedule should be part of a validation plan/schedule.

Now, all that should have been taken into consideration at design stage: testing something against requirements it has not been designed for is always weird: your tests are presumably here to test comprehensively your product.

Regards,

Patrice

It seems a daft question to me...it presupposes you've spontaneously invented something with no specification...
I can't imagine this happens very often... last time was when Fred Mac Murray invented Flubber .

Test it to the spec' initially and then test to find it's limits (within safe parameters).
Ok...sometimes I build stuff just to see how it works and what it will do, but I still have an expectation which is effectively the spec'.
Exception testing is an infinite set of tests.
You can test that 'It does all the things it should do'
But the set of tests 'It doesn't do all the things it shouldn't do' is infinite.
Should I test that my new chemical dosing unit to see if it is suitable for chocking aircraft on a carrier flight deck?
Nuff said..
Del _{(Are there an unusual amount of daft threads about, or is it just me being grumpy?)}

I would re-read this question to be

".....what do you do when you invent something vastly Superior to existing instruments...."

I know its superior but convincing the customer who is used to 'doing things the way they've always been done' is a different matter.

Existing air gauging techniques are fundamentally flawed in that the reading is based on arbitrary units and no real connection to any national standard is possible....

So in the early 90's I designed an air gauging instrument which was more sensitive, accurate and had a digital readout calibrated to national standards.... GREAT!!!

Companies loved it..... BUT.... when they bought one they wanted to know how to relate their existing readings with the readings mine gave them!!! As their instruments weren't calibrated to any standard units, let alone any national standards and all their old manufacturing drawings specified their instrument's units as an inspection tolerance....

Most realised the vast improvement that my instrument gave them but couldn't find out how to convert their existing testing methods from their existing units to national accredited units of measurement!!!!

Not wishing to point the finger at anyone but if you saw the measuring systems they are still using for gas appliance manufacturing and safety you would be horrified!!!