Previous in Forum: Joining ABS Plastic Sheet   Next in Forum: Vista Password Bank
Close
Close
Close
20 comments
Rate Comments: Nested
Associate
Engineering Fields - Marine Engineering - New Member

Join Date: May 2006
Posts: 36

Experimental Standards?

02/08/2009 5:20 PM

there are many journals published with good looking results on paper and valid claims. however is there any such standard that is kept in mind while doing experiments. although the idea may be new, is there any international standard which determines that the experiment has been done well enough for its results to be accepted worldwide?...i am doing some simple experiments in fluid dynamics relating to osscillating flow....and i was wondering if there were any such standards which i have to keep in mind?

Register to Reply
Interested in this topic? By joining CR4 you can "subscribe" to
this discussion and receive notification when new comments are added.

Good Answers:

These comments received enough positive votes to make them "good answers".
Guru

Join Date: Feb 2008
Location: Milky Way galaxy, Sol solar system, Earth (not Giaha), USA, WA, N.E.
Posts: 691
Good Answers: 13
#1

Re: experimental standards?

02/08/2009 6:23 PM

Why not Google 'International Scientific Standards'?

It would be interesting to find out about.

Let us know what you find. or.

I, of course, am too lazy to look it up myself.

__________________
They that do not learn from history and apply those lessons to the present are bound to repeat its failures.
Register to Reply
Guru
Hobbies - CNC - New Member Popular Science - Biology - New Member Hobbies - Musician - New Member

Join Date: Dec 2008
Location: Canada
Posts: 3523
Good Answers: 146
#2

Re: experimental standards?

02/08/2009 7:48 PM

Basic experimental design:

1. The basic scientific method is to test different values of one experimental variable, while all other variables are kept constant. This allows you to collect meaningful data about the effects attributable to your experimental variable.

2. In designing the experiment, you have to consider possible sources of error. Your setup should include a plan to mitigate any identified sources of error: this method is written up in your paper.

3. Statistical methods are built into the experimental design, in order to determine your margins of error and to show how stable your setup is and how reproducible your results are expected to be. So, for example, instead of taking a measurement once, you collect a data set of 30 or more measurements of the same test, and so on for each value of the experimental variable.

4. Your data should be collected in writing (digitally nowadays, in some permanent format), and should be kept not only to perform analysis, but to back up your analysis if it is challenged. Where applicable, physical data should also be retained.

These general procedures apply to any branch of science. If you discuss your specific research project, or ask about any of these design procedures, I'm sure you will get good advice about each or any of these issues.

Did I leave anything out?

__________________
incus opella
Register to Reply
Guru

Join Date: Oct 2008
Location: I'm outa here
Posts: 1924
Good Answers: 196
#3
In reply to #2

Re: experimental standards?

02/09/2009 12:12 AM

Perhaps a bit of elaboration on the subject of designed experiments. It's a statistical analysis subject I have some overarching awareness of but certainly not enough thorough knowledge to attempt an explanation. I'd sure like to read more basic commentary on the subject; but don't want to narrow the topic at this point. (I have a book on the subject; but these days I tend to fall asleep before I get more than a page or two into the second chapter of most books which start the math at about that point)

Hmmm........please don't answer this for a couple of days.....

Ed Weldon

Register to Reply
Guru

Join Date: Mar 2007
Posts: 2550
Good Answers: 103
#11
In reply to #3

Re: experimental standards?

02/10/2009 2:45 AM

I was going to add this aspect, but you have already beaten me

This aspect must be taken care to prove the experiment (we call is industrial experiment but is equally valid in scientific ones- since as we always teach our so called students (trainees) - that there is no mathematics in real life - only statistics.

So ehat ever experiment you do you have to prove the result statistically.

The difficult part i find in industry is to carry it out , convincing the persons to change parameters that were there from dinosaur age. But for the lab persons no such constraint .

The stat experiment is very simple. As far as design and analysis is there.

But before that did you thorough yourself with ANOVA and Test Of Hypothesis?

(necessary esp Chi square and Students -F test

why student? there is explanation i think even wiki has the answer.

__________________
Fantastic ideas for a Fantastic World, I make the illogical logical.They put me in cars,they put me in yer tv.They put me in stereos and those little radios you stick in your ears.They even put me in watches, they have teeny gremlins for your watches
Register to Reply Off Topic (Score 5)
Guru

Join Date: Mar 2007
Location: City of Light
Posts: 3943
Good Answers: 183
#4
In reply to #2

Re: experimental standards?

02/09/2009 7:41 PM

It is not always necessary to make 30 measurements. this number was chosen to be sure that the equations for the normal distribution (Gauss) can be used since in fact they are valid only for an infinite number of elements in the sample.

There is an other approach called the Student method which permits to deal with smaller sets of values and which has as limit the Gauss distribution if the sample is big enough.

Depending on the limits of uncertainty the size can be reduced to less than30.

Register to Reply
Guru
Hobbies - CNC - New Member Popular Science - Biology - New Member Hobbies - Musician - New Member

Join Date: Dec 2008
Location: Canada
Posts: 3523
Good Answers: 146
#5
In reply to #4

Re: experimental standards?

02/09/2009 8:25 PM

Good point. Much depends on the specific objective. I personally have a great interest in "small n" research. Sometimes it is the only practical approach: sometimes the necessary knowledge can be obtained with a small sample.

If the objective is to have your findings published and recognized world-wide, the stronger statistical basis is the way to go. Then again, the statistical expectations vary from one branch of science to another. I know nothing about fluid dynamics or the standard statistical approaches to this type of experiment. If the setup is really top notch, maybe there's little variation in the measurement taken, and no need to repeat each measurement so many times. So the student variable would be fine.

For general ideas about experimental design standards, I also found this page, (although the examples are medical the general outline is pretty comprehensive)

http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage2.htm

here's one that uses standard experimental designs
http://www.camo.com/rt/Resources/design_of_experiment.html

__________________
incus opella
Register to Reply
Guru
Engineering Fields - Electromechanical Engineering - Technical Services Manager Canada - Member - Army brat Popular Science - Cosmology - What is Time and what is Energy? Technical Fields - Architecture - Draftsperson Hobbies - RC Aircraft - New Member

Join Date: Sep 2006
Location: Clive, Alberta, Canada
Posts: 5916
Good Answers: 204
#6

Re: Experimental Standards?

02/09/2009 11:33 PM

I don't know, try here, here and here.

but I'm slowing building a new application for a genre that I'm calling "Computer Aided Specification and Compliance". I need a foundation specification to work with, so if you are going to write one (sounds like a great project), I would happily incorporate it as a base module, and you would have a bit more marketing for your work.

Chris

Register to Reply
3
Guru
Panama - Member - New Member Hobbies - CNC - New Member Engineering Fields - Marine Engineering - New Member Engineering Fields - Retired Engineers / Mentors - New Member

Join Date: Dec 2006
Location: Panama
Posts: 4273
Good Answers: 213
#7

Re: Experimental Standards?

02/10/2009 12:12 AM

The most important standard of any report of research is that anyone, anywhere, can reproduce the results that you report, following the detailed instructions you provide. If you do not provide sufficient detail as to your experimental setup such that others can produce similar results using a similar setup and similar controls, no matter how sophisticated your analysis or profound your conclusions, you will not achieve credibility.

Register to Reply Good Answer (Score 3)
Guru
Engineering Fields - Electromechanical Engineering - Technical Services Manager Canada - Member - Army brat Popular Science - Cosmology - What is Time and what is Energy? Technical Fields - Architecture - Draftsperson Hobbies - RC Aircraft - New Member

Join Date: Sep 2006
Location: Clive, Alberta, Canada
Posts: 5916
Good Answers: 204
#8
In reply to #7

Re: Experimental Standards?

02/10/2009 12:38 AM

awesome.

Register to Reply Off Topic (Score 5)
4
Anonymous Poster
#9

Re: Experimental Standards?

02/10/2009 1:55 AM

many good answers and #7 hit it on right the mark.

this post is to remind that when doing research that you can publish in international journals, you should also consider the following practicalities

1. your work will have to go through peer review. reviewers may frown on unusual methods and view them with suspicion. this means that before starting your own experiment, you should look at how previous similar experiments were conducted. identify their strengths (and weaknesses). then carefully justify to yourself why you choose your particular method. if you can defend your choice successfully, then ok (generally, it helps to research with the eventual peer review in mind...)

2. absolutely try to keep a record of everything, incl any seemingly useless data (it is sometimes the 'useless data' that, upon further investigation, brings the new insights)

3. be realistic and honest about reporting errors, sources of bias etc

4. take photographs and make sketches and drawings. your experimental rig may be taken apart afterwards (if only to save space), but these will be very valuable to you in the mid/long-term

5. try to not mix measuring with analysing (although in some tests this is unavoidable). 'measure as you analyse' can be a major source of bias, if not done very very carefully. just do the test and then do the math. you can always follow up with a modified (well-documented) test if the math 'tells' you to.

6. the golden rule: question (attack?) your method and findings relentlessly. if they survive, then they're ok

best of luck

Register to Reply Good Answer (Score 4)
Guru
Engineering Fields - Electromechanical Engineering - Technical Services Manager Canada - Member - Army brat Popular Science - Cosmology - What is Time and what is Energy? Technical Fields - Architecture - Draftsperson Hobbies - RC Aircraft - New Member

Join Date: Sep 2006
Location: Clive, Alberta, Canada
Posts: 5916
Good Answers: 204
#10
In reply to #9

Re: Experimental Standards?

02/10/2009 2:08 AM

good followup answer

Register to Reply Off Topic (Score 5)
2
Commentator

Join Date: May 2005
Location: Las Cruces, NM
Posts: 81
Good Answers: 4
#13
In reply to #9

Re: Experimental Standards?

02/10/2009 10:04 AM

There is the Nitty and there is the Gritty. Guest handles the Nitty pretty well. By the way, welcome Guest, Please join us.

As to the Gritty, the issue of reproducibility involves care not to be "Fooled by Randomness" (a good read by N. Talib, by the way). The issues here are 1. Understanding the nature of the experimental noise (natural variability in the outcomes, due to random variations in the control of the experimental mechanics and in the measurement errors), and 2. Establishing the Statistical Confidence of the results (taking enough samples to provide confidence that the results are probably not a fluke, at a stated level). The number of samples required is a function of the noise level (sigma) and the statistical distribution of the noise.

This is not an easy thing to do, and most experimenters come up short on these issues. Some tips: If your outcome can be expressed as a number, plot the results to see if the distribution LOOKS NORMAL. If so, then there are tests to estimate your confidence that the noise is normally distributed. In this case, estimate sigma. Again, there are guidelines for expressing your confidence in the sigma estimate. Finally, State the Null Hypothesis, the condition that describes the failure of your experiment. Now compute the probability that the Null Hypothesis can be rejected.

If the noise distribution is clearly not Normal, this becomes much more difficult. But this is well worth the effort.

BEWARE OF BLINDLY COMPUTING AVERAGES. The average human has one ball and one tit ;-)

__________________
Is there any knowledge in the world which is so certain that no reasonable man could doubt it? Bertrand Russell
Register to Reply Good Answer (Score 2)
Commentator

Join Date: May 2005
Location: Las Cruces, NM
Posts: 81
Good Answers: 4
#14
In reply to #13

Re: Experimental Standards?

02/10/2009 10:18 AM

I just re-read the opening question, and note the the application is Fluid Dynamics. WOE TO YOU!!!

If your flow is laminar, there is some hope. If there is significant turbulence, it will be nearly impossible to make measurements that are sensible in the spirit of my previous comment. Good Luck!

__________________
Is there any knowledge in the world which is so certain that no reasonable man could doubt it? Bertrand Russell
Register to Reply
2
Power-User

Join Date: Dec 2007
Posts: 120
Good Answers: 12
#15
In reply to #9

Re: Experimental Standards?

02/10/2009 1:17 PM

The answer above was excellent. I want to expand on one aspect of it and try to drive a point home at the same time.

The main reason for separating the sampling and analysis is an issue called "Curve Fitting". It is probably one of the trickiest issues for anyone doing research in anything. As you are working on your experiment you will notice things. It is a strong and natural tendency to tweak your experiment to take these noted effects into account. What you are actually doing is modifying the experiment to fit the data you are finding. In other words you are designing your experiment to fit the curve of observed data and that data may be completely irrelevant to what you are trying to prove. Worse yet, the fit to random data may cause you to believe you have proven your hypothesis when you actually haven't. If you intend to publish your results the peer review process will tear you apart on this and completely discount your results.

The proper way to avoid this is to state your theory before designing your experiment. You then carefully design your experiment to prove or disprove your idea. When this has been carefully done and documented you can begin your testing. Do not change the experiment while you are testing at all. After all is complete, then analyse the results and live with the conclusions. Sometimes being able to reject an idea is just as powerful as proving an idea. After all is done you can go back and look at what you have learned. At this time you may need to start the process all over again.

It is a long hard process to get credible results, but the value of those results can be immeasurable.

As a little side note. My day job is as a Quantitative Analyst for a stock trading company. I see the advertisements for tools to quickly and easily "Back Test" your trading ideas against historical market data. This is an extremely manipulative trick on their part. These tools encourage people to quickly try out lots of ideas until they find one that the analysis says will work. These are just curve fitting tools and they are designed to give people a feeling that the trading idea has scientific validity. This drives up trading volume and very often leads to significant losses for the people involved. This is just an example of the dangerous lure of curve fitting.

Good luck with your experiment.

-Doug

__________________
A robot must risk his neck for his brother man, and may not cop out when there's danger all about. - Isaac Hayes' First Law of Robotics
Register to Reply Good Answer (Score 2)
Guru
Engineering Fields - Electromechanical Engineering - Technical Services Manager Canada - Member - Army brat Popular Science - Cosmology - What is Time and what is Energy? Technical Fields - Architecture - Draftsperson Hobbies - RC Aircraft - New Member

Join Date: Sep 2006
Location: Clive, Alberta, Canada
Posts: 5916
Good Answers: 204
#16
In reply to #15

Re: Experimental Standards?

02/10/2009 1:27 PM

Dear drobertson,

Well said... however, what I think you have really done here is define the difference between an Engineer and a Scientist.

A scientist 'should' do exactly as you say, as having an experiment, with a corresponding theory, with reproducible results, is the purpose scientific method.

An engineer curve fits continously, because they are all about figuring out the best use of natural and real effects, and putting it to work. So they will modify and tweak all over the place.. and if the theory goes out the window, oh well. just have to come up with a new theory. In the meantime, "how cool is this new gizmo I just engineered!"

Chris

Register to Reply
Commentator

Join Date: May 2005
Location: Las Cruces, NM
Posts: 81
Good Answers: 4
#17
In reply to #16

Re: Experimental Standards?

02/10/2009 1:39 PM

"So they will modify and tweak all over the place.. and if the theory goes out the window, oh well..."

OOPS there goes another Rubber Tree Plant, or bridge, or reactor, or ..........................

__________________
Is there any knowledge in the world which is so certain that no reasonable man could doubt it? Bertrand Russell
Register to Reply
Power-User

Join Date: Dec 2007
Posts: 120
Good Answers: 12
#18
In reply to #16

Re: Experimental Standards?

02/10/2009 1:55 PM

I agree completely. I happen to like being both at different times. There are abut 7 different projects in my garage that are in different states of "Tweak". Personally I enjoy the thrill of tweaking a design until it works just the way I want. It is a very satisfying process.

As for the answer above I was just trying to make a point relative to the thread title "Experimental Standards". He seems to be questioning from a more scientific point of view and hoping to get results that are credible to outside people. A scientific process is better for him in that context. On the other hand, when the boss wants something last week the scientific process goes out the window. Tweaking and gut instinct take over and you get it done.

-Doug

__________________
A robot must risk his neck for his brother man, and may not cop out when there's danger all about. - Isaac Hayes' First Law of Robotics
Register to Reply
Guru

Join Date: Oct 2008
Location: I'm outa here
Posts: 1924
Good Answers: 196
#19
In reply to #16

Re: Experimental Standards?

02/10/2009 2:18 PM

Ahhhh......It may be cool; but is it good? Experience brings with it the filter through which the engineer tests the efficacy of his invention sometime before he gets too far into the execution stage.

My wife (45years) is good at helping me build and maintain filters. I've applied a number of different more or less technical solutions to mundane household problems over the years. Some worked well, others didn't. The worst one fell on her (about 30 years ago). I wasn't in the room at the time; but my two sons aged 6 and 10 were. My wife told me that immediately following the event the youngest asked the oldest "Is she dead". To this day I must endure yet another filter building trial whenever the subject of model train layouts comes up.

Ed Weldon

Register to Reply Off Topic (Score 5)
Guru
Engineering Fields - Electromechanical Engineering - Technical Services Manager Canada - Member - Army brat Popular Science - Cosmology - What is Time and what is Energy? Technical Fields - Architecture - Draftsperson Hobbies - RC Aircraft - New Member

Join Date: Sep 2006
Location: Clive, Alberta, Canada
Posts: 5916
Good Answers: 204
#20
In reply to #19

Re: Experimental Standards?

02/10/2009 2:23 PM

"Ahhhh......It may be cool; but is it good?"... Well now Ed, thats the difference between an Engineer and an Inventor!

A good engineer is part scientist, part inventor, part production manager, part quality manager, part accountant, part secretary... (part husband?)

Chris

Register to Reply Off Topic (Score 5)
Guru
Safety - ESD - New Member Popular Science - Cosmology - Amateur Astronomer Technical Fields - Technical Writing - Writer India - Member - Regular CR4 participant Engineering Fields - Optical Engineering - New Member

Join Date: Dec 2007
Location: 18 29 N 73 57E
Posts: 1390
Good Answers: 31
#12

Re: Experimental Standards?

02/10/2009 7:44 AM

I am doubtful, if there is any international standard for this subject. But, there whole science for it which is called Design of Experiment. This goes on ellimil\nating uncertainties of various parameters and goes of defining effect of each parameter on functioning of object user study

Register to Reply
Register to Reply 20 comments

Good Answers:

These comments received enough positive votes to make them "good answers".
Copy to Clipboard

Users who posted comments:

Anonymous Poster (1); artsmith (2); chrisg288 (5); cwarner7_11 (1); drobertson (2); Ed Weldon (2); fwes (3); gsuhas (1); nick name (1); sb (1); Shadetree (1)

Previous in Forum: Joining ABS Plastic Sheet   Next in Forum: Vista Password Bank

Advertisement