Previous in Forum: Power Calculation   Next in Forum: Head Pressure for Differential Pressure Level Transmitter
Close
Close
Close
10 comments
Rate Comments: Nested
Associate

Join Date: Sep 2011
Posts: 47

Consistency of Data

05/26/2015 12:18 AM

Hi,

I have data set of length 5000 (collected at same frequecny). I need to find whether the data is consistent along its length (i thought of comparing the first half with the second half).

Will Coherency calculation between first 2500 data and last 2500 data will help me to achieve this?

Register to Reply
Interested in this topic? By joining CR4 you can "subscribe" to
this discussion and receive notification when new comments are added.

"Almost" Good Answers:

Check out these comments that don't yet have enough votes to be "official" good answers and, if you agree with them, vote them!
Guru
New Zealand - Member - Kiwi Popular Science - Weaponology - New Member Engineering Fields - Power Engineering - New Member Engineering Fields - Electrical Engineering - New Member

Join Date: Sep 2006
Location: Auckland, New Zealand
Posts: 8777
Good Answers: 376
#1

Re: Consistency of data

05/26/2015 12:39 AM

Put the data set into an excel spread sheet (or similar) and plot the data (I am assuming the data is in a format that can be imported easily).

Checking the first 2500 against the last 2500 is likely to give you no help whatsoever for all but the most simple processes being monitored.

Plot all the data, don't be stingy and try and shortcut the analysis otherwise the results will be worse than a random guess, you could be lending weight to the wrong conclusion. This is basic statistical and data analysis.

__________________
jack of all trades
Register to Reply
Guru

Join Date: Mar 2012
Location: Out of your mind! Not in sight!
Posts: 4424
Good Answers: 108
#2

Re: Consistency of data

05/26/2015 3:38 AM

No!

How about comparing the first and the last value?

Check one in the middle too!

__________________
Common Sense Dictates
Register to Reply
Guru
Engineering Fields - Electrical Engineering - Been there, done that. Engineering Fields - Control Engineering - New Member

Join Date: Dec 2008
Location: Long Island NY
Posts: 15600
Good Answers: 981
#3

Re: Consistency of data

05/26/2015 8:19 AM

5000 samples collected at the same frequency is a nice start. After you plot these data points and visually examined for a trend, I would perform a Fast Fourier Transform analysis of the data. Remember, you will have to either add about 3000 null points or remove about 900 points to make this a Fast Transform. Adding null points is better because nulls will also be added in the analysis. This should help you to see if your data set has any periodic component. Next I would do a purely statistical analysis and see if the distribution resembles any of the known Probability Density Functions.

__________________
"Don't disturb my circles." translation of Archimedes last words
Register to Reply
Guru

Join Date: Mar 2007
Location: by the beach in Florida
Posts: 33392
Good Answers: 1817
#4

Re: Consistency of Data

05/26/2015 8:56 AM

You must define the purpose of your analysis and choose a method of analysing your data that best suits the purpose....

http://www.excel-easy.com/data-analysis.html

http://www.qtiplot.com/doc/manual-en/c6433.html

__________________
All living things seek to control their own destiny....this is the purpose of life
Register to Reply Score 1 for Good Answer
Guru

Join Date: Dec 2008
Location: Cd. Juarez, Chihuahua, Mexico.
Posts: 1023
Good Answers: 69
#5
In reply to #4

Re: Consistency of Data

05/26/2015 12:47 PM

I think he said that the samples were taken at the same frequency, so a Furier analysis won't apply here because it is intended to show the amplitude of each of the fundamental frequencies that compose a complex signal (waveform).

His best option is to feed his data into a graph in excel, he can establish trend and consistency from it.

__________________
No hay conocimiento ni herramienta que sustituya al sentido comun.
Register to Reply
Guru
Engineering Fields - Electrical Engineering - Been there, done that. Engineering Fields - Control Engineering - New Member

Join Date: Dec 2008
Location: Long Island NY
Posts: 15600
Good Answers: 981
#6
In reply to #5

Re: Consistency of Data

05/26/2015 2:10 PM

You're correct that the OP made an ambiguous statement about the frequency of the samples taken. I assumed that they meant that the sampled data was taken at the same, repeated time interval (inverse of frequency). This is a mandatory requirement for performing a Fourier analysis. The highest frequency detectable would also have to comply with Nyquist's theorem. Only 5000 samples would also not produce a very fine frequency resolution at all in the analysis, too.

If instead the OP meant your understanding that the magnitude of a frequency was measured 5000 times then a Fourier analysis cannot be performed at all.

I take that back, one can always perform any analysis on any data set to produce a result. That result maybe useless or worse, misleading if you don't know the validity of the data or the limitations of the analysis technique.

__________________
"Don't disturb my circles." translation of Archimedes last words
Register to Reply
Guru

Join Date: Apr 2010
Location: About 4000 miles from the center of the earth (+/-100 mi)
Posts: 9910
Good Answers: 1141
#7

Re: Consistency of Data

05/27/2015 3:41 PM

It would be helpful to know what the 5000 data points represent and what you mean by consistent.

Register to Reply Score 1 for Good Answer
Guru
Safety - Hazmat - New Member United States - US - Statue of Liberty - New Member Engineering Fields - Chemical Engineering - Old Hand

Join Date: Mar 2011
Location: Lubbock, Texas
Posts: 14331
Good Answers: 162
#8

Re: Consistency of Data

05/28/2015 11:17 AM

Why not plot the difference function over time domain? One can then perform FFT analysis on this to reveal periodic systematic errors in the data. If the FFT spectrum is low amplitude and mostly noise, then congratulations, there is essentially none of the periodic systematic errors. IF there is a constant offset error, no analysis can help you other than taking the median and the mean and comparing these to what the expectation value is. If the expectation value is 10.50001, but you have a mean of 5000 pts that is 10.60001, you still have a very precise, but inaccurate measurement. In such a case the error is (10.60001-10.50001)/10.50001 = 0.952%

You also must define (for yourself) what level of consistency is considered to be sufficient. If you want an accuracy into the ppm (parts per million) range, you must not only have a singularly precise instrument, it must also be found by measurements and calculations (not based on estimates, or approximations) to be accurate at that level prior to reporting any finalized data based upon the data set. You must incorporate a strict propagation of errors treatment all the way throughout the data analysis and final answer calculation. Otherwise, this will mostly be an exercise of a futile nature yielding nothing.

__________________
If it ain't broke, don't fix it. Just build a better one.
Register to Reply
Anonymous Poster #1
#9

???? ????? ??????? 0509945905 ????? ????? ???? ?????Re: Consistency of Data

10/29/2015 4:48 PM
Register to Reply Off Topic (Score 5)
Participant

Join Date: Jul 2015
Posts: 1
#10

Re: Consistency of Data

01/20/2016 1:14 AM

Central Tendency and Dispersion are kind of statistics whereas central tendency gets at the typical score on the variable and while dispersion gets at how much variety there is in the scores. Through this online course on Central Tendency & Dispersion using Data Analysis you will learn how to calculate central tendency & dispersion in practical using data analysis.
educba

Register to Reply
Register to Reply 10 comments

"Almost" Good Answers:

Check out these comments that don't yet have enough votes to be "official" good answers and, if you agree with them, vote them!
Copy to Clipboard

Users who posted comments:

Anonymous Poster (1); FelixClouse (1); IdeaSmith (1); jack of all trades (1); James Stewart (1); redfred (2); Rixter (1); SolarEagle (1); Yahlasit (1)

Previous in Forum: Power Calculation   Next in Forum: Head Pressure for Differential Pressure Level Transmitter

Advertisement