LabVIEW
Cross-correlation values are too good
Hi Dave,
@DaveSA wrote:
I cannot figure out why this output is so high.
You use the "default" crosscorrelation with "none" normalization. what happens with any of the other normalization modes? (They are described in the help for this function!)
@DaveSA wrote:
I am also not certain what the difference between the crosscorrelation.vi and TSA cross-correlation function.vi would be - these give similar results.
One function comes with "default" LabVIEW (maybe with Professional edition due to SignalAnalysis package), the other is part of a specific toolkit for specific tasks...
@DaveSA wrote:
Attached is a VI with default values for the input arrays (real data and another arrays of 1s).
Unfortunately you use a very recent LabVIEW version. You would reach a broader audience when downconverting the VI before attaching. (File->Save for previous, I prefer LV2019.)
As your "real" data aren't the same as the ones shown in your screenshot I don't know the expected result for those "real" data. And I don't have that toolkit installed so you need to read the help for the TSA function on your own...
Cross-correlation is a measure of the similarity between two data sets.
(simplified) RXY = ∑i Xi Yi
The larger the absolute value of RXY, the higher the correlation.
A value of 0 indicates no correlation.
if you use unbiased or biasd normalization, and compare real data 1 and real data 2 their max crosscorrelation is 0,06..
if you use biasd normalization, and compare signal 1 with "array of 1s" their max crosscorrelation is 0,27..
but if you'd compare array of 1s with array of 1s, the crosscorrelation is 1,00
Just like all correlations, it does not represent causation, only statistical association.
so in terms of max correlation value, signal 1 and signal 2 are less correlated than signal 1 and "array of 1s"
which apperas to be counter-intuitiv - or is it?
keep in mind:
- Cross-correlation analysis is usually conducted to understand the relationship between two stochastic processes.
- I'd say the array of 1s is hardly or very unlikely the outcome of a stochastic process
- Cross-correlation measures the temporal similarity for two data series, and it assesses the information between peaks (Derrick and Thomas, 2004; Ruppert and Matteson, 2015).
- arrayof1s is just one big plateau, with no peaks at all ...
this is signal 2 (red) compared to arrays of 1 (white):
this is signal 2(red) compared to signal 1 (white):
as far as i am concerned, crosscorrelation vi works as expected:
I used 2d convolution to reverse engineer this small CNN (convolutional neueral network, on the page saerch for Convolution Demo). as 2d convolution is almost the same as 2d crosscorrelation, but with a flipped convolution-mask, I also tried successfully to exchange those .vis - which did work successfully. I also tried the correlation .vis in the free sparse matrix toolkit, as it provides a double and a single datatype version of crosscorrelation.vi.
@DaveSA wrote:The question I have left is why the cross correlation code is not returning a max value of 1 when cross correlating the signal with itself?
I don't know the answear to this - I suppose this is rather an expectation, not a fact, which is generally true, but is true under certain conditons.
however, the cross correlation of signal 1 with an array of 0 is obviously always zero 😉
 |
 |
so, if we do this with different values for x, max cross-corr value is about 1 when x=4,45
| x | max cross-corr value |
| 0 | 0 |
| 0,01 | 0,00223537 |
| 0,1 | 0,0223537 |
| 0,25 | 0,0558843 |
| 0,5 | 0,111769 |
| 1 ( auto correlation) | 0,22 |
| 4,45 | 0,994741 |
@DaveSA wrote:
For a little more context - I added two more .csv files which represent the mean of 15 trials each. If I use the correlation coefficient VI, correlation between real data 3.csv & real data 4.csv is 0.949. And perhaps I am overthinking this, and the correlation coefficient VI is sufficient since I know there is no lag between the signals.
It looks like crosscorrelation.vi is the wrong tool for your task, but linear correlation coefficient is the proper tool.
so basically you learn a linear model and compare how well the model fits both signals.
if the signals are identically, this correlation coefficient will be 1. if you do it with an array of 1s or an array 0s and one of the two signal, this will result in NaN.
but this also appears to some-what scale-invariant, when multiplying one the two sgnals with a scalar factor, corr-eff stays exactly the same, however the signals look completly different:
x = 0,2; cross-corr-max = 13,5; corr-coeff = 0,9492
x = 1; cross-corr-max = 67,54 ; corr-coeff = 0,9492
x = 10; cross-corr-max = 675,46 ; corr-coeff = 0,9492
- which is propably mostly due to z-normalization, which involves subtracting mean-values... -
