Removing artifact and outlier values

[Coq_Rouge]

---

Mayflowers wrote:
Yes! That looks great! - now how do I do that?

---

Well first you must gain 15 years of experience working with medical signals analysis, then I shall consider to tell you my secrets. It like magicians. They never tell other than magicians how a trick is done

Or can we take it tomorrow my time is cet+2 I am going to bed

[Mayflower2400]

Very well, I will look forward to being enlightened! Sleep well.

[CoastalMaineBird]

I whipped up  a quick version that looks almost as good as Coq Rouge's

Basically, you're looking for DELTA outside of 20, where DELTA is defined as Y(i) - Y(i-1).

If it changes that rapidly, then it's likely to be a dropout, and you just duplicate the last good value instead.

You may have to tweak the tolerances in the code.

The 1st graph is your original data, the 2nd graph is Coq Rouge's, the third is mine. 

 

Message Edited by CoastalMaineBird on 05-28-2009 04:10 PM

[Mayflower2400]

Thank you! My existing files are all lvm text files with TDMS logs saved as well from signal express. What is the best way to input the signal to what you created?

[CoastalMaineBird]

You can either read a complete file, then process the complete signal, or if you look at my code, you could easily process it point by point as you read it in.

Your example showed 36000 samples - that's an hour at 10 Hz.  Is that a complete file, or only a portion?  If the file's much larger than that, I would process it as I read it.  Otherwise it doesn't much matter.

How did you get the example data in? 

[Coq_Rouge]

Here is my solution, It is pretty much the same idea that Steve Bird stole from me and presented as his idea:smileyvery-happy: 

I also tested some other filter methods added after this but it did not help any thing. The problem is that you have data dropout not data+noise 

[CoastalMaineBird]

Yes, I took your VI with data on it (and no code), and added some code to help the guy out.

[Mayflower2400]

That was the full data set and most are about this size or just slightly larger. I read the TDMS data file with the "read from measurement file" vi. I then wired that to a table and a signal generator vi.  The express vi's are easiest for me to understand because they are the least like "code" which is almost as foreign to me as "engineer"

When I wired this to your vi it failed - I understand this to be because it is the wrong signal or data type? I then tried converting it or reading it in differently but that didnt work either.

[Coq_Rouge]

try this it may help you.

[CoastalMaineBird · Accepted solution]

I don't know if you're talking to me or not - this forum doesn't indicate who you're responding to, Mayflower.

My VI isn't intended to be used as is, since it treats the graph as a control (data source), when a graph is usually an indicator (data sink).

I have attached a VI for drop-in use.  Connect your data to the input, take your corrected data from the output.

You might want to play with the tolerance, I set it to 20 by default, but you can wire in a different number.