09-22-2008 06:56 PM
I am trying to trim some graphical data using two cursors. I only want the data between the cursors to appear in the new graph.
It seems that some combination of reading cursor positions and using search 1d array to find the index in the graph data improperly returns a result of -1. The weird thing is it only fails when the cursor position is 6.6, or 5.6, or 4.6.
To test the attached example:
Just run the vi once without making any changes. XY graph 2 should come up blank. Then move the yellow cursor (cursor 1) to a different data point (anything not ending in .6) and run again. XY graph 2 will show properly trimmed data. You can see the search element is definitely in the array but it still returns -1. Also make sure to keep the yellow cursor to the right of the blue cursor or the trim will not work.
Thanks for any help!
09-22-2008 07:41 PM - edited 09-22-2008 07:44 PM
You should never do exact comparisons with DBL. (equal, search array, etc)
Instead of "search 1D array" use "threshold array". It is a drop-in replacement that will not fail in a situation like this.
Even better, instead of gettting cursorX, get the cursor index and use it directly to set the cut indices. (see image)
(btw: What's up with all the sequence structures and local variables?)
09-23-2008 12:05 PM
09-23-2008 12:50 PM
Just as a side-note so you understand what is happening:
Element 33 of your array is actually "6599.999999999999990" (showing 15 decimal digits), while your numeric is "6600.000000000000900". They are not equal!
The main problem are componding rounding error due to the axis multiplier of 0.001 and your external multiplications of 1000.