Exclusion of undesirable segments of splitted series

[dan07]

Hello...

I have a VI to perform analysis of series and its spectral analysis results. The VI loads two ascii files: the first ascii loaded has one single column with the whole series (9001 values), in the sequence, the second ascii file loaded has 34 columns, each column representing the spectral analysis results of each one of the 34 segments that the whole series were splitted.

After loaded, the first ascii file (9001 values) is shown in 34 different graphs with 512 points each, and the 34 segments are overlapped by 50 %. In the left top of each graph there is a LED, that when is GREEN indicates that the spectral results provided from the analysis of that specific segment, will be used to compute the final result. For example: if I set (just clicking on them) to GREEN only the LED of segments (19, 27, 31, 33), only the columns 19, 27, 31, 33 of the ascii file with spectral analysis results, will be take into consideration to compute the final result. All the other columns (1, 2, 3, 4, ......) will not be used to the calculus.

My problem is, I performed the analysis with this VI of a large amount of files, but I only take note of the final values (Very low, Low, High, Low/High), but I did not take note of which segments were labeled with RED or labeled with GREEN, but now I need this data and will be impossible to try all the possible combinations of GREEN or RED led of the 34 segments, because there are 17.179.869.184 (2^34) possible combinations.

My idea to solve this issue: I need to build a function where I show to the VI the values of Low and High, and the VI by itself promotes the 17.179.869.184 combinations with the LEDS and when the expected values are reached the VI stops. Next, I can see which of the LEDS are GREEN or RED. I think that the COMPARASION vi might be used to test the to values inputed and when the correct combinations provides a value that matchs with the entered value, the VI stops.

One imporant thing to remember is that the LED provides values 0 and 1 like. This way, the sequence might look like this 00000110001000100.... (with 34 digits).

Lets use as example a situation where I have the value 12432 for Low and 36450 for high, which combination of LEDS promotes these results?

Are there any way to do this?

Attached is the VI, and the two ascii files that should be loaded in sequence.

Thanks in advance

Dan07


Message Edited by dan07 on 06-12-2008 11:47 PM

[altenbach]

---

dan07 wrote:
Lets use as example a situation where I have the value 12432 for Low and 36450 for high, which combination of LEDS promotes these results?

Are there any way to do this?

---

Yes, this is a very simple search, but it will still take a very long time due to the complexity of the problem. Even if you could check 1 million combinations/second (realistically doable), the search will take a few hours. It's definitely possible, but since you have many file, it won't be quick.

Another problem is that you only know the integer value, meaning there are many matches for "Low" that also need to be checked for "High", slowing down the search. For example, searching for a low=12432, gives me already more than 100000 matches in a few minutes that would need to be also checked for High.. If you know a few more decimal digits, the search would be more unique.

Now to your program itself. It is hard to believe how incredibly convoluted your code is. I made a quick rewrite and was able to fit all functionality on a the size of a postcard. You have race conditions, you recalculate a huge number of computations 10x per second that you only need to calculate once after reading the data. You have huge explicit constructs, hidden terminals with the same name as visible terminals, You write the same four property nodes with the same old value over and over, etc. etc.

(see also http://forums.ni.com/ni/board/message?board.id=BreakPoint&view=by_date_ascending&message.id=5640#M56...)

Attached is a quick alternative. Instead of all these individual terminals, booleans, and graphs, you could use an array where each element is a cluster of a number, a boolean, and a graph. Now you can do everything in loops. See if it makes sense. Do you agree that it's slighly simpler code? 😄

I have a version that searches for a given result pattern, but it probably needs some code cleanup first.

Message Edited by altenbach on 06-13-2008 02:25 AM

[altenbach]

---

dan07 wrote:
Lets use as example a situation where I have the value 12432 for Low and 36450 for high, which combination of LEDS promotes these results?

---

OK, the code to do this is quite trivial, but your values are not unique enough. I made a quick search routine and it finds a match with the current data after about 1 minutes (running on a single core).

The problem is that it will continue to find additional matches at about the same rate, so you will have a huge number of correct answer. You either need to know more significant decimal digits or use additional search criteria.

Here are for example the first four matches. All give the same output values.

Message Edited by altenbach on 06-13-2008 10:15 AM

[dan07]

Hello....

First of all: thank you very much for the attention and help. You helped me too much cleaning up my code.

I know that my old code was very complicated, but as I am a begginer user of Labview, I was not able to change it to an easier one. Thanks!

Concerning the search, please send me the VI with the search code because I will try to find someway to maximize the search, with some additional information.

What I don't undestand is why your search engine did not return only segments 19, 27, 31, 33 as RED, because in these ascii files that I sent to you, the values 12432 for Low and 36450 for high, were obtained with segments 19, 27, 31, 33 RED, and all the other segments GREEN. Do you know why this result was not showed?

Thanks

Dan

[dan07]

Sorry I make a mistake about the segments.. the segments marked with RED that produced the values 12432 for low and 36450 for high were: 12, 17, 23, 24, 31

Try to add to the search the LF/HF value 0.33

Thanks

[dan07]

Another additional information...

To obtain that values (12432 for low, 36450 for high, and 0.33 for LF/HF), I have certainty that at least 17 segments were GREEN

Thanks

Dan

[altenbach]

---

dan07 wrote:
Try to add to the search the LF/HF value 0.33

---

Wouldn't the ratio be highly correlated to the low and high value?

Since we only have two significant digits, the ratio will not help much anyway. Can we search for "very low" in addition?

[dan07]

You are right about the correlation between LF and HF with LF/HF, but we have a little little little advantage using LF/HF in the search because the LF/HF were generate using the LF and HF of each segment and not the final result for LF and HF. In rare conditions it will help. When I performed the analysis, the very low results were not take into consideration

The information that at least 17 segments were GREEN does not help?

Send me the VI because I can make some tests with my files...

Thanks

Dan

[altenbach]

---

dan07 wrote:
The information that at least 17 segments were GREEN does not help?

---

OK, I let it search overnight using [Low=12432, High=36450 , L/H=0.33, Min# of segments=17]

and if found 291 solutions. So, unless you have more significant digits or some other information, you are pretty much out of luck.

The search took under five hours (4:40), running on a single core on my laptop (2.33GHz core 2 Duo T7600).

The code is a bit rough and I am sure one could speed it up quite a bit. It can also be split so it can utilize multiple cores. One performance issue is due to the need for of 64 bit integers and boolean arrays in the current implementation, but I did not want to make the code more complicated. 😉

I update the progress every 2^18 inspections, which is about 250ms on my rig. If you have a slower computer, you might want to adjust accordingly. Due to the limited updates, the final values in the progress indicators is not accurate.

Your default values and the search results are built into the code, so you don't need to load any files to start testing. Of course you can load different files if needed. For searching, the low and high values are truncated to integers and the ratio is truncated to two decimal places.

You can pick any search result with the slider on the right and everything will update accordingly. A selection of -1 is whatever booleans you select manually.

Sorry, this was just a very quick effort last night while watching TV. I haven't had a chance to clean up the code, maybe this can get you started. (there are probably bugs). The sizes are hardwired, so if you have data with a number of sets different to 34, you would need to correct that.

Modify as needed. Good luck!

[dan07]

Dear Altenbach,

First of all, I would like to thank you very much for your help and great attention to my problem. As a begginer user, some simple tasks became very difficult to me. With your help, my VI is running properly and my data analysis very quick and clean. I wish that someday I can help somebody like you did to me.

Thanks

Dan07