very slow loop execution

[RVR]

Hello,

This subset of my program is running extremely slow.

There is a fair bit of computation involved with medium size data sets (1000 rows and 1500 columns) but this should not be the issue I think...

I am running a regression algorithm and I am cross validating the results using a single training data set...so basically you have a a data file (2D array) with my response and predictor variables and the algorithm takes the first row out builds the training model with the remaining data and predicts onto the row that was left out, then the first row is included back in the analysis and the algorithm moves onto the second row, it take it out and makes the training model with the remaining data and predicts onto the second row that was taken out...etc... (sorry if this is not explained clearly but it may not be so important) using the code shown in the attached image.

I get the feeling the slowness is due to my programming.

As you can see I am running 2 for loops, one inside another. The orange vi contains my model and the yellow vi contains the algorithm for predictions.

The outer loop subsets the number of data points to leave out of the analysis (i.e. it could be a single row or it could be any number of rows at a time) and the inner loop tells the program to run the regression algorithm up to a maximum number of runs.

I guess my questions are: (i) is there problems with programming with 2 for loops like I have done? and (ii) does my programming with 2 for loops look efficient? or is there a better way to do this?

I am sorry if this is not clearly explained. Please let me know if you want some more information.

Best wishes

RVR

[altenbach]

please try again to attach your code. 🙂

[RVR]

not sure what happened. Here I attach the image of my code again.

than you.

[Gabi1]

Hello RVR,

i am not sure how much iterations your loop does, but it seems at each single iteration you wait 350ms...multiply that by 100 iterations... you are in it for 6 minutes!

except that, i think we would need to look at the subvis to see where problem could be.

Message Edited by Gabi1 on 05-01-2007 01:23 AM

[RVR]

Hi Gabi,

thanks for the reply.

Sorry that 350 is not actually there. I was testing something and forgot to remove it before I did the print screen.

So it runs like an old dog without the wait 350 ms.

Here I attach the code that is inside the orange vi where most of the computation is done.

I will appreciate whatever comments anyone has on this and on how I may recode to 'speed thing up' a bit.

Thank you.

Note that the vi's inside this orange sub vi are straight forward matrix computations - no loops inside.

[altenbach]

---

@RVR wrote:

I guess my questions are: (i) is there problems with programming with 2 for loops like I have done? and (ii) does my programming with 2 for loops look efficient? or is there a better way to do this?

---

it would really help if you could attach some real code. It is difficult to tell much from a reduced image.

1. stacked loops are no problem, but there is no obvious reason why you place a delay inside the innermost loop. As Gabi1 already mentioned, this is a drag.
2. your constant resizing of large arrays ( e.g. delete from array) is expensive. you should try to operate on the full array in place, simply skipping over the row/column you want to disregard.
3. why are you outputting to local variables in the innermost loop? is there some other code that reads these elsewhere?

[RVR]

Sorry Altenbach...the delay is not there. It is just that i had it there to test part of the execution and forgot to remove it when I did the printscreen.

About your other points:

- the reason I used the 'delete from array' is that the rows that are deleted and then used for the predictions. I am not sure of any other way to do this.

- yes the locals are necessary as there is other parts of the program that uses these elsewhere.

It is a fairly large program and hence the reason why I do not post it. Other parts of the program run great. It is this bit that is very slow.

I will appreciate further suggestions and if you need more informatin then please ask.

RVR

[Gabi1]

Hei RVR,

as Altenbach pointed already, without the actual vi, we cant see much. your orange subvi is filled with other ones...

that is too bad, because he has the touch for making things faster.

here what you can do if you want to spare us the full prog:

fill your input arrays, make values defaults, save as a new llb. we then get only this vi and subvis.

[altenbach]

---

@RVR wrote:
Note that the vi's inside this orange sub vi are straight forward matrix computations - no loops inside.

---

At last attach your code images at full size. These scaled down version are pretty unreadable. From what I can tell, you're running in circles, doing a lot of unecessary operations. for example, inside the orange subVI, you delete a constant row from the array, but since the input is the same during all iterations of the inner loop, you're just doing repetitive work. a lot of code can be pulled outside the loop.

Then you are jumping through all these flaming hoops in the orange vI, constantly converting between 1d and 2d arrays without really adding any extra data. Look at the code segment on the picture. You are creating a 1d array of size N at the autoindexing output tunnel, then you convert it to a 1xN 2d array which you subsequently reshape to a 1d array of size N, giving you back what you had in the first place. Why????

 What is the purpose??

Message Edited by altenbach on 05-01-2007 01:25 AM

[RVR]

Hi Gabi and Altenbach,

thanks for pointing those errors out.

It is something that i did when I first was using lview and i did not revise my code since.

I have done so now, thanks to you, but it did not significantly speed things up.

I am currently reading some materian on embedded design from the NI web pages.

I will give that a go and try to optimise things further.

If I don't get anywhere, i may call on your (the NI community) help again...and this time with the vi.

Thank you very much