Deleting blocks of values in several channels - Is there a better way than this?

[Matthew.cf828br]

Hi All,

I've got two groups of data in my data portal. In the second group there are 3 channels each containing roughly 2.5 million data points. I want to delete for instance the first 10000 points in each channel. I'm currently doing it like this:

Dim Chn_No_Str

Chn_No_Str = "'" & "[" & GroupCount & "]/[1]" & "'-'[" & GroupCount & "]/[" & Data.Root.ChannelGroups(GroupCount).Channels.Count & "]'"
Call DataBlDel(Chn_No_Str,1,10000,1)

Is there any better (Faster) way to do it? Currently it takes roughly 203 seconds to do it. However there could be 100+ channels in the second group, all of which would need to have the same operation performed on them. As you can see it would then take considerably longer to delete the 10000 points. By my calculations 6777 seconds (1.9 hours).

Looking forward to reading any replies and thanks for looking.

Matthew

[Brad_Turpin]

Hi Matthew,

Actually, that's already the fastest way to do it.  You could reduce the size of your code a bit by using the older GroupChnCount() function, like this:

Chn_No_Str = "'" & "[" & GroupCount & "]/[1]" & "'-'[" & GroupCount & "]/[" & GroupChnCount(GroupCount) & "]'"
Call DataBlDel(Chn_No_Str, 1, 10000, 1)

I created 3 channels with 2.5 million values on my 2 year old laptop and ran the above code in well under a second.  So something goofy is going on at your end if it takes minutes to execute.  Perhaps the first group of channels is taking up all your RAM and the second group is loaded with virtual memory?  What happens if you load only the second group of 3 channels, does the above code run in about a second then?

Brad Turpin

DIAdem Product Support Engineer

National Instruments

[Matthew.cf828br]

Hi Brad,

Thanks for the reply. I think your comment regarding the first group of channels taking up all the RAM and the second group being loaded into virtual memory is correct. The second group was in fact a subset of the first group. Is there any workaround that you are aware of in this scenario other than deleting the first group.

With the first group deleted the code executed in about 80ms.

Thanks for you assistance in advance.

Regards

Matthew

[Hossein128]

Hi,

I hope that you already solved your problem but I found some information about the using of a large amount of data in DIAdem which is given below.

In general, if you have files over 1 GB in size it is recommended that you register the data on disk instead of actually loading it into memory. Even if your machine as 3+ GB of memory, not all of it will be devoted to the DIAdem process because of limitations in memory allocation in Windows. To register the data, right click the TDMS file in the navigator and select Register Data. Registered data does not perform as well as loaded data but it will still be much faster than 37 minutes!
You said that DIADem is using 20 GB of ram, did you mean 20 MB? if you attempt to load a 3+ GB file in DIAdem it should be using about 1.2 GB of memory (depending on how much ram your machine has).

http://digital.ni.com/public.nsf/allkb/70E5B1A4AED125D78625731A005874F3?OpenDocument

I hope this helps.

Best regards,

Hossein