DataFinder slowing DIAdem 2017 down

[TamerZero]

I never used to use DataFinder, but I decided to start using it recently. It is very convenient. However, recently loading and removing TDMs has become very, very slow. The software sits and spins for 30-45 sec before the file loads or removes, it used to be instant.

Any thoughts on this? Is the indexing slowing things down? 

[apriltalksrobots]

TamerDTS,

How are you adding and removing the TDM files? Also, how many files do you have in your DataFinder?

Thanks,

aprillest

[TamerZero]

I'm just dragging and dropping them from the DataFinder to the Data Portal.

I have 50-100 TDMs, not sure on the exact count.

[Brad_Turpin]

Hi Tamer,

I'm not convinced that the DataFinder is the culprit here.  It could be, if it's actively indexing a bunch of data files and hogging the hard drive or the CPUs, but other than resource battles there's no blocking mechanism I can think of between the DataFinder and DIAdem.  The DataFinder does not participate in the loading of TDM files into the Data Portal of DIAdem.

What happens if you drag the same TDM file from one of the nodes of the NAVIGATOR tree that is NOT part of the Search Areas?  How large are these TDM/TDX data files you're loading?  Please check the "Loading Behavior" tab in the "DIAdem Settings" dialog you get by choosing the menu "Settings >> DIAdem Settings..."

If you now see "Always load bulk data" and before had "Load bulk data on modification of channel data", that would explain the loading speed difference (assuming your data files are large).

Brad Turpin

DIAdem Product Support Engineer

National Instruments

[TamerZero]

Doesn't matter if the TDM is in the search area or not, it takes forever. Like something that took seconds is taking minutes now. It's also taking forever to plot anything in the viewer tab.

My loading behavior is set to "Load bulk data on modification of channel data" Not sure what that means, or what I had before.

This is getting really frustrating. Processing that should take me 10-15 minutes is taking hours now.

[TamerZero]

I deleted all the search areas in my DataFinder and it sped right back up. I have a feeling DataFinder is trying to index the file I'm working on as I make changes, bogs the whole process down. I guess I'll have to go back to not using DataFinder for now.

[StefanR]

Hi Tamer,

it seems your process is making a lot of changes to files contained in a search area of DataFinder.

Because DataFinder immediately reacts on changes (including deleting) to files in a search area, it can have a negative impact on the process (when DataFinder index a file it blocks concurrent writing access) when the process continues working on that file.

However, there is possibly a solution to it: Pause the indexer for the time your script is running.

You can do this by using the following code:

dim oMyDataFinder : Set oMyDataFinder = Navigator.ConnectDataFinder("My DataFinder")
oMyDataFinder.Indexer.Status = eIndexerPaused

To start indexer again use

oMyDataFinder.Indexer.Status = eIndexerRunning

A more elegant way would be to create a class, which remembers the current state of the indexer and then pause the indexer. When the class terminates (gets destroyed) the indexer is set back to its initial state. The class terminates automatically for instance when your script terminates / or when using it within a sub, when the sub exits. So you will never forget to restart DataFinder even when your scripts exists in an unexpected way....

Class cPauseMyDataFinder
Private myDataFinder_Indexer_Status
'=== Constructor and deconstructor ===
Private Sub Class_Initialize()
  dim oMyDataFinder : Set oMyDataFinder = Navigator.ConnectDataFinder("My DataFinder")
  myDataFinder_Indexer_Status = oMyDataFinder.Indexer.Status
  oMyDataFinder.Indexer.Status = eIndexerPaused
End Sub
Private Sub Class_Terminate()
  oMyDataFinder.Indexer.Status = myDataFinder_Indexer_Status
End Sub
End Class

dim PauseMyDataFinder : Set PauseMyDataFinder = new cPauseMyDataFinder