synchronous call

[MikeBowen]

Greetings all,

I am working on an application where a single network file has to have read/write access from myultiple machines.  This is for a manufacturing test environment.

Initial development work was with the file being stored locally (single machine), and that worked well.  We want to store 50K to 100K records, and rather than do a resize check once a day (or similar) we opted to have size processing where records are deleted (if needed) on each call.  I also did the coding such that after the file was read it was kept open, file size resized to zero, and fresh data written back to the file.

The next step I put the file on a network location, and copied the file back and forth to a local machine.  I knew what was going to happen, and processing time was on the order of a minute - not acceptable.

I think the solution is this:

1) Have the code that interfaces to the file resident in one place, on the network machine.

2) Have the local machines make a synchronous call to the network VIs (One for write and one for read).   This way the loca code is forced to wait for the network program to run, and this prevents large text files being continuously have read/write operations done over the network.

The issue I am runnning into is how to make the synchronous call.

Mike

[BowenM]

Have you looked in to just running a little demon on the server with a shared variable?

http://www.ni.com/white-paper/4679/en/

-M

[MikeBowen]

I'll look into it.  So your name is Bowen too?

[BowenM]

Yep! The "M" in my name isn't for "Mike" though 🙂

[MikeBowen]

I would say no, because the data file would still have to be transferred to the local machine, processed, then copied back.  The method I am working on means only very little network traffic is involved, which should speed up operation.

[BowenM]

Perhaps I'm not understanding what you need. The shared variable doesn't need to be the entire dataset. You basically would be utilizing the FIFO properties of the variables to make sure that multiple instances aren't trying to write to the file at once. Isn't that what you were trying to solve?

[BowenM]

Perhaps I'm not understanding what you need. The shared variable doesn't need to be the entire dataset. You basically would be utilizing the FIFO properties of the variables to make sure that multiple instances aren't trying to write to the file at once. Isn't that what you were trying to solve?

[MikeBowen]

The write operation is one data record.  When I read all that I want to do is query to see if a particular sn has passed at a specific station.  Does the shared variable engine have any processing capabilities?

[BowenM]

Oh, I see.  No, you'd need a demon on the server accessing the shared variable and processing read/write requests.

[MikeBowen]

I was able to read 50K records on the server and return a result in 5 seconds.  We are working with a 30 second test window, so the time for these operations needs to be about 1 second for both the read and write operation.  If I cut my record size to25K that would be a week's worth of data, and assuming a linear relationship between record size and time that would make it 2-3 seconds.

I'll keep looking.  Thanks for the input.