save arcnet file

[Kito]

Hi everybody,

I have got file from Arcnet (10MB) and I need to save all of those data in a file. But my problem is: If I use the structure below my labview cannot getting data at real time (Remember that I'm talking about a 512 Byte packet per aqcuisition) . Does anyone know or teach me something better that a i can do.

I do need, that the sequence of files being in a column way, like for example:

1                     2              3     ......................... X

Acceleration     Velocity    A/C info                    F/W version

Acceleration1   Velocity1  A/C info1                  F/W version1

and so on....

But each data is differente, I mean, I've got 32, 16 and 8 uint in my structure (As you can see in the picture).

I tried to use spreadsheet, but as I mentioned above, It's a lot of packets and then I receive a message of Memory is full (i think that is because of the limit of the spreadsheet).

Just to sumirize: I need to get data from arcnet in a real time and save it in a file.

Regards

P.S.>The picture is ilustrative, because the file is very big and I think it's just to emphasize my example.

[Cory_K]

The memory error you are getting is not related to the spreadsheet you are writng being too large.
In fact, I dont think there is a size limit.

The memory error is occurring because you are trying to load all 10 MB of data in at once.

Not only that, but you are reading this data in a while loop with no wait, so it is spinning as fast as it can,

using up all your CPU.

[Kito]

I didn't put a wait because the rate that I'm getting the data is 10ms. is is very fast, If I put a wait I can't run in real time.

[Kito]

I use a structure to check if there is any data available in the Arcnet. If there is, I get tha data.

This is not a problem. My problem is that I can't save using the structure that I showed in the picture. So I'd like to know if there is another way to save the data, maybe using a buffer (RAM, not a temp file)  or something like that.

Regards

[johnsold]

Typically it is better to separate the reading of data from the instrument from the writing to file.  File writing delays can easily exceed the 10 ms you expect between data packets.  Also remove the indicators from the reading loop.  Because they are in the UI thread and the screen updates are slower than 10 ms, this could also be a bottleneck.

Look at the Producer/Consumer Design Patterns which ship with LV for an idea about how to separate the loops.  The loops can run at different speeds.  The Producer loop (with the arcnet read) would run as fast as the data is received.  The Consumer loop (User display and write to file) would run much slower.

Accumulate and parse several packets before writing to the file, perhaps one write every few seconds. Put a delay or wait (even zero ms) in each loop to assure that the loops share CPU resources.  The read function may perform the wait if it waits for a complete packet.

Watch out for unnecessary data copies.  Search for a white paper on Large Data Sets in LabVIEW.

Lynn 

[smercurio_fc]

Is that little blue VI returning 10MB of data, or 512 bytes, or the 20-or-so bytes that you seem to be indexing out?

You have to keep in mind that if you're running this on a computer with a "regular" operating system, like MS Windows, you will never get real-time. It's simply not possible since the OS can preempt you at any time. If you need to gather data quickly, then you should separate the data collection from the writing to file since the writing to file is an expensive operation in terms of time. Thus, you basically have 2 options:

1. Create a large initial buffer (i.e., array of bytes) that you populate in the loop to get all your data. You want to preallocate this memory so you don't do memory shuffling to continuously build a growing array. Then, when it's done write it out to file.
2. Have 2 loops. One to gather the data into a buffer, and the other loop to pull the data out of the buffer and write it to file at a slower rate. You will need to determine empirically how big of a buffer to use in order to be able to collect data quickly, keeping in mind that the file write will be pulling it out at a slower rate.

[Kito]

Lynn

Could you give me an example of how to separate that, because I don't know how to do that in a way that assure I'm receiving and then showing in a display and writing in a file.

Regards

[smercurio_fc]

You can read over the Producer/Consumer KB article, which contains an example. As Lynn pointed out, you can create a new one using the LabVIEW templates (File -> New).

[johnsold]

File >> New.. >> VI >> From Template >> Frameworks >> Design Patterns >> Producer/Consumer Design Pattern (Data).

Note that this is the second item in the file menu, not New VI.

Put the Read arcnet subVI in the Producer loop.  Every time it receives a packet, put the packet into the  queue.  Nothing else goes into this loop.  Obviously you will need to reduce the Wait time.

Do all the decoding, parsing, display, and write to file in the Consumer loop. 

Lynn 

[Kito]

I don't know how to use the data that I'm getting in another loop. Could you let me know how to do that?