topic Re: named queue stops working? in LabVIEW

named queue stops working?

Ha-Jo — Fri, 13 Jul 2007 16:27:49 GMT

Hello, I've got a problem using named queues. I've wrote a program for a Measurement set-up, which is installed on a ferry to collect environmental data along the route between the home- and the destination port over a long period. In automatic mode the set-up is controlled by the ships GPS-position. If it leaves an defined area around the port, a calibration of some instruments is performed. After a succesfully ended calibration a "water-measurement"-mode starts in a timed loop (period 1 minute) till the ship reaches a specified latitude to perform some "air-measurments" and then go back to "water-measurement"-mode again till it reaches a defined area around the destination port and so on. In "water-measurement"-mode two readings have to be delayed for 2 minutes, to compensate the time, the water needs to flow from an outer sensor to the set-up. I thougt it was a good idea to use a named queue for each reading to delay it for two periods of the timed loop. A third named queue collects error informations around the programm. This error information is queued out, displayed and stored in a file in a separate loop. The program works fine at the first journey. At the run back only calibration and air-measurements are stored. I've got neither water nor error information. My next step was adding an additional named queue to collect status information about the programm parts which have passed through and additional every five minutes the actual GPS-position. Now all informations from sources, which are including a queue, is ending already about 20 hours past program start. About 45 minutes after truncation of water measurements, errorlog and infolog, the air measurements again are performed correctly. Before installation on board, we tested the set-up whith a simulated GPS-signal. In these tests we had accelerate time for a journey. Whith 60 minutes for one run between the ports we performed multiple runs without any problem. So I guess, there is a problem caused by my usage of queues. I tested my DataDelay.vi in a loop and got some strange results after I forgot to stop my test. It seems that labview is consuming an increasing amount of CPU-time after a high number of loop cycles. In the attachment I tried to speed up the effect by using 3 of these VIs in a loop. On my PC after about one hour the labview part of the CPU-time reaches nearly 100%. This may not be related to the problem at the ship, because the conditions are too different, but I don't understand it. My next trial will be to replace the queues for data delay by shift registers, "build array" and "delete from array". Since the set-up is installed at the ferry, it it costly and time-consuming to get access. I have to ride about 150 km to the port at tuesday evening to get some hours around midnight to make changes. Please excuse my english. I hope, someone can give me some hints to understand and solve the problem. Ha-Jo

Re: named queue stops working?

Ha-Jo — Fri, 13 Jul 2007 16:32:06 GMT

Sorry, I've forgotten to attach the excample Ha-Jo

Re: named queue stops working?

Evan — Fri, 13 Jul 2007 17:00:13 GMT

I took a very quick look at it and I am a little confused. Why do you obtain the queue twice in the same subVI? In the delay case you have the reference already but you obtain it again.

I wonder if your issue is that you are basically leaking Queue references. You would have obtained a reference an lot of times after hours of operation.

Re: named queue stops working?

Ha-Jo — Fri, 13 Jul 2007 17:28:03 GMT

Sorry again, seems that I attached the wrong VI. My login was timed out and Iwas a little bit confused. I had send the vi to a colleque, and I think he was playing a little bit around with it. Here comes the original excample. Ha-Jo

Re: named queue stops working?

johnsold — Fri, 13 Jul 2007 18:02:39 GMT

Wiring the queue references rather than repeatedly calling Obtain Queue in the DataDelay.vi speeds it up substantially. I am using LV 8.2 so I cannot easily give you a LV 7.1 version. I have attached an image of the Delay Case. By moving the Data valid indicator outside the case I have eliminated the local variables. This probably speeds things up also.

Lynn

Re: named queue stops working?

Ha-Jo — Fri, 13 Jul 2007 18:50:36 GMT

Sorry, sorry, sorry
can't get rid of the wrong attachment. Now I renamed it and tried again. Thank you for the hint with the location of the data valid indicator, I will change it. The rest of my original VI, I think, isn't so much different from Your suggestion.
But I think, the main part of my problem isn't speed. In the original program the Datadelay.vi runs inside a timed loop with a period of one minute for a long time. After more than 20 hours of operation it seems, that the Data valid indicator turns false, preventing the rest of that vi to calculate and store data. In addition all my attempts to get infos about the state of the program or errors, where I used other named queues seems to truncate at the same time.

Ha-Jo

Re: named queue stops working?

Anonymous — Fri, 13 Jul 2007 19:10:03 GMT

On Jul 13, 12:40 pm, Ha-Jo <x...@no.email> wrote:
> Hello,
> I've got a problem using named queues. I've wrote a program for a Measurement set-up, which is installed on a ferry to collect environmental data along the route between the home- and the destination port over a long period. In automatic mode the set-up is controlled by the ships GPS-position. If it leaves an defined area around the port, a calibration of some instruments is performed. After a succesfully ended calibration a "water-measurement"-mode starts in a timed loop (period 1 minute) till the ship reaches a specified latitude to perform some "air-measurments" and then go back to "water-measurement"-mode again till it reaches a defined area around the destination port and so on.
> In "water-measurement"-mode two readings have to be delayed for 2 minutes, to compensate the time, the water needs to flow from an outer sensor to the set-up. I thougt it was a good idea to use a named queue for each reading to delay it for two periods of the timed loop. A third named queue collects error informations around the programm. This error information is queued out, displayed and stored in a file in a separate loop.
> The program works fine at the first journey. At the run back only calibration and air-measurements are stored. I've got neither water nor error information.
> My next step was adding an additional named queue to collect status information about the programm parts which have passed through and additional every five minutes the actual GPS-position. Now all informations from sources, which are including a queue, is ending already about 20 hours past program start. About 45 minutes after truncation of water measurements, errorlog and infolog, the air measurements again are performed correctly.
> Before installation on board, we tested the set-up whith a simulated GPS-signal. In these tests we had accelerate time for a journey. Whith 60 minutes for one run between the ports we performed multiple runs without any problem. So I guess, there is a problem caused by my usage of queues.
> I tested my DataDelay.vi in a loop and got some strange results after I forgot to stop my test. It seems that labview is consuming an increasing amount of CPU-time after a high number of loop cycles. In the attachment I tried to speed up the effect by using 3 of these VIs in a loop. On my PC after about one hour the labview part of the CPU-time reaches nearly 100%. This may not be related to the problem at the ship, because the conditions are too different, but I don't understand it.
>
> My next trial will be to replace the queues for data delay by shift registers, "build array" and "delete from array".
> Since the set-up is installed at the ferry, it it costly and time-consuming to get access. I have to ride about 150 km to the port at tuesday evening to get some hours around midnight to make changes.
>
> Please excuse my english. I hope, someone can give me some hints to understand and solve the problem.
> Ha-Jo

It seems like there is a memory leak due to queue not being emptied or
initialized. make sure that you initialize the queue first, use it in
the loop and delete the queue before quitting the program. If
synchronization is not important, try using shift registers or global
variables. hope it helps.

Re: named queue stops working?

johnsold — Fri, 13 Jul 2007 19:23:20 GMT

I wonder if your problem has to do with memory allocation? The queues keep growing in your application. Arrays will require memory re-allocation if they increase in size and I suspect that queues do also. Can you predict how much data will be collected over the length of your run? Or at least determine an upper limit on the amount of data? If so, create the named queue with a maximum size greater than or equal to the largest amount of data you need to collect. If the queues preallocate memory for a specified maximum length, then this would avoid the re-allocation issue. The help file does not specify how the queue handles its dataspace memory.

If you switch to arrays as you mentioned in a earlier posting, use Initialize array outside the loop with the size large enough for all your data and use Replace array subset inside the loop. Never use Build Array or Autoindexing on array which can get large or you will likely have memory problems.

Another option might be to only save a small amount of data in memory, write it to a file, then collect the next set of data. After the run is finished calculate the results from the data saved in the file(s). This also prevents loss of data if the computer loses power or the freezes.

Lynn

Re: named queue stops working?

Ha-Jo — Fri, 13 Jul 2007 19:53:54 GMT

Thank you, unfortunately I missed to check the task manager to get some information about the state of the PC before I exit the program at my last visit on board the ship. I will overtake it when I visit the ship again next Tuesday night. Because the data delay is called every minute during the measurements, I will replace it by shift registers. I hope this helps me to get stable readings for some consecutive journeys. Now I will go offline till Sunday evening (it's already 9:47 PM here in Germany and I have to go to my hometown this weekend) Thank You all again Ha-Jo

Re: named queue stops working?

Ha-Jo — Fri, 13 Jul 2007 20:22:53 GMT

Thank you Lynn, I didn't specify the length of the queues, but each of the two delay lines actually is holding only two doubles and all calculated values are written to a file on harddisk and additional to an USB-stick (for a convenient access) at the end of each cycle of the timed loop. The queues for my logs are also examined very often in a separate loop and each time, there is an element in it, it will be dequeued and written to disk (exceptly the errorLog, which is additional displayed on screen, but it doesn't containe much entries), so I didn't expect any memory problems. I will rethink my planed solution including the shift registers and the build array function to avoid calling the memory manager. I think, it's possible to build something like an fixed array and rotating pointers for access, but it has to wait at least till Sunday evening. Thank you all again Ha-Jo

Re: named queue stops working?

mikeporter — Sat, 14 Jul 2007 00:31:52 GMT

I guess the thing I don't understand is why you are using queues in the first place. This is a pretty straight-forward low speed data logging application. Why bother with queues when you can just write the data out to some sort of temporary file or database? When you have a measurement process that's going to take a long time you don't want to hold all the data in memory because it's too easily lost.

Mike...

Re: named queue stops working?

Ha-Jo — Sun, 15 Jul 2007 22:02:47 GMT

Hello Mike,
thank you for answering to my message. I didn't keep much data in the queue, but during water measurements it is essential to compensate some time, the water needs to flow through pipes from outside the ship to the setup. Along this way the temperature of the probe is changing. To calculate some gas saturations in an equilibration prozess accurate, the scientist, who developt the set-up, needs to know the original temperature of the probe. As a compromise he wants to be able to change the delay in units of the measurement cycle, which is one minute. Actually two readings are hold for two cycles each, to compensate about two minutes the probe needs on its way through a pipe.
I thought, it is a convenient way to use a queue as some sort of shiftregister with an adjustable length. I didn't expect any problems because the maximum needed length may be about 10 cycles and there are no special requirements for timing.
A third queue was used to collect strings from LabVIEW error messages around the VIs of the program to write them in a log file centrally. This queue is examined very often to dequeue any element in it immediately.
The program works fine on the first journey to the destination port but at the run back there were only calibration- an air-data stored in the files. My first idea was a bug in my program, but I couldn't find it. So I decided to put little strings in nearly all branches of the program flow to give me some hints about what really happens. Again I used a named queue to collect them before writing on hard disk. Now the stored water measurements are disappearing about 45 minutes before the mode changes from water to air measurement the first time and my new "InfoLog.txt"-file stops at the same time. Thats why I'm thinking, the problem is related to my queues, but honestly, I don't really understand it.
Tuesday night I will replace my "DataDelay"-queues by an array solution. Actual I'm not at work this week, but it's my last chance to provide a functioning program till next week. Then the scientist will be on board during the journey to make some fine-tuning with his instruments.

Ha-Jo