High speed data transfer internally, between loops. Nothing seems fast enough.

[JørgenSteen]

---

@Kevin_Price wrote:

As others have said, the producer part of the code would first accumulate single samples until a fixed size array is full, then write that whole array to the queue / RTFIFO as a single queue element.

-Kevin P

---

This might be the solution, i have just been reluctant to put addition tasks on the control loop as i want to push it to go even faster later.

[Jamosgee]

I agree with the others on sending an array in the queue.

If that doesn't work, you could try implementing a constant size buffer your self. I'd make it an circular buffer using an array. You could wrap it into an FGV or maybe also OO Singleton. You could integrate into the reading of the buffer a check if it's already full or not and only return the data when it's full. The FGV is blocking, so it cannot overflow. With the constant size buffer it could be fast enough to not cause a problem. In a circular buffer, clearing is extremely fast. Reading the whole buffer could take some time, but that you have to test.

[James_McN]

---

@JørgenSteen wrote:

Another option on a similar note i have been thinking about is having 2 normal queues and then write say 4k samples to one then 4k to the other, and then back to the first queue. With this i could flush queue 1 while the control loop is writing to queue 2 and vice versa. What do you think of an approach like this? (writing to a normal queue has never been a problem, just the flushing that blocks.)

---

While this could work I would be tempted to stick to the RT FIFOs since they are inherently designed for predictable timing, which is key here. Standard queues will use a system of locks internally to block access which means that what your other loops do can impact the performance of the control loops.

The RT FIFOs are probably (I don't know the exact details) designed to be lock-less so whatever is going on elsewhere they should have a predictable performance. They are designed for exactly this kind of scenario

With the concern of additional tasks on the control loop - these should both be very fast, as long as you pre-allocate the array so I wouldn't be too concerned. Anything other than this is likely to result in another loop having to run at the same speed, hogging another CPU so as a system it will probably run much faster.

[JørgenSteen]

---

@James_McN wrote:

---

@JørgenSteen wrote:

Another option on a similar note i have been thinking about is having 2 normal queues and then write say 4k samples to one then 4k to the other, and then back to the first queue. With this i could flush queue 1 while the control loop is writing to queue 2 and vice versa. What do you think of an approach like this? (writing to a normal queue has never been a problem, just the flushing that blocks.)

---

While this could work I would be tempted to stick to the RT FIFOs since they are inherently designed for predictable timing, which is key here. Standard queues will use a system of locks internally to block access which means that what your other loops do can impact the performance of the control loops.

The RT FIFOs are probably (I don't know the exact details) designed to be lock-less so whatever is going on elsewhere they should have a predictable performance. They are designed for exactly this kind of scenario

With the concern of additional tasks on the control loop - these should both be very fast, as long as you pre-allocate the array so I wouldn't be too concerned. Anything other than this is likely to result in another loop having to run at the same speed, hogging another CPU so as a system it will probably run much faster.

---

Thanks for the answer.

I will set up a few proper tests and then give the results in full tomorrow or monday.

If anyone has any other suggestions or comments on my dummy code, please feel free to add your thoughts.

[JørgenSteen]

Hei everyone who is interessted.

I ended up going with what James suggested (and many of you suggested).

I made the control loop gather up data in 100 element arrays before sending it into the RT FIFO, and it worked great. No, negativ impact on the reader nor the control loop. The control loop was running at 8kHz, which meant the reader was running at a comfortable 80Hz(12.5ms).

I was happy with these result and a bit suprised as i am sending a cluster with more clusters inside it and was afraid i needed to change that and make the code less readable. An important notice for anyone who will do this is to configure the reading to blocking and the writing to polling. When i did this i had less issues and used a lot less CPU power. It might seem obvious but i think it is an important note.

I also tried my alternative which was to switch between two normal queues so one queue could be emptied while the other one was written to. This also worked, but was a lot less elegant as it needed a mechanism of telling the reader which queue to read and even more logic, than James suggestion.

Thanks to everyone who commented. I want to mark this as answered, but has not done this before. Should i mark James original comment as the solution or this(my own) comment as the solution, since it breaks it down?

[Bob_Schor]

I recommend you give James credit for the Solution.  Anyone who "finds the solution" will probably read the entire post, and see you say "Yes, that works!".

Bob Schor

[JørgenSteen]

Thanks for the answer. My first post so i just wanted to follow the best practice here.