01-13-2025 02:50 AM - edited 01-13-2025 02:52 AM
Hi Pinco,
@PincoG wrote:
my problem now it is that same binary data, in the conversion, contain just a comma.
I'm thinking to replace commas into the data with some special char.
What Jolly do you me suggest to use?
Or any other idea you have to solve the problem.
When you are working with "binary data" then you cannot use a delimiter char between data elements at all.
Instead you should define a fixed message structure, maybe with a start/end char and some checksum inside. Something along "STX, 2 bytes = u16 = length of data, 1/2/4/8 bytes = u8/u16/u32/u64/DBL, ... , 2 bytes = u16 = checksum, ETX"… (STX/ETX = start/end of transmisson, explained here…)
(I used just 2 bytes for data length to handle "ordinary length" messages. When you need really large messages then you should use an u3 for this parameter.)
01-13-2025 04:03 AM
Hi GerdW,
are you sure I really need such protocol?
I mean that string type already include length in first bytes.
I'm only afraid of other formats. Like Pictures and real.
Because now I'm trying using nothing in betwwen data, and seems to work well.
01-13-2025 05:38 AM
Hi Pinco,
@PincoG wrote:
are you sure I really need such protocol?
I'm only afraid of other formats. Like Pictures and real.
You should use a well-defined format because your are afraid of "other formats"…
(Using a flattened string may contain the length, but doesn't tell you about the format of the data content.)
01-13-2025 07:24 AM
@PincoG wrote:
are you sure I really need such protocol?
I mean that string type already include length in first bytes.
I'm only afraid of other formats. Like Pictures and real.
Because now I'm trying using nothing in between data and seems to work well.
The Flatten To String and Unflatten From String will handle all of that for you. You just need to know what data is in the message. For this reason, you may want to include a message ID of some kind. This could be a simple Byte ID or a descriptive string. Then using ID, you know what the format of the data should be and you can handle it properly.
02-05-2025 09:49 AM
Everything is working well BUT some images converted in binary contains CRLF, that broke the message receive.
How can I do to avoid this mess ?
02-05-2025 10:10 AM
Hi Pinco,
@PincoG wrote:
some images converted in binary contains CRLF, that broke the message receive.
How can I do to avoid this mess ?
By not converting any CRLF chars in the message receiver routine!
02-06-2025 01:37 AM
Hi gerd,
probably I did not explained very well. The CRLFs that are doing this mess are encapsulated in the data, in my case are inside an image converted to binary.
I found a workaround: sending the message I replace x0D0A to ascii CRLF, all of them but the last one a t the end of the message, that is the only one that I have to find to close the receive operation.
Reciving routine convert again ascii CRLF to x0D0A, and then it's all ok: my images are fine and working.
The only thing is that, in this way, I cannot send ascii CRLF into strings or others, because they are my special cchractr sequence. Maybe I can find and use a more special sequence, like a kind of "***CRLF***", but I think it's not so different.
02-06-2025 03:18 AM
Hi Pinco,
@PincoG wrote:
probably I did not explained very well. The CRLFs that are doing this mess are encapsulated in the data, in my case are inside an image converted to binary.
An image usually is always "binary" data… (I guess you don't send ASCII Art.)
When sendind "binary" data (aka "data that can contain any bytes from 0x00 to 0xFF") you usually:
The receiver does:
No need to fiddle with image data, not in the sender and not in the receiver!
02-06-2025 03:23 AM - edited 02-06-2025 03:29 AM
@PincoG wrote:
The only thing is that, in this way, I cannot send ascii CRLF into strings or others, because they are my special cchractr sequence. Maybe I can find and use a more special sequence, like a kind of "***CRLF***", but I think it's not so different.
Well, and in a few months from now you come across a file that contains exactly that magic bit pattern for whatever reason and doesn't mean it should be replaced by a \r\n: Lets start to do even more escaping above the already existing escaping to escape the escape that is not an escape!!!
That is why you should have started with a proper protocol in the first place. Either it is binary or it is not! Do not try to mix and match!
Embedding binary data in an ASCII stream is absolutely insane. Trying to indicate EndOfMessage with CRLF for a stream that contains binary data, is the same!
Keep it all binary or all ASCII. Either define a sane binary protocol where any ASCII strings are simply binary data too, or escape the entire binary data to base64 encoded ASCII or something when including it in an ASCII stream. Anything else is beseeching the Gods and causing whoever is coming after you to consider sending you anonymous bomb letters, if they are able to find out who created that mess.
A binary protocol starts with a binary length value and a well known header that tells you what is following in the body. Then you can switch off the CRLF mode on the TCP Read and start reading first that length parameter of a exactly known size, then read that many bytes again without CRLF mode, and then if your header indicates there is more to follow, read that too.
But, but, that costs performance! Scanning your data for special characters to be replaced with something else on both the sender and receiver side costs a multitude more performance than breaking your read into multiple reads to read first the length, then the header, and than the body. You can even go for a fixed size header so you just do a read for the header, analyze its content and then read the body based on what the header told you!
02-06-2025 05:36 AM
Let me put on my chaotic hat: use base64 encoding. It will increase your data size and eat performance but you will still be able to send anything you want and still use CRLF mode.
Otherwise, you could show some code. Just prepending the message with a size field and switching away from CRLF mode will make your problems go away.