[Solved] Serial port connection
[Solved] Serial port connection
Hi,
I'm testing the serial port in VBox (5.2.2 r119230 (Qt5.6.3)). The VM is configured with a client mode COM1 port (i.e. Port Mode: TCP, "connecting to existing pipe/socket", "Path/Address: 192.168.1.2:2023").
I run a simple TCP server that runs on 192.168.1.2 port 2023. It reads and then prints whatever the client sends. I want to use it as a log.
The weird thing is the log server doesn't receive/print anything from client unless the client sends a LF byte (i.e. ascii 0x0A), or the VM is shutdown.
For example, I can write 600 bytes to the serial port's THR register in a tight loop, nothing shows up on my TCP server side. However, if I send an LF byte after these 600 bytes, all bytes are printed on log server immediately.
The COM1 was configured as: 115200/8-bit-1-stop-no-parity/14-byte-receive-trigger. TX/RX interrupts are enabled.
The log server runs on a Linux VM. I tried read() and select()/recv(), also set/unset the TCP_NODELAY option. Same results.
It seems the data sent thru the VBox serial port are being cached by the device driver. The cache is flushed to remote only if an LF is written to the COM1's THR register.
But is this correct behavior? I don't think I have to do this with a real serial port.
I'm testing the serial port in VBox (5.2.2 r119230 (Qt5.6.3)). The VM is configured with a client mode COM1 port (i.e. Port Mode: TCP, "connecting to existing pipe/socket", "Path/Address: 192.168.1.2:2023").
I run a simple TCP server that runs on 192.168.1.2 port 2023. It reads and then prints whatever the client sends. I want to use it as a log.
The weird thing is the log server doesn't receive/print anything from client unless the client sends a LF byte (i.e. ascii 0x0A), or the VM is shutdown.
For example, I can write 600 bytes to the serial port's THR register in a tight loop, nothing shows up on my TCP server side. However, if I send an LF byte after these 600 bytes, all bytes are printed on log server immediately.
The COM1 was configured as: 115200/8-bit-1-stop-no-parity/14-byte-receive-trigger. TX/RX interrupts are enabled.
The log server runs on a Linux VM. I tried read() and select()/recv(), also set/unset the TCP_NODELAY option. Same results.
It seems the data sent thru the VBox serial port are being cached by the device driver. The cache is flushed to remote only if an LF is written to the COM1's THR register.
But is this correct behavior? I don't think I have to do this with a real serial port.
-
- Site Moderator
- Posts: 27330
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: Serial port connection
From my experience with serial ports many, many years ago, there's always a "Termination character" that you have to set in order to indicate the termination of the string sent. This is typically set to 0x0a. So I guess, the answer to your last question is yes, you have to include the termination character.
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
-
- Site Moderator
- Posts: 39156
- Joined: 4. Sep 2008, 17:09
- Primary OS: MS Windows 10
- VBox Version: PUEL
- Guest OSses: Mostly XP
Re: Serial port connection
That is not the behaviour of a serial port (which is character based, not line or packet based). However it may be the behaviour of the guest OS serial drivers. I.e. it sounds Unixy to me.
-
- Site Moderator
- Posts: 27330
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: Serial port connection
You are most probably absolutely right, since all the serial port connection encounters that I had, were instrument based, and they definitely like a line-based approach. As in:mpack wrote:That is not the behaviour of a serial port (which is character based, not line or packet based).
SNAP?1,2,4,10,11;\0x0a
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Re: Serial port connection
I'm not using any guest OS. I wrote a very simple MBR and jump to UART code immediately.mpack wrote:That is not the behaviour of a serial port (which is character based, not line or packet based). However it may be the behaviour of the guest OS serial drivers. I.e. it sounds Unixy to me.
Ideally, even if I only write a single byte to VM's COM1 THB register, this byte should be delivered to the remote TCP server immediately. Using 0x0a as "Termination Char" leads to problems for sending binary data.
Is it possible the Nagle's algorithm(or something similar -- e.g. an internal cache) on VM side delays the data transmission?
@socratis, I read National Semiconductor's PC 16550D UART spec (June 1995), Xilinx's AXI UART 16550 spec v2.0 (Oct 2016), and Xilinx's 16550 device driver (https://github.com/Xilinx/embeddedsw/tr ... tns550/src). None of them mentioned or implemented any "Termination Char".
-
- Site Moderator
- Posts: 39156
- Joined: 4. Sep 2008, 17:09
- Primary OS: MS Windows 10
- VBox Version: PUEL
- Guest OSses: Mostly XP
Re: Serial port connection
I think you misunderstood my previous comment. I responded to Socratis when he said that this was his recollection of how serial ports worked back in the day. It wasn't a comment on how a virtual UART works.gtq wrote: Is it possible the Nagle's algorithm(or something similar -- e.g. an internal cache) on VM side delays the data transmission?
I have no problem believing that a virtual UART must buffer data for acceptable performance, and no problem accepting that in "pipe mode" there are additional constraints. However I have no long experience of using pipes for communications so I've no idea what the norms are with them. Quite possibly they use the Unix text terminal model, and only do a tx on EOL.
-
- Site Moderator
- Posts: 27330
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: Serial port connection
You remember what I said about controlling instruments? Maybe it was a requirement for every instrument driver out there. I wasn't talking to the UART, I was using the UART to talk to instruments. The example of the SR830 DSP Lock-in Amplifier that I brought up, was from 1993 working on a 80486! Quite a few differences from last year's spec...gtq wrote:@socratis, I read ... spec v2.0 (Oct 2016)... None of them mentioned or implemented any "Termination Char".
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Re: Serial port connection
Sorry for the misunderstanding. @socratis, so the "termination char" is part of the DCE device protocol (e.g. a modem connected to the serial port, or the Amplifier you mentioned)?
I believe you're right VBox is emulating this "termination char" or Unix text terminal mode, but I think it should only emulate the UART chipset, not the "termination char". If I connect 2 VMs via RS232 cable, I should be able to send data without "termination char".
If my understanding is correct this time, should this be considered a bug?
I believe you're right VBox is emulating this "termination char" or Unix text terminal mode, but I think it should only emulate the UART chipset, not the "termination char". If I connect 2 VMs via RS232 cable, I should be able to send data without "termination char".
If my understanding is correct this time, should this be considered a bug?
-
- Site Moderator
- Posts: 39156
- Joined: 4. Sep 2008, 17:09
- Primary OS: MS Windows 10
- VBox Version: PUEL
- Guest OSses: Mostly XP
Re: Serial port connection
I would suspect that it's the stream driver in the guest OS, not in VirtualBox. I'm no Unix expert, but ISTR that stream oriented ports can be configured for byte mode.
-
- Site Moderator
- Posts: 27330
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: Serial port connection
I did a quick search and I found a "Linux Serial Programming HOWTO" (or in searhcable PDF). In there they talk about three modes of communicating:
Generally speaking the termination character has nothing to do with the communication, it's a "convention" between the sender and the receiver. They got to speak the same language and have the same expectations.
The HOWTO includes code and examples, so it might be worth taking a look at it...
which seems to coincide with my memory cells that still remember a couple of things about serial ports. So, what you're looking for is the "Non-canonical form" with a 1-byte character per read, no timeout and I'm not quite sure about the asynchronous vs. synchronous mode, but that would depend on how your daemon is implemented. Or, you could append a termination character to each and every byte to make the communications immediate. That will add an overhead to your communications, but you'll be the judge of its importance.
- 2.3.1. Canonical Input Processing
- This is the normal processing mode for terminals, but can also be useful for communicating with other dl input is processed in units of lines, which means that a read will only return a full line of input. A line is by default terminated by a NL (ASCII LF), an end of file, or an end of line character. A CR (the DOS/Windows default end−of−line) will not terminate a line with the default settings.
- 2.3.2. Non−Canonical Input Processing
- Non−Canonical Input Processing will handle a fixed amount of characters per read, and allows for a character timer. This mode should be used if your application will always read a fixed number of characters, or if the connected device sends bursts of characters.
- 2.3.3. Asynchronous Input
- The two modes described above can be used in synchronous and asynchronous mode. Synchronous is the default, where a read statement will block, until the read is satisfied. In asynchronous mode the read statement will return immediatly and send a signal to the calling program upon completion. This signal can be received by a signal handler.
Generally speaking the termination character has nothing to do with the communication, it's a "convention" between the sender and the receiver. They got to speak the same language and have the same expectations.
The HOWTO includes code and examples, so it might be worth taking a look at it...
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
-
- Site Moderator
- Posts: 39156
- Joined: 4. Sep 2008, 17:09
- Primary OS: MS Windows 10
- VBox Version: PUEL
- Guest OSses: Mostly XP
Re: Serial port connection
The only question is who is doing this queuing. The OP claims that there is no OS as such, that he's booting straight into his own code. Now I know for a fact that VirtualBox doesn't require a LF before it will transmit on the serial port (*), so I guess that makes it a feature of the pipe. I suggest doing some digging into pipes on the host OS.
(*) The 16550D UART has receive and transmit FIFOs, from memory they are either 14 or 16 characters. The UART doesn't know anything about lines or EOLs, it just sends bytes. Transmit happens either when the tx fifo fill level is reached, or when 4 character times have elapsed since the last write. The 16550D was great, I miss it. Most of the modern MCUs I work with have a half assed microcode UART that requires me to set up a DMA channel to get the equivalent of FIFOs, but all so far have overlooked the vital timeout feature.
(*) The 16550D UART has receive and transmit FIFOs, from memory they are either 14 or 16 characters. The UART doesn't know anything about lines or EOLs, it just sends bytes. Transmit happens either when the tx fifo fill level is reached, or when 4 character times have elapsed since the last write. The 16550D was great, I miss it. Most of the modern MCUs I work with have a half assed microcode UART that requires me to set up a DMA channel to get the equivalent of FIFOs, but all so far have overlooked the vital timeout feature.
Re: Serial port connection
Just figured out it's my bad. When I receive the data on TCP server, I printed the data with "std::cout << data". I didn't know "std::cout" caches data by lines. Same thing with printf and Python sys.stdout.write().
I usually use these print statements to print large amount data. Never noticed this cache behavior before. What a shame!
Thank you guys for help,
I usually use these print statements to print large amount data. Never noticed this cache behavior before. What a shame!
Thank you guys for help,
-
- Site Moderator
- Posts: 27330
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: Serial port connection
OK, you got my curiosity up...gtq wrote:I usually use these print statements to print large amount data. Never noticed this cache behavior before.
What do you use instead of those statements then?
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Re: Serial port connection
You mean how to fix the print caching problem? Flush the output stream like this: std::cout << data << std::flush. fflush() should work for printf().
-
- Site Moderator
- Posts: 27330
- Joined: 22. Oct 2010, 11:03
- Primary OS: Mac OS X other
- VBox Version: PUEL
- Guest OSses: Win(*>98), Linux*, OSX>10.5
- Location: Greece
Re: Serial port connection
Flush the buffers... Great, thanks for the reply.
Do NOT send me Personal Messages (PMs) for troubleshooting, they are simply deleted.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.
Do NOT reply with the "QUOTE" button, please use the "POST REPLY", at the bottom of the form.
If you obfuscate any information requested, I will obfuscate my response. These are virtual UUIDs, not real ones.