Serial port (TCP mode) garbage characters at connection

[RicV]

---

[ModEdit: Posts split from the Discuss the 6.0.4 release thread, this discussion was getting its own life]

---

I am having a problem with virtual serial port in 6.0.4. I configure serial port 1 as a TCP socket server, as shown in the attachment. My PDM plugin connects to this socket, to receive input from the user who writes it to /dev/ttyS0 (or COM1). This has worked from VirtualBox 5.0.x through 5.2.26, but with 6.0.4 my socket client gets garbage data (actually a buffer of incrementing bytes).

I notice the release notes talk about having fixed some problem with the serial port, but it's unclear whether that problem relates to my problem.

Anyone else having such a problem? I am running this on RHEL7.3.

[socratis]

@RicV
Is this something with 6.0.4 only, or have you noticed it since 6.0.0? Did you try with 6.0.0/2? Because there was a ticket, #18319 Serial ports (uart) don't work after upgrading from 5.2.22 to 6.0.0 and 6.0.2, but it was fixed with 6.0.4.

Can you try the previous 6.0.x versions and see if the fix for #18319 may have broken your setup?

[RicV]

@socratis
I had not tried 6.0.0 or 6.0.2 as I was otherwise occupied when they came out. So I jumped right from 5.2.22 to 6.0.4. But you make a good point, so I just tried 6.0.2. Unfortunately, I cannot build 6.0.2. In a new terminal, I did the configure, then sourced env.sh, then kmk all and got this:

kBuild: Installing VBoxDD2 => /home/rvilbig/vbox/VirtualBox-6.0.2/out/linux.amd64/release/bin/VBoxDD2.so
The failing command:
@g++ '-Wl,-rpath,/opt/VirtualBox' '-Wl,-rpath,/opt/VirtualBox' -Wl,-z,noexecstack,-z,relro -Wl,--as-needed -m64 -o /home/rvilbig/vbox/VirtualBox-6.0.2/out/linux.amd64/release/obj/tstGuestControlSvc/tstGuestControlSvc /home/rvilbig/vbox/VirtualBox-6.0.2/out/linux.amd64/release/obj/tstGuestControlSvc/dt/service.o /home/rvilbig/vbox/VirtualBox-6.0.2/out/linux.amd64/release/obj/tstGuestControlSvc/tstGuestControlSvc.o /home/rvilbig/vbox/VirtualBox-6.0.2/out/linux.amd64/release/bin/VBoxRT.so /home/rvilbig/vbox/VirtualBox-6.0.2/out/linux.amd64/release/bin/VBoxVMM.so /home/rvilbig/vbox/VirtualBox-6.0.2/out/linux.amd64/release/bin/VBoxRT.so -lpthread -lm -lrt -ldl
kBuild: Linking tstGuestControlSvc
/opt/VirtualBox/VBoxREM.so: undefined reference to `HMR3CanExecuteGuest'
collect2: error: ld returned 1 exit status
kmk: *** [/home/rvilbig/vbox/VirtualBox-6.0.2/kBuild/footer-pass2-compiling-targets.kmk:739: /home/rvilbig/vbox/VirtualBox-6.0.2/out/linux.amd64/release/obj/tstGuestControlSvc/tstGuestControlSvc] Error 1
kmk: *** Waiting for unfinished jobs....

I will try 6.0.0

[RicV]

Sorry I didn't think to mention it. My job includes maintaining a PDM plugin, and building that requires a VBox build tree. Plus sometimes I have had to dig into VBox code to understand why mine isn't working right. Once done, my plug-in works with a standard build, but as I am the developer I am working with the source code tar ball for a given release.

Anyway, 6.0.0 doesn't build either, differently. It's surprising, since 6.0.4 gave me no surprises. I will have to take a look to see what's going on here.

[RicV]

@socratis,

Glad I persevered. Here is what I learned. There is definitely something odd about the 6.0.x OSE tarballs, compared to 5.2.x or prior. But that is off topic. I can report my findings on that part in a different thread, or email to vbox_dev if you like.

As for the virtual serial port, it behaves the same in 6.0.0, 6.0.2 and 6.0.4. When my socket client first opens the socket, it receives a bunch of garbage characters which appear to be an incrementing byte pattern. The attached log shows what the socket client is receiving from VirtualBox. The last four lines of the log show the text which I actually wrote to /dev/ttyS0, specifically "-logmsg ABC the log" (without the quotes).

If I revert to 5.2.26, then I do not see any garbage characters coming through the socket, and the only message I see is "-logmsg ABC the log" (without the quotes).

[RicV]

I would be willing to try the virtual serial port on an official release, but as I understand it the official release and OSE builds are incompatible. So I need somehow to completely uninstall my OSE build before I can attempt installing an official release. This is something that gave me considerable trouble in the past, so I have steadfastly avoided doing that. I would need some guidance on making the transition, and then going back again.

[klaus]

There have been substantial changes in the serial port code during development of 6.0, so if you have a use case where the serial port doesn't work as it did in 5.2.x then it's quite plausibly a regression which came in through the changes. If you can provide details about the use case it might help figuring out what's broken and how to fix it.

[RicV]

Hi Klaus. My use case, in a nut shell, is as follows. I have a PDM plug-in which simulates a PCIe device below a PCI/PCIe bridge. This allows one to start developing drivers on the VM before the actual hardware exists. I configure the virtual serial port as a TCP server, and use RTThreadCreate() in my plug-in to start a client thread which receives messages from the virtual serial port. This allows the user to set test modes and parameters of the plug-in, or control logging features, that sort of thing.

I have found that in 6.0.x there are garbage characters coming through the socket. In my investigation, I removed my plug-in completely so there is no RicV code in the picture at all, just vbox code. Then I wrote a simple little socket client app that echos everything it gets from the virtual serial socket out to stdout. First I start the VM (Ubuntu 16 guest), and then immediately in another terminal on the host, start the skt-echo program. After the guest finishes booting, I open a terminal in the VM and echo a text message to /dev/ttyS0.

When using 5.2.26 or prior, the message that I echoed to /dev/ttyS0 comes through fine. But in 6.0.x there are a bunch of garbage characters coming through the socket before my message comes through. I have found the gobbledygook comes through all at once, right as the VM starts to boot.

Everything you need to duplicate this is attached.

[socratis]

@RicV
I split your posts from the Discuss the 6.0.4 release thread, as your issue

1. was getting a life of its own,
2. it wasn't strictly speaking a 6.0.4 issue.

I hope that the new title covers your issue...

[RicV]

well put

[RicV]

Is this thread sufficient to have someone take a look at the problem? Should I write an email to vbox-dev?

[andyp73]

I've had a quick look at this using my Fedora 27 guest running under VirtualBox 6.0.4 on my macOS 10.14.3 host. I made a small change to the skt-echo.c so that when it receives data it dumps out the individual hex bytes as well as the string. Here is what I have seen:

echo "Hello" > /dev/ttyS0
Client received 5 bytes:  Hello
0x48 0x65 0x6c 0x6c 0x6f 
Client received 2 bytes:  

0x0d 0x0a 

echo -n "Hello" > /dev/ttyS0
Client received 5 bytes:  Hello
0x48 0x65 0x6c 0x6c 0x6f

That looks as though it is correct in that "echo" without the -n option also outputs a CR/LF pair (as in the first test) and with -n doesn't (as in the second test).

The only potential anomaly is that sometimes when I start skt-echo it tells me that it receives 1 byte (0x00).

-Andy.

[RicV]

Well, that's strange. My message does come through correctly, but only after a lot of garbage characters, as seen in the skt-echo_vbox604.log file. The only obvious difference is that you have a Mac OS host whereas I am using RHEL7 as the host.

But are you building VBox 6.0.4 from the OSE tarball, or are you installing VBox 6.0.4 from distribution?

[andyp73]

But are you building VBox 6.0.4 from the OSE tarball, or are you installing VBox 6.0.4 from distribution

I am using the distribution version. Looking at the output you are seeing (skt-echo_vbox604.log) it looks like there is some test code being included that looks a bit like:

Code: Select all

for (i = 0; i < 0xff; i++)
{
    serial_write_char(i);
}

I'll have a look through the sources and see if I can find it anywhere.

-Andy.

[aeichner]

This was reported in the public bugtracker and will be fixed in the next maintenance release. There are 6.0.x testbuilds out which include the fix: https://www.virtualbox.org/wiki/Testbuilds

The garbage you were seeing was due to incorrectlyh andling loopback mode in the rewritten serial device emulation. Certain guests use that to probe the FIFO size writing incrementing counters to the FIFO and reading back the content.