Ubuntu guest Lost FTDI USB Serial Device after disconnect and reconnect

[Mike D]

I'm having a problem and posted to following at the Ubuntu support forum but the more I dig into it the more I think it might be a VB issue. The Ubuntu thread is under AskUbuntu, topic 929389 "Lost FTDI USB Serial Device after disconnect and reconnect".

- Win10 host, VirtualBox 5.1.22, installed extension pack 5.1.22
- Ubuntu 16.04 guest, installed 5.1.22 of VirtualBox Guest Additions

Under settings have checked Enable USB Controller, selected USB 2.0 (EHCI) and added my FTDI Quad RS232-HS [0800] filter.

I start Ubuntu. Connect my device. I see my four USB /dev/ttyUSB# serial ports. I can use them just fine.

The problem comes if I remove my USB cable and then re-connect it. When I remove the USB, the ttyUSBs go away as expected. I reconnect the USB cable and the ttyUSBs do not get recreated. I need to restart Ubuntu to get the ports to return.

I have the same VM machine hosted on Win7 under the same VBox configuration and it works. I can connect and reconnect the USB cable over and over and the terminal ports are always recreated.

I tried creating a new Ubuntu guest from scratch (rather than copying the existing one from my Win7 system) and the newly created one has the same problem.

I notice on my Win10 device manager that the Comm Ports are not returned after shutting down Ubuntu.

I'm not sure how to approach solving this issue; is it a hardware issue, a configuration issue with Ubuntu, VB or Win10?! Not sure how to search for answers because it works the first time but fails on retries. I'd be very happy to be pointed to an existing thread or solution.

I'll post a paste from my dmesg below. Perhaps it has a hint? (I am sorry but I can't seem to paste with LFs). The last message shows the USB as recognized attached but nothing further as far as IDing what it is and re-creating the ttyUSBs.

Thanks for taking the time to read this plea.

Mike

Code: Select all

[ 50.582532] usb 1-1: new high-speed USB device number 2 using ehci-pci
[ 50.935061] usb 1-1: New USB device found, idVendor=0403, idProduct=6011
[ 50.935064] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 50.935067] usb 1-1: Product: Quad RS232-HS
[ 50.935069] usb 1-1: Manufacturer: FTDI
[ 51.006339] usbcore: registered new interface driver usbserial
[ 51.006347] usbcore: registered new interface driver usbserial_generic
[ 51.006353] usbserial: USB Serial support registered for generic
[ 51.023554] usbcore: registered new interface driver ftdi_sio
[ 51.023589] usbserial: USB Serial support registered for FTDI USB Serial Device
[ 51.023606] ftdi_sio 1-1:1.0: FTDI USB Serial Device converter detected
[ 51.023644] usb 1-1: Detected FT4232
[ 51.028247] usb 1-1: FTDI USB Serial Device converter now attached to ttyUSB0
[ 51.028261] ftdi_sio 1-1:1.1: FTDI USB Serial Device converter detected
[ 51.028311] usb 1-1: Detected FT4232H
[ 51.032871] usb 1-1: FTDI USB Serial Device converter now attached to ttyUSB1
[ 51.032889] ftdi_sio 1-1:1.2: FTDI USB Serial Device converter detected
[ 51.032927] usb 1-1: Detected FT4232H
[ 51.037967] usb 1-1: FTDI USB Serial Device converter now attached to ttyUSB2
[ 51.037983] ftdi_sio 1-1:1.3: FTDI USB Serial Device converter detected
[ 51.038019] usb 1-1: Detected FT4232H
[ 51.041199] usb 1-1: FTDI USB Serial Device converter now attached to ttyUSB3
[ 80.269589] usb 1-1: USB disconnect, device number 2
[ 80.269806] ftdi_sio ttyUSB0: FTDI USB Serial Device converter now disconnected from ttyUSB0
[ 80.269828] ftdi_sio 1-1:1.0: device disconnected
[ 80.272317] ftdi_sio ttyUSB1: FTDI USB Serial Device converter now disconnected from ttyUSB1
[ 80.272342] ftdi_sio 1-1:1.1: device disconnected
[ 80.272659] ftdi_sio ttyUSB2: FTDI USB Serial Device converter now disconnected from ttyUSB2
[ 80.272684] ftdi_sio 1-1:1.2: device disconnected
[ 80.273004] ftdi_sio ttyUSB3: FTDI USB Serial Device converter now disconnected from ttyUSB3
[ 80.273031] ftdi_sio 1-1:1.3: device disconnected
[ 89.035073] usb 1-1: new high-speed USB device number 3 using ehci-pci

Additional testing : I've removed the USB filter for the guest VM. Now when I run the guest, I can check Devices->USB and select the FTDI device. Sometimes, after checking it and returning to the Devices->USB menu, the line is still there with a Check on it (which I can uncheck) but most other times the line isn't there at all. The outcome is the same, after the first time, I can no longer reconnect the FTDI Quad device. I also found that after trying the reconnect and waiting several minutes? the FTDI Quad device is released back to Win10 (it show up again there as four comm ports). And every once in a while, very infrequently, the reconnect works.

Mike

[Mike D]

Observation:
It just worked again. I checked Devices-USB->FTDI Quad and then returned to the Devices->USB and the line was still thre with a check mark. Checking dmesg in Ubuntu show the ttyUSB attached. Success. But then uncheckcing, removing and trying to reconnect, same problem, the menu item under Devices->USB is now gone (not still there with a check).

[scottgus1]

I don't know how to solve your problem, regrettably. Socratis is the big USB guy. But you do report that:

have the same VM machine hosted on Win7 under the same VBox configuration and it works. I can connect and reconnect the USB cable over and over and the terminal ports are always recreated.

By chance are you using the same Virtualbox version 5.1.22 on the Windows 7 host, too?

Could be just a difference between 7 and 10 in how the host OS handles this particular kind of device, or in a difference in how Virtualbox is programmed between 7 and 10. How to troubleshoot I have no clue.

But the reason why I posted is different: Why not Not take out the USB device? Do you really need to? Can you remember to shut down the guest after you do take it out? Might just be one of those "glitches" that we learn to compensate for.

[Mike D]

Thank you for pondering my question.

I'm doing development work on the device I am attaching and removing. So I am flashing new firmware onto it and ckecking connection events on the device. One of the serial ports is used for flashing and serial port terminal. So yes, I need to connect/disconnect/re-connect repeatedly and having to reboot the development Ubuntu environment would be bad. So bad I'd go back to the old and slower Win7 computer.

[mpack]

This is standard Windows behaviour. Every time you plug in a USB device Windows creates a new "object handle" to identify it, which will not be the same as the object handle it had last time it was plugged in. If an application is using the device then it is indirectly using that object handle. Once you unplug the USB device, that object handle is dead forever: it doesn't matter if you subsequently plug the device in again.

If you can find a way to reset your devboard without unplugging the USB connection then that would solve your problem.

In my own PC apps where users have adopted this annoying habit, I have to detect the serial port failure, and one of my remedies is to close my current handle and open a new one, see if that gets me going again. For my apps that was easy because I knew that serial I/O should be happening. VirtualBox doesn't know that, so I'm not sure that a fix at the VirtualBox end would be so easy.


BTW, you mention that the behaviour of the port is different on a Win7 host. I don't see how that can possibly be the case. Windows USB serial devices have behaved this way for as long as I've been using them, which is from XP onwards to the current day (Win10-64bit).

[Mike D]

I am pretty certain this is a VirtualBox problem.

I can do the following:
Under the Ubuntu guest setting: Enable USB Controller checked, USB 2.0 selected and no USB Device Filter.
Start the VB Ubuntu guest.
Do nothing in the guest as far as opening the port, any terminal or anything. Just auto log on.
Plug in my device.
Go to the Devices->USB menu and see the FDTI Quad RS232-HS [0800].
Hover over it and see the vender id, device id, revision and State = busy.
Click on the Devices->USB menu and see the FDTI Quad RS232-HS [0800]
- menu closes -
- at this point the COM ports disappear from the Win10 host Comm port listing - as expected.
Go back again to Devices->USB menu and see the FDTI Quad RS232-HS [0800] but now checked [as expected]
Click on - at this point the COM ports disappear from the Win10 host Comm port listing - to release the port.
- menu closes -
- at this point the COM ports re-appear on the Win10 host Comm port listing - as expected.

TWO THINGS HAPPEN -
a) Go back again to Devices->USB menu and the FDTI Quad RS232-HS [0800] IS NO LONGER ON THE MENU. It should be.
b) The entry is there. I selected it and the port comes back to my guest and the next time I click to release I get a VirtualBox.exe Application Error (crash) with "The instruction at 0x..... referenced memory at 0x0000000000E8. The memory could not be read. Click on OK to terminate the program"

Clearly - that is a VirtualBox problem.

I guess there was a c) - it worked for several times before stopping. Sometime when rebooting the Win10 host (to try and fix the problem) I get a message about VB still attached to a guest - continue?

And yes - it did work perfectly on Win7.

[Mike D]

I have a Logitech USB Receiver that experiences a very similar behavior.

I find that by repeatably selecting/deselecting between that one and my FDTI, VB eventual passed all the info on to Ubuntu to allow me to use the ttyUSB ports again. How many times I need to ping pong back and forth seems like anywhere from once to several times but it always seems to work. A real pain to be sure but better than rebooting my development VM.

I'd very much like to get a proper solution and I'm willing to provide testing, logs, whatever to anyone that might be able to help.

[socratis]

Clearly - that is a VirtualBox problem.

And yes - it did work perfectly on Win7.

You see the contradiction, right?

So, to summarize: you've enabled USB2. You're manually capturing/uncapturing the device from the Devices » USB menu (are you pulling the plug to free it?). Sometimes it's not recognized if plugged in under Win10, but it works all the time under Win7.

Here are a couple of thoughts:

• Enable USB3 in your VM. It's much more robust compared to USB2.
• Use a filter. Your host won't have a chance of even seeing the FTDI. See USB basics and troubleshooting, #4: USB Filters. Actually read the whole thing...
• Post a VBox.log.zip where: you start the VM from a cold boot, insert/remove the FTDI (as you would do normally), several times until it fails. When it fails, shut down the VM, right-click on it in the VBox Manager, show log, save the first one, ZIP IT, and attach it to your response.
• If you attach the dmesg.log from the same session, I wouldn't mind. As long as it's zipped...

A couple of times you hinted that the whole behavior might be time-sensitive, as in it may work after some time. Can you observe this behavior a little closer and see if a pattern emerges?

I'm doing development work on the device I am attaching and removing. So I am flashing new firmware onto it and ckecking connection events on the device.

Any chance that the IDs change? Any chance that the device needs some time to "boot" or to come back to it's normal state?

[Mike D]

Thank you very much for taking the time to consider my issue.

Some answers first:

* No I do not have to unplug the device between Devices→USB→check/uncheck to demonstrate the problem.
* I am not changing the device IDs (but I think some how VB is not passing them through to Ubuntu because with dmesg – see the dmesg log I originally posted).
* I have the same problem when I do the test with USB mouse capture. (on Win10 – I’ve not tried Win7). So the problem is not exclusive to my device.
* Originally I was using the Filter (as you are suggesting) but if I do that now, while I am having this new problem, I have no way to try and release/recapture, i.e., I am really forced to reboot my guest to give it another attempt. Someone in this thread or perhaps the ubuntu one suggested I try this approach. I’ve since gone back and tried both ways. Currently back with no filter so I can give multiple attempts because sometimes, after going back and forth with the usb logitech driver, it will work again, or crash.

I tried USB3 using a Filter and the behavior seemed the same. I did not do extensive testing or attempts like I did with settings set to USB2 once I saw it didn’t help. And since I was using the Filter I was plugging/unplugging/plugging in the device when I tried. (I just re-did that test – dmesg output is similar except Ubuntu recognizes the attached device as: [ 61.421030] usb 1-2: new high-speed USB device number 4 using xhci_hcd instead of ehci-pci

I will try the test sequence you describe and send the logs using USB2. I’ll just send the end of dmesg output from the usb device connect forward.

Thank you very much. Like I said, I have a work around hack (ping-pong between logitech and my FTDI but it takes time and still occasionally crashes)

Sincere thanks,
Mike
VB on different host is a different test bed. So working on Win7 doesn’t mean it is bug free on a Win7 does it? As a software developer, if my software crashes for any reason, I consider that is has a problem. Right?

[Mike D]

Ok attaching two tests. I did four total. The Test1 is only using the Logitech receiver. This was the first time I tried this and the Guest did not capture the device as it should have. There are detailed notes as well as logs in the zip. I repeated the test and got the same results with one additional observation, after some minute or so, the mouse control returned to Win10. I did not attach this test details but can send it you want.

[i did a another test to try and show how I could get my device to show up but capturing and releasing the logitech it wasn't working until many attempts and it all got too confusing so I am not sending it. I was eventual able to get ti working. I can send if helpful]

The second test I am sending is from the FTDI Quad device showing what I am dealing with. Meaning attach/detach/re-attach. First time works. Remove works. Re-attach fails. (all the time with the device physically remaining attached). Again, notes and logs are in the zip.

All tests were done from clean Win10 boots.

Additional note; the FTDI device is a Motorola production device. Not some of our hardware. It is this reference design: https://developer.motorola.com/build/mdk-user-guide

[socratis]

(all the time with the device physically remaining attached)

I told you to plug and unplug the device. To *NOT* use the Device menu. Could you do that please?

[Mike D]

Sorry about that. I was trying to demonstrate the problem w/o involving my device. Attached are the results. This time the FTDI device ID info was never passed through to Ubuntu so the ttyUSBs didn't even get created.

I'm going to try USB2 again.

Thank you.

[Mike D]

Same test but Settings for USB2. My device is recognized in Ubuntu when connected. I disconnect and reconnect. ttyUSBs fail to get recreated. Test details and logs attached.
Thanks again.

[Mike D]

VB info:

Code: Select all

c:\Program Files\Oracle\VirtualBox>VBoxManage -version
5.1.22r115126

c:\Program Files\Oracle\VirtualBox>VBoxManage list extpacks
Extension Packs: 1
Pack no. 0:   Oracle VM VirtualBox Extension Pack
Version:      5.1.22
Revision:     115126
Edition:
Description:  USB 2.0 and USB 3.0 Host Controller, Host Webcam, VirtualBox RDP, PXE ROM, Disk Encryption, NVMe.
VRDE Module:  VBoxVRDP
Usable:       true
Why unusable:

c:\Program Files\Oracle\VirtualBox>VBoxManage list usbhost
Host USB Devices:

UUID:               69bfcc2e-cab8-40d6-803c-96d55d9353cf
VendorId:           0x1bcf (1BCF)
ProductId:          0x2b96 (2B96)
Revision:           104.37 (10437)
Port:               0
USB version/speed:  2/High
Manufacturer:       Sunplus Innovation Technology Inc.
Address:            {36fc9e60-c465-11cf-8056-444553540000}\0002
Current State:      Busy

UUID:               16bb1399-c622-481b-9d93-717a7de23e69
VendorId:           0x0cf3 (0CF3)
ProductId:          0xe007 (E007)
Revision:           0.1 (0001)
Port:               0
USB version/speed:  2/High
Manufacturer:       Qualcomm Atheros Communications
Address:            {e0cbf06c-cd8b-4647-bb8a-263b43f0f974}\0000
Current State:      Busy

UUID:               074b200d-f015-4a6f-be59-2cb1822ba4ba
VendorId:           0x046d (046D)
ProductId:          0xc52f (C52F)
Revision:           34.0 (3400)
Port:               0
USB version/speed:  2/High
Manufacturer:       Logitech
Product:            USB Receiver
Address:            {36fc9e60-c465-11cf-8056-444553540000}\0016
Current State:      Busy

UUID:               ee9da2b6-009c-4237-9a29-ee0f0ca81542
VendorId:           0x0403 (0403)
ProductId:          0x6011 (6011)
Revision:           8.0 (0800)
Port:               0
USB version/speed:  2/High
Manufacturer:       FTDI
Product:            Quad RS232-HS
Address:            {36fc9e60-c465-11cf-8056-444553540000}\0005
Current State:      Busy

c:\Program Files\Oracle\VirtualBox>VBoxManage list usbfilters
Global USB Device Filters:

<none>


c:\Program Files\Oracle\VirtualBox>VBoxManage showvminfo "MotoX Dev"
Name:            MotoX Dev
Groups:          /
Guest OS:        Ubuntu (64-bit)
UUID:            9a153d29-0aa4-40ff-b78b-9a0e8d19917b
Config file:     C:\Users\mdann\VirtualBox VMs\MotoX Dev\MotoX Dev.vbox
Snapshot folder: C:\Users\mdann\VirtualBox VMs\MotoX Dev\Snapshots
Log folder:      C:\Users\mdann\VirtualBox VMs\MotoX Dev\Logs
Hardware UUID:   9a153d29-0aa4-40ff-b78b-9a0e8d19917b
Memory size:     2048MB
Page Fusion:     off
VRAM size:       16MB
CPU exec cap:    100%
HPET:            off
Chipset:         piix3
Firmware:        BIOS
Number of CPUs:  1
PAE:             off
Long Mode:       on
Triple Fault Reset: off
APIC:            on
X2APIC:          on
CPUID Portability Level: 0
CPUID overrides: None
Boot menu mode:  message and menu
Boot Device (1): Floppy
Boot Device (2): DVD
Boot Device (3): HardDisk
Boot Device (4): Not Assigned
ACPI:            on
IOAPIC:          on
BIOS APIC mode:  APIC
Time offset:     0ms
RTC:             UTC
Hardw. virt.ext: on
Nested Paging:   on
Large Pages:     on
VT-x VPID:       on
VT-x unr. exec.: on
Paravirt. Provider: Default
Effective Paravirt. Provider: KVM
State:           powered off (since 2017-06-30T19:52:12.856000000)
Monitor count:   1
3D Acceleration: off
2D Video Acceleration: off
Teleporter Enabled: off
Teleporter Port: 0
Teleporter Address:
Teleporter Password:
Tracing Enabled: off
Allow Tracing to Access VM: off
Tracing Configuration:
Autostart Enabled: off
Autostart Delay: 0
Default Frontend:
Storage Controller Name (0):            IDE
Storage Controller Type (0):            PIIX4
Storage Controller Instance Number (0): 0
Storage Controller Max Port Count (0):  2
Storage Controller Port Count (0):      2
Storage Controller Bootable (0):        on
Storage Controller Name (1):            SATA
Storage Controller Type (1):            IntelAhci
Storage Controller Instance Number (1): 0
Storage Controller Max Port Count (1):  30
Storage Controller Port Count (1):      1
Storage Controller Bootable (1):        on
IDE (1, 0): Empty
SATA (0, 0): C:\Users\mdann\VirtualBox VMs\MotoX Dev\Snapshots/{1c0c6f94-435b-43f2-8430-f6f90fc96d36}.vdi (UUID: 1c0c6f94-435b-43f2-8430-f6f90fc96d36)
NIC 1:           MAC: 080027D28FF2, Attachment: NAT, Cable connected: on, Trace: off (file: none), Type: 82540EM, Reported speed: 0 Mbps, Boot priority: 0, Promisc Policy: deny, Bandwidth group: none
NIC 1 Settings:  MTU: 0, Socket (send: 64, receive: 64), TCP Window (send:64, receive: 64)
NIC 2:           disabled
NIC 3:           disabled
NIC 4:           disabled
NIC 5:           disabled
NIC 6:           disabled
NIC 7:           disabled
NIC 8:           disabled
Pointing Device: USB Tablet
Keyboard Device: PS/2 Keyboard
UART 1:          disabled
UART 2:          disabled
UART 3:          disabled
UART 4:          disabled
LPT 1:           disabled
LPT 2:           disabled
Audio:           enabled (Driver: DSOUND, Controller: AC97, Codec: AD1980)
Clipboard Mode:  Bidirectional
Drag and drop Mode: Bidirectional
VRDE:            disabled
USB:             enabled
EHCI:            enabled
XHCI:            disabled

USB Device Filters:

Index:            0
Active:           yes
Name:             FTDI Quad RS232-HS [0800]
VendorId:         0403
ProductId:        6011
Revision:         0800
Manufacturer:     FTDI
Product:          Quad RS232-HS
Remote:           0
Serial Number:

Bandwidth groups:  <none>

Shared folders:

Name: 'Echo_Ridge', Host path: 'C:\Users\mdann\Documents\DASTEC\Dell_Development\Echo Ridge' (machine mapping), writable

Video capturing:    not active
Capture screens:    0
Capture file:       C:\Users\mdann\VirtualBox VMs\MotoX Dev\MotoX Dev.webm
Capture dimensions: 1024x768
Capture rate:       512 kbps
Capture FPS:        25

Guest:

Configured memory balloon size:      0 MB

Snapshots:

   Name: Initial Envirnoment (UUID: 8f4a4878-2066-403d-823c-b4f6ad67e1d9)
   Description:
5/23/2017
Environment setup in ~
      Name: Span 06/29/2017 (UUID: 1ff8e81d-6931-4d61-af26-3b093f160ae1) *
      Description:
Running on Win10
Crashing on USB disconnect?



c:\Program Files\Oracle\VirtualBox>
c:\Program Files\Oracle\VirtualBox>

[Mike D]

Is it clear now from the prior three posts with complete log files that indeed there is a problem with how FTDI is getting handled in my VB configuration? This worked fine under Win7.