Transcript
Paravirtualized USB Support for Xen
Noboru Iwamatsu
[email protected] FUJITSU LABORATORIES LTD. Copyright 2008 FUJITSU LIMITED
Background: Client-side virtualization and USB In client-side virtualization, special Service-VMs work in the background of User-VM, and provide various functions. To give some examples,
Authentication VM: Authenticate with USB-connected smartcard or biometrics.
Communication VM: VoIP/Messenger application using USB-connected webcam or audio.
In this situation, both User-VM and Service-VMs require using the USB devices at the same time by the same USB host controller. Usage example of client-side virtualization
User VM
Service VM
Service VM
User OS
Service OS
Service OS
Office/Music apps
Authentication
VoIP/ Messenger
Hypervisor USB Host Controller
Laptop
USB devices 1
Copyright 2008 FUJITSU LIMITED
History and Motivation: Xen and USB support Xen 2.0.x (in PV-only age)
Xen 3.0 Xen 3.2
Paravirtualized USB support was temporarily into the tree. But eventually removed and not released. Not well supported Qemu-dm supported UHCI emulation for HVM domain. USB1.1 only PCI pass-through with IOMMU supported attaching USB Host Controller to HVM domain. Works well. But the entire controller is assigned to single domain and can’t be used from other domains. Existing options are not suitable for the client-side virtualization. So, we have started developing paravirtualized USB driver, and proposed in XCI.
Xen 3.4 (current unstable)
Now, “PV USB support” is on the roadmap. Today, we would like to report our implementation details and development status. 2
Copyright 2008 FUJITSU LIMITED
Overview: What’s paravirtualized USB driver? A split drivers for handling USB devices from guest domains.
frontend: Virtual USB 2.0 Host Controller driver acting as a proxy for backend. backend: USB function driver handling the USB devices. Dom 0 Dom0 OS PV USB backend driver
USB native driver
Xen Hypervisor USB Host Controller
USB devices
Guest VM 1
Guest VM 2
Guest OS
Guest OS
PV USB frontend driver
PV USB frontend driver
urb transferring urb (USB request block): In linux kernel, all USB drivers and subsystems communicate with urb. Urb is a USB request block structure (described in include/linux/usb.h). It is much like “struct skbuff” in the networking code.
Frontend transfers urbs to backend through Xen, and backend transfers the received urbs to the USB devices. 3
Copyright 2008 FUJITSU LIMITED
Implementation details(1/5): USB basics: Driver architecture and urb lifecycle in kernel User-space or other kernel subsystems urb
urb
usb_alloc_urb()
USB Function Driver
usb_submit_urb()
usb_free_urb() calling completion handler
USB Core .urb_enqueue()
Kernel-space
usb_hcd_giveback_urb()
USB Host Controller Driver Hardware (USB Host Controller)
USB drivers in linux kernel is implemented as 3-layer module stacks. USB Function Driver - Driver for each USB device (e.g. storage, printer, hid) A urb is created by this driver, and is submitted to USB core. If the urb is successfully transferred to the USB device, the completion handler of the urb is called.
USB Core - USB subsystem in linux kernel The urb is enqueued to specific host controller driver for the specified device by this layer.
USB Host Controller Driver – Driver for USB host controller hardware The enqueued urb is transferred to the hardware, and is backed to USB core after the driver get response. 4
Copyright 2008 FUJITSU LIMITED
Implementation details(2/5): Paravirtualized USB driver architecture and urb lifecycle Dom0 kernel-space
DomU kernel-space
PV USB backend
urb
USB Function Driver
Xen backend interface xenbus
USB Core
RING
.urb_enqueue()
alloc Cloned urb
USB Function Driver
usb_submit_urb ()
urb
usb_hcd_giveback_urb()
Virtual USB 2.0 Host Controller Driver
free Cloned urb
completion handler
RING
USB Core
xenbus
Xen frontend interface Host Controller Driver
PV USB frontend
Hardware (USB Host Controller)
Frontend driver - Implemented as a USB host controller driver
The enqueued urb is mapped to RING request, and is sent to backend. When the RING response is received, the urb is backed to USB core after substituting that status-value.
Backend driver - Implemented as a USB function driver
The urb is re-created from the RING request, and is submitted to the specified device. When the urb is completed, the status of the urb is mapped to RING response and returned to frontend.
All existing code need not be modified, and maybe all USB drivers in kernel can work. 5
Copyright 2008 FUJITSU LIMITED
Implementation details(3/5): How to map urb to RING request/response? The fields of the urb structure is mapped to the RING as follows. The data buffers of urb are sharing between frontend and backend by using the grant table operations. If the number of shared pages is 10, RING_SIZE is set to 32. Most of function drivers may be all right by this setting :-) struct usbif_request {
struct urb { unsigned int unsigned int void int unsigned char int int int struct usb_iso_packet_descriptor int int int … };
…
Mapping to RING request pipe; transfer_flags; *transfer_buffer; transfer_buffer_length; *setup_packet; interval; start_frame; number_of_packets; iso_frame_desc[0];
segs[];
};
Sharing pages with grant table
status; actual_length; error_count;
struct usbif_response { ...
Mapping to RING response }; 6
Copyright 2008 FUJITSU LIMITED
Implementation details(4/5): Virtual USB host controller internals Virtual host controller has 3 queues for scheduling urbs.
submit_waiting_queue The enqueued urb is added to the tail of this queue, and waits its turn to be sent to backend.
submit_in_progress_queue While the RING request is sent and waits for the response, the urb is added to this queue.
giveback_waiting_queue While the RING response is received and waits for backing to the USB core, the urb is added to this queue. .urb_enqueue() usb_hcd_giveback_urb() urb
urb
Virtual USB Host Controller submit_waiting_queue RING request
urb
urb
urb
RING responses are called from interrupt context and submit_wating_queue and givback_wating_queue are flushed by timer functions.
add to tail
after send, move to tail
submit_in_progress_queue urb
urb
urb
urb
RING response giveback_waiting_queue move matched urb to tail
urb 7
delete from queue Copyright 2008 FUJITSU LIMITED
Implementation details(5/5): How to plug the devices to guest domains? Hotplug-rule is set from sysfs interface in the backend driver.
The hotplug-rule format •
:::
Example settings % echo 1-2.3:1:0:3 > /sys/bus/usb/drivers/usbback/new_vport % echo 1-4:2:0:1 > /sys/bus/usb/drivers/usbback/new_vport % echo 1-2.1:2:0:2 > /sys/bus/usb/drivers/usbback/new_vport
Dom 0
Guest VM 1 (domain ID:1)
Guest VM 2 (domain ID:2)
Host Controller (usb1)
Virtual Host Controller (vusb-0)
Virtual Host Controller (vusb-0)
Virtual roothub
Virtual roothub
roothub
1-4 device
1-2.1 device
hub
device
device
device
device
1-2.3 device
When new device is connected, the backend driver claims the device if its busname is found on the settings. The frontend driver is notified of hotplug by xenbus. 8
Copyright 2008 FUJITSU LIMITED
Development Status Paravirtualized USB driver has just started working.
Tested devices Type
Name
Manufacturer
Driver
Status
Keyboard
FKB-108-EU
FILCO
usbhid
Mouse
Cordless Notebook Mouse
Logitech
usbhid
Flash drive
RUF2-R2GS
Buffalo
usb-storage
Flash drive
RUF-C1G/U2
Buffalo
usb-storage
HDD
HDCN-U500
IO DATA
usb-storage
[1]
Webcam
WebCam 3 USB
Creative Labs
ov511
[2]
Works
Works with issues
Not worked yet
1. SCSI command [READ_CAPCITY] fails. 2. Horizontal stripes are into video streaming image.
The performance is about 6MB/s(Read/Write throughput of flash drive). • 1/5 of native performance, but more than 4 times faster than USB1.1.
• There is still room for improvement.
Implementation Status • Hotplug and disconnect functions are not completed yet. 9
Copyright 2008 FUJITSU LIMITED
TODO & Future work TODO
Many cleanups and bugfixes. Performance upgrade. Modern webcams support • The UVC (USB Video Class) driver requires high bandwidth.
Suspend / Resume support. Xend support.
Future work
Netchannel2 support. Porting the frontend driver to stub domain.
We will post the code by the end of this year! 10
Copyright 2008 FUJITSU LIMITED
11
Copyright 2008 FUJITSU LIMITED
