Hi,
As it's Friday I thought I'd better summarise where I've gotten to with booting Xen. TLDR, got something working with GICv2, GICv3 seems broken.
Platform:
QEMU with: -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=3 \ -device virtio-net-pci,netdev=unet,id=virt-net,disable-legacy=on \ -device virtio-scsi-pci,id=virt-scsi,disable-legacy=on \ -blockdev driver=raw,node-name=hd,discard=unmap,file.driver=host_device,file.filename=/dev/zen-disk/debian-buster-arm64 \ -device scsi-hd,drive=hd,id=virt-scsi-hd \ -drive if=pflash,file=/usr/share/AAVMF/AAVMF_CODE.fd,format=raw,readonly \ -drive if=pflash,file=/home/alex/models/qemu-arm64-efivars,format=raw
The firmware is the stock Debian EFI:
Package: qemu-efi-aarch64 Status: install ok installed Installed-Size: 133154 Maintainer: Debian QEMU Team pkg-qemu-devel@lists.alioth.debian.org Architecture: all Source: edk2 Version: 0~20181115.85588389-3+deb10u1 Description: UEFI firmware for 64-bit ARM virtual machines qemu-efi-aarch64 is a build of EDK II for 64-bit ARM virtual machines. It includes full support for UEFI, including Secure Boot. Homepage: http://www.tianocore.org
I've also turned on semihosting (-semihosting) to get earlycon output.
Xen:
Current master with the following patches:
e680dde6fd (HEAD -> review/arm-xen-acpi-changes) xen/arm: gic-v2: acpi: Use the correct length for the GICC structure a34a93acb9 xen/arm: acpi: add BAD_MADT_GICC_ENTRY() macro f66afeb018 (linaro/review/arm-xen-acpi-changes) xen/arm: acpi: relax FADT revision requirements to 5.1 72f3d495d0 xen/arm: Introduce fw_unreserved_regions() and use it 8d447fa48a xen/arm: Check if the platform is not using ACPI before initializing Dom0less fa03236a08 xen/arm: acpi: The fixmap area should always be cleared during failure/unmap 505b82a8be xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()
I've tried various kernels:
- The default buster kernel (4.19.0-11) would panic in memblock_alloc_base - The backports kernel (5.8.0-0.bpo.2) didn't panic AFAICT but never came up either
In the end I built my own image so I could enable semihosting support and finally I was able to get some output from the kernel:
ing Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 ST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90328 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90328 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902D0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000194 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E902A8 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 ACPI: [APIC:0x0b] Invalid zero length percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: GIC system register CPU interface CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kerel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927524K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153760K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0e] Invalid zero length ACPI: [APIC:0x0b] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length Kernel panic - not syncing: No interrupt controller found. CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.8.11-ajb-00010-ge059ce7b131b #42 Call trace: dump_backtrace+0x0/0x1a8 show_stack+0x18/0x28 dump_stack+0xc8/0x128 panic+0x154/0x30c init_IRQ+0xd0/0xe8 start_kernel+0x260/0x400 ---[ end Kernel panic - not syncing: No interrupt controller found. ]---
If I boot the same kernel with:
-cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=2
We get a lot further:
Booting Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 BST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90308 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90308 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902B0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000174 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E90288 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kernel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh printk: log_buf_len individual max cpu contribution: 4096 bytes printk: log_buf_len total cpu_extra contributions: 12288 bytes printk: log_buf_len min size: 16384 bytes printk: log_buf_len: 32768 bytes printk: early log buf free: 12432(75%) Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927316K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153968K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 rcu: Offload RCU callbacks from CPUs: (none). random: get_random_bytes called from start_kernel+0x2a4/0x400 with crng_init=0 arch_timer: cp15 timer(s) running at 62.50MHz (virt). clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns sched_clock: 56 its at 62MHz, resolution 16ns, wraps every 4398046511096ns Console: colour dummy device 80x25 printk: console [hvc0] enabled printk: bootconsole [smh0] disabled
which completes the Buster boot and allows me to ssh into Dom0 - I haven't yet tried to boot a DomU.
Alex Bennée alex.bennee@linaro.org writes:
(forgot to CC Andre)
Hi,
As it's Friday I thought I'd better summarise where I've gotten to with booting Xen. TLDR, got something working with GICv2, GICv3 seems broken.
Platform:
QEMU with: -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=3 \ -device virtio-net-pci,netdev=unet,id=virt-net,disable-legacy=on \ -device virtio-scsi-pci,id=virt-scsi,disable-legacy=on \ -blockdev driver=raw,node-name=hd,discard=unmap,file.driver=host_device,file.filename=/dev/zen-disk/debian-buster-arm64 \ -device scsi-hd,drive=hd,id=virt-scsi-hd \ -drive if=pflash,file=/usr/share/AAVMF/AAVMF_CODE.fd,format=raw,readonly \ -drive if=pflash,file=/home/alex/models/qemu-arm64-efivars,format=raw
The firmware is the stock Debian EFI:
Package: qemu-efi-aarch64 Status: install ok installed Installed-Size: 133154 Maintainer: Debian QEMU Team pkg-qemu-devel@lists.alioth.debian.org Architecture: all Source: edk2 Version: 0~20181115.85588389-3+deb10u1 Description: UEFI firmware for 64-bit ARM virtual machines qemu-efi-aarch64 is a build of EDK II for 64-bit ARM virtual machines. It includes full support for UEFI, including Secure Boot. Homepage: http://www.tianocore.org
I've also turned on semihosting (-semihosting) to get earlycon output.
Xen:
Current master with the following patches:
e680dde6fd (HEAD -> review/arm-xen-acpi-changes) xen/arm: gic-v2: acpi: Use the correct length for the GICC structure a34a93acb9 xen/arm: acpi: add BAD_MADT_GICC_ENTRY() macro f66afeb018 (linaro/review/arm-xen-acpi-changes) xen/arm: acpi: relax FADT revision requirements to 5.1 72f3d495d0 xen/arm: Introduce fw_unreserved_regions() and use it 8d447fa48a xen/arm: Check if the platform is not using ACPI before initializing Dom0less fa03236a08 xen/arm: acpi: The fixmap area should always be cleared during failure/unmap 505b82a8be xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()
I've tried various kernels:
- The default buster kernel (4.19.0-11) would panic in memblock_alloc_base
- The backports kernel (5.8.0-0.bpo.2) didn't panic AFAICT but never came up either
In the end I built my own image so I could enable semihosting support and finally I was able to get some output from the kernel:
ing Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 ST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90328 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90328 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902D0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000194 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E902A8 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 ACPI: [APIC:0x0b] Invalid zero length percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: GIC system register CPU interface CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kerel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927524K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153760K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0e] Invalid zero length ACPI: [APIC:0x0b] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length Kernel panic - not syncing: No interrupt controller found. CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.8.11-ajb-00010-ge059ce7b131b #42 Call trace: dump_backtrace+0x0/0x1a8 show_stack+0x18/0x28 dump_stack+0xc8/0x128 panic+0x154/0x30c init_IRQ+0xd0/0xe8 start_kernel+0x260/0x400 ---[ end Kernel panic - not syncing: No interrupt controller found. ]---
If I boot the same kernel with:
-cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=2We get a lot further:
Booting Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 BST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90308 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90308 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902B0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000174 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E90288 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kernel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh printk: log_buf_len individual max cpu contribution: 4096 bytes printk: log_buf_len total cpu_extra contributions: 12288 bytes printk: log_buf_len min size: 16384 bytes printk: log_buf_len: 32768 bytes printk: early log buf free: 12432(75%) Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927316K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153968K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 rcu: Offload RCU callbacks from CPUs: (none). random: get_random_bytes called from start_kernel+0x2a4/0x400 with crng_init=0 arch_timer: cp15 timer(s) running at 62.50MHz (virt). clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns sched_clock: 56 its at 62MHz, resolution 16ns, wraps every 4398046511096ns Console: colour dummy device 80x25 printk: console [hvc0] enabled printk: bootconsole [smh0] disabled
which completes the Buster boot and allows me to ssh into Dom0 - I haven't yet tried to boot a DomU.
On Fri, 2 Oct 2020, Alex Bennée wrote:
Alex Bennée alex.bennee@linaro.org writes:
(forgot to CC Andre)
Hi,
As it's Friday I thought I'd better summarise where I've gotten to with booting Xen. TLDR, got something working with GICv2, GICv3 seems broken.
Platform:
QEMU with: -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=3 \ -device virtio-net-pci,netdev=unet,id=virt-net,disable-legacy=on \ -device virtio-scsi-pci,id=virt-scsi,disable-legacy=on \ -blockdev driver=raw,node-name=hd,discard=unmap,file.driver=host_device,file.filename=/dev/zen-disk/debian-buster-arm64 \ -device scsi-hd,drive=hd,id=virt-scsi-hd \ -drive if=pflash,file=/usr/share/AAVMF/AAVMF_CODE.fd,format=raw,readonly \ -drive if=pflash,file=/home/alex/models/qemu-arm64-efivars,format=raw
The firmware is the stock Debian EFI:
Package: qemu-efi-aarch64 Status: install ok installed Installed-Size: 133154 Maintainer: Debian QEMU Team pkg-qemu-devel@lists.alioth.debian.org Architecture: all Source: edk2 Version: 0~20181115.85588389-3+deb10u1 Description: UEFI firmware for 64-bit ARM virtual machines qemu-efi-aarch64 is a build of EDK II for 64-bit ARM virtual machines. It includes full support for UEFI, including Secure Boot. Homepage: http://www.tianocore.org
I've also turned on semihosting (-semihosting) to get earlycon output.
Xen:
Current master with the following patches:
e680dde6fd (HEAD -> review/arm-xen-acpi-changes) xen/arm: gic-v2: acpi: Use the correct length for the GICC structure a34a93acb9 xen/arm: acpi: add BAD_MADT_GICC_ENTRY() macro f66afeb018 (linaro/review/arm-xen-acpi-changes) xen/arm: acpi: relax FADT revision requirements to 5.1 72f3d495d0 xen/arm: Introduce fw_unreserved_regions() and use it 8d447fa48a xen/arm: Check if the platform is not using ACPI before initializing Dom0less fa03236a08 xen/arm: acpi: The fixmap area should always be cleared during failure/unmap 505b82a8be xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()
I've tried various kernels:
- The default buster kernel (4.19.0-11) would panic in memblock_alloc_base
- The backports kernel (5.8.0-0.bpo.2) didn't panic AFAICT but never came up either
In the end I built my own image so I could enable semihosting support and finally I was able to get some output from the kernel:
ing Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 ST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90328 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90328 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902D0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000194 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E902A8 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 ACPI: [APIC:0x0b] Invalid zero length percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: GIC system register CPU interface CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kerel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927524K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153760K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0e] Invalid zero length ACPI: [APIC:0x0b] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length Kernel panic - not syncing: No interrupt controller found. CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.8.11-ajb-00010-ge059ce7b131b #42 Call trace: dump_backtrace+0x0/0x1a8 show_stack+0x18/0x28 dump_stack+0xc8/0x128 panic+0x154/0x30c init_IRQ+0xd0/0xe8 start_kernel+0x260/0x400 ---[ end Kernel panic - not syncing: No interrupt controller found. ]---
If I boot the same kernel with:
-cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=2We get a lot further:
Booting Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 BST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90308 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90308 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902B0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000174 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E90288 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kernel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh printk: log_buf_len individual max cpu contribution: 4096 bytes printk: log_buf_len total cpu_extra contributions: 12288 bytes printk: log_buf_len min size: 16384 bytes printk: log_buf_len: 32768 bytes printk: early log buf free: 12432(75%) Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927316K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153968K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 rcu: Offload RCU callbacks from CPUs: (none). random: get_random_bytes called from start_kernel+0x2a4/0x400 with crng_init=0 arch_timer: cp15 timer(s) running at 62.50MHz (virt). clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns sched_clock: 56 its at 62MHz, resolution 16ns, wraps every 4398046511096ns Console: colour dummy device 80x25 printk: console [hvc0] enabled printk: bootconsole [smh0] disabled
which completes the Buster boot and allows me to ssh into Dom0 - I haven't yet tried to boot a DomU.
Excellent!
For your information, I could boot Xen on QEMU with GICv3 in the past, with something like:
-machine virt,gic_version=3 \ -machine virtualization=true \ -cpu cortex-a57 -machine type=virt \
However it was a long time ago: QEMU 2.11.
Stefano Stabellini stefano.stabellini@xilinx.com writes:
On Fri, 2 Oct 2020, Alex Bennée wrote:
Alex Bennée alex.bennee@linaro.org writes:
(forgot to CC Andre)
Hi,
As it's Friday I thought I'd better summarise where I've gotten to with booting Xen. TLDR, got something working with GICv2, GICv3 seems broken.
Platform:
QEMU with: -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=3 \ -device virtio-net-pci,netdev=unet,id=virt-net,disable-legacy=on \ -device virtio-scsi-pci,id=virt-scsi,disable-legacy=on \ -blockdev driver=raw,node-name=hd,discard=unmap,file.driver=host_device,file.filename=/dev/zen-disk/debian-buster-arm64 \ -device scsi-hd,drive=hd,id=virt-scsi-hd \ -drive if=pflash,file=/usr/share/AAVMF/AAVMF_CODE.fd,format=raw,readonly \ -drive if=pflash,file=/home/alex/models/qemu-arm64-efivars,format=raw
<snip>
Excellent!
For your information, I could boot Xen on QEMU with GICv3 in the past, with something like:
-machine virt,gic_version=3 \ -machine virtualization=true \ -cpu cortex-a57 -machine type=virt \However it was a long time ago: QEMU 2.11.
So with my recent direct boot hack:
https://patchew.org/QEMU/20201009170742.23695-1-alex.bennee@linaro.org/
which avoids going through both EDK2 and Grub I can confirm I can boot Xen with GICv3:
./aarch64-softmmu/qemu-system-aarch64 -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=3 -display none -serial mon:stdio -netdev user,id=unet,hostfwd=tcp::22 22-:22 -device virtio-net-pci,netdev=unet,id=virt-net,disable-legacy=on -device virtio-scsi-pci,id=virt-scsi,disable-legacy=on -blockdev driver=raw,node-name=hd,discard=unma p,file.driver=host_device,file.filename=/dev/zen-disk/debian-buster-arm64 -device scsi-hd,drive=hd,id=virt-scsi-hd -smp 4 -m 4096 -kernel ~/lsrc/xen.git/xen/xen -append "dom 0_mem=1G,max:1G loglvl=all guest_loglvl=all" -device loader,addr=0x47000000,file=$HOME/lsrc/linux.git/builds/arm64/arch/arm64/boot/Image,len-fdt-compat=2,fdt-compat[0]='mult iboot,,module',fdt-compat[1]='multiboot,,kernel',fdt-bootargs="root=/dev/sda2 ro console=hvc0 earlyprintk=xen" - UART enabled - - Boot CPU booting - - Current EL 00000008 - - Initialize CPU - - Turning on paging - - Zero BSS - - Ready - (XEN) Checking for initrd in /chosen (XEN) RAM: 0000000040000000 - 000000013fffffff (XEN) (XEN) MODULE[0]: 0000000040200000 - 000000004034b0c8 Xen (XEN) MODULE[1]: 0000000048000000 - 0000000048100000 Device Tree (XEN) MODULE[2]: 0000000047000000 - 0000000047dc6200 Kernel (XEN) (XEN) CMDLINE[0000000047000000]:chosen root=/dev/sda2 ro console=hvc0 earlyprintk=xen (XEN) (XEN) Command line: dom0_mem=1G,max:1G loglvl=all guest_loglvl=all (XEN) parameter "dom0_mem" has invalid value "1G,max:1G", rc=-22! (XEN) Domain heap initialised (XEN) Booting using Device Tree (XEN) Platform: Generic System (XEN) Taking dtuart configuration from /chosen/stdout-path (XEN) Looking for dtuart at "/pl011@9000000", options "" Xen 4.15-unstable (XEN) Xen version 4.15-unstable (alex@) (gcc (Debian 8.3.0-6) 8.3.0) debug=y Wed Sep 30 12:58:57 BST 2020 (XEN) Latest ChangeSet: Wed Sep 30 10:26:05 2020 +0100 git:f66afeb018-dirty (XEN) build-id: e2e95cc5d131b7659a57b8d10212647c3c514e7e (XEN) Processor: 411fd070: "ARM Limited", variant: 0x1, part 0xd07, rev 0x0 (XEN) 64-bit Execution: (XEN) Processor Features: 0000000001000222 0000000000000000 (XEN) Exception Levels: EL3:No EL2:64+32 EL1:64+32 EL0:64+32 (XEN) Extensions: FloatingPoint AdvancedSIMD GICv3-SysReg (XEN) Debug Features: 0000000010305106 0000000000000000 (XEN) Auxiliary Features: 0000000000000000 0000000000000000 (XEN) Memory Model Features: 0000000000001124 0000000000000000 (XEN) ISA Features: 0000000000011120 0000000000000000 (XEN) 32-bit Execution: (XEN) Processor Features: 00000131:10011001 (XEN) Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle (XEN) Extensions: GenericTimer (XEN) Debug Features: 03010066 (XEN) Auxiliary Features: 00000000 (XEN) Memory Model Features: 10101105 40000000 01260000 02102211 (XEN) ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121 (XEN) Using SMC Calling Convention v1.0 (XEN) Using PSCI v0.2 (XEN) SMP: Allowing 4 CPUs (XEN) enabled workaround for: ARM erratum 832075 (XEN) enabled workaround for: ARM erratum 834220 (XEN) enabled workaround for: ARM erratum 1319367 (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 62500 KHz (XEN) GICv3 initialization: (XEN) gic_dist_addr=0x00000008000000 (XEN) gic_maintenance_irq=25 (XEN) gic_rdist_stride=0 (XEN) gic_rdist_regions=1 (XEN) redistributor regions: (XEN) - region 0: 0x000000080a0000 - 0x00000009000000 (XEN) GICv3: 256 lines, (IID 0000043b). (XEN) GICv3: CPU0: Found redistributor in region 0 @000000004001c000
Hi,
I've succeeded to make the X.org running on Dom0. It seems that Xorg's nouveau driver caused SIGBUS issue. Custom nouveau kernel driver + Xorg fbdev driver seems stable. (Even if it doesn't work again, I'll try to use USB-HDMI adaptor next time)
So, I would like to test the virtio-video for the next step. Alex, how can I help you to test it?
Thank you,
2020年10月2日(金) 23:59 Alex Bennée alex.bennee@linaro.org:
Hi,
As it's Friday I thought I'd better summarise where I've gotten to with booting Xen. TLDR, got something working with GICv2, GICv3 seems broken.
Platform:
QEMU with: -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=3 \ -device virtio-net-pci,netdev=unet,id=virt-net,disable-legacy=on \ -device virtio-scsi-pci,id=virt-scsi,disable-legacy=on \ -blockdev driver=raw,node-name=hd,discard=unmap,file.driver=host_device,file.filename=/dev/zen-disk/debian-buster-arm64 \ -device scsi-hd,drive=hd,id=virt-scsi-hd \ -drive if=pflash,file=/usr/share/AAVMF/AAVMF_CODE.fd,format=raw,readonly \ -drive if=pflash,file=/home/alex/models/qemu-arm64-efivars,format=raw
The firmware is the stock Debian EFI:
Package: qemu-efi-aarch64 Status: install ok installed Installed-Size: 133154 Maintainer: Debian QEMU Team pkg-qemu-devel@lists.alioth.debian.org Architecture: all Source: edk2 Version: 0~20181115.85588389-3+deb10u1 Description: UEFI firmware for 64-bit ARM virtual machines qemu-efi-aarch64 is a build of EDK II for 64-bit ARM virtual machines. It includes full support for UEFI, including Secure Boot. Homepage: http://www.tianocore.org
I've also turned on semihosting (-semihosting) to get earlycon output.
Xen:
Current master with the following patches:
e680dde6fd (HEAD -> review/arm-xen-acpi-changes) xen/arm: gic-v2: acpi: Use the correct length for the GICC structure a34a93acb9 xen/arm: acpi: add BAD_MADT_GICC_ENTRY() macro f66afeb018 (linaro/review/arm-xen-acpi-changes) xen/arm: acpi: relax FADT revision requirements to 5.1 72f3d495d0 xen/arm: Introduce fw_unreserved_regions() and use it 8d447fa48a xen/arm: Check if the platform is not using ACPI before initializing Dom0less fa03236a08 xen/arm: acpi: The fixmap area should always be cleared during failure/unmap 505b82a8be xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()
I've tried various kernels:
- The default buster kernel (4.19.0-11) would panic in memblock_alloc_base
- The backports kernel (5.8.0-0.bpo.2) didn't panic AFAICT but never came up either
In the end I built my own image so I could enable semihosting support and finally I was able to get some output from the kernel:
ing Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 ST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90328 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90328 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902D0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000194 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E902A8 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 ACPI: [APIC:0x0b] Invalid zero length percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: GIC system register CPU interface CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kerel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927524K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153760K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0e] Invalid zero length ACPI: [APIC:0x0b] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length Kernel panic - not syncing: No interrupt controller found. CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.8.11-ajb-00010-ge059ce7b131b #42 Call trace: dump_backtrace+0x0/0x1a8 show_stack+0x18/0x28 dump_stack+0xc8/0x128 panic+0x154/0x30c init_IRQ+0xd0/0xe8 start_kernel+0x260/0x400 ---[ end Kernel panic - not syncing: No interrupt controller found. ]---
If I boot the same kernel with:
-cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=2We get a lot further:
Booting Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 BST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90308 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90308 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902B0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000174 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E90288 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kernel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh printk: log_buf_len individual max cpu contribution: 4096 bytes printk: log_buf_len total cpu_extra contributions: 12288 bytes printk: log_buf_len min size: 16384 bytes printk: log_buf_len: 32768 bytes printk: early log buf free: 12432(75%) Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927316K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153968K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 rcu: Offload RCU callbacks from CPUs: (none). random: get_random_bytes called from start_kernel+0x2a4/0x400 with crng_init=0 arch_timer: cp15 timer(s) running at 62.50MHz (virt). clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns sched_clock: 56 its at 62MHz, resolution 16ns, wraps every 4398046511096ns Console: colour dummy device 80x25 printk: console [hvc0] enabled printk: bootconsole [smh0] disabled
which completes the Buster boot and allows me to ssh into Dom0 - I haven't yet tried to boot a DomU.
-- Alex Bennée
Masami Hiramatsu masami.hiramatsu@linaro.org writes:
Hi,
I've succeeded to make the X.org running on Dom0. It seems that Xorg's nouveau driver caused SIGBUS issue. Custom nouveau kernel driver + Xorg fbdev driver seems stable. (Even if it doesn't work again, I'll try to use USB-HDMI adaptor next time)
So, I would like to test the virtio-video for the next step. Alex, how can I help you to test it?
In one window you need the vhost-user gpu daemon:
./vhost-user-gpu --socket-path=vgpu.sock -v
and then on the QEMU command line you need the memory sharing and the socket connection as well as the device:
$QEMU $ARGS \ -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on \ -numa node,memdev=mem \ -chardev socket,path=vgpu.sock,id=vgpu \ -device vhost-user-gpu-pci,chardev=vgpu
However currently although my guest kernel detects the PCI device it seems to hang the kernel on boot up.
Thank you,
2020年10月2日(金) 23:59 Alex Bennée alex.bennee@linaro.org:
Hi,
As it's Friday I thought I'd better summarise where I've gotten to with booting Xen. TLDR, got something working with GICv2, GICv3 seems broken.
Platform:
QEMU with: -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=3 \ -device virtio-net-pci,netdev=unet,id=virt-net,disable-legacy=on \ -device virtio-scsi-pci,id=virt-scsi,disable-legacy=on \ -blockdev driver=raw,node-name=hd,discard=unmap,file.driver=host_device,file.filename=/dev/zen-disk/debian-buster-arm64 \ -device scsi-hd,drive=hd,id=virt-scsi-hd \ -drive if=pflash,file=/usr/share/AAVMF/AAVMF_CODE.fd,format=raw,readonly \ -drive if=pflash,file=/home/alex/models/qemu-arm64-efivars,format=raw
The firmware is the stock Debian EFI:
Package: qemu-efi-aarch64 Status: install ok installed Installed-Size: 133154 Maintainer: Debian QEMU Team pkg-qemu-devel@lists.alioth.debian.org Architecture: all Source: edk2 Version: 0~20181115.85588389-3+deb10u1 Description: UEFI firmware for 64-bit ARM virtual machines qemu-efi-aarch64 is a build of EDK II for 64-bit ARM virtual machines. It includes full support for UEFI, including Secure Boot. Homepage: http://www.tianocore.org
I've also turned on semihosting (-semihosting) to get earlycon output.
Xen:
Current master with the following patches:
e680dde6fd (HEAD -> review/arm-xen-acpi-changes) xen/arm: gic-v2: acpi: Use the correct length for the GICC structure a34a93acb9 xen/arm: acpi: add BAD_MADT_GICC_ENTRY() macro f66afeb018 (linaro/review/arm-xen-acpi-changes) xen/arm: acpi: relax FADT revision requirements to 5.1 72f3d495d0 xen/arm: Introduce fw_unreserved_regions() and use it 8d447fa48a xen/arm: Check if the platform is not using ACPI before initializing Dom0less fa03236a08 xen/arm: acpi: The fixmap area should always be cleared during failure/unmap 505b82a8be xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()
I've tried various kernels:
- The default buster kernel (4.19.0-11) would panic in memblock_alloc_base
- The backports kernel (5.8.0-0.bpo.2) didn't panic AFAICT but never came up either
In the end I built my own image so I could enable semihosting support and finally I was able to get some output from the kernel:
ing Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 ST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90328 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90328 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902D0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000194 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E902A8 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 ACPI: [APIC:0x0b] Invalid zero length percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: GIC system register CPU interface CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kerel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927524K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153760K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0e] Invalid zero length ACPI: [APIC:0x0b] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length Kernel panic - not syncing: No interrupt controller found. CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.8.11-ajb-00010-ge059ce7b131b #42 Call trace: dump_backtrace+0x0/0x1a8 show_stack+0x18/0x28 dump_stack+0xc8/0x128 panic+0x154/0x30c init_IRQ+0xd0/0xe8 start_kernel+0x260/0x400 ---[ end Kernel panic - not syncing: No interrupt controller found. ]---
If I boot the same kernel with:
-cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=2We get a lot further:
Booting Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 BST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90308 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90308 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902B0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000174 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E90288 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kernel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh printk: log_buf_len individual max cpu contribution: 4096 bytes printk: log_buf_len total cpu_extra contributions: 12288 bytes printk: log_buf_len min size: 16384 bytes printk: log_buf_len: 32768 bytes printk: early log buf free: 12432(75%) Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927316K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153968K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 rcu: Offload RCU callbacks from CPUs: (none). random: get_random_bytes called from start_kernel+0x2a4/0x400 with crng_init=0 arch_timer: cp15 timer(s) running at 62.50MHz (virt). clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns sched_clock: 56 its at 62MHz, resolution 16ns, wraps every 4398046511096ns Console: colour dummy device 80x25 printk: console [hvc0] enabled printk: bootconsole [smh0] disabled
which completes the Buster boot and allows me to ssh into Dom0 - I haven't yet tried to boot a DomU.
-- Alex Bennée
Hi Alex,
2020年10月16日(金) 2:01 Alex Bennée alex.bennee@linaro.org:
Masami Hiramatsu masami.hiramatsu@linaro.org writes:
Hi,
I've succeeded to make the X.org running on Dom0. It seems that Xorg's nouveau driver caused SIGBUS issue. Custom nouveau kernel driver + Xorg fbdev driver seems stable. (Even if it doesn't work again, I'll try to use USB-HDMI adaptor next time)
So, I would like to test the virtio-video for the next step. Alex, how can I help you to test it?
In one window you need the vhost-user gpu daemon:
./vhost-user-gpu --socket-path=vgpu.sock -v
Hmm, I couldn't find vhost-user-gpu (I've installed xen tools under /usr/local, but I can not find vhost* tools)
and then on the QEMU command line you need the memory sharing and the socket connection as well as the device:
$QEMU $ARGS \ -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on \ -numa node,memdev=mem \ -chardev socket,path=vgpu.sock,id=vgpu \ -device vhost-user-gpu-pci,chardev=vgpu
I'm using xl command ( xl create -dc CONFIGFILE ) to boot up guest domain. Can I boot it up via qemu too?
However currently although my guest kernel detects the PCI device it seems to hang the kernel on boot up.
Might it come from the difference of guest boot command?
Thank you,
Thank you,
2020年10月2日(金) 23:59 Alex Bennée alex.bennee@linaro.org:
Hi,
As it's Friday I thought I'd better summarise where I've gotten to with booting Xen. TLDR, got something working with GICv2, GICv3 seems broken.
Platform:
QEMU with: -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=3 \ -device virtio-net-pci,netdev=unet,id=virt-net,disable-legacy=on \ -device virtio-scsi-pci,id=virt-scsi,disable-legacy=on \ -blockdev driver=raw,node-name=hd,discard=unmap,file.driver=host_device,file.filename=/dev/zen-disk/debian-buster-arm64 \ -device scsi-hd,drive=hd,id=virt-scsi-hd \ -drive if=pflash,file=/usr/share/AAVMF/AAVMF_CODE.fd,format=raw,readonly \ -drive if=pflash,file=/home/alex/models/qemu-arm64-efivars,format=raw
The firmware is the stock Debian EFI:
Package: qemu-efi-aarch64 Status: install ok installed Installed-Size: 133154 Maintainer: Debian QEMU Team pkg-qemu-devel@lists.alioth.debian.org Architecture: all Source: edk2 Version: 0~20181115.85588389-3+deb10u1 Description: UEFI firmware for 64-bit ARM virtual machines qemu-efi-aarch64 is a build of EDK II for 64-bit ARM virtual machines. It includes full support for UEFI, including Secure Boot. Homepage: http://www.tianocore.org
I've also turned on semihosting (-semihosting) to get earlycon output.
Xen:
Current master with the following patches:
e680dde6fd (HEAD -> review/arm-xen-acpi-changes) xen/arm: gic-v2: acpi: Use the correct length for the GICC structure a34a93acb9 xen/arm: acpi: add BAD_MADT_GICC_ENTRY() macro f66afeb018 (linaro/review/arm-xen-acpi-changes) xen/arm: acpi: relax FADT revision requirements to 5.1 72f3d495d0 xen/arm: Introduce fw_unreserved_regions() and use it 8d447fa48a xen/arm: Check if the platform is not using ACPI before initializing Dom0less fa03236a08 xen/arm: acpi: The fixmap area should always be cleared during failure/unmap 505b82a8be xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()
I've tried various kernels:
- The default buster kernel (4.19.0-11) would panic in memblock_alloc_base
- The backports kernel (5.8.0-0.bpo.2) didn't panic AFAICT but never came up either
In the end I built my own image so I could enable semihosting support and finally I was able to get some output from the kernel:
ing Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 ST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90328 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90328 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902D0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000194 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E902A8 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 ACPI: [APIC:0x0b] Invalid zero length percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: GIC system register CPU interface CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kerel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927524K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153760K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0e] Invalid zero length ACPI: [APIC:0x0b] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length Kernel panic - not syncing: No interrupt controller found. CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.8.11-ajb-00010-ge059ce7b131b #42 Call trace: dump_backtrace+0x0/0x1a8 show_stack+0x18/0x28 dump_stack+0xc8/0x128 panic+0x154/0x30c init_IRQ+0xd0/0xe8 start_kernel+0x260/0x400 ---[ end Kernel panic - not syncing: No interrupt controller found. ]---
If I boot the same kernel with:
-cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=2We get a lot further:
Booting Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 BST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90308 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90308 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902B0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000174 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E90288 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kernel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh printk: log_buf_len individual max cpu contribution: 4096 bytes printk: log_buf_len total cpu_extra contributions: 12288 bytes printk: log_buf_len min size: 16384 bytes printk: log_buf_len: 32768 bytes printk: early log buf free: 12432(75%) Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927316K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153968K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 rcu: Offload RCU callbacks from CPUs: (none). random: get_random_bytes called from start_kernel+0x2a4/0x400 with crng_init=0 arch_timer: cp15 timer(s) running at 62.50MHz (virt). clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns sched_clock: 56 its at 62MHz, resolution 16ns, wraps every 4398046511096ns Console: colour dummy device 80x25 printk: console [hvc0] enabled printk: bootconsole [smh0] disabled
which completes the Buster boot and allows me to ssh into Dom0 - I haven't yet tried to boot a DomU.
-- Alex Bennée
-- Alex Bennée
Masami Hiramatsu masami.hiramatsu@linaro.org writes:
Hi Alex,
2020年10月16日(金) 2:01 Alex Bennée alex.bennee@linaro.org:
Masami Hiramatsu masami.hiramatsu@linaro.org writes:
Hi,
I've succeeded to make the X.org running on Dom0. It seems that Xorg's nouveau driver caused SIGBUS issue. Custom nouveau kernel driver + Xorg fbdev driver seems stable. (Even if it doesn't work again, I'll try to use USB-HDMI adaptor next time)
So, I would like to test the virtio-video for the next step. Alex, how can I help you to test it?
In one window you need the vhost-user gpu daemon:
./vhost-user-gpu --socket-path=vgpu.sock -v
Hmm, I couldn't find vhost-user-gpu (I've installed xen tools under /usr/local, but I can not find vhost* tools)
The vhost-user-gpu tool is part of the QEMU source tree (contrib/vhost-user-gpu).
and then on the QEMU command line you need the memory sharing and the socket connection as well as the device:
$QEMU $ARGS \ -object memory-backend-file,id=mem,size=4G,mem-path=/dev/shm,share=on \ -numa node,memdev=mem \ -chardev socket,path=vgpu.sock,id=vgpu \ -device vhost-user-gpu-pci,chardev=vgpu
I'm using xl command ( xl create -dc CONFIGFILE ) to boot up guest domain. Can I boot it up via qemu too?
Hmm so this is where we might need some extra tooling. I'm not sure how QEMU gets invoked by the xl tooling but QEMU for Xen is a fairly different beast to the normal hypervisor interaction as rather than handling vmexits from the hypervisor it just gets commands via the Xen control interface to service emulation and I/O requests.
The above QEMU commands ensure that:
- the guest memory space is shared wit the vhost-user-gpu daemon - a control path is wired up so vhost-user messages can be sent during setup (inititalise the device etc) - the same socket path is fed eventfd messages as the guest triggers virtio events. These can all come from QEMU if the kernel isn't translating mmio access to the virtqueues to eventfd events.
Steffano, how does the XL tooling invoke QEMU and can the command line be modified?
However currently although my guest kernel detects the PCI device it seems to hang the kernel on boot up.
Might it come from the difference of guest boot command?
As far as I can tell there is a hang caused by an access to the GPU that locks up the guest waiting for a response that never can come from the vhost-user-gpu process. If I kill the userspace daemon things un-wedge so it seems like some sort of live lock. I will keep digging.
Thank you,
Thank you,
2020年10月2日(金) 23:59 Alex Bennée alex.bennee@linaro.org:
Hi,
As it's Friday I thought I'd better summarise where I've gotten to with booting Xen. TLDR, got something working with GICv2, GICv3 seems broken.
Platform:
QEMU with: -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=3 \ -device virtio-net-pci,netdev=unet,id=virt-net,disable-legacy=on \ -device virtio-scsi-pci,id=virt-scsi,disable-legacy=on \ -blockdev driver=raw,node-name=hd,discard=unmap,file.driver=host_device,file.filename=/dev/zen-disk/debian-buster-arm64 \ -device scsi-hd,drive=hd,id=virt-scsi-hd \ -drive if=pflash,file=/usr/share/AAVMF/AAVMF_CODE.fd,format=raw,readonly \ -drive if=pflash,file=/home/alex/models/qemu-arm64-efivars,format=raw
The firmware is the stock Debian EFI:
Package: qemu-efi-aarch64 Status: install ok installed Installed-Size: 133154 Maintainer: Debian QEMU Team pkg-qemu-devel@lists.alioth.debian.org Architecture: all Source: edk2 Version: 0~20181115.85588389-3+deb10u1 Description: UEFI firmware for 64-bit ARM virtual machines qemu-efi-aarch64 is a build of EDK II for 64-bit ARM virtual machines. It includes full support for UEFI, including Secure Boot. Homepage: http://www.tianocore.org
I've also turned on semihosting (-semihosting) to get earlycon output.
Xen:
Current master with the following patches:
e680dde6fd (HEAD -> review/arm-xen-acpi-changes) xen/arm: gic-v2: acpi: Use the correct length for the GICC structure a34a93acb9 xen/arm: acpi: add BAD_MADT_GICC_ENTRY() macro f66afeb018 (linaro/review/arm-xen-acpi-changes) xen/arm: acpi: relax FADT revision requirements to 5.1 72f3d495d0 xen/arm: Introduce fw_unreserved_regions() and use it 8d447fa48a xen/arm: Check if the platform is not using ACPI before initializing Dom0less fa03236a08 xen/arm: acpi: The fixmap area should always be cleared during failure/unmap 505b82a8be xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()
I've tried various kernels:
- The default buster kernel (4.19.0-11) would panic in memblock_alloc_base
- The backports kernel (5.8.0-0.bpo.2) didn't panic AFAICT but never came up either
In the end I built my own image so I could enable semihosting support and finally I was able to get some output from the kernel:
ing Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 ST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90328 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90328 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902D0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000194 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E902A8 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 ACPI: [APIC:0x0b] Invalid zero length percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: GIC system register CPU interface CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kerel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927524K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153760K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0e] Invalid zero length ACPI: [APIC:0x0b] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length Kernel panic - not syncing: No interrupt controller found. CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.8.11-ajb-00010-ge059ce7b131b #42 Call trace: dump_backtrace+0x0/0x1a8 show_stack+0x18/0x28 dump_stack+0xc8/0x128 panic+0x154/0x30c init_IRQ+0xd0/0xe8 start_kernel+0x260/0x400 ---[ end Kernel panic - not syncing: No interrupt controller found. ]---
If I boot the same kernel with:
-cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=2We get a lot further:
Booting Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 BST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90308 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90308 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902B0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000174 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E90288 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kernel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh printk: log_buf_len individual max cpu contribution: 4096 bytes printk: log_buf_len total cpu_extra contributions: 12288 bytes printk: log_buf_len min size: 16384 bytes printk: log_buf_len: 32768 bytes printk: early log buf free: 12432(75%) Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927316K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153968K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 rcu: Offload RCU callbacks from CPUs: (none). random: get_random_bytes called from start_kernel+0x2a4/0x400 with crng_init=0 arch_timer: cp15 timer(s) running at 62.50MHz (virt). clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns sched_clock: 56 its at 62MHz, resolution 16ns, wraps every 4398046511096ns Console: colour dummy device 80x25 printk: console [hvc0] enabled printk: bootconsole [smh0] disabled
which completes the Buster boot and allows me to ssh into Dom0 - I haven't yet tried to boot a DomU.
-- Alex Bennée
-- Alex Bennée
Hi Alex,
2020年10月16日(金) 19:49 Alex Bennée alex.bennee@linaro.org:
Masami Hiramatsu masami.hiramatsu@linaro.org writes:
Hi Alex,
2020年10月16日(金) 2:01 Alex Bennée alex.bennee@linaro.org:
Masami Hiramatsu masami.hiramatsu@linaro.org writes:
Hi,
I've succeeded to make the X.org running on Dom0. It seems that Xorg's nouveau driver caused SIGBUS issue. Custom nouveau kernel driver + Xorg fbdev driver seems stable. (Even if it doesn't work again, I'll try to use USB-HDMI adaptor next time)
So, I would like to test the virtio-video for the next step. Alex, how can I help you to test it?
In one window you need the vhost-user gpu daemon:
./vhost-user-gpu --socket-path=vgpu.sock -v
Hmm, I couldn't find vhost-user-gpu (I've installed xen tools under /usr/local, but I can not find vhost* tools)
The vhost-user-gpu tool is part of the QEMU source tree (contrib/vhost-user-gpu).
How can I build it? I've tried to build the latest qemu with
mhiramat@develbox:~/ksrc/qemu$ ./configure --prefix=/usr/local --enable-vhost-user mhiramat@develbox:~/ksrc/qemu$ make -j 24
But it didn't build the vhost-user-gpu command...
Thank you,
On Wed, Oct 21, 2020 at 07:00:22PM +0900, Masami Hiramatsu wrote:
Hi Alex,
2020年10月16日(金) 19:49 Alex Bennée alex.bennee@linaro.org:
Masami Hiramatsu masami.hiramatsu@linaro.org writes:
Hi Alex,
2020年10月16日(金) 2:01 Alex Bennée alex.bennee@linaro.org:
Masami Hiramatsu masami.hiramatsu@linaro.org writes:
Hi,
I've succeeded to make the X.org running on Dom0. It seems that Xorg's nouveau driver caused SIGBUS issue. Custom nouveau kernel driver + Xorg fbdev driver seems stable. (Even if it doesn't work again, I'll try to use USB-HDMI adaptor next time)
So, I would like to test the virtio-video for the next step. Alex, how can I help you to test it?
In one window you need the vhost-user gpu daemon:
./vhost-user-gpu --socket-path=vgpu.sock -v
Hmm, I couldn't find vhost-user-gpu (I've installed xen tools under /usr/local, but I can not find vhost* tools)
The vhost-user-gpu tool is part of the QEMU source tree (contrib/vhost-user-gpu).
How can I build it? I've tried to build the latest qemu with
mhiramat@develbox:~/ksrc/qemu$ ./configure --prefix=/usr/local --enable-vhost-user mhiramat@develbox:~/ksrc/qemu$ make -j 24
But it didn't build the vhost-user-gpu command...
According to Makefile,
ifeq ($(CONFIG_LINUX)$(CONFIG_VIRGL)$(CONFIG_GBM)$(CONFIG_TOOLS),yyyy) HELPERS-y += vhost-user-gpu$(EXESUF) vhost-user-json-y += contrib/vhost-user-gpu/50-qemu-gpu.json endif
there are some package dependencies: - libvirglrenderer-dev - libgbm-dev and a required configure option: - --enable-tools"
# I've never tried it though.
-Takahiro Akashi
Thank you,
-- Masami Hiramatsu
Hi Akashi-san,
2020年10月21日(水) 20:45 AKASHI Takahiro takahiro.akashi@linaro.org:
On Wed, Oct 21, 2020 at 07:00:22PM +0900, Masami Hiramatsu wrote:
Hi Alex,
2020年10月16日(金) 19:49 Alex Bennée alex.bennee@linaro.org:
Masami Hiramatsu masami.hiramatsu@linaro.org writes:
Hi Alex,
2020年10月16日(金) 2:01 Alex Bennée alex.bennee@linaro.org:
Masami Hiramatsu masami.hiramatsu@linaro.org writes:
Hi,
I've succeeded to make the X.org running on Dom0. It seems that Xorg's nouveau driver caused SIGBUS issue. Custom nouveau kernel driver + Xorg fbdev driver seems stable. (Even if it doesn't work again, I'll try to use USB-HDMI adaptor next time)
So, I would like to test the virtio-video for the next step. Alex, how can I help you to test it?
In one window you need the vhost-user gpu daemon:
./vhost-user-gpu --socket-path=vgpu.sock -v
Hmm, I couldn't find vhost-user-gpu (I've installed xen tools under /usr/local, but I can not find vhost* tools)
The vhost-user-gpu tool is part of the QEMU source tree (contrib/vhost-user-gpu).
How can I build it? I've tried to build the latest qemu with
mhiramat@develbox:~/ksrc/qemu$ ./configure --prefix=/usr/local --enable-vhost-user mhiramat@develbox:~/ksrc/qemu$ make -j 24
But it didn't build the vhost-user-gpu command...
According to Makefile,
ifeq ($(CONFIG_LINUX)$(CONFIG_VIRGL)$(CONFIG_GBM)$(CONFIG_TOOLS),yyyy) HELPERS-y += vhost-user-gpu$(EXESUF) vhost-user-json-y += contrib/vhost-user-gpu/50-qemu-gpu.json endif
there are some package dependencies:
- libvirglrenderer-dev
- libgbm-dev
and a required configure option:
- --enable-tools"
Thanks! I found that the dependency and built it right now :) Let me try it works on xen DomU.
Thank you,
# I've never tried it though.
-Takahiro Akashi
Thank you,
-- Masami Hiramatsu
stratos-dev@op-lists.linaro.org
-
AKASHI Takahiro -
Alex Bennée -
Masami Hiramatsu -
Stefano Stabellini