Alex Bennée alex.bennee@linaro.org writes:
(forgot to CC Andre)
Hi,
As it's Friday I thought I'd better summarise where I've gotten to with booting Xen. TLDR, got something working with GICv2, GICv3 seems broken.
Platform:
QEMU with: -cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=3 \ -device virtio-net-pci,netdev=unet,id=virt-net,disable-legacy=on \ -device virtio-scsi-pci,id=virt-scsi,disable-legacy=on \ -blockdev driver=raw,node-name=hd,discard=unmap,file.driver=host_device,file.filename=/dev/zen-disk/debian-buster-arm64 \ -device scsi-hd,drive=hd,id=virt-scsi-hd \ -drive if=pflash,file=/usr/share/AAVMF/AAVMF_CODE.fd,format=raw,readonly \ -drive if=pflash,file=/home/alex/models/qemu-arm64-efivars,format=raw
The firmware is the stock Debian EFI:
Package: qemu-efi-aarch64 Status: install ok installed Installed-Size: 133154 Maintainer: Debian QEMU Team pkg-qemu-devel@lists.alioth.debian.org Architecture: all Source: edk2 Version: 0~20181115.85588389-3+deb10u1 Description: UEFI firmware for 64-bit ARM virtual machines qemu-efi-aarch64 is a build of EDK II for 64-bit ARM virtual machines. It includes full support for UEFI, including Secure Boot. Homepage: http://www.tianocore.org
I've also turned on semihosting (-semihosting) to get earlycon output.
Xen:
Current master with the following patches:
e680dde6fd (HEAD -> review/arm-xen-acpi-changes) xen/arm: gic-v2: acpi: Use the correct length for the GICC structure a34a93acb9 xen/arm: acpi: add BAD_MADT_GICC_ENTRY() macro f66afeb018 (linaro/review/arm-xen-acpi-changes) xen/arm: acpi: relax FADT revision requirements to 5.1 72f3d495d0 xen/arm: Introduce fw_unreserved_regions() and use it 8d447fa48a xen/arm: Check if the platform is not using ACPI before initializing Dom0less fa03236a08 xen/arm: acpi: The fixmap area should always be cleared during failure/unmap 505b82a8be xen/acpi: Rework acpi_os_map_memory() and acpi_os_unmap_memory()
I've tried various kernels:
- The default buster kernel (4.19.0-11) would panic in memblock_alloc_base
- The backports kernel (5.8.0-0.bpo.2) didn't panic AFAICT but never came up either
In the end I built my own image so I could enable semihosting support and finally I was able to get some output from the kernel:
ing Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 ST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90328 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90328 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902D0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000194 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E902A8 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 ACPI: [APIC:0x0b] Invalid zero length percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: GIC system register CPU interface CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kerel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927524K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153760K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length ACPI: [APIC:0x0e] Invalid zero length ACPI: [APIC:0x0b] Invalid zero length ACPI: [APIC:0x0c] Invalid zero length Kernel panic - not syncing: No interrupt controller found. CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.8.11-ajb-00010-ge059ce7b131b #42 Call trace: dump_backtrace+0x0/0x1a8 show_stack+0x18/0x28 dump_stack+0xc8/0x128 panic+0x154/0x30c init_IRQ+0xd0/0xe8 start_kernel+0x260/0x400 ---[ end Kernel panic - not syncing: No interrupt controller found. ]---
If I boot the same kernel with:
-cpu cortex-a57 -machine type=virt,virtualization=on,gic-version=2We get a lot further:
Booting Linux on physical CPU 0x0000000000 [0x411fd070] Linux version 5.8.11-ajb-00010-ge059ce7b131b (alex@zen) (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #42 SMP Fri Oct 2 14:19:57 BST 2020 earlycon: smh0 at I/O port 0x0 (options '') printk: bootconsole [smh0] enabled Xen XEN_VERSION.XEN_SUBVERSION support found efi: EFI v2.50 by Xen efi: ACPI 2.0=0xf7e90308 cma: Reserved 16 MiB at 0x00000000bf000000 ACPI: Early table checksum verification disabled ACPI: RSDP 0x00000000F7E90308 000024 (v02 BOCHS ) ACPI: XSDT 0x00000000F7E902B0 000054 (v01 BOCHS BXPCFACP 00000001 01000013) ACPI: FACP 0x00000000F7E90000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001) ACPI: DSDT 0x0000000138520000 0014A2 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001) ACPI: APIC 0x00000000F7E90110 000174 (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: GTDT 0x00000001384F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001) ACPI: MCFG 0x00000001384E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001) ACPI: SPCR 0x00000001384D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001) ACPI: STAO 0x00000000F7E90288 000025 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001) ACPI: SPCR: console: pl011,mmio,0x9000000,9600 Zone ranges: DMA [mem 0x0000000040000000-0x000000007fffffff] DMA32 [mem 0x0000000080000000-0x00000000ffffffff] Normal [mem 0x0000000100000000-0x000000013856ffff] Movable zone start for each node Early memory node ranges node 0: [mem 0x0000000040000000-0x00000000bfffffff] node 0: [mem 0x00000000f7e90000-0x00000000f7e90fff] node 0: [mem 0x00000001384d0000-0x000000013852ffff] node 0: [mem 0x0000000138550000-0x000000013856ffff] Initmem setup node 0 [mem 0x0000000040000000-0x000000013856ffff] psci: probing for conduit method from ACPI. psci: PSCIv1.1 detected in firmware. psci: Using standard PSCI v0.2 function IDs psci: Trusted OS migration not required psci: SMC Calling Convention v1.1 percpu: Embedded 22 pages/cpu s51288 r8192 d30632 u90112 Detected PIPT I-cache on CPU0 CPU features: detected: ARM erratum 832075 CPU features: detected: ARM erratum 834220 CPU features: detected: EL2 vector hardening CPU features: detected: Speculative Store Bypass Disable CPU features: detected: ARM errata 1165522, 1319367, or 1530923 Built 1 zonelists, mobility grouping on. Total pages: 516222 Kernel command line: placeholder root=/dev/sda2 ro console=hvc0 earlycon=smh printk: log_buf_len individual max cpu contribution: 4096 bytes printk: log_buf_len total cpu_extra contributions: 12288 bytes printk: log_buf_len min size: 16384 bytes printk: log_buf_len: 32768 bytes printk: early log buf free: 12432(75%) Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) mem auto-init: stack:off, heap alloc:off, heap free:off software IO TLB: mapped [mem 0x7bfff000-0x7ffff000] (64MB) Memory: 1927316K/2097668K available (8700K kernel code, 536K rwdata, 3280K rodata, 1536K init, 248K bss, 153968K reserved, 16384K cma-reserved) SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 rcu: Hierarchical RCU implementation. rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=4. rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 rcu: Offload RCU callbacks from CPUs: (none). random: get_random_bytes called from start_kernel+0x2a4/0x400 with crng_init=0 arch_timer: cp15 timer(s) running at 62.50MHz (virt). clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns sched_clock: 56 its at 62MHz, resolution 16ns, wraps every 4398046511096ns Console: colour dummy device 80x25 printk: console [hvc0] enabled printk: bootconsole [smh0] disabled
which completes the Buster boot and allows me to ssh into Dom0 - I haven't yet tried to boot a DomU.