diff --git a/CHANGELOG.md b/CHANGELOG.md index 2e90a59f1..6ae73d664 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,16 @@ hashes are given where a change is easiest to understand by reading the commit. ## November 2026 +### virtio + +- virtio-serial and virtio-9p on PCI (Octane, Tezro) offer + `VIRTIO_F_ACCESS_PLATFORM`, so Linux gives them bus addresses they can reach + instead of physical ones; Linux's `virtio_console` hung at load. + +### Fibre Channel + +- Linux's qla2xxx brings the ISP2200's loop online and finds its disks on the + Octane (Linux 7.3); it stopped at the port login, which the adapter refused. ### Octane - The Bridges' 64 GB windows reach PCI: the 32-bit and 64-bit aliases of PCI diff --git a/clanker/INDEX.md b/clanker/INDEX.md index 4e0985785..bc13a810f 100644 --- a/clanker/INDEX.md +++ b/clanker/INDEX.md @@ -147,6 +147,8 @@ Non-graphics devices: MC/MEMCFG and RAM banks, HPC3, VDMA, EEPROMs and NVRAM, PS | [ip30-module-name-must-match-active-cpus.md](devices/ip30-module-name-must-match-active-cpus.md) | rule | Octane: the module record must say PM10 for one CPU and PM20 for two, or the PROM fails its NIC diagnostic | | [ip30-low-memory-aliases-sdram.md](devices/ip30-low-memory-aliases-sdram.md) | rule | Octane: the 16 KiB at physical 0 is the first 16 KiB of SDRAM, which the processor reaches only there (0x2000_0000..+16 KiB bus-errors, as the stock IDE memory diagnostics expect); shares its JIT page generations; the guard holds on the table path only (the ppmem window maps whole units); DMA reach unknown | | [ip30-prom-stores-into-flash-stay-in-cache.md](devices/ip30-prom-stores-into-flash-stay-in-cache.md) | rule | Octane: the PROM's cached stores into its own flash never reach the Bridge; `CachedFlashLines` holds them (approximation: no write-back) | +| [ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md](devices/ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md) | open | Octane: the PROM loading an 11 MB ELF (Linux vmlinux) from the volume header writes its 16 KB-page DMA list over its adapter list and panics; TFTP boot works | +| [ip30-linux-7.3-guest.md](devices/ip30-linux-7.3-guest.md) | howto | Octane: mainline Linux 7.3 config, the PROM/Linux PCI window mismatch and the root-bus scan past device 7 (kernel patch), TFTP boot, no power-off | | [ip30-device-timing-on-hptimer.md](devices/ip30-device-timing-on-hptimer.md) | design | Octane: IOC3, RAD1 and ISP1040 pace themselves on `hptimer` one-shots, with threads only where something blocks (the ISP1040 ring, serial receive, NVRAM writer) | | [ip30-openbsd-needs-64gb-window-and-slot-4-empty.md](devices/ip30-openbsd-needs-64gb-window-and-slot-4-empty.md) | rule | Octane: OpenBSD 6.9 reaches PCI through the Bridges' 64 GB windows (IOC3 at 0xF0_4050_0000) and takes the SuperIO interrupt from the first empty Bridge slot, so virtio goes in slots 6 and 7 (`bridge_slot`); what it attaches | | [ip30-ps2-keyboard-runs-set-3.md](devices/ip30-ps2-keyboard-runs-set-3.md) | rule | Octane: PROM 4.17 and IRIX's `ioc3_pckm` put the PS/2 keyboard in scan code set 3 (Linux/OpenBSD use 2); K_RD slot order, CLAMP_THREE, role by F2 answer; how the PROM's keyboard sequence was traced | @@ -172,6 +174,7 @@ WD33C93A, SCSI targets (disk, CD-ROM, DaynaPort) and guest SCSI drivers. |---|---|---| | [a-write-cdb-must-outlive-its-data-request.md](scsi/a-write-cdb-must-outlive-its-data-request.md) | rule | A write CDB must outlive the chip's request for its data | | [daynaport-target-gotchas.md](scsi/daynaport-target-gotchas.md) | rule | DaynaPort target: the three things that bite | +| [isp2200-linux-logs-in-through-ring-mailbox-entries.md](scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md) | rule | Linux qla2xxx logs in to ISP2200 loop ports with 0x0074 and with type 0x39 mailbox entries in the request ring; both must be answered or no FC disk appears | | [linux-lun-scan-and-mode-sense-gaps.md](scsi/linux-lun-scan-and-mode-sense-gaps.md) | rule | Linux SCSI probe noise: phantom LUNs, missing Caching page, no MODE SENSE(10) | | [miniroot-install-hang-scsi0-dma-irq-storm.md](scsi/miniroot-install-hang-scsi0-dma-irq-storm.md) | rule | IRIX miniroot install hangs on iris: SCSI0_DMA IRQ storm (FIXED) | | [windows-cannot-fsync-a-directory.md](scsi/windows-cannot-fsync-a-directory.md) | rule | Windows cannot fsync a directory: raw COW overlays failed every sidecar rewrite | diff --git a/clanker/devices/ip30-linux-7.3-guest.md b/clanker/devices/ip30-linux-7.3-guest.md new file mode 100644 index 000000000..6b53d0ea4 --- /dev/null +++ b/clanker/devices/ip30-linux-7.3-guest.md @@ -0,0 +1,42 @@ +# Running mainline Linux (7.3) on the Octane + +Status: howto (Linux 7.3.4, PROM 4.17, 2026-11-16). Boots to a shell on two +CPUs with the onboard SCSI, IOC3 serial and Ethernet, the cage's ISP2200 and +NVMe behind 21152s, virtio-serial and virtio-9p. + +**Kernel.** `ip30_defconfig` plus: W1, W1_MASTER_SGI, W1_SLAVE_DS250X (the +Bridge driver defers its probe for ever until it can read each board's +part number from the one-wire NIC); RTC_DRV_DS1685_FAMILY; DEVTMPFS; +`qlogic/1040.bin` built in for qla1280 (it calls request_firmware before any +root exists); qla2xxx wants `ql2200_fw.bin`. The PROM loads the ELF64 +`vmlinux` (load address 0xa800000020004000) from `bootp()` but not from the +volume header (`ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md`). + +**Kernel-side problems, not iris's** (fixed with a patch to +`arch/mips/pci/pci-xtalk-bridge.c`): + +- The driver sets `PCI_PROBE_ONLY` and claims what firmware assigned, and + assumes PCI bus address = small-window offset. The PROM gives BaseIO + windows 2 and up a base 1 MB below their offset (IOC3 BAR 0x500000 through + window 2 at 0x600000) and leaves an option board's Bridge at reset values + with nothing assigned behind its 21152s. Pointing each Device(x) window at + its own offset (except the one the PROM console's IOC3 uses; `prom_putchar` + hard-codes widget 15 + 0x600000) and calling + `pci_assign_unassigned_bus_resources` after the claim makes the cage work. +- The root-bus scan goes to device 31; device 8 of type 0 space is the type 1 + aperture, so with the type 1 register at bus 0 the unnumbered 21152 claims + the cycle and a phantom ISP2200 appears at 00:08.0 (a bridge with + secondary = subordinate = 0 does claim a type 1 cycle for bus 0; iris is + right). Stop the scan at `BRIDGE_DEV_CNT`. + +**iris fixes it took:** ISP2200 Login Loop Port and +request-ring mailbox entries +(`scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md`); virtio-pci +offering ACCESS_PLATFORM; virtio-serial off slot 4's interrupt pin +(`virtio-serial-design.md`, a separate change). + +**Other behaviour to expect.** No software power-off (`poweroff` halts; the +DS1685 driver powers down only on the power button). The 21152's I/O window +lands at bus 0x200000, which the CPU cannot reach (all Bridge windows are +memory space); qla2xxx runs on the memory BAR. ext4 sends WRITE SAME(10), which the SCSI disk refuses +(noted, harmless). `make depend` of OpenSSL 1.0.2 takes hours without the JIT. diff --git a/clanker/devices/ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md b/clanker/devices/ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md new file mode 100644 index 000000000..8f67d8f58 --- /dev/null +++ b/clanker/devices/ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md @@ -0,0 +1,29 @@ +# Octane PROM: loading an 11 MB ELF from the volume header overruns a DMA list + +Status: open (PROM 4.17 as iris runs it, 2026-11-15). Not established whether +real hardware does the same; network boot is the workaround. + +**Symptom.** `boot -f dksc(0,1,8)vmlinux` with a Linux 7.3 kernel of 11.5 MB +(one PT_LOAD of 0xaf3e30 bytes at 0xa800000020004000) prints the size and +then `Read address error exception, bad address: 0x40001f400008` at PROM PC +0x9fc422a4, followed by `PANIC: driver_acquire: io_state.active deadlock` +for ever. The same file loads and runs from `bootp()vmlinux` (iris's NAT +TFTP). The SCSI trace ends with READ(10) of the ELF header (block 8, one +block): the crash comes while the PROM issues the segment's read. + +**What was found.** The PROM polls its SCSI adapters through a list whose +head is at 0xa800000020f17e40; the node for controller 1 at 0x20f0e9a0 had +next = 0x...20f0dfc0, base = 0x900000001f400000 before the boot and read +`0x00000000a0128000, 0x000040001f400000, 0x00000000a012c000, +0x0000400000050000` after it. Those are 16-byte records `(0, PCI address, +0x4000)` whose PCI addresses are the Bridge's direct window (0x8000_0000 + +physical 0x2012_8000, +0x4000, ...): a DMA page list for the 11 MB segment, +one record per 16 KB page, written past its buffer into the adapter node. +No CPU store to the node hit a physical write breakpoint, and iris's cache +model here is the shadow model (no dirty lines to lose), so the list is +written by the PROM (through the fast store path) or by a DMA. + +**Open.** What sizes that list on real hardware, i.e. whether the PROM +splits a large read by something iris answers (INQUIRY, MODE SENSE pages, +the ISP1040 firmware's limits) or real Octanes cannot load a kernel this +large from the volume header either. diff --git a/clanker/scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md b/clanker/scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md new file mode 100644 index 000000000..b843baebe --- /dev/null +++ b/clanker/scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md @@ -0,0 +1,27 @@ +# Linux's qla2xxx logs in to ISP2200 loop ports through request-ring mailbox entries + +Status: rule (Linux 7.3.4 qla2xxx 12.00.00, ISP2200 on the Octane's PCI cage, 2026-11-15). + +**Symptom.** qla2xxx loads, the loop comes up ("LOOP UP detected"), and no +disk ever appears. `scsi2 notes` shows `loop command 0x0074 not modelled`. + +**Cause.** On the parts before the 24xx (not FWI2-capable) the driver logs in +to each loop port two ways: synchronously with mailbox command 0x0074 (Login +Loop Port, `qla2x00_login_local_device`), and from its async discovery state +machine with a *mailbox entry* in the request ring, entry type 0x39 +(`MBX_IOCB_TYPE`, `struct mbx_entry` in `qla_def.h`), built by +`qla2x00_login_iocb` (m0 = 0x006f), `qla2x00_logout_iocb` (0x0071) and +`qla2x00_adisc_iocb` (0x0064 into `async_pd_dma`). The firmware answers in the +response ring with an entry of the same type: handle copied, `entry_status` +and `status` 0, the output mailboxes at the same offsets +(`qla2x00_mbx_iocb_entry`). Both paths were refused, so no port was ever +logged in. IRIX and the BSDs never send either, which is why IRIX 6.5.30 +worked. + +**Fix.** `Isp2200::mailbox_entry` serves the four commands Linux puts in an +entry and answers anything else there with 0x4001; 0x0074 answers COMPLETE +for a present port. Test: `isp2200::tests::loop_tests::port_login`. + +Layout of `struct mbx_entry` (little-endian): type 0, count 1, handle 4, +loop ID 8 (the high byte on parts without extended IDs), status 10, m0 24, +m1 26, m2 28, m3 30, m6 32, m7 34, m9 36, m10 38. diff --git a/docs/dev/virtio-serial.md b/docs/dev/virtio-serial.md index b48f15f45..dadfd9b4b 100644 --- a/docs/dev/virtio-serial.md +++ b/docs/dev/virtio-serial.md @@ -125,6 +125,15 @@ not offered. Everything in the BAR is little-endian as the device sees it; the bridge's byte-lane rules apply as for any other function on that machine. +On PCI the device also offers `VIRTIO_F_ACCESS_PLATFORM` (bit 33): ring and +buffer addresses are PCI bus addresses, which go through the host bridge's +DMA translation like any other function's (on the Octane, the Bridge's direct +windows; a physical address below 1 GB is PCI-local space and reaches no +memory). Linux accepts the bit and gives the addresses its DMA mapping +returns; without the bit it hands over physical addresses and the device never +sees its rings. A driver that does not accept the bit is still taken. The same +holds for virtio-9p on PCI. + ## Limits - **No snapshot support yet**: a restored snapshot leaves the device idle and diff --git a/docs/fibre-channel.md b/docs/fibre-channel.md index 1e28a0c88..ba4153429 100644 --- a/docs/fibre-channel.md +++ b/docs/fibre-channel.md @@ -61,9 +61,11 @@ and the comment in `src/dev/isp2200.rs` says so. ## Limits -- Only IRIX 6.5.30 has been tried. Linux logs in through request-ring mailbox - entries the model does not serve, so mainline Linux cannot bring a port - online. +- IRIX 6.5.30 and Linux 7.3 (qla2xxx, with the stock `ql2200_fw.bin`) have + been tried. Linux logs in to each port with Login Loop Port and with + mailbox commands it puts in the request ring (login, logout and the port + database for ADISC); the model answers those four in a ring entry and + refuses any other command sent that way. - Commands run one at a time. An abort or reset matches the command running, not the ones queued behind it. - A disk attached or removed after start is not announced; there is no loop diff --git a/src/dev/isp2200.rs b/src/dev/isp2200.rs index 8e79e2f64..be75ea670 100644 --- a/src/dev/isp2200.rs +++ b/src/dev/isp2200.rs @@ -196,6 +196,9 @@ pub mod mbox { pub const GET_LINK_STATUS: u16 = 0x006b; pub const LIP_RESET: u16 = 0x006c; pub const FULL_LOGIN_LIP: u16 = 0x0072; + pub const LOGIN_FABRIC_PORT: u16 = 0x006f; + pub const LOGOUT_FABRIC_PORT: u16 = 0x0071; + pub const LOGIN_LOOP_PORT: u16 = 0x0074; pub const PORT_NAME_LIST: u16 = 0x0075; pub const GET_ID_LIST: u16 = 0x007c; pub const LUN_RESET: u16 = 0x007e; @@ -223,6 +226,9 @@ const ENTRY_TYPE_CONTINUATION_A64: u8 = 0x0a; const ENTRY_TYPE_COMMAND: u8 = 0x11; const ENTRY_TYPE_COMMAND_LIST: u8 = 0x15; const ENTRY_TYPE_COMMAND_A64: u8 = 0x19; +/// A mailbox command carried in the request ring (Linux's `MBX_IOCB_TYPE`), +/// answered by an entry of the same type in the response ring. +const ENTRY_TYPE_MAILBOX: u8 = 0x39; /// Entry status flags a response entry can carry. const ENTRY_FLAG_BAD_TYPE: u8 = 0x04; @@ -1741,6 +1747,7 @@ impl Core { } // A marker resynchronises firmware state; nothing is posted. ENTRY_TYPE_MARKER => {} + ENTRY_TYPE_MAILBOX => self.mailbox_entry(st, &req), ENTRY_TYPE_CONTINUATION | ENTRY_TYPE_CONTINUATION_A64 => st.reject_entry(&req, ENTRY_FLAG_BAD_ORDER), _ => st.reject_entry(&req, ENTRY_FLAG_BAD_TYPE), } @@ -1752,6 +1759,48 @@ impl Core { true } + /// A mailbox command in the request ring (Linux's `struct mbx_entry`: + /// handle at 4, loop ID at 8, status at 10, then m0, m1, m2, m3, m6, m7, + /// m9 and m10 as words from 24). Linux 7.3 sends its port logins, + /// logouts and ADISC this way on the parts before the 24xx + /// (qla2x00_login_iocb, qla2x00_logout_iocb, qla2x00_adisc_iocb). The + /// answer is the entry itself with the output mailboxes in the same + /// places and status 0; a command the ring does not carry comes back + /// with an invalid-command m0. Which commands the firmware takes in an + /// entry is not public: these are the ones Linux sends, and fabric + /// login is answered as a loop port login, since the loop has no fabric. + fn mailbox_entry(&self, st: &mut Locked<'_>, req: &[u8; ENTRY_SIZE]) { + const SLOTS: [(usize, usize); 8] = [(0, 24), (1, 26), (2, 28), (3, 30), (6, 32), (7, 34), (9, 36), (10, 38)]; + let mut input: Mailboxes = [0; MAILBOX_COUNT]; + for (m, off) in SLOTS { + input[m] = le16(req, off); + } + let mut out: Mailboxes = [0; MAILBOX_COUNT]; + out[0] = match input[0] { + mbox::LOGIN_FABRIC_PORT | mbox::LOGIN_LOOP_PORT => { + if !st.loop_ready() || !st.target(usize::from(input[1] >> 8)) { + mbox::COMMAND_ERROR + } else { + mbox::COMPLETE + } + } + // Nothing is logged out: every port stays logged in. + mbox::LOGOUT_FABRIC_PORT => mbox::COMPLETE, + mbox::GET_PORT_DB => self.get_port_db(st, &input), + cmd => { + self.note(format!("mailbox command {cmd:#06x} in a request entry not modelled")); + mbox::INVALID + } + }; + let mut entry = *req; + entry[ENTRY_OFF_STATUS] = 0; + entry[10..16].fill(0); + for (m, off) in SLOTS { + entry[off..off + 2].copy_from_slice(&out[m].to_le_bytes()); + } + st.held.push_back(entry); + } + /// A command entry and the continuations its entry count claims. The /// count, not each entry's type, says how many slots the command takes: /// a 64-bit continuation has no handle, only segment data from byte 4. @@ -1832,8 +1881,8 @@ impl Core { /// The mailbox commands the loop answers: those it implements, then /// those it knows of and refuses (0x0040, 0x0041 and 0x0047, whose use is - /// not established here, and the name-server and login commands 0x006e, - /// 0x006f, 0x0071 and 0x0074), which `fc_mailbox` notes and fails. + /// not established here, and the name-server and fabric login commands + /// 0x006e, 0x006f and 0x0071), which `fc_mailbox` notes and fails. fn fc_owned(cmd: u16) -> bool { matches!( cmd, @@ -1859,9 +1908,9 @@ impl Core { | 0x0041 | 0x0047 | 0x006e - | 0x006f - | 0x0071 - | 0x0074 + | mbox::LOGIN_FABRIC_PORT + | mbox::LOGOUT_FABRIC_PORT + | mbox::LOGIN_LOOP_PORT ) } @@ -2072,6 +2121,18 @@ impl Core { self.start_bringup(st, false); mbox::COMPLETE } + // Login to a port on the loop: every disk is logged in already, so + // the login succeeds for any port present; m1 bit 0 (initiator) + // and bit 1 (FCP-2 device) stay clear, the FCP target role the port + // database gives. A loop ID with no port is the command error + // Linux reads as a failed login (qla2x00_local_device_login). + mbox::LOGIN_LOOP_PORT => { + if !st.loop_ready() || !st.target(usize::from(input[1] >> 8)) { + return mbox::COMMAND_ERROR; + } + out[1] = 0; + mbox::COMPLETE + } // The control block's retry count and delay, the order Linux and // NetBSD read them in; that the firmware echoes the bytes rather // than keeping values of its own is inferred. IRIX 6.5.30 scans diff --git a/src/dev/isp2200/tests/loop_tests.rs b/src/dev/isp2200/tests/loop_tests.rs index 7b11339b5..8520b4840 100644 --- a/src/dev/isp2200/tests/loop_tests.rs +++ b/src/dev/isp2200/tests/loop_tests.rs @@ -389,3 +389,55 @@ fn port_name_list_retains_m7() { assert_eq!(r.rd(mb(1)), 20); r.ack(); } + +/// Linux's port login: Login Loop Port as a mailbox command, and the +/// mailbox commands it carries in request entries (login, logout and the +/// port database ADISC reads), each answered in an entry of the same type. +#[test] +fn port_login() { + let r = Rig::with_disks("login", &[5]); + r.loop_up(ICB_OPT_PORT_DB, 0); + assert_eq!(r.call(mbox::LOGIN_LOOP_PORT, 5 << 8, 1, 0), mbox::COMPLETE); + assert_eq!(r.rd(mb(1)), 0); + r.ack(); + assert_eq!(r.call(mbox::LOGIN_LOOP_PORT, 6 << 8, 1, 0), mbox::COMMAND_ERROR); + r.ack(); + + let entry = |handle: u32, mbx: &[(usize, u16)]| { + let mut e = [0u8; ENTRY_SIZE]; + e[0] = ENTRY_TYPE_MAILBOX; + e[1] = 1; + p32(&mut e, 4, handle); + e[9] = 5; + for &(off, v) in mbx { + p16(&mut e, off, v); + } + e + }; + let answer = |r: &Rig| { + r.settle(); + let e = r.response(r.rs_next.get()); + r.consume(1); + e + }; + // Login (fabric login of a loop port, as qla2x00_login_iocb sends it). + r.publish(&entry(0x1234, &[(24, mbox::LOGIN_FABRIC_PORT), (26, 5 << 8), (30, 0x00e0)])); + let e = answer(&r); + assert_eq!((e[0], e[3], g32(&e, 4), g16(&e, 10)), (ENTRY_TYPE_MAILBOX, 0, 0x1234, 0)); + assert_eq!((g16(&e, 24), g16(&e, 26)), (mbox::COMPLETE, 0)); + // No port at loop ID 6. + r.publish(&entry(0x1235, &[(24, mbox::LOGIN_FABRIC_PORT), (26, 6 << 8)])); + assert_eq!(g16(&answer(&r), 24), mbox::COMMAND_ERROR); + // ADISC: the port database to m2:m3 (m6:m7 the high words). + r.mem.put(0x9100, &[0xa5; 128]); + r.publish(&entry(0x1236, &[(24, mbox::GET_PORT_DB), (26, (5 << 8) | 1), (30, 0x9100)])); + let e = answer(&r); + assert_eq!((g32(&e, 4), g16(&e, 24)), (0x1236, mbox::COMPLETE)); + let pd = r.mem.get(0x9100, 128); + assert_eq!(pd[PDB_OFF_PORT_NAME..PDB_OFF_PORT_NAME + 8], port_wwn(5).to_be_bytes()); + // Logout completes; a command the ring does not carry is invalid. + r.publish(&entry(0x1237, &[(24, mbox::LOGOUT_FABRIC_PORT), (26, 5 << 8)])); + assert_eq!(g16(&answer(&r), 24), mbox::COMPLETE); + r.publish(&entry(0x1238, &[(24, mbox::GET_RESOURCE_COUNTS)])); + assert_eq!(g16(&answer(&r), 24), mbox::INVALID); +} diff --git a/src/dev/virtio/mod.rs b/src/dev/virtio/mod.rs index 8a61bcedf..664a6f4af 100644 --- a/src/dev/virtio/mod.rs +++ b/src/dev/virtio/mod.rs @@ -42,7 +42,7 @@ pub mod p9 { pub mod pci; pub mod queue; -use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; use std::sync::{Arc, OnceLock}; use std::thread::JoinHandle; use std::time::Duration; @@ -63,6 +63,10 @@ pub const STATUS_FAILED: u8 = 0x80; /// `VIRTIO_F_VERSION_1`: the device is a 1.x device. Bit 32. pub const F_VERSION_1: u64 = 1 << 32; +/// `VIRTIO_F_ACCESS_PLATFORM`: the device reaches memory through the +/// platform's translation, so the driver must give it the addresses the +/// platform's DMA mapping returns rather than physical ones. Bit 33. +pub const F_ACCESS_PLATFORM: u64 = 1 << 33; /// ISR / interrupt-status bits. pub const ISR_QUEUE: u32 = 1; @@ -229,7 +233,9 @@ struct Inner { struct Shared { inner: Mutex, device_id: u32, - device_features: u64, + /// Offered features. Set before the device is attached; a transport + /// may add its own bits ([`VirtioCore::offer`]). + device_features: AtomicU64, mem: OnceLock>, irq: Mutex, /// Serialises "compute the level, drive the line" so two threads cannot @@ -267,7 +273,7 @@ impl VirtioCore { notified: 0, }), device_id, - device_features, + device_features: AtomicU64::new(device_features), mem: OnceLock::new(), irq: Mutex::new(IrqLine::unconnected()), irq_lock: Mutex::new(()), @@ -291,6 +297,12 @@ impl VirtioCore { // ---- identity ---- + /// Offer `bits` as well as the device's own features: a transport's + /// bits, added before any driver reads the features. + pub fn offer(&self, bits: u64) { + self.sh.device_features.fetch_or(bits, Ordering::Relaxed); + } + pub fn device_id(&self) -> u32 { self.sh.device_id } @@ -298,7 +310,7 @@ impl VirtioCore { /// Offered feature bits `32*sel .. 32*sel+32`. pub fn device_features_word(&self, sel: u32) -> u32 { match sel { - 0 | 1 => (self.sh.device_features >> (32 * sel)) as u32, + 0 | 1 => (self.sh.device_features.load(Ordering::Relaxed) >> (32 * sel)) as u32, _ => 0, } } @@ -337,7 +349,7 @@ impl VirtioCore { // offered and the 1.x bit: otherwise the device clears it so the // driver reads it back and gives up (ยง3.1.1). if new & STATUS_FEATURES_OK != 0 && g.status & STATUS_FEATURES_OK == 0 { - let ok = g.driver_features & !self.sh.device_features == 0 && g.driver_features & F_VERSION_1 != 0; + let ok = g.driver_features & !self.sh.device_features.load(Ordering::Relaxed) == 0 && g.driver_features & F_VERSION_1 != 0; if !ok { new &= !STATUS_FEATURES_OK; } diff --git a/src/dev/virtio/pci.rs b/src/dev/virtio/pci.rs index a439ae77f..741df1124 100644 --- a/src/dev/virtio/pci.rs +++ b/src/dev/virtio/pci.rs @@ -95,6 +95,15 @@ impl VirtioPci { .with_capabilities(16, &caps); // Subsystem: vendor and the virtio device ID. pci.set_subsystem_identity(((id as u32) << 16) | VENDOR_ID as u32); + // The function's DMA goes through the host bridge, where a bus + // address need not be the physical one: behind the Octane's Bridge a + // physical address below 1 GB is PCI-local space, and memory is + // reached through the direct windows. A driver that + // knows this flag (Linux) then gives the queue addresses its DMA + // mapping returns; without it Linux hands over physical addresses, + // which no bridge carries to memory. Not required: a driver that + // does not accept it is still taken. + core.offer(super::F_ACCESS_PLATFORM); Self { core, pci: PciFunctionState::new(pci), sel: Default::default() } } @@ -326,6 +335,25 @@ mod tests { assert_eq!(c.read(4), !(BAR_SIZE - 1)); } + /// The PCI transport offers ACCESS_PLATFORM (bit 33) beside VERSION_1, + /// and takes a driver that accepts it as well as one that does not. + #[test] + fn offers_access_platform() { + use crate::dev::virtio::F_ACCESS_PLATFORM; + for accept in [F_VERSION_1 | F_ACCESS_PLATFORM, F_VERSION_1] { + let (dev, ..) = rig(); + let w = |off: u32, v: u32, n: usize| dev.bar_write(0, off, v, n); + let r = |off: u32, n: usize| dev.bar_read(0, off, n); + w(0x00, 1, 4); // device_feature_select = 1 + assert_eq!(r(0x04, 4), ((F_VERSION_1 | F_ACCESS_PLATFORM) >> 32) as u32); + w(0x14, 3, 1); + w(0x08, 1, 4); + w(0x0c, (accept >> 32) as u32, 4); + w(0x14, 0xb, 1); + assert_eq!(r(0x14, 1) & 8, 8, "FEATURES_OK with {accept:#x}"); + } + } + #[test] fn driver_brings_up_queue_and_sends() { let (dev, mem, port, irq) = rig();