From 71dee7c8be7cbe6276b3c9f8841667404bf54cef Mon Sep 17 00:00:00 2001 From: steigr Date: Sun, 15 Nov 2026 22:26:55 +0100 Subject: [PATCH 1/4] isp2200: answer linux's loop port logins Linux's qla2xxx logs in to each port on the loop with Login Loop Port (0x0074) and, from its discovery state machine, with mailbox commands carried in a request-ring entry (type 0x39: login, logout and the port database for ADISC). The model refused both, so the loop came up and no disk ever appeared. Answer 0x0074 for a present port, and serve those four commands in a ring entry, answered in an entry of the same type in the response ring; anything else sent that way is an invalid command. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 5 ++ clanker/INDEX.md | 1 + ...ux-logs-in-through-ring-mailbox-entries.md | 27 +++++++ docs/fibre-channel.md | 8 ++- src/dev/isp2200.rs | 71 +++++++++++++++++-- src/dev/isp2200/tests/loop_tests.rs | 52 ++++++++++++++ 6 files changed, 156 insertions(+), 8 deletions(-) create mode 100644 clanker/scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md diff --git a/CHANGELOG.md b/CHANGELOG.md index fd55fa9e5..3f5f5a55c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,11 @@ hashes are given where a change is easiest to understand by reading the commit. ## November 2026 +### Fibre Channel + +- Linux's qla2xxx brings the ISP2200's loop online and finds its disks on the + Octane (Linux 7.3); it stopped at the port login, which the adapter refused. + ### Display - VPro (Octane, Fuel, Tezro): 16-bit colour-index images put by the X diff --git a/clanker/INDEX.md b/clanker/INDEX.md index b18403e01..e105b10f9 100644 --- a/clanker/INDEX.md +++ b/clanker/INDEX.md @@ -170,6 +170,7 @@ WD33C93A, SCSI targets (disk, CD-ROM, DaynaPort) and guest SCSI drivers. |---|---|---| | [a-write-cdb-must-outlive-its-data-request.md](scsi/a-write-cdb-must-outlive-its-data-request.md) | rule | A write CDB must outlive the chip's request for its data | | [daynaport-target-gotchas.md](scsi/daynaport-target-gotchas.md) | rule | DaynaPort target: the three things that bite | +| [isp2200-linux-logs-in-through-ring-mailbox-entries.md](scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md) | rule | Linux qla2xxx logs in to ISP2200 loop ports with 0x0074 and with type 0x39 mailbox entries in the request ring; both must be answered or no FC disk appears | | [linux-lun-scan-and-mode-sense-gaps.md](scsi/linux-lun-scan-and-mode-sense-gaps.md) | rule | Linux SCSI probe noise: phantom LUNs, missing Caching page, no MODE SENSE(10) | | [miniroot-install-hang-scsi0-dma-irq-storm.md](scsi/miniroot-install-hang-scsi0-dma-irq-storm.md) | rule | IRIX miniroot install hangs on iris: SCSI0_DMA IRQ storm (FIXED) | | [windows-cannot-fsync-a-directory.md](scsi/windows-cannot-fsync-a-directory.md) | rule | Windows cannot fsync a directory: raw COW overlays failed every sidecar rewrite | diff --git a/clanker/scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md b/clanker/scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md new file mode 100644 index 000000000..b843baebe --- /dev/null +++ b/clanker/scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md @@ -0,0 +1,27 @@ +# Linux's qla2xxx logs in to ISP2200 loop ports through request-ring mailbox entries + +Status: rule (Linux 7.3.4 qla2xxx 12.00.00, ISP2200 on the Octane's PCI cage, 2026-11-15). + +**Symptom.** qla2xxx loads, the loop comes up ("LOOP UP detected"), and no +disk ever appears. `scsi2 notes` shows `loop command 0x0074 not modelled`. + +**Cause.** On the parts before the 24xx (not FWI2-capable) the driver logs in +to each loop port two ways: synchronously with mailbox command 0x0074 (Login +Loop Port, `qla2x00_login_local_device`), and from its async discovery state +machine with a *mailbox entry* in the request ring, entry type 0x39 +(`MBX_IOCB_TYPE`, `struct mbx_entry` in `qla_def.h`), built by +`qla2x00_login_iocb` (m0 = 0x006f), `qla2x00_logout_iocb` (0x0071) and +`qla2x00_adisc_iocb` (0x0064 into `async_pd_dma`). The firmware answers in the +response ring with an entry of the same type: handle copied, `entry_status` +and `status` 0, the output mailboxes at the same offsets +(`qla2x00_mbx_iocb_entry`). Both paths were refused, so no port was ever +logged in. IRIX and the BSDs never send either, which is why IRIX 6.5.30 +worked. + +**Fix.** `Isp2200::mailbox_entry` serves the four commands Linux puts in an +entry and answers anything else there with 0x4001; 0x0074 answers COMPLETE +for a present port. Test: `isp2200::tests::loop_tests::port_login`. + +Layout of `struct mbx_entry` (little-endian): type 0, count 1, handle 4, +loop ID 8 (the high byte on parts without extended IDs), status 10, m0 24, +m1 26, m2 28, m3 30, m6 32, m7 34, m9 36, m10 38. diff --git a/docs/fibre-channel.md b/docs/fibre-channel.md index 21b7a2c71..559aba3e1 100644 --- a/docs/fibre-channel.md +++ b/docs/fibre-channel.md @@ -61,9 +61,11 @@ and the comment in `src/dev/isp2200.rs` says so. ## Limits -- Only IRIX 6.5.30 has been tried. Linux logs in through request-ring mailbox - entries the model does not serve, so mainline Linux cannot bring a port - online. +- IRIX 6.5.30 and Linux 7.3 (qla2xxx, with the stock `ql2200_fw.bin`) have + been tried. Linux logs in to each port with Login Loop Port and with + mailbox commands it puts in the request ring (login, logout and the port + database for ADISC); the model answers those four in a ring entry and + refuses any other command sent that way. - Commands run one at a time. An abort or reset matches the command running, not the ones queued behind it. - A disk attached or removed after start is not announced; there is no loop diff --git a/src/dev/isp2200.rs b/src/dev/isp2200.rs index 8e79e2f64..be75ea670 100644 --- a/src/dev/isp2200.rs +++ b/src/dev/isp2200.rs @@ -196,6 +196,9 @@ pub mod mbox { pub const GET_LINK_STATUS: u16 = 0x006b; pub const LIP_RESET: u16 = 0x006c; pub const FULL_LOGIN_LIP: u16 = 0x0072; + pub const LOGIN_FABRIC_PORT: u16 = 0x006f; + pub const LOGOUT_FABRIC_PORT: u16 = 0x0071; + pub const LOGIN_LOOP_PORT: u16 = 0x0074; pub const PORT_NAME_LIST: u16 = 0x0075; pub const GET_ID_LIST: u16 = 0x007c; pub const LUN_RESET: u16 = 0x007e; @@ -223,6 +226,9 @@ const ENTRY_TYPE_CONTINUATION_A64: u8 = 0x0a; const ENTRY_TYPE_COMMAND: u8 = 0x11; const ENTRY_TYPE_COMMAND_LIST: u8 = 0x15; const ENTRY_TYPE_COMMAND_A64: u8 = 0x19; +/// A mailbox command carried in the request ring (Linux's `MBX_IOCB_TYPE`), +/// answered by an entry of the same type in the response ring. +const ENTRY_TYPE_MAILBOX: u8 = 0x39; /// Entry status flags a response entry can carry. const ENTRY_FLAG_BAD_TYPE: u8 = 0x04; @@ -1741,6 +1747,7 @@ impl Core { } // A marker resynchronises firmware state; nothing is posted. ENTRY_TYPE_MARKER => {} + ENTRY_TYPE_MAILBOX => self.mailbox_entry(st, &req), ENTRY_TYPE_CONTINUATION | ENTRY_TYPE_CONTINUATION_A64 => st.reject_entry(&req, ENTRY_FLAG_BAD_ORDER), _ => st.reject_entry(&req, ENTRY_FLAG_BAD_TYPE), } @@ -1752,6 +1759,48 @@ impl Core { true } + /// A mailbox command in the request ring (Linux's `struct mbx_entry`: + /// handle at 4, loop ID at 8, status at 10, then m0, m1, m2, m3, m6, m7, + /// m9 and m10 as words from 24). Linux 7.3 sends its port logins, + /// logouts and ADISC this way on the parts before the 24xx + /// (qla2x00_login_iocb, qla2x00_logout_iocb, qla2x00_adisc_iocb). The + /// answer is the entry itself with the output mailboxes in the same + /// places and status 0; a command the ring does not carry comes back + /// with an invalid-command m0. Which commands the firmware takes in an + /// entry is not public: these are the ones Linux sends, and fabric + /// login is answered as a loop port login, since the loop has no fabric. + fn mailbox_entry(&self, st: &mut Locked<'_>, req: &[u8; ENTRY_SIZE]) { + const SLOTS: [(usize, usize); 8] = [(0, 24), (1, 26), (2, 28), (3, 30), (6, 32), (7, 34), (9, 36), (10, 38)]; + let mut input: Mailboxes = [0; MAILBOX_COUNT]; + for (m, off) in SLOTS { + input[m] = le16(req, off); + } + let mut out: Mailboxes = [0; MAILBOX_COUNT]; + out[0] = match input[0] { + mbox::LOGIN_FABRIC_PORT | mbox::LOGIN_LOOP_PORT => { + if !st.loop_ready() || !st.target(usize::from(input[1] >> 8)) { + mbox::COMMAND_ERROR + } else { + mbox::COMPLETE + } + } + // Nothing is logged out: every port stays logged in. + mbox::LOGOUT_FABRIC_PORT => mbox::COMPLETE, + mbox::GET_PORT_DB => self.get_port_db(st, &input), + cmd => { + self.note(format!("mailbox command {cmd:#06x} in a request entry not modelled")); + mbox::INVALID + } + }; + let mut entry = *req; + entry[ENTRY_OFF_STATUS] = 0; + entry[10..16].fill(0); + for (m, off) in SLOTS { + entry[off..off + 2].copy_from_slice(&out[m].to_le_bytes()); + } + st.held.push_back(entry); + } + /// A command entry and the continuations its entry count claims. The /// count, not each entry's type, says how many slots the command takes: /// a 64-bit continuation has no handle, only segment data from byte 4. @@ -1832,8 +1881,8 @@ impl Core { /// The mailbox commands the loop answers: those it implements, then /// those it knows of and refuses (0x0040, 0x0041 and 0x0047, whose use is - /// not established here, and the name-server and login commands 0x006e, - /// 0x006f, 0x0071 and 0x0074), which `fc_mailbox` notes and fails. + /// not established here, and the name-server and fabric login commands + /// 0x006e, 0x006f and 0x0071), which `fc_mailbox` notes and fails. fn fc_owned(cmd: u16) -> bool { matches!( cmd, @@ -1859,9 +1908,9 @@ impl Core { | 0x0041 | 0x0047 | 0x006e - | 0x006f - | 0x0071 - | 0x0074 + | mbox::LOGIN_FABRIC_PORT + | mbox::LOGOUT_FABRIC_PORT + | mbox::LOGIN_LOOP_PORT ) } @@ -2072,6 +2121,18 @@ impl Core { self.start_bringup(st, false); mbox::COMPLETE } + // Login to a port on the loop: every disk is logged in already, so + // the login succeeds for any port present; m1 bit 0 (initiator) + // and bit 1 (FCP-2 device) stay clear, the FCP target role the port + // database gives. A loop ID with no port is the command error + // Linux reads as a failed login (qla2x00_local_device_login). + mbox::LOGIN_LOOP_PORT => { + if !st.loop_ready() || !st.target(usize::from(input[1] >> 8)) { + return mbox::COMMAND_ERROR; + } + out[1] = 0; + mbox::COMPLETE + } // The control block's retry count and delay, the order Linux and // NetBSD read them in; that the firmware echoes the bytes rather // than keeping values of its own is inferred. IRIX 6.5.30 scans diff --git a/src/dev/isp2200/tests/loop_tests.rs b/src/dev/isp2200/tests/loop_tests.rs index 7b11339b5..8520b4840 100644 --- a/src/dev/isp2200/tests/loop_tests.rs +++ b/src/dev/isp2200/tests/loop_tests.rs @@ -389,3 +389,55 @@ fn port_name_list_retains_m7() { assert_eq!(r.rd(mb(1)), 20); r.ack(); } + +/// Linux's port login: Login Loop Port as a mailbox command, and the +/// mailbox commands it carries in request entries (login, logout and the +/// port database ADISC reads), each answered in an entry of the same type. +#[test] +fn port_login() { + let r = Rig::with_disks("login", &[5]); + r.loop_up(ICB_OPT_PORT_DB, 0); + assert_eq!(r.call(mbox::LOGIN_LOOP_PORT, 5 << 8, 1, 0), mbox::COMPLETE); + assert_eq!(r.rd(mb(1)), 0); + r.ack(); + assert_eq!(r.call(mbox::LOGIN_LOOP_PORT, 6 << 8, 1, 0), mbox::COMMAND_ERROR); + r.ack(); + + let entry = |handle: u32, mbx: &[(usize, u16)]| { + let mut e = [0u8; ENTRY_SIZE]; + e[0] = ENTRY_TYPE_MAILBOX; + e[1] = 1; + p32(&mut e, 4, handle); + e[9] = 5; + for &(off, v) in mbx { + p16(&mut e, off, v); + } + e + }; + let answer = |r: &Rig| { + r.settle(); + let e = r.response(r.rs_next.get()); + r.consume(1); + e + }; + // Login (fabric login of a loop port, as qla2x00_login_iocb sends it). + r.publish(&entry(0x1234, &[(24, mbox::LOGIN_FABRIC_PORT), (26, 5 << 8), (30, 0x00e0)])); + let e = answer(&r); + assert_eq!((e[0], e[3], g32(&e, 4), g16(&e, 10)), (ENTRY_TYPE_MAILBOX, 0, 0x1234, 0)); + assert_eq!((g16(&e, 24), g16(&e, 26)), (mbox::COMPLETE, 0)); + // No port at loop ID 6. + r.publish(&entry(0x1235, &[(24, mbox::LOGIN_FABRIC_PORT), (26, 6 << 8)])); + assert_eq!(g16(&answer(&r), 24), mbox::COMMAND_ERROR); + // ADISC: the port database to m2:m3 (m6:m7 the high words). + r.mem.put(0x9100, &[0xa5; 128]); + r.publish(&entry(0x1236, &[(24, mbox::GET_PORT_DB), (26, (5 << 8) | 1), (30, 0x9100)])); + let e = answer(&r); + assert_eq!((g32(&e, 4), g16(&e, 24)), (0x1236, mbox::COMPLETE)); + let pd = r.mem.get(0x9100, 128); + assert_eq!(pd[PDB_OFF_PORT_NAME..PDB_OFF_PORT_NAME + 8], port_wwn(5).to_be_bytes()); + // Logout completes; a command the ring does not carry is invalid. + r.publish(&entry(0x1237, &[(24, mbox::LOGOUT_FABRIC_PORT), (26, 5 << 8)])); + assert_eq!(g16(&answer(&r), 24), mbox::COMPLETE); + r.publish(&entry(0x1238, &[(24, mbox::GET_RESOURCE_COUNTS)])); + assert_eq!(g16(&answer(&r), 24), mbox::INVALID); +} From 557d35b4146bf19c13b21d477f4097345cca550a Mon Sep 17 00:00:00 2001 From: steigr Date: Sun, 15 Nov 2026 22:30:26 +0100 Subject: [PATCH 2/4] clanker: record the octane prom's crash loading a large elf from disk Co-Authored-By: Claude Opus 5.5 --- clanker/INDEX.md | 1 + ...load-of-a-large-elf-overruns-a-dma-list.md | 29 +++++++++++++++++++ 2 files changed, 30 insertions(+) create mode 100644 clanker/devices/ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md diff --git a/clanker/INDEX.md b/clanker/INDEX.md index e105b10f9..0f6a2db93 100644 --- a/clanker/INDEX.md +++ b/clanker/INDEX.md @@ -146,6 +146,7 @@ Non-graphics devices: MC/MEMCFG and RAM banks, HPC3, VDMA, EEPROMs and NVRAM, PS | [ip30-module-name-must-match-active-cpus.md](devices/ip30-module-name-must-match-active-cpus.md) | rule | Octane: the module record must say PM10 for one CPU and PM20 for two, or the PROM fails its NIC diagnostic | | [ip30-low-memory-aliases-sdram.md](devices/ip30-low-memory-aliases-sdram.md) | rule | Octane: the 16 KiB at physical 0 is the first 16 KiB of SDRAM, which the processor reaches only there (0x2000_0000..+16 KiB bus-errors, as the stock IDE memory diagnostics expect); shares its JIT page generations; the guard holds on the table path only (the ppmem window maps whole units); DMA reach unknown | | [ip30-prom-stores-into-flash-stay-in-cache.md](devices/ip30-prom-stores-into-flash-stay-in-cache.md) | rule | Octane: the PROM's cached stores into its own flash never reach the Bridge; `CachedFlashLines` holds them (approximation: no write-back) | +| [ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md](devices/ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md) | open | Octane: the PROM loading an 11 MB ELF (Linux vmlinux) from the volume header writes its 16 KB-page DMA list over its adapter list and panics; TFTP boot works | | [ip30-device-timing-on-hptimer.md](devices/ip30-device-timing-on-hptimer.md) | design | Octane: IOC3, RAD1 and ISP1040 pace themselves on `hptimer` one-shots, with threads only where something blocks (the ISP1040 ring, serial receive, NVRAM writer) | | [ip30-ps2-keyboard-runs-set-3.md](devices/ip30-ps2-keyboard-runs-set-3.md) | rule | Octane: PROM 4.17 and IRIX's `ioc3_pckm` put the PS/2 keyboard in scan code set 3 (Linux/OpenBSD use 2); K_RD slot order, CLAMP_THREE, role by F2 answer; how the PROM's keyboard sequence was traced | | [ip30-promdrive.py](devices/ip30-promdrive.py) | howto | Drives the Octane PROM on its TCP serial console: waits for the maintenance menu, enters the command monitor, runs commands, echoes everything | diff --git a/clanker/devices/ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md b/clanker/devices/ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md new file mode 100644 index 000000000..8f67d8f58 --- /dev/null +++ b/clanker/devices/ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md @@ -0,0 +1,29 @@ +# Octane PROM: loading an 11 MB ELF from the volume header overruns a DMA list + +Status: open (PROM 4.17 as iris runs it, 2026-11-15). Not established whether +real hardware does the same; network boot is the workaround. + +**Symptom.** `boot -f dksc(0,1,8)vmlinux` with a Linux 7.3 kernel of 11.5 MB +(one PT_LOAD of 0xaf3e30 bytes at 0xa800000020004000) prints the size and +then `Read address error exception, bad address: 0x40001f400008` at PROM PC +0x9fc422a4, followed by `PANIC: driver_acquire: io_state.active deadlock` +for ever. The same file loads and runs from `bootp()vmlinux` (iris's NAT +TFTP). The SCSI trace ends with READ(10) of the ELF header (block 8, one +block): the crash comes while the PROM issues the segment's read. + +**What was found.** The PROM polls its SCSI adapters through a list whose +head is at 0xa800000020f17e40; the node for controller 1 at 0x20f0e9a0 had +next = 0x...20f0dfc0, base = 0x900000001f400000 before the boot and read +`0x00000000a0128000, 0x000040001f400000, 0x00000000a012c000, +0x0000400000050000` after it. Those are 16-byte records `(0, PCI address, +0x4000)` whose PCI addresses are the Bridge's direct window (0x8000_0000 + +physical 0x2012_8000, +0x4000, ...): a DMA page list for the 11 MB segment, +one record per 16 KB page, written past its buffer into the adapter node. +No CPU store to the node hit a physical write breakpoint, and iris's cache +model here is the shadow model (no dirty lines to lose), so the list is +written by the PROM (through the fast store path) or by a DMA. + +**Open.** What sizes that list on real hardware, i.e. whether the PROM +splits a large read by something iris answers (INQUIRY, MODE SENSE pages, +the ISP1040 firmware's limits) or real Octanes cannot load a kernel this +large from the volume header either. From 01a2303793d8e55856d722cc72913f58a4220691 Mon Sep 17 00:00:00 2001 From: steigr Date: Mon, 16 Nov 2026 00:31:18 +0100 Subject: [PATCH 3/4] virtio: offer access_platform on pci so linux gives bus addresses Without VIRTIO_F_ACCESS_PLATFORM Linux bypasses the DMA API and writes physical addresses into the queue registers. Behind the Octane's Bridge a physical address below 1 GB is PCI-local space, so the device never read its rings and virtio_console spun for ever on its first control message. The PCI transport now offers the bit (a driver that does not accept it is still taken) and Linux hands over the Bridge's direct-window addresses. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 6 ++++++ docs/dev/virtio-serial.md | 9 +++++++++ src/dev/virtio/mod.rs | 22 +++++++++++++++++----- src/dev/virtio/pci.rs | 28 ++++++++++++++++++++++++++++ 4 files changed, 60 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3f5f5a55c..931aa27b4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,12 @@ hashes are given where a change is easiest to understand by reading the commit. ## November 2026 +### virtio + +- virtio-serial and virtio-9p on PCI (Octane, Tezro) offer + `VIRTIO_F_ACCESS_PLATFORM`, so Linux gives them bus addresses they can reach + instead of physical ones; Linux's `virtio_console` hung at load. + ### Fibre Channel - Linux's qla2xxx brings the ISP2200's loop online and finds its disks on the diff --git a/docs/dev/virtio-serial.md b/docs/dev/virtio-serial.md index 986e3b2b5..ef6420fa3 100644 --- a/docs/dev/virtio-serial.md +++ b/docs/dev/virtio-serial.md @@ -122,6 +122,15 @@ Interrupts are INTx on pin A; MSI-X is not offered. Everything in the BAR is little-endian as the device sees it; the bridge's byte-lane rules apply as for any other function on that machine. +On PCI the device also offers `VIRTIO_F_ACCESS_PLATFORM` (bit 33): ring and +buffer addresses are PCI bus addresses, which go through the host bridge's +DMA translation like any other function's (on the Octane, the Bridge's direct +windows; a physical address below 1 GB is PCI-local space and reaches no +memory). Linux accepts the bit and gives the addresses its DMA mapping +returns; without the bit it hands over physical addresses and the device never +sees its rings. A driver that does not accept the bit is still taken. The same +holds for virtio-9p on PCI. + ## Limits - **No snapshot support yet**: a restored snapshot leaves the device idle and diff --git a/src/dev/virtio/mod.rs b/src/dev/virtio/mod.rs index 8a61bcedf..664a6f4af 100644 --- a/src/dev/virtio/mod.rs +++ b/src/dev/virtio/mod.rs @@ -42,7 +42,7 @@ pub mod p9 { pub mod pci; pub mod queue; -use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::atomic::{AtomicBool, AtomicU64, Ordering}; use std::sync::{Arc, OnceLock}; use std::thread::JoinHandle; use std::time::Duration; @@ -63,6 +63,10 @@ pub const STATUS_FAILED: u8 = 0x80; /// `VIRTIO_F_VERSION_1`: the device is a 1.x device. Bit 32. pub const F_VERSION_1: u64 = 1 << 32; +/// `VIRTIO_F_ACCESS_PLATFORM`: the device reaches memory through the +/// platform's translation, so the driver must give it the addresses the +/// platform's DMA mapping returns rather than physical ones. Bit 33. +pub const F_ACCESS_PLATFORM: u64 = 1 << 33; /// ISR / interrupt-status bits. pub const ISR_QUEUE: u32 = 1; @@ -229,7 +233,9 @@ struct Inner { struct Shared { inner: Mutex, device_id: u32, - device_features: u64, + /// Offered features. Set before the device is attached; a transport + /// may add its own bits ([`VirtioCore::offer`]). + device_features: AtomicU64, mem: OnceLock>, irq: Mutex, /// Serialises "compute the level, drive the line" so two threads cannot @@ -267,7 +273,7 @@ impl VirtioCore { notified: 0, }), device_id, - device_features, + device_features: AtomicU64::new(device_features), mem: OnceLock::new(), irq: Mutex::new(IrqLine::unconnected()), irq_lock: Mutex::new(()), @@ -291,6 +297,12 @@ impl VirtioCore { // ---- identity ---- + /// Offer `bits` as well as the device's own features: a transport's + /// bits, added before any driver reads the features. + pub fn offer(&self, bits: u64) { + self.sh.device_features.fetch_or(bits, Ordering::Relaxed); + } + pub fn device_id(&self) -> u32 { self.sh.device_id } @@ -298,7 +310,7 @@ impl VirtioCore { /// Offered feature bits `32*sel .. 32*sel+32`. pub fn device_features_word(&self, sel: u32) -> u32 { match sel { - 0 | 1 => (self.sh.device_features >> (32 * sel)) as u32, + 0 | 1 => (self.sh.device_features.load(Ordering::Relaxed) >> (32 * sel)) as u32, _ => 0, } } @@ -337,7 +349,7 @@ impl VirtioCore { // offered and the 1.x bit: otherwise the device clears it so the // driver reads it back and gives up (ยง3.1.1). if new & STATUS_FEATURES_OK != 0 && g.status & STATUS_FEATURES_OK == 0 { - let ok = g.driver_features & !self.sh.device_features == 0 && g.driver_features & F_VERSION_1 != 0; + let ok = g.driver_features & !self.sh.device_features.load(Ordering::Relaxed) == 0 && g.driver_features & F_VERSION_1 != 0; if !ok { new &= !STATUS_FEATURES_OK; } diff --git a/src/dev/virtio/pci.rs b/src/dev/virtio/pci.rs index 250ed4c60..61db76ad0 100644 --- a/src/dev/virtio/pci.rs +++ b/src/dev/virtio/pci.rs @@ -88,6 +88,15 @@ impl VirtioPci { .with_capabilities(16, &caps); // Subsystem: vendor and the virtio device ID. pci.set_subsystem_identity(((id as u32) << 16) | VENDOR_ID as u32); + // The function's DMA goes through the host bridge, where a bus + // address need not be the physical one: behind the Octane's Bridge a + // physical address below 1 GB is PCI-local space, and memory is + // reached through the direct windows. A driver that + // knows this flag (Linux) then gives the queue addresses its DMA + // mapping returns; without it Linux hands over physical addresses, + // which no bridge carries to memory. Not required: a driver that + // does not accept it is still taken. + core.offer(super::F_ACCESS_PLATFORM); Self { core, pci: PciFunctionState::new(pci), sel: Default::default() } } @@ -319,6 +328,25 @@ mod tests { assert_eq!(c.read(4), !(BAR_SIZE - 1)); } + /// The PCI transport offers ACCESS_PLATFORM (bit 33) beside VERSION_1, + /// and takes a driver that accepts it as well as one that does not. + #[test] + fn offers_access_platform() { + use crate::dev::virtio::F_ACCESS_PLATFORM; + for accept in [F_VERSION_1 | F_ACCESS_PLATFORM, F_VERSION_1] { + let (dev, ..) = rig(); + let w = |off: u32, v: u32, n: usize| dev.bar_write(0, off, v, n); + let r = |off: u32, n: usize| dev.bar_read(0, off, n); + w(0x00, 1, 4); // device_feature_select = 1 + assert_eq!(r(0x04, 4), ((F_VERSION_1 | F_ACCESS_PLATFORM) >> 32) as u32); + w(0x14, 3, 1); + w(0x08, 1, 4); + w(0x0c, (accept >> 32) as u32, 4); + w(0x14, 0xb, 1); + assert_eq!(r(0x14, 1) & 8, 8, "FEATURES_OK with {accept:#x}"); + } + } + #[test] fn driver_brings_up_queue_and_sends() { let (dev, mem, port, irq) = rig(); From e31d49080bb6b7bb114281e34e1c8c838eb4bd9c Mon Sep 17 00:00:00 2001 From: steigr Date: Mon, 16 Nov 2026 00:47:57 +0100 Subject: [PATCH 4/4] clanker: record what mainline linux needs on the octane Co-Authored-By: Claude Opus 5.5 --- clanker/INDEX.md | 1 + clanker/devices/ip30-linux-7.3-guest.md | 42 +++++++++++++++++++++++++ 2 files changed, 43 insertions(+) create mode 100644 clanker/devices/ip30-linux-7.3-guest.md diff --git a/clanker/INDEX.md b/clanker/INDEX.md index 0f6a2db93..482419afb 100644 --- a/clanker/INDEX.md +++ b/clanker/INDEX.md @@ -147,6 +147,7 @@ Non-graphics devices: MC/MEMCFG and RAM banks, HPC3, VDMA, EEPROMs and NVRAM, PS | [ip30-low-memory-aliases-sdram.md](devices/ip30-low-memory-aliases-sdram.md) | rule | Octane: the 16 KiB at physical 0 is the first 16 KiB of SDRAM, which the processor reaches only there (0x2000_0000..+16 KiB bus-errors, as the stock IDE memory diagnostics expect); shares its JIT page generations; the guard holds on the table path only (the ppmem window maps whole units); DMA reach unknown | | [ip30-prom-stores-into-flash-stay-in-cache.md](devices/ip30-prom-stores-into-flash-stay-in-cache.md) | rule | Octane: the PROM's cached stores into its own flash never reach the Bridge; `CachedFlashLines` holds them (approximation: no write-back) | | [ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md](devices/ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md) | open | Octane: the PROM loading an 11 MB ELF (Linux vmlinux) from the volume header writes its 16 KB-page DMA list over its adapter list and panics; TFTP boot works | +| [ip30-linux-7.3-guest.md](devices/ip30-linux-7.3-guest.md) | howto | Octane: mainline Linux 7.3 config, the PROM/Linux PCI window mismatch and the root-bus scan past device 7 (kernel patch), TFTP boot, no power-off | | [ip30-device-timing-on-hptimer.md](devices/ip30-device-timing-on-hptimer.md) | design | Octane: IOC3, RAD1 and ISP1040 pace themselves on `hptimer` one-shots, with threads only where something blocks (the ISP1040 ring, serial receive, NVRAM writer) | | [ip30-ps2-keyboard-runs-set-3.md](devices/ip30-ps2-keyboard-runs-set-3.md) | rule | Octane: PROM 4.17 and IRIX's `ioc3_pckm` put the PS/2 keyboard in scan code set 3 (Linux/OpenBSD use 2); K_RD slot order, CLAMP_THREE, role by F2 answer; how the PROM's keyboard sequence was traced | | [ip30-promdrive.py](devices/ip30-promdrive.py) | howto | Drives the Octane PROM on its TCP serial console: waits for the maintenance menu, enters the command monitor, runs commands, echoes everything | diff --git a/clanker/devices/ip30-linux-7.3-guest.md b/clanker/devices/ip30-linux-7.3-guest.md new file mode 100644 index 000000000..6b53d0ea4 --- /dev/null +++ b/clanker/devices/ip30-linux-7.3-guest.md @@ -0,0 +1,42 @@ +# Running mainline Linux (7.3) on the Octane + +Status: howto (Linux 7.3.4, PROM 4.17, 2026-11-16). Boots to a shell on two +CPUs with the onboard SCSI, IOC3 serial and Ethernet, the cage's ISP2200 and +NVMe behind 21152s, virtio-serial and virtio-9p. + +**Kernel.** `ip30_defconfig` plus: W1, W1_MASTER_SGI, W1_SLAVE_DS250X (the +Bridge driver defers its probe for ever until it can read each board's +part number from the one-wire NIC); RTC_DRV_DS1685_FAMILY; DEVTMPFS; +`qlogic/1040.bin` built in for qla1280 (it calls request_firmware before any +root exists); qla2xxx wants `ql2200_fw.bin`. The PROM loads the ELF64 +`vmlinux` (load address 0xa800000020004000) from `bootp()` but not from the +volume header (`ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md`). + +**Kernel-side problems, not iris's** (fixed with a patch to +`arch/mips/pci/pci-xtalk-bridge.c`): + +- The driver sets `PCI_PROBE_ONLY` and claims what firmware assigned, and + assumes PCI bus address = small-window offset. The PROM gives BaseIO + windows 2 and up a base 1 MB below their offset (IOC3 BAR 0x500000 through + window 2 at 0x600000) and leaves an option board's Bridge at reset values + with nothing assigned behind its 21152s. Pointing each Device(x) window at + its own offset (except the one the PROM console's IOC3 uses; `prom_putchar` + hard-codes widget 15 + 0x600000) and calling + `pci_assign_unassigned_bus_resources` after the claim makes the cage work. +- The root-bus scan goes to device 31; device 8 of type 0 space is the type 1 + aperture, so with the type 1 register at bus 0 the unnumbered 21152 claims + the cycle and a phantom ISP2200 appears at 00:08.0 (a bridge with + secondary = subordinate = 0 does claim a type 1 cycle for bus 0; iris is + right). Stop the scan at `BRIDGE_DEV_CNT`. + +**iris fixes it took:** ISP2200 Login Loop Port and +request-ring mailbox entries +(`scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md`); virtio-pci +offering ACCESS_PLATFORM; virtio-serial off slot 4's interrupt pin +(`virtio-serial-design.md`, a separate change). + +**Other behaviour to expect.** No software power-off (`poweroff` halts; the +DS1685 driver powers down only on the power button). The 21152's I/O window +lands at bus 0x200000, which the CPU cannot reach (all Bridge windows are +memory space); qla2xxx runs on the memory BAR. ext4 sends WRITE SAME(10), which the SCSI disk refuses +(noted, harmless). `make depend` of OpenSSL 1.0.2 takes hours without the JIT.