Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,16 @@ hashes are given where a change is easiest to understand by reading the commit.

## November 2026

### virtio

- virtio-serial and virtio-9p on PCI (Octane, Tezro) offer
`VIRTIO_F_ACCESS_PLATFORM`, so Linux gives them bus addresses they can reach
instead of physical ones; Linux's `virtio_console` hung at load.

### Fibre Channel

- Linux's qla2xxx brings the ISP2200's loop online and finds its disks on the
Octane (Linux 7.3); it stopped at the port login, which the adapter refused.
### Octane

- The Bridges' 64 GB windows reach PCI: the 32-bit and 64-bit aliases of PCI
Expand Down
3 changes: 3 additions & 0 deletions clanker/INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,8 @@ Non-graphics devices: MC/MEMCFG and RAM banks, HPC3, VDMA, EEPROMs and NVRAM, PS
| [ip30-module-name-must-match-active-cpus.md](devices/ip30-module-name-must-match-active-cpus.md) | rule | Octane: the module record must say PM10 for one CPU and PM20 for two, or the PROM fails its NIC diagnostic |
| [ip30-low-memory-aliases-sdram.md](devices/ip30-low-memory-aliases-sdram.md) | rule | Octane: the 16 KiB at physical 0 is the first 16 KiB of SDRAM, which the processor reaches only there (0x2000_0000..+16 KiB bus-errors, as the stock IDE memory diagnostics expect); shares its JIT page generations; the guard holds on the table path only (the ppmem window maps whole units); DMA reach unknown |
| [ip30-prom-stores-into-flash-stay-in-cache.md](devices/ip30-prom-stores-into-flash-stay-in-cache.md) | rule | Octane: the PROM's cached stores into its own flash never reach the Bridge; `CachedFlashLines` holds them (approximation: no write-back) |
| [ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md](devices/ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md) | open | Octane: the PROM loading an 11 MB ELF (Linux vmlinux) from the volume header writes its 16 KB-page DMA list over its adapter list and panics; TFTP boot works |
| [ip30-linux-7.3-guest.md](devices/ip30-linux-7.3-guest.md) | howto | Octane: mainline Linux 7.3 config, the PROM/Linux PCI window mismatch and the root-bus scan past device 7 (kernel patch), TFTP boot, no power-off |
| [ip30-device-timing-on-hptimer.md](devices/ip30-device-timing-on-hptimer.md) | design | Octane: IOC3, RAD1 and ISP1040 pace themselves on `hptimer` one-shots, with threads only where something blocks (the ISP1040 ring, serial receive, NVRAM writer) |
| [ip30-openbsd-needs-64gb-window-and-slot-4-empty.md](devices/ip30-openbsd-needs-64gb-window-and-slot-4-empty.md) | rule | Octane: OpenBSD 6.9 reaches PCI through the Bridges' 64 GB windows (IOC3 at 0xF0_4050_0000) and takes the SuperIO interrupt from the first empty Bridge slot, so virtio goes in slots 6 and 7 (`bridge_slot`); what it attaches |
| [ip30-ps2-keyboard-runs-set-3.md](devices/ip30-ps2-keyboard-runs-set-3.md) | rule | Octane: PROM 4.17 and IRIX's `ioc3_pckm` put the PS/2 keyboard in scan code set 3 (Linux/OpenBSD use 2); K_RD slot order, CLAMP_THREE, role by F2 answer; how the PROM's keyboard sequence was traced |
Expand All @@ -172,6 +174,7 @@ WD33C93A, SCSI targets (disk, CD-ROM, DaynaPort) and guest SCSI drivers.
|---|---|---|
| [a-write-cdb-must-outlive-its-data-request.md](scsi/a-write-cdb-must-outlive-its-data-request.md) | rule | A write CDB must outlive the chip's request for its data |
| [daynaport-target-gotchas.md](scsi/daynaport-target-gotchas.md) | rule | DaynaPort target: the three things that bite |
| [isp2200-linux-logs-in-through-ring-mailbox-entries.md](scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md) | rule | Linux qla2xxx logs in to ISP2200 loop ports with 0x0074 and with type 0x39 mailbox entries in the request ring; both must be answered or no FC disk appears |
| [linux-lun-scan-and-mode-sense-gaps.md](scsi/linux-lun-scan-and-mode-sense-gaps.md) | rule | Linux SCSI probe noise: phantom LUNs, missing Caching page, no MODE SENSE(10) |
| [miniroot-install-hang-scsi0-dma-irq-storm.md](scsi/miniroot-install-hang-scsi0-dma-irq-storm.md) | rule | IRIX miniroot install hangs on iris: SCSI0_DMA IRQ storm (FIXED) |
| [windows-cannot-fsync-a-directory.md](scsi/windows-cannot-fsync-a-directory.md) | rule | Windows cannot fsync a directory: raw COW overlays failed every sidecar rewrite |
Expand Down
42 changes: 42 additions & 0 deletions clanker/devices/ip30-linux-7.3-guest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Running mainline Linux (7.3) on the Octane

Status: howto (Linux 7.3.4, PROM 4.17, 2026-11-16). Boots to a shell on two
CPUs with the onboard SCSI, IOC3 serial and Ethernet, the cage's ISP2200 and
NVMe behind 21152s, virtio-serial and virtio-9p.

**Kernel.** `ip30_defconfig` plus: W1, W1_MASTER_SGI, W1_SLAVE_DS250X (the
Bridge driver defers its probe for ever until it can read each board's
part number from the one-wire NIC); RTC_DRV_DS1685_FAMILY; DEVTMPFS;
`qlogic/1040.bin` built in for qla1280 (it calls request_firmware before any
root exists); qla2xxx wants `ql2200_fw.bin`. The PROM loads the ELF64
`vmlinux` (load address 0xa800000020004000) from `bootp()` but not from the
volume header (`ip30-prom-disk-load-of-a-large-elf-overruns-a-dma-list.md`).

**Kernel-side problems, not iris's** (fixed with a patch to
`arch/mips/pci/pci-xtalk-bridge.c`):

- The driver sets `PCI_PROBE_ONLY` and claims what firmware assigned, and
assumes PCI bus address = small-window offset. The PROM gives BaseIO
windows 2 and up a base 1 MB below their offset (IOC3 BAR 0x500000 through
window 2 at 0x600000) and leaves an option board's Bridge at reset values
with nothing assigned behind its 21152s. Pointing each Device(x) window at
its own offset (except the one the PROM console's IOC3 uses; `prom_putchar`
hard-codes widget 15 + 0x600000) and calling
`pci_assign_unassigned_bus_resources` after the claim makes the cage work.
- The root-bus scan goes to device 31; device 8 of type 0 space is the type 1
aperture, so with the type 1 register at bus 0 the unnumbered 21152 claims
the cycle and a phantom ISP2200 appears at 00:08.0 (a bridge with
secondary = subordinate = 0 does claim a type 1 cycle for bus 0; iris is
right). Stop the scan at `BRIDGE_DEV_CNT`.

**iris fixes it took:** ISP2200 Login Loop Port and
request-ring mailbox entries
(`scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md`); virtio-pci
offering ACCESS_PLATFORM; virtio-serial off slot 4's interrupt pin
(`virtio-serial-design.md`, a separate change).

**Other behaviour to expect.** No software power-off (`poweroff` halts; the
DS1685 driver powers down only on the power button). The 21152's I/O window
lands at bus 0x200000, which the CPU cannot reach (all Bridge windows are
memory space); qla2xxx runs on the memory BAR. ext4 sends WRITE SAME(10), which the SCSI disk refuses
(noted, harmless). `make depend` of OpenSSL 1.0.2 takes hours without the JIT.
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Octane PROM: loading an 11 MB ELF from the volume header overruns a DMA list

Status: open (PROM 4.17 as iris runs it, 2026-11-15). Not established whether
real hardware does the same; network boot is the workaround.

**Symptom.** `boot -f dksc(0,1,8)vmlinux` with a Linux 7.3 kernel of 11.5 MB
(one PT_LOAD of 0xaf3e30 bytes at 0xa800000020004000) prints the size and
then `Read address error exception, bad address: 0x40001f400008` at PROM PC
0x9fc422a4, followed by `PANIC: driver_acquire: io_state.active deadlock`
for ever. The same file loads and runs from `bootp()vmlinux` (iris's NAT
TFTP). The SCSI trace ends with READ(10) of the ELF header (block 8, one
block): the crash comes while the PROM issues the segment's read.

**What was found.** The PROM polls its SCSI adapters through a list whose
head is at 0xa800000020f17e40; the node for controller 1 at 0x20f0e9a0 had
next = 0x...20f0dfc0, base = 0x900000001f400000 before the boot and read
`0x00000000a0128000, 0x000040001f400000, 0x00000000a012c000,
0x0000400000050000` after it. Those are 16-byte records `(0, PCI address,
0x4000)` whose PCI addresses are the Bridge's direct window (0x8000_0000 +
physical 0x2012_8000, +0x4000, ...): a DMA page list for the 11 MB segment,
one record per 16 KB page, written past its buffer into the adapter node.
No CPU store to the node hit a physical write breakpoint, and iris's cache
model here is the shadow model (no dirty lines to lose), so the list is
written by the PROM (through the fast store path) or by a DMA.

**Open.** What sizes that list on real hardware, i.e. whether the PROM
splits a large read by something iris answers (INQUIRY, MODE SENSE pages,
the ISP1040 firmware's limits) or real Octanes cannot load a kernel this
large from the volume header either.
27 changes: 27 additions & 0 deletions clanker/scsi/isp2200-linux-logs-in-through-ring-mailbox-entries.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Linux's qla2xxx logs in to ISP2200 loop ports through request-ring mailbox entries

Status: rule (Linux 7.3.4 qla2xxx 12.00.00, ISP2200 on the Octane's PCI cage, 2026-11-15).

**Symptom.** qla2xxx loads, the loop comes up ("LOOP UP detected"), and no
disk ever appears. `scsi2 notes` shows `loop command 0x0074 not modelled`.

**Cause.** On the parts before the 24xx (not FWI2-capable) the driver logs in
to each loop port two ways: synchronously with mailbox command 0x0074 (Login
Loop Port, `qla2x00_login_local_device`), and from its async discovery state
machine with a *mailbox entry* in the request ring, entry type 0x39
(`MBX_IOCB_TYPE`, `struct mbx_entry` in `qla_def.h`), built by
`qla2x00_login_iocb` (m0 = 0x006f), `qla2x00_logout_iocb` (0x0071) and
`qla2x00_adisc_iocb` (0x0064 into `async_pd_dma`). The firmware answers in the
response ring with an entry of the same type: handle copied, `entry_status`
and `status` 0, the output mailboxes at the same offsets
(`qla2x00_mbx_iocb_entry`). Both paths were refused, so no port was ever
logged in. IRIX and the BSDs never send either, which is why IRIX 6.5.30
worked.

**Fix.** `Isp2200::mailbox_entry` serves the four commands Linux puts in an
entry and answers anything else there with 0x4001; 0x0074 answers COMPLETE
for a present port. Test: `isp2200::tests::loop_tests::port_login`.

Layout of `struct mbx_entry` (little-endian): type 0, count 1, handle 4,
loop ID 8 (the high byte on parts without extended IDs), status 10, m0 24,
m1 26, m2 28, m3 30, m6 32, m7 34, m9 36, m10 38.
9 changes: 9 additions & 0 deletions docs/dev/virtio-serial.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,15 @@ not offered. Everything in the BAR is
little-endian as the device sees it; the bridge's byte-lane rules apply as for
any other function on that machine.

On PCI the device also offers `VIRTIO_F_ACCESS_PLATFORM` (bit 33): ring and
buffer addresses are PCI bus addresses, which go through the host bridge's
DMA translation like any other function's (on the Octane, the Bridge's direct
windows; a physical address below 1 GB is PCI-local space and reaches no
memory). Linux accepts the bit and gives the addresses its DMA mapping
returns; without the bit it hands over physical addresses and the device never
sees its rings. A driver that does not accept the bit is still taken. The same
holds for virtio-9p on PCI.

## Limits

- **No snapshot support yet**: a restored snapshot leaves the device idle and
Expand Down
8 changes: 5 additions & 3 deletions docs/fibre-channel.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,11 @@ and the comment in `src/dev/isp2200.rs` says so.

## Limits

- Only IRIX 6.5.30 has been tried. Linux logs in through request-ring mailbox
entries the model does not serve, so mainline Linux cannot bring a port
online.
- IRIX 6.5.30 and Linux 7.3 (qla2xxx, with the stock `ql2200_fw.bin`) have
been tried. Linux logs in to each port with Login Loop Port and with
mailbox commands it puts in the request ring (login, logout and the port
database for ADISC); the model answers those four in a ring entry and
refuses any other command sent that way.
- Commands run one at a time. An abort or reset matches the command running,
not the ones queued behind it.
- A disk attached or removed after start is not announced; there is no loop
Expand Down
71 changes: 66 additions & 5 deletions src/dev/isp2200.rs
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,9 @@ pub mod mbox {
pub const GET_LINK_STATUS: u16 = 0x006b;
pub const LIP_RESET: u16 = 0x006c;
pub const FULL_LOGIN_LIP: u16 = 0x0072;
pub const LOGIN_FABRIC_PORT: u16 = 0x006f;
pub const LOGOUT_FABRIC_PORT: u16 = 0x0071;
pub const LOGIN_LOOP_PORT: u16 = 0x0074;
pub const PORT_NAME_LIST: u16 = 0x0075;
pub const GET_ID_LIST: u16 = 0x007c;
pub const LUN_RESET: u16 = 0x007e;
Expand Down Expand Up @@ -223,6 +226,9 @@ const ENTRY_TYPE_CONTINUATION_A64: u8 = 0x0a;
const ENTRY_TYPE_COMMAND: u8 = 0x11;
const ENTRY_TYPE_COMMAND_LIST: u8 = 0x15;
const ENTRY_TYPE_COMMAND_A64: u8 = 0x19;
/// A mailbox command carried in the request ring (Linux's `MBX_IOCB_TYPE`),
/// answered by an entry of the same type in the response ring.
const ENTRY_TYPE_MAILBOX: u8 = 0x39;

/// Entry status flags a response entry can carry.
const ENTRY_FLAG_BAD_TYPE: u8 = 0x04;
Expand Down Expand Up @@ -1741,6 +1747,7 @@ impl Core {
}
// A marker resynchronises firmware state; nothing is posted.
ENTRY_TYPE_MARKER => {}
ENTRY_TYPE_MAILBOX => self.mailbox_entry(st, &req),
ENTRY_TYPE_CONTINUATION | ENTRY_TYPE_CONTINUATION_A64 => st.reject_entry(&req, ENTRY_FLAG_BAD_ORDER),
_ => st.reject_entry(&req, ENTRY_FLAG_BAD_TYPE),
}
Expand All @@ -1752,6 +1759,48 @@ impl Core {
true
}

/// A mailbox command in the request ring (Linux's `struct mbx_entry`:
/// handle at 4, loop ID at 8, status at 10, then m0, m1, m2, m3, m6, m7,
/// m9 and m10 as words from 24). Linux 7.3 sends its port logins,
/// logouts and ADISC this way on the parts before the 24xx
/// (qla2x00_login_iocb, qla2x00_logout_iocb, qla2x00_adisc_iocb). The
/// answer is the entry itself with the output mailboxes in the same
/// places and status 0; a command the ring does not carry comes back
/// with an invalid-command m0. Which commands the firmware takes in an
/// entry is not public: these are the ones Linux sends, and fabric
/// login is answered as a loop port login, since the loop has no fabric.
fn mailbox_entry(&self, st: &mut Locked<'_>, req: &[u8; ENTRY_SIZE]) {
const SLOTS: [(usize, usize); 8] = [(0, 24), (1, 26), (2, 28), (3, 30), (6, 32), (7, 34), (9, 36), (10, 38)];
let mut input: Mailboxes = [0; MAILBOX_COUNT];
for (m, off) in SLOTS {
input[m] = le16(req, off);
}
let mut out: Mailboxes = [0; MAILBOX_COUNT];
out[0] = match input[0] {
mbox::LOGIN_FABRIC_PORT | mbox::LOGIN_LOOP_PORT => {
if !st.loop_ready() || !st.target(usize::from(input[1] >> 8)) {
mbox::COMMAND_ERROR
} else {
mbox::COMPLETE
}
}
// Nothing is logged out: every port stays logged in.
mbox::LOGOUT_FABRIC_PORT => mbox::COMPLETE,
mbox::GET_PORT_DB => self.get_port_db(st, &input),
cmd => {
self.note(format!("mailbox command {cmd:#06x} in a request entry not modelled"));
mbox::INVALID
}
};
let mut entry = *req;
entry[ENTRY_OFF_STATUS] = 0;
entry[10..16].fill(0);
for (m, off) in SLOTS {
entry[off..off + 2].copy_from_slice(&out[m].to_le_bytes());
}
st.held.push_back(entry);
}

/// A command entry and the continuations its entry count claims. The
/// count, not each entry's type, says how many slots the command takes:
/// a 64-bit continuation has no handle, only segment data from byte 4.
Expand Down Expand Up @@ -1832,8 +1881,8 @@ impl Core {

/// The mailbox commands the loop answers: those it implements, then
/// those it knows of and refuses (0x0040, 0x0041 and 0x0047, whose use is
/// not established here, and the name-server and login commands 0x006e,
/// 0x006f, 0x0071 and 0x0074), which `fc_mailbox` notes and fails.
/// not established here, and the name-server and fabric login commands
/// 0x006e, 0x006f and 0x0071), which `fc_mailbox` notes and fails.
fn fc_owned(cmd: u16) -> bool {
matches!(
cmd,
Expand All @@ -1859,9 +1908,9 @@ impl Core {
| 0x0041
| 0x0047
| 0x006e
| 0x006f
| 0x0071
| 0x0074
| mbox::LOGIN_FABRIC_PORT
| mbox::LOGOUT_FABRIC_PORT
| mbox::LOGIN_LOOP_PORT
)
}

Expand Down Expand Up @@ -2072,6 +2121,18 @@ impl Core {
self.start_bringup(st, false);
mbox::COMPLETE
}
// Login to a port on the loop: every disk is logged in already, so
// the login succeeds for any port present; m1 bit 0 (initiator)
// and bit 1 (FCP-2 device) stay clear, the FCP target role the port
// database gives. A loop ID with no port is the command error
// Linux reads as a failed login (qla2x00_local_device_login).
mbox::LOGIN_LOOP_PORT => {
if !st.loop_ready() || !st.target(usize::from(input[1] >> 8)) {
return mbox::COMMAND_ERROR;
}
out[1] = 0;
mbox::COMPLETE
}
// The control block's retry count and delay, the order Linux and
// NetBSD read them in; that the firmware echoes the bytes rather
// than keeping values of its own is inferred. IRIX 6.5.30 scans
Expand Down
52 changes: 52 additions & 0 deletions src/dev/isp2200/tests/loop_tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -389,3 +389,55 @@ fn port_name_list_retains_m7() {
assert_eq!(r.rd(mb(1)), 20);
r.ack();
}

/// Linux's port login: Login Loop Port as a mailbox command, and the
/// mailbox commands it carries in request entries (login, logout and the
/// port database ADISC reads), each answered in an entry of the same type.
#[test]
fn port_login() {
let r = Rig::with_disks("login", &[5]);
r.loop_up(ICB_OPT_PORT_DB, 0);
assert_eq!(r.call(mbox::LOGIN_LOOP_PORT, 5 << 8, 1, 0), mbox::COMPLETE);
assert_eq!(r.rd(mb(1)), 0);
r.ack();
assert_eq!(r.call(mbox::LOGIN_LOOP_PORT, 6 << 8, 1, 0), mbox::COMMAND_ERROR);
r.ack();

let entry = |handle: u32, mbx: &[(usize, u16)]| {
let mut e = [0u8; ENTRY_SIZE];
e[0] = ENTRY_TYPE_MAILBOX;
e[1] = 1;
p32(&mut e, 4, handle);
e[9] = 5;
for &(off, v) in mbx {
p16(&mut e, off, v);
}
e
};
let answer = |r: &Rig| {
r.settle();
let e = r.response(r.rs_next.get());
r.consume(1);
e
};
// Login (fabric login of a loop port, as qla2x00_login_iocb sends it).
r.publish(&entry(0x1234, &[(24, mbox::LOGIN_FABRIC_PORT), (26, 5 << 8), (30, 0x00e0)]));
let e = answer(&r);
assert_eq!((e[0], e[3], g32(&e, 4), g16(&e, 10)), (ENTRY_TYPE_MAILBOX, 0, 0x1234, 0));
assert_eq!((g16(&e, 24), g16(&e, 26)), (mbox::COMPLETE, 0));
// No port at loop ID 6.
r.publish(&entry(0x1235, &[(24, mbox::LOGIN_FABRIC_PORT), (26, 6 << 8)]));
assert_eq!(g16(&answer(&r), 24), mbox::COMMAND_ERROR);
// ADISC: the port database to m2:m3 (m6:m7 the high words).
r.mem.put(0x9100, &[0xa5; 128]);
r.publish(&entry(0x1236, &[(24, mbox::GET_PORT_DB), (26, (5 << 8) | 1), (30, 0x9100)]));
let e = answer(&r);
assert_eq!((g32(&e, 4), g16(&e, 24)), (0x1236, mbox::COMPLETE));
let pd = r.mem.get(0x9100, 128);
assert_eq!(pd[PDB_OFF_PORT_NAME..PDB_OFF_PORT_NAME + 8], port_wwn(5).to_be_bytes());
// Logout completes; a command the ring does not carry is invalid.
r.publish(&entry(0x1237, &[(24, mbox::LOGOUT_FABRIC_PORT), (26, 5 << 8)]));
assert_eq!(g16(&answer(&r), 24), mbox::COMPLETE);
r.publish(&entry(0x1238, &[(24, mbox::GET_RESOURCE_COUNTS)]));
assert_eq!(g16(&answer(&r), 24), mbox::INVALID);
}
Loading
Loading