sv6160: fix the lifecycle and accounting bugs a reload harness never hits

A seven-pass review of this branch (testing, maintainability, security,
performance, simplification, adversarial, red team) found 13 crash-, deadlock-
or leak-class defects.  Every one was verified by reading the code on both
sides of the defect before it was touched.

They share a reason for surviving the 8 h soak: the harness recovers by
rmmod/insmod, so it never exercises ifdown/ifup, mac_stop, mac_flush,
remove(), or the restart path's own state.  Those are exactly where these live.

Use-after-free and ordering:

  - remove() called ieee80211_free_hw() while the threaded host-wake IRQ was
    still registered.  devm frees that IRQ only AFTER remove() returns, i.e.
    after the softc is gone, and rung 2 of the recovery ladder runs remove()
    precisely when the level line is stuck high with the thread mid-drain.
    Free both IRQs and flush rx_work first, the way the probe error path
    already did; the probe path now flushes rx_work too.

  - is_firmware_loaded went true inside boot_firmware, which probe calls
    BEFORE core_init allocates rx_buf and INIT_WORKs rx_work -- with both IRQ
    paths already registered and gating on that flag.  A host-wake in that
    window drained into a NULL rx_buf.  It is now raised in core_init once the
    drain has somewhere to land, and re-raised by fw_reboot on success.

  - The wt_trace WT53 RX line read ((struct rpu_msg_hdr *)skb->data)->id ten
    lines before skb was allocated: uninitialised stack on chain link 0, and
    on later links the previous link's skb, already consumed by
    process_event().  Read the id from sc->rx_buf, which is what was just
    filled.

  - tx_pending[slot] was published before sv6160_sdio_write_tx() read
    bodies[i]->data, and tx_retire_all() took no lock, so the resync could
    claim the descriptor and hand every frame in it to mac80211 while the
    CMD53 was still copying from them.  The worker now marks tx_writing[id]
    before publishing and clears it when the write returns; the reclaim path
    skips a descriptor that is being written.

Deadlock:

  - The CMD53 writers guarded recursion with current_work() != &sc->rx_work.
    current_work() is NULL in a threaded IRQ, which is where the drain runs
    since it moved there, so a write reached from inside the drain (the
    cmd_gate kick on COMMAND_PROC_DONE) re-took the mutex it already held and
    the IRQ thread slept on itself for good.  cmd_gate defaults off, which is
    the only reason this never fired.  sc->lock now records its owner task and
    the guard compares against current.

Accounting that never recovers:

  - mac_stop and mac_flush(drop) zeroed tx_token_bmp but never
    tx_outstanding_ac[], whose only decrement is gated on a bit they had just
    cleared.  Any AC with descriptors in flight at stop time kept its count
    for the module's life: get_token refuses it, the worker stops every queue,
    and tx_resync cannot help because !tx_token_bmp returns early.  They also
    never freed the batched companions in tx_batch[][], and stored NULL over
    tx_pending[] with plain stores that raced the cmpxchg completion path.
    One sv6160_tx_drop_all() now does all of it under the token lock, and
    mac_flush cancels tx_work first like its siblings.

  - tx_outstanding_ac[] is a plain u8 incremented in the TX worker and
    decremented from the IRQ thread and the resync work.  A lost update is
    permanent and is exactly the "<-- GATED" state the debugfs ledger was
    written to diagnose.  Added tx_token_lock around the pool.

  - tx_outstanding was added AFTER the slot was published, so a completion
    claiming it in between decremented what was never added and drifted the
    counter negative for the rest of the session.  Count first, undo on the
    write-failure path.

  - RPU_EVENT_HOST_DATA_TX_DONE (id 32, which the firmware does emit per the
    comment at its own case label) bulk-completed with plain stores, never
    freed the tokens, and ignored tx_batch[][].  If it ever arrived it gated
    every AC for good.  Routed through the same claim path and counted as
    tx_bulk_done in debugfs, because whether this firmware emits it is the
    thing worth knowing next time TX dies.

Firmware-supplied lengths:

  - An RX_SERIAS sub-packet shorter than struct rpu_msg_hdr (28) had its
    hdr->length read AND rewritten past the aggregate's tail, which has 0-63
    bytes of tailroom and skb_shared_info behind it.  Same for a top-level
    RECV_LEN of 1..27.  Both are now rejected, and process_event() drops a
    runt rather than reading an event id out of allocation slack.

  - Any early exit from the sub-packet loop left serias_next_len holding a
    sub-packet's in-aggregate "next", which the drain then read as a chain
    link the firmware never queued: the -110 that wedges the controller.  The
    chain is stopped instead.

  - The OOM path set gpio_irq_pending via trigger_rx and then cleared it at
    clear_and_done, so the rescheduled drain returned at its own gate and the
    aggregate was stranded.

Recovery ladder semantics:

  - ieee80211_restart_hw() re-enters .start WITHOUT calling .stop --
    ieee80211_reconfig() calls drv_start only.  Two comments here claimed the
    opposite, and the consequence was that is_rpu_ready stayed true so the TX
    worker kept writing descriptors to SDIO address 0, the address a WAIT_FW
    MCU takes image bytes at, while fw_reboot re-uploaded the firmware there.
    The resync kept firing, the command gate and fwpeek kept sending.  That is
    very likely the "restart_hw made things worse (4->8 resyncs, 6->13 lost)"
    recorded in KNOWN-ISSUES.  mac_start now quiesces its own state first.

  - The 60 s rung-2 counter counted every trigger_recovery() call, including
    the tx_resync_giveup escalation and a debugfs level-1 write.  Three
    TX-completion link deaths inside a minute -- a mode whose median gap is
    28 s -- therefore re-created the dw_mmc host instead of the restart_hw
    that was measured to fix that failure, conflating the two failure modes
    the rest of this driver's notes insist on keeping apart.  Only sdio_wedged
    faults count now.

  - mac_start refused with -EIO when is_firmware_loaded was false, which is
    the state an exhausted bring-up leaves, so the interface could never come
    back without rmmod.  fw_bringup re-uploads the firmware itself; the gate
    is gone.

  - recover_work aborts a hardware scan before restart_hw (mac80211 WARNs
    otherwise), and hw_scan now fails cleanly instead of scheduling a sweep
    fallback that could not run: rpu_scan() had already dropped scan_request,
    so scan_work bailed without completing anything and mac80211 stayed in
    SCAN_HW_SCANNING until a module reload -- no scan, no roam.

mac80211 contracts:

  - tx_status_skb() was mixed with tx_status_irqsafe() on one hw, which
    mac80211.h forbids.  All completions are irqsafe now.
  - Frames delivered after .stop hit WARN_ON(!local->started) per frame.
  - connection_loss() is STA-only; on an AP/IBSS vif it scribbles the u.mgd
    union.
  - RX BA sessions were programmed against vif->bss_conf.bssid, which is our
    own address in AP/IBSS mode.  Use the STA's.
  - remain_on_channel never called ieee80211_ready_on_channel(), so every ROC
    expired without the off-channel work ever starting.
  - The PMF branch set encrypt=1 with a hardcoded iv_len=8 without consulting
    hw_key, so a frame mac80211 had already encrypted in software would have
    been encrypted again.

Debug surface, before anyone else reads this tree:

  - wiredump and wt_trace>=2 hex-dumped RPU_CMD_SETKEY, i.e. the PTK/GTK, and
    the WBODY/RXDATA/RXTAIL dumps carried plaintext payload.  Keys are
    redacted and the frame dumps stop after the header plus IV.
  - fwpoke/fwpoke_when are 0400: they write firmware memory.  The charp is
    read under kernel_param_lock().
  - fw_peek and fw_addr_ok compared addr + len, which wraps; addr=0xFFFFFFFC
    len=4 passed both.
  - chan_settle and sweep_len are clamped (a negative chan_settle was a 49-day
    msleep under the wiphy lock; a negative sweep_len was ~2^30 iterations of
    msleep(60) that cancel_delayed_work_sync would then wait for).  Negative
    tx_stall_ms, host_rebind and tx_resync_giveup mean off rather than "fire
    on the first event".
  - The debugfs recover knob returns -EINVAL/-EFAULT rather than silently
    running the shallow rung, and -EBUSY when the deep one is a no-op.
  - The TX bounce buffer was 3 bytes short of what write_tx could put in it.
  - The beacon RXCH line is dev_dbg: it was one printk per received beacon.

Behavioural, applied as experiments -- A/B these against a57846c before
trusting either direction (see KNOWN-ISSUES section 5):

  - The -EBUSY back-off now drops sc->lock, not just the host claim.  The
    comment claimed releasing the claim "lets the RX drain interleave", but
    the drain takes sc->lock BEFORE the claim, so for up to 16 x 25 ms the
    chip was held busy by the very frames it was waiting for us to collect,
    and the tries ran out into a declared wedge.  Candidate mechanism for the
    open wedge mode.
  - IRQF_NO_SUSPEND is gone from the host-wake IRQ (it is documented as
    incompatible with the enable_irq_wake() next to it) and the thread refuses
    to drain while is_suspended, where it was issuing CMD53s to a host being
    clock-gated.  Suspend needs re-verifying.

Dead state removed: gpio_irq_disabled and the three enable_irq blocks that
tested it (nothing set it since the IRQ went threaded), rx_queue, need_reinit,
last_drain_jiffies, dwmmc_regs, PS_MODE_*.

Cross-compiles clean for aarch64, zero warnings.  NOT yet soaked: the
lifecycle fixes want an ifdown/ifup cycle in the harness, which is the gap
that hid them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JSNniou44nN9EFGqTmZ6my
This commit is contained in:
Paul KilarandClaude Opus 5 committed 2026-09-18 13:59:03 -04:00
1 parent 09fd7881db
commit 9de36d7731
3 files changed
+497 -270

No files matched your search

+20 -16
View File
@@ -246,11 +246,6 @@ enum rpu_event_tag {
#define SV6160_MAX_TX_RATES 4
#define MARK_RATE_AS_MCS 0x80
/* Power save modes */
#define PS_MODE_DISABLED 0
#define PS_MODE_ENABLED 1
#define PS_MODE_DEEP 2
/* VIF control operations */
#define VIF_ADD 1
#define VIF_REMOVE 2
@@ -881,6 +876,11 @@ struct sv6160_softc {
struct ieee80211_hw *hw;
struct device *dev;
struct mutex lock;
/* Task holding sc->lock through sv6160_lock()/the RX drain. A path
* reached from inside the drain (either entry: IRQ thread or rx_work)
* compares against current and skips the mutex instead of deadlocking
* on it -- the old current_work() test was NULL in the IRQ thread. */
struct task_struct *lock_owner;
bool is_firmware_loaded;
bool is_started;
bool is_rpu_ready;
@@ -889,7 +889,6 @@ struct sv6160_softc {
struct work_struct recover_work; /* runtime SDIO-wedge recovery */
struct work_struct rx_work;
struct delayed_work rx_poll;
struct sk_buff_head rx_queue;
struct sv6160_coalescing coalescing;
struct workqueue_struct *wq;
@@ -979,8 +978,6 @@ struct sv6160_softc {
u32 txs_lost_desc[SV6160_NUM_TX_DESCS];
u32 txs_lost_ac[SV6160_NUM_ACS];
u32 txs_resync_age_max; /* ms, oldest ever lost */
/* Times the worker stopped ALL mac80211 queues because one AC had no
* descriptor left, and which AC caused it. */
/* Rate asked for vs rate the firmware reports using. */
u32 txs_ask_mcs[16];
u32 txs_ask_legacy;
@@ -989,6 +986,8 @@ struct sv6160_softc {
u32 txs_use_none;
u32 txs_rate_match;
u32 txs_rate_nomatch;
/* Times the worker stopped ALL mac80211 queues because one AC had no
* descriptor left, and which AC caused it. */
u32 txs_qstop;
u32 txs_qstop_ac[SV6160_NUM_ACS];
/* How long a genuine TX_DONE takes, and how many arrive for a descriptor
@@ -1033,6 +1032,17 @@ struct sv6160_softc {
* for (spare tokens are not AC-derivable from the id alone). */
u8 tx_outstanding_ac[SV6160_NUM_ACS];
u8 tx_token_ac[SV6160_NUM_TX_DESCS];
/* Guards tx_token_bmp / tx_token_ac / tx_outstanding_ac / tx_quarantine.
* get_token runs in the TX worker, free_token from the IRQ-thread drain
* and the resync work, and a lost ++/-- on a per-AC count is permanent:
* the AC either sits at its limit with nothing in flight (get_token
* refuses it for good) or admits more than the firmware tolerates. */
spinlock_t tx_token_lock;
/* Set by the worker just before it publishes a descriptor, cleared when
* sv6160_sdio_write_tx() returns. A reclaim path that sees it leaves
* that descriptor alone: the CMD53 is still reading those skbs. */
bool tx_writing[SV6160_NUM_TX_DESCS];
u32 txs_bulk_done; /* event-32 bulk completions seen */
/* Control-command flow control, mirroring the vendor's rpu_send()
* (rk915 src/rpu_if.c:755): never put more than
@@ -1138,13 +1148,9 @@ struct sv6160_softc {
u64 txs_retry_n;
u32 txs_tx_failed; /* frm_status != SUCCESS */
/* DW MMC controller registers (for debug diagnostics) */
void __iomem *dwmmc_regs;
/* GPIO interrupt */
int host_wake_irq;
bool gpio_irq_pending; /* GPIO fired, data ready */
bool gpio_irq_disabled; /* level IRQ disabled by handler */
u32 gpio_irq_count; /* total GPIO IRQs received */
/* debugfs */
@@ -1152,7 +1158,6 @@ struct sv6160_softc {
/* Smart RECV_LEN polling — detect stale vs new data without GPIO */
u16 last_recv_len; /* RECV_LEN of the last drained aggregate */
unsigned long last_drain_jiffies;/* when we last drained real RX data */
/* After an RX_SERIAS batch the firmware re-presents the batch's FIRST
* sub-packet length on a later lone host-wake, with no data behind it.
* Measured twice: a 352B batch of (204,28,28,28) was followed ~690ms
@@ -1160,7 +1165,7 @@ struct sv6160_softc {
* by one advertising 408. Reading the absent bytes times out and wedges
* the bus. last_recv_len cannot catch this -- it holds the aggregate,
* not the sub-packet -- so remember the one length that actually gets
* re-presented, and when. */
* re-presented. */
bool recv_len_failed; /* last CMD53 at this RECV_LEN timed out */
@@ -1217,7 +1222,6 @@ struct sv6160_softc {
bool scan_in_progress;
bool operating; /* a BSSID is programmed (connecting/associated) — hold the operating channel, don't sweep off it */
bool assoc; /* association completed (STA_CONN sent) */
bool need_reinit; /* a scan de-committed the RF; re-bring-up on the operating channel before connect */
int scan_ch_idx; /* current channel in sweep */
struct delayed_work scan_work; /* channel-sweep scan worker */
struct delayed_work scan_timeout;
@@ -1295,7 +1299,7 @@ void sv6160_start_queues(struct sv6160_softc *sc);
/* Last-resort wedge recovery: ask the driver core to re-create the dw_mmc host
* that owns our card. Takes the sdio_func, not the softc, because the wedge
* that most needs it is the one where PROBE fails -16 and no softc exists. */
void sv6160_rebind_sdio_host(struct sdio_func *func);
bool sv6160_rebind_sdio_host(struct sdio_func *func);
/* PM Support */
void sv6160_notify_pm(struct sv6160_softc *sc, int wakeup);
File diff suppressed because it is too large. Load diff
@@ -10,6 +10,7 @@
#include <linux/mmc/host.h>
#include <linux/mmc/core.h>
#include <linux/pm_runtime.h>
#include <linux/delay.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
#include <linux/platform_device.h>
@@ -34,12 +35,20 @@ static irqreturn_t sv6160_gpio_irq_thread(int irq, void *dev_id)
{
struct sv6160_softc *sc = dev_id;
if (sc && sc->is_firmware_loaded) {
sc->gpio_irq_count++;
dev_dbg(&sc->func->dev, "GPIOIRQ #%u\n", sc->gpio_irq_count);
sc->gpio_irq_pending = true;
sv6160_rx_drain(sc);
/* Nothing to drain into yet (firmware being uploaded, core_init not done)
* or the host is on its way into suspend: leave the wake where it is.
* The line is level-high and ONESHOT, so returning re-fires this thread
* as soon as it is unmasked; the sleep turns that spin into a 1 ms poll
* for the few ms either window lasts. Draining while suspended did
* CMD53s against a host being clock-gated and set sdio_wedged. */
if (!sc || !sc->is_firmware_loaded || sc->is_suspended) {
usleep_range(1000, 2000);
return IRQ_HANDLED;
}
sc->gpio_irq_count++;
dev_dbg(&sc->func->dev, "GPIOIRQ #%u\n", sc->gpio_irq_count);
sc->gpio_irq_pending = true;
sv6160_rx_drain(sc);
return IRQ_HANDLED;
}
@@ -171,6 +180,8 @@ int sv6160_sdio_fw_reboot(struct sv6160_softc *sc)
ret = sv6160_boot_firmware(sc);
dev_info(&func->dev, "fw_reboot: fresh firmware boot ret=%d\n", ret);
if (!ret)
sc->is_firmware_loaded = true; /* IRQ paths may drain again */
return ret;
}
@@ -347,23 +358,6 @@ static int sv6160_probe(struct sdio_func *func,
sc->dev = &func->dev;
mutex_init(&sc->lock);
/* Map the DW MMC controller CTRL register for FIFO/DMA reset.
* The SDIO host controller's parent device has the MMIO resource.
* DW MMC CTRL register is at offset 0x000 from the controller base. */
{
struct device *host_dev = func->card->host->parent;
struct resource *res;
res = platform_get_resource(to_platform_device(host_dev), IORESOURCE_MEM, 0);
if (res) {
sc->dwmmc_regs = devm_ioremap(&func->dev, res->start, 0x100);
if (sc->dwmmc_regs)
dev_info(&func->dev, "DW MMC regs mapped at %pa\n",
&res->start);
else
dev_warn(&func->dev, "Failed to map DW MMC regs\n");
}
}
sdio_set_drvdata(func, sc);
@@ -397,11 +391,14 @@ static int sv6160_probe(struct sdio_func *func,
/* Vendor model: THREADED IRQ, NULL primary + IRQF_ONESHOT.
* The DTS trigger (LEVEL_HIGH) is kept; the kernel masks the
* level line on assert and unmasks after the thread drains
* and ACKs the firmware. IRQF_NO_SUSPEND keeps it live across
* system suspend for wake. */
* and ACKs the firmware. No IRQF_NO_SUSPEND: it is documented as
* incompatible with the enable_irq_wake() below (suspend-and-
* interrupts.rst), and it kept the thread draining a suspended
* host. The PM core masks the line for the suspend and latches a
* wake; the thread also refuses to drain while is_suspended. */
ret = devm_request_threaded_irq(&func->dev, sc->host_wake_irq,
NULL, sv6160_gpio_irq_thread,
IRQF_ONESHOT | IRQF_NO_SUSPEND,
IRQF_ONESHOT,
"sv6160", sc);
if (ret) {
dev_warn(&func->dev, "GPIO IRQ %d failed: %d\n",
@@ -415,7 +412,11 @@ static int sv6160_probe(struct sdio_func *func,
}
}
/* Boot firmware and register with mac80211 */
/* Boot firmware and register with mac80211. Both IRQ paths are already
* registered and gate on is_firmware_loaded, which core_init raises only
* once the drain has its buffer, works and completion to land in. It
* used to flip inside boot_firmware, before any of that existed, so an
* early host-wake could drain into a NULL rx_buf. */
ret = sv6160_boot_firmware(sc);
if (ret == 0)
ret = sv6160_core_init(sc);
@@ -447,6 +448,13 @@ err_free:
sdio_claim_host(func);
sdio_release_irq(func);
sdio_release_host(func);
/* Freeing the IRQs synchronises their handlers but does not flush a
* work item one of them already queued (the in-band handler and the
* drain's OOM re-arm both schedule rx_work), and core_init has run far
* enough to have INIT_WORK'd it whenever the failure came from
* register_hw. */
sc->is_firmware_loaded = false;
cancel_work_sync(&sc->rx_work);
pm_runtime_disable(&func->dev);
ieee80211_free_hw(hw);
@@ -465,12 +473,21 @@ static void sv6160_remove(struct sdio_func *func)
return;
sv6160_core_exit(sc);
/* GPIO IRQ is freed automatically by devm */
pm_runtime_disable(&func->dev);
/* Both IRQ paths must be dead BEFORE ieee80211_free_hw() frees sc. devm
* would free the host-wake IRQ only after this function returns, i.e.
* after the softc is gone, and on the rung-2 host re-create that runs
* this remove() the level line is typically stuck high with the thread
* mid-drain. Same order as the probe error path. */
sc->is_firmware_loaded = false;
if (sc->host_wake_irq > 0) {
disable_irq_wake(sc->host_wake_irq);
devm_free_irq(&func->dev, sc->host_wake_irq, sc);
}
sdio_claim_host(func);
sdio_release_irq(func);
sdio_release_host(func);
cancel_work_sync(&sc->rx_work); /* an in-band IRQ may have re-queued it */
pm_runtime_disable(&func->dev);
ieee80211_free_hw(sc->hw);