Skip to content

[PW_SID:1151269] [BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter - #2521

Closed
linux-riscv-bot wants to merge 8 commits into
workflow__riscv__fixesfrom
pw1151269
Closed

[PW_SID:1151269] [BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter#2521
linux-riscv-bot wants to merge 8 commits into
workflow__riscv__fixesfrom
pw1151269

Conversation

@linux-riscv-bot

Copy link
Copy Markdown

PR for series 1151269 applied to workflow__riscv__fixes

Name: [BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter
URL: https://patchwork.kernel.org/project/linux-riscv/list/?series=1151269
Version: 1

ThreeBottleCap and others added 8 commits August 20, 2026 17:56
…_RWX

When CONFIG_STRICT_MODULE_RWX is not set, execmem cannot create
temporary writable mappings for read-only executable pages. In this
case, the execmem ranges must already have writable permissions.

Currently EXECMEM_KPROBES unconditionally uses PAGE_KERNEL_READ_EXEC,
which causes kprobe instruction slot writes to trigger page faults
on systems where CONFIG_STRICT_MODULE_RWX is not enabled.

Fix this by using PAGE_KERNEL_EXEC when CONFIG_STRICT_MODULE_RWX
is not available.

Signed-off-by: Xiaofeng Yuan <xiaofengmian@163.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Reviewed-by: Nam Cao <namcao@linutronix.de>
Link: https://patch.msgid.link/20260814082742.148403-2-xiaofengmian@163.com
Signed-off-by: Paul Walmsley <pjw@kernel.org>
patch_map() always creates a temporary writable mapping via fixmap for
kernel text addresses, even when CONFIG_STRICT_KERNEL_RWX is disabled
and the kernel text is already mapped with _PAGE_WRITE.

This is unnecessary overhead at best, and on minimal configurations
it can cause page faults.

Skip the fixmap path for kernel text when CONFIG_STRICT_KERNEL_RWX
is not enabled, since the text pages are already writable in that case.
The module text path is already gated on CONFIG_STRICT_MODULE_RWX and
is kept unchanged.

Reported-by: Klara Modin <klara@kasm.eu>
Closes: https://lore.kernel.org/all/ant_8TaBbov_GS4i@soda.int.kasm.eu/
Reported-by: Lad Prabhakar <prabhakar.csengg@gmail.com>
Closes: https://lore.kernel.org/all/CA+V-a8tQK8rih9SGGTyqrEBGpNkx4H0eX2YccCRrgkVAPr+EBg@mail.gmail.com/
Tested-by: Klara Modin <klarasmodin@gmail.com>
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Link: https://patch.msgid.link/20260814082742.148403-3-xiaofengmian@163.com
Signed-off-by: Paul Walmsley <pjw@kernel.org>
This reverts commit 3033b2b.

The reverted patch is userspace-visible behavior change, not a bug fix.
The two variables here (pmm and pmlen) control two independent features:
pmm is the _hardware_ pointer masking mode that applies while executing
in userspace. pmlen is the shift amount that the _kernel_ uses when
untagging addresses; PMLEN_0 means no untagging occurs, so the kernel
does not accept tagged addresses in syscall arguments.

It is valid (as documented and tested by the self test) to enable
pointer masking without enabling the tagged address ABI. This separation
is necessary to allow userspace to create an execution environment
similar to what the kernel supports on arm64 by default, where TBI is
enabled but the tagged address ABI is not. (On arm64, there is no
equivalent to PR_PMLEN_MASK because TBI is always enabled.)

Signed-off-by: Samuel Holland <samuel.holland@sifive.com>
Link: https://patch.msgid.link/20260820014551.1979772-1-samuel.holland@sifive.com
Cc: stable@vger.kernel.org
Fixes: 3033b2b ("riscv: Reset pmm when PR_TAGGED_ADDR_ENABLE is not set")
Signed-off-by: Paul Walmsley <pjw@kernel.org>
Prefer the convenient string choice 'str_disabled_enabled()'
helper over hardcoded strings in 'setup_global_riscv_enable()'.

Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Link: https://patch.msgid.link/20260819160546.3219942-1-dmantipov@yandex.ru
Signed-off-by: Paul Walmsley <pjw@kernel.org>
The vendor-extension handlers reached from hwprobe_one_pair()
(hwprobe_isa_vendor_ext_thead_0() and friends) only OR the present bits
into pair->value via VENDOR_EXTENSION_SUPPORTED() and clear their own
missing bits; they assume the caller has already zeroed pair->value.

That holds for hwprobe_get_values() (it zeroes each pair) and
hwprobe_get_cpus() (it re-initializes its scratch pair per key), but not
for complete_hwprobe_vdso_data(), which reuses a single pair across all
keys without re-zeroing. A vendor key therefore inherits stale bits from
the previously probed key, and the wrong value is cached in the vDSO
all_cpu_hwprobe_values[] and handed to userspace on the fast patih.

Zero pair->value once at the top of hwprobe_one_pair() so every handler
starts from a clean value regardless of the caller, and drop the now
redundant zeroing in the *_BLOCK_SIZE cases. hwprobe_isa_ext0() keeps its
own zeroing because hwprobe_ext0_has() calls it directly, bypassing
hwprobe_one_pair().

Fixes: a5ea53d ("riscv: hwprobe: Add thead vendor extension probing")
Signed-off-by: Andy Chiu <tchiu@tenstorrent.com>
Reviewed-by: Jesse Taube <jtaubepe@redhat.com>
Link: https://patch.msgid.link/20260725001614.2578617-2-tchiu@tenstorrent.com
Cc: stable@vger.kernel.org
Signed-off-by: Paul Walmsley <pjw@kernel.org>
x86 did this in commit b0a848f ("x86/bugs: Make i386 use
GENERIC_BUG_RELATIVE_POINTERS")

powerpc did this in commit 1baa1f7 ("powerpc: Allow relative
pointers in bug table entries")

Similar as x86 and powerpc does, make RV32 use
GENERIC_BUG_RELATIVE_POINTERS for "there is only one code path." and
"less #ifdef is more better".

Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
Link: https://patch.msgid.link/20260221024255.3552-1-jszhang@kernel.org
Signed-off-by: Paul Walmsley <pjw@kernel.org>
…Milk-V Jupiter

Hi,

On a Milk-V Jupiter, probing the PCIe controller at ca800000 (port C, the
card slot) permanently wedges the CPU that runs the probe when one
particular add-in card is installed.  The CPU stops responding to NMI, and
because the probe is asynchronous, kernel_init() then blocks forever in
async_synchronize_full() and the machine never finishes booting.

The same card, in the same slot, on the same board, does *not* hang the
vendor 6.6 kernel -- it reports "Phy link never came up" and boots normally.
Six other cards do not hang mainline either.  So whatever the electrical
cause, this looks like a robustness problem in pcie-spacemit-k1: an endpoint
should not be able to hang a host-side DBI register access.

HARDWARE
--------
  Milk-V Jupiter v1.1, SpacemiT M1 (socinfo: CPU[M1-8571] REV[C] DRO[130]),
  16 GiB LPDDR4X.
  Firmware: stock vendor U-Boot 2022.10 (k1-bl-v2.2.9), unmodified.
  Port B (ca400000, M.2) has a Samsung PM9B1 NVMe and works throughout.
  Port C (ca800000) is the card slot -- an x8-length connector, silkscreened
  PCIE_X2, wired x2.

REPRODUCED ON
-------------
  v7.1 and v7.2, riscv defconfig (plus PHY_SPACEMIT_K1_USB2, USB_DWC3,
  SPACEMIT_K1_TSENSOR, IGB, IGC, NVMe/ext4 built in).
  gcc 13.3.0 (cross) and gcc 16.2.0 (native, Debian sid).
  Identical failure in all combinations.  Not a regression -- port C has
  never worked with this card on mainline.

Command line:
  console=ttyS0,115200 earlycon root=/dev/nvme0n1p2 rootwait rw
  swiotlb=65536 clk_ignore_unused pd_ignore_unused

SYMPTOM
-------
Port C prints its address ranges and then never speaks again.  (Log below is
from a run with port B disabled in DT, so nothing is interleaved.)

  [ 1.290074] spacemit-k1-pcie ca800000.pcie: host bridge /soc/pcie-bus/pcie@ca800000 ranges:
  [ 1.297283] spacemit-k1-pcie ca800000.pcie:       IO 0x00b7002000..0x00b7101fff -> 0x0000000000
  [ 1.312783] spacemit-k1-pcie ca800000.pcie:      MEM 0x00a0000000..0x00afffffff -> 0x00a0000000
  [ 1.326753] spacemit-k1-pcie ca800000.pcie:      MEM 0x00b0000000..0x00b6ffffff -> 0x00b0000000
  [22.348490] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
  [22.351764] rcu:     4-...0: (12 GPs behind) idle=051c/1/0x4000000000000000 softirq=43/43 fqs=1908
  [22.367040] Sending NMI from CPU 2 to CPUs 4:
  [32.367049] After 10 seconds, these CPUS still haven't responded to the NMI: 4

The CPU ignoring an NMI for ten seconds is why I read this as an MMIO access
that never receives a completion rather than a spin or a deadlock.

LOCALISATION
------------
I added a dev_info() before each step of k1_pcie_init() (patch at the end of
this mail).  The last marker port C prints is the one immediately before the
first DBI access:

  [1.347050] spacemit-k1-pcie ca800000.pcie: K1DBG 1 toggle_soft_reset
  [1.362635] spacemit-k1-pcie ca800000.pcie: K1DBG 2 enable_resources
  [1.370918] spacemit-k1-pcie ca800000.pcie: K1DBG 3 first DBI write (vendor/device ID)
  <nothing further>

i.e. it dies in

        dw_pcie_dbi_ro_wr_en(pci);
        dw_pcie_writew_dbi(pci, PCI_VENDOR_ID, PCI_VENDOR_ID_SPACEMIT);

which is the first register access to the controller after
k1_pcie_enable_resources() has enabled the clocks and deasserted the resets.

I also tried moving phy_init() ahead of that block, in case the DBI domain
needed the PHY running (port B is masked here, because vendor U-Boot
initialises port B's PHY and never touches port C's).  phy_init() returned
success and the DBI access still hung:

  K1DBG 1 toggle_soft_reset
  K1DBG 2 enable_resources
  K1DBG 4 assert PERST# + 100ms
  K1DBG 5 RC mode + AUX_PWR_DET
  K1DBG 6 phy_init
  K1DBG 3 first DBI write (vendor/device ID)     <- still the last line

CARD MATRIX (port C, mainline 7.2, otherwise identical boots)
-------------------------------------------------------------
  empty slot                                          boots
  Intel I210        [8086:1533]  Gen1 x1   link up, enumerates
  HP NC360T         [8086:105e]  Gen1 x2   link up, enumerates
  NVIDIA Quadro P400[10de:1cb3]  Gen1 x2   link up, enumerates
  NVIDIA Quadro T400[10de:1fb2]  Gen1 x1   link up, enumerates
  AMD Radeon RX 550 [1002:699f]  Gen1 x2   link up, enumerates
  Sun ATLS1QGE                             "Device found, but not active"
  HP NC375T                                "Device found, but not active"
  Intel I225-V rev 01                      *** CPU HANG ***

Note the two cards that do not train fail *politely* -- the DWC core logs
"Device found, but not active", the probe completes, an empty bus 0002:00 is
created and the machine boots.  So port C is perfectly capable of handling a
link that never comes up.  The I225-V is different, and it fails long before
link training is reached.

The I225-V card itself is good: it works on a MACCHIATObin (Armada 8040) in
the same office, and it works on this same Jupiter under the vendor kernel.

THE VENDOR DRIVER SURVIVES THE SAME CARD
----------------------------------------
Bianbu 2.3.5 / Linux 6.6.63, vendor k1x-dwc-pcie driver, same board, same
slot, same I225-V:

  [ 1.657482] k1x-dwc-pcie ca400000.pcie: PCIe Gen.2 x2 link up
  [ 2.949762] k1x-dwc-pcie ca800000.pcie: Phy link never came up
  [ 2.952808] k1x-dwc-pcie ca800000.pcie: PCI host bridge to bus 0002:00
  ... boots to a login prompt

The vendor driver does not do the "set the PCI vendor and device ID" DBI
write at that point in its sequence.

HYPOTHESES ELIMINATED BY EXPERIMENT
-----------------------------------
  nvme driver          initcall_blacklist=nvme_init  -- still hangs
  power domains        pd_ignore_unused              -- still hangs
  link training        dies before dw_pcie_iatu_detect()
  kernel version       identical on v7.1 and v7.2
  compiler             identical with gcc 13.3 and gcc 16.2
  PHY init ordering    phy_init() first -- returns 0, still hangs
  CLKREQ# pinmux       CLKREQ# removed from pcie2_4_cfg -- still hangs
  port B interference  port B disabled in DT -- still hangs
  power supply         12 V / 12.5 A bench supply, not USB-C PD

A POSSIBLY RELATED OBSERVATION
------------------------------
k1_pcie_init() writes PCI_VENDOR_ID_SPACEMIT / PCI_DEVICE_ID_SPACEMIT_K1 to
both ports.  On this board the write takes effect on port B but not on port C
-- including on the boots where port C works fine:

  pci 0001:00:00.0: [201f:0001] type 01 class 0x060400 PCIe Root Port
  pci 0002:00:00.0: [1e5d:3003] type 01 class 0x060400 PCIe Root Port

1e5d:3003 is the hardware default (ASR Microelectronics).  So that same DBI
read-only write is being silently dropped on port C even when it does not
hang.  I do not know whether this is the same underlying issue, but it is in
the same function and on the same port, so it seemed worth mentioning.

WORKAROUND
----------
        &pcie2 { status = "disabled"; };

or, from U-Boot, before booti:

        fdt set /soc/pcie-bus/pcie@ca800000 status disabled

With that, mainline 7.2 boots Debian happily on this board with root on the
M.2 NVMe.  Nobody's board is stuck; the slot is just unusable with this card.

Note this is not a problem with the DTS enablement of &pcie2 (added in 7.1) --
an empty slot, and six of seven cards, work fine.

WHAT I AM ASKING
----------------
I do not have the K1 documentation, so I have gone as far as I can from
outside.  I would appreciate a pointer to what could gate that first DBI
access on port C -- CLK_PCIE2_DBI/MASTER/SLAVE, RESET_PCIE2_*, or something
in the PMU/APMU block -- and I am happy to run any test you like on this
board.

Separately, and regardless of the cause: a hard CPU hang on an unanswered
DBI read is an unpleasant failure mode, since there is no completion timeout
and no machine check to abort it on RISC-V.  If there is a sane way to bound
it, that seems worth having.

Full logs for every boot referenced above are available on request.

Thanks,
Bruno Banelli

--------------------------------------------------------------------------------
Instrumentation used for the localisation above (not for merging):

Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>
@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
build-rv32-defconfig
Desc: Builds riscv32 defconfig
Duration: 146.09 seconds
Result: PASS

@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
build-rv64-clang-allmodconfig
Desc: Builds riscv64 allmodconfig with Clang, and checks for errors and added warnings
Duration: 1472.63 seconds
Result: PASS

@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
build-rv64-gcc-allmodconfig
Desc: Builds riscv64 allmodconfig with GCC, and checks for errors and added warnings
Duration: 1471.97 seconds
Result: PASS

@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
build-rv64-nommu-k210-defconfig
Desc: Builds riscv64 defconfig with NOMMU for K210
Duration: 28.31 seconds
Result: PASS

@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
build-rv64-nommu-k210-virt
Desc: Builds riscv64 defconfig with NOMMU for the virt platform
Duration: 30.31 seconds
Result: PASS

@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
checkpatch
Desc: Runs checkpatch.pl on the patch
Duration: 1.99 seconds
Result: ERROR
Output:

ERROR: Invalid commit separator - some tools may have problems applying this
#22: 
--------

ERROR: Invalid commit separator - some tools may have problems applying this
#31: 
-------------

ERROR: Invalid commit separator - some tools may have problems applying this
#43: 
-------

ERROR: Invalid commit separator - some tools may have problems applying this
#60: 
------------

ERROR: Invalid commit separator - some tools may have problems applying this
#91: 
-------------------------------------------------------------

ERROR: Invalid commit separator - some tools may have problems applying this
#112: 
----------------------------------------

ERROR: Invalid commit separator - some tools may have problems applying this
#125: 
-----------------------------------

ERROR: Invalid commit separator - some tools may have problems applying this
#137: 
------------------------------

ERROR: Invalid commit separator - some tools may have problems applying this
#151: 
----------

ERROR: Invalid commit separator - some tools may have problems applying this
#165: 
----------------

ERROR: Invalid commit separator - some tools may have problems applying this
#182: 
--------------------------------------------------------------------------------

total: 11 errors, 0 warnings, 0 checks, 52 lines checked

NOTE: For some of the reported defects, checkpatch may be able to
      mechanically convert to the typical style using --fix or --fix-inplace.

Commit 7ed7ddde3c23 ("PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter") has style problems, please review.

NOTE: Ignored message types: ALLOC_SIZEOF_STRUCT CAMELCASE COMMIT_LOG_LONG_LINE GIT_COMMIT_ID MACRO_ARG_REUSE NO_AUTHOR_SIGN_OFF

NOTE: If any of the errors are false positives, please report
      them to the maintainer, see CHECKPATCH in MAINTAINERS.
ERROR: Invalid commit separator - some tools may have problems applying this


@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
dtb-warn-rv64
Desc: Checks for Device Tree warnings/errors
Duration: 89.41 seconds
Result: PASS

@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
header-inline
Desc: Detects static functions without inline keyword in header files
Duration: 0.23 seconds
Result: PASS

@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
kdoc
Desc: Detects for kdoc errors
Duration: 0.92 seconds
Result: PASS

@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
module-param
Desc: Detect module_param changes
Duration: 0.25 seconds
Result: PASS

@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
verify-fixes
Desc: Verifies that the Fixes: tags exist
Duration: 0.23 seconds
Result: PASS

@linux-riscv-bot

Copy link
Copy Markdown
Author

Patch 1: "[BUG] PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter"
verify-signedoff
Desc: Verifies that Signed-off-by: tags are correct
Duration: 1.29 seconds
Result: ERROR
Output:

Commit 7ed7ddde3c23 ("PCI: spacemit-k1: port C probe hard-hangs a CPU with one endpoint on Milk-V Jupiter")
	author Signed-off-by missing
	author email:    bbanelli@gmail.com
	committer email: linux.riscv.bot@gmail.com
	Signed-off-by: Linux RISC-V bot <linux.riscv.bot@gmail.com>

Errors in tree with Signed-off-by, please fix!


@linux-riscv-bot
linux-riscv-bot force-pushed the workflow__riscv__fixes branch 4 times, most recently from 495bff0 to c27f74f Compare September 2, 2026 00:49
@linux-riscv-bot
linux-riscv-bot deleted the pw1151269 branch September 2, 2026 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants