From: Gaurav Batra <gbatra@linux.ibm.com>
To: mpe@ellerman.id.au
Cc: brking@linux.vnet.ibm.com, Gaurav Batra <gbatra@linux.ibm.com>,
linuxppc-dev@lists.ozlabs.org
Subject: [PATCH v3] powerpc/pseries/iommu: DLPAR ADD of pci device doesn't completely initialize pci_controller structure
Date: Thu, 15 Feb 2024 16:18:33 -0600 [thread overview]
Message-ID: <20240215221833.4817-1-gbatra@linux.ibm.com> (raw)
When a PCI device is Dynamically added, LPAR OOPS with NULL pointer
exception.
Complete stack is as below
[ 211.239206] BUG: Kernel NULL pointer dereference on read at 0x00000030
[ 211.239210] Faulting instruction address: 0xc0000000006bbe5c
[ 211.239214] Oops: Kernel access of bad area, sig: 11 [#1]
[ 211.239218] LE PAGE_SIZE=64K MMU=Radix SMP NR_CPUS=2048 NUMA pSeries
[ 211.239223] Modules linked in: rpadlpar_io rpaphp rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace fscache netfs xsk_diag bonding nft_compat nf_tables nfnetlink rfkill binfmt_misc dm_multipath rpcrdma sunrpc rdma_ucm ib_srpt ib_isert iscsi_target_mod target_core_mod ib_umad ib_iser libiscsi scsi_transport_iscsi ib_ipoib rdma_cm iw_cm ib_cm mlx5_ib ib_uverbs ib_core pseries_rng drm drm_panel_orientation_quirks xfs libcrc32c mlx5_core mlxfw sd_mod t10_pi sg tls ibmvscsi ibmveth scsi_transport_srp vmx_crypto pseries_wdt psample dm_mirror dm_region_hash dm_log dm_mod fuse
[ 211.239280] CPU: 17 PID: 2685 Comm: drmgr Not tainted 6.7.0-203405+ #66
[ 211.239284] Hardware name: IBM,9080-HEX POWER10 (raw) 0x800200 0xf000006 of:IBM,FW1060.00 (NH1060_008) hv:phyp pSeries
[ 211.239289] NIP: c0000000006bbe5c LR: c000000000a13e68 CTR: c0000000000579f8
[ 211.239293] REGS: c00000009924f240 TRAP: 0300 Not tainted (6.7.0-203405+)
[ 211.239298] MSR: 8000000000009033 <SF,EE,ME,IR,DR,RI,LE> CR: 24002220 XER: 20040006
[ 211.239306] CFAR: c000000000a13e64 DAR: 0000000000000030 DSISR: 40000000 IRQMASK: 0
[ 211.239306] GPR00: c000000000a13e68 c00000009924f4e0 c0000000015a2b00 0000000000000000
[ 211.239306] GPR04: c0000000013c5590 0000000000000000 c000000006d07970 c0000000d8f8f180
[ 211.239306] GPR08: 00000000000006ec c0000000d8f8f180 c000000002c35d58 0000000024002228
[ 211.239306] GPR12: c0000000000579f8 c0000003ffeb3880 0000000000000000 0000000000000000
[ 211.239306] GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 211.239306] GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 211.239306] GPR24: c0000000919460c0 0000000000000000 fffffffffffff000 c0000000010088e8
[ 211.239306] GPR28: c0000000013c5590 c000000006d07970 c0000000919460c0 c0000000919460c0
[ 211.239354] NIP [c0000000006bbe5c] sysfs_add_link_to_group+0x34/0x94
[ 211.239361] LR [c000000000a13e68] iommu_device_link+0x5c/0x118
[ 211.239367] Call Trace:
[ 211.239369] [c00000009924f4e0] [c000000000a109b8] iommu_init_device+0x26c/0x318 (unreliable)
[ 211.239376] [c00000009924f520] [c000000000a13e68] iommu_device_link+0x5c/0x118
[ 211.239382] [c00000009924f560] [c000000000a107f4] iommu_init_device+0xa8/0x318
[ 211.239387] [c00000009924f5c0] [c000000000a11a08] iommu_probe_device+0xc0/0x134
[ 211.239393] [c00000009924f600] [c000000000a11ac0] iommu_bus_notifier+0x44/0x104
[ 211.239398] [c00000009924f640] [c00000000018dcc0] notifier_call_chain+0xb8/0x19c
[ 211.239405] [c00000009924f6a0] [c00000000018df88] blocking_notifier_call_chain+0x64/0x98
[ 211.239411] [c00000009924f6e0] [c000000000a250fc] bus_notify+0x50/0x7c
[ 211.239416] [c00000009924f720] [c000000000a20838] device_add+0x640/0x918
[ 211.239421] [c00000009924f7f0] [c0000000008f1a34] pci_device_add+0x23c/0x298
[ 211.239427] [c00000009924f840] [c000000000077460] of_create_pci_dev+0x400/0x884
[ 211.239432] [c00000009924f8e0] [c000000000077a08] of_scan_pci_dev+0x124/0x1b0
[ 211.239437] [c00000009924f980] [c000000000077b0c] __of_scan_bus+0x78/0x18c
[ 211.239442] [c00000009924fa10] [c000000000073f90] pcibios_scan_phb+0x2a4/0x3b0
[ 211.239447] [c00000009924fad0] [c0000000001007a8] init_phb_dynamic+0xb8/0x110
[ 211.239453] [c00000009924fb40] [c008000006920620] dlpar_add_slot+0x170/0x3b8 [rpadlpar_io]
[ 211.239461] [c00000009924fbe0] [c008000006920d64] add_slot_store.part.0+0xb4/0x130 [rpadlpar_io]
[ 211.239468] [c00000009924fc70] [c000000000fb4144] kobj_attr_store+0x2c/0x48
[ 211.239473] [c00000009924fc90] [c0000000006b90e4] sysfs_kf_write+0x64/0x78
[ 211.239479] [c00000009924fcb0] [c0000000006b7b78] kernfs_fop_write_iter+0x1b0/0x290
[ 211.239485] [c00000009924fd00] [c0000000005b6fdc] vfs_write+0x350/0x4a0
[ 211.239491] [c00000009924fdc0] [c0000000005b7450] ksys_write+0x84/0x140
[ 211.239496] [c00000009924fe10] [c000000000030a04] system_call_exception+0x124/0x330
[ 211.239502] [c00000009924fe50] [c00000000000cedc] system_call_vectored_common+0x15c/0x2ec
Commit a940904443e4 ("powerpc/iommu: Add iommu_ops to report capabilities
and allow blocking domains") broke DLPAR ADD of pci devices.
The above added iommu_device structure to pci_controller. During
system boot, pci devices are discovered and this newly added iommu_device
structure initialized by a call to iommu_device_register().
During DLPAR ADD of a PCI device, a new pci_controller structure is
allocated but there are no calls made to iommu_device_register()
interface.
Fix would be to register iommu device during DLPAR ADD as well.
Fixes: a940904443e4 ("powerpc/iommu: Add iommu_ops to report capabilities and allow blocking domains")
Signed-off-by: Gaurav Batra <gbatra@linux.ibm.com>
Reviewed-by: Brian King <brking@linux.vnet.ibm.com>
---
arch/powerpc/include/asm/ppc-pci.h | 10 ++++++++++
arch/powerpc/kernel/iommu.c | 23 ++++++++++++++++------
arch/powerpc/platforms/pseries/pci_dlpar.c | 4 ++++
3 files changed, 31 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/include/asm/ppc-pci.h b/arch/powerpc/include/asm/ppc-pci.h
index ce2b1b5eebdd..a8b7e8682f5b 100644
--- a/arch/powerpc/include/asm/ppc-pci.h
+++ b/arch/powerpc/include/asm/ppc-pci.h
@@ -30,6 +30,16 @@ void *pci_traverse_device_nodes(struct device_node *start,
void *data);
extern void pci_devs_phb_init_dynamic(struct pci_controller *phb);
+#if defined(CONFIG_IOMMU_API) && (defined(CONFIG_PPC_PSERIES) || \
+ defined(CONFIG_PPC_POWERNV))
+extern void ppc_iommu_register_device(struct pci_controller *phb);
+extern void ppc_iommu_unregister_device(struct pci_controller *phb);
+#else
+static inline void ppc_iommu_register_device(struct pci_controller *phb) { }
+static inline void ppc_iommu_unregister_device(struct pci_controller *phb) { }
+#endif
+
+
/* From rtas_pci.h */
extern void init_pci_config_tokens (void);
extern unsigned long get_phb_buid (struct device_node *);
diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c
index ebe259bdd462..3425b917c8d6 100644
--- a/arch/powerpc/kernel/iommu.c
+++ b/arch/powerpc/kernel/iommu.c
@@ -1339,7 +1339,7 @@ static struct iommu_device *spapr_tce_iommu_probe_device(struct device *dev)
struct pci_controller *hose;
if (!dev_is_pci(dev))
- return ERR_PTR(-EPERM);
+ return ERR_PTR(-ENODEV);
pdev = to_pci_dev(dev);
hose = pdev->bus->sysdata;
@@ -1388,6 +1388,21 @@ static const struct attribute_group *spapr_tce_iommu_groups[] = {
NULL,
};
+void ppc_iommu_register_device(struct pci_controller *phb)
+{
+ iommu_device_sysfs_add(&phb->iommu, phb->parent,
+ spapr_tce_iommu_groups, "iommu-phb%04x",
+ phb->global_number);
+ iommu_device_register(&phb->iommu, &spapr_tce_iommu_ops,
+ phb->parent);
+}
+
+void ppc_iommu_unregister_device(struct pci_controller *phb)
+{
+ iommu_device_unregister(&phb->iommu);
+ iommu_device_sysfs_remove(&phb->iommu);
+}
+
/*
* This registers IOMMU devices of PHBs. This needs to happen
* after core_initcall(iommu_init) + postcore_initcall(pci_driver_init) and
@@ -1398,11 +1413,7 @@ static int __init spapr_tce_setup_phb_iommus_initcall(void)
struct pci_controller *hose;
list_for_each_entry(hose, &hose_list, list_node) {
- iommu_device_sysfs_add(&hose->iommu, hose->parent,
- spapr_tce_iommu_groups, "iommu-phb%04x",
- hose->global_number);
- iommu_device_register(&hose->iommu, &spapr_tce_iommu_ops,
- hose->parent);
+ ppc_iommu_register_device(hose);
}
return 0;
}
diff --git a/arch/powerpc/platforms/pseries/pci_dlpar.c b/arch/powerpc/platforms/pseries/pci_dlpar.c
index 4ba824568119..4448386268d9 100644
--- a/arch/powerpc/platforms/pseries/pci_dlpar.c
+++ b/arch/powerpc/platforms/pseries/pci_dlpar.c
@@ -35,6 +35,8 @@ struct pci_controller *init_phb_dynamic(struct device_node *dn)
pseries_msi_allocate_domains(phb);
+ ppc_iommu_register_device(phb);
+
/* Create EEH devices for the PHB */
eeh_phb_pe_create(phb);
@@ -76,6 +78,8 @@ int remove_phb_dynamic(struct pci_controller *phb)
}
}
+ ppc_iommu_unregister_device(phb);
+
pseries_msi_free_domains(phb);
/* Keep a reference so phb isn't freed yet */
base-commit: ecb1b8288dc7ccbdcb3b9df005fa1c0e0c0388a7
--
2.39.3 (Apple Git-145)
next reply other threads:[~2024-02-15 22:19 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-15 22:18 Gaurav Batra [this message]
2024-02-20 12:28 ` [PATCH v3] powerpc/pseries/iommu: DLPAR ADD of pci device doesn't completely initialize pci_controller structure Michael Ellerman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240215221833.4817-1-gbatra@linux.ibm.com \
--to=gbatra@linux.ibm.com \
--cc=brking@linux.vnet.ibm.com \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=mpe@ellerman.id.au \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.