Firefly开源社区

标题: 加载xilinx的xdma驱动出错 [打印本页]

作者: rudy_jenny 时间: 2022-6-24 17:41
标题: 加载xilinx的xdma驱动出错
Ubuntu系统，出错log如下：
interrupt_selection 1.
Loading driver...insmod xdma.ko interrupt_mode=2 ...
[ 79.039826] xdma:probe_one: pdev 0x00000000805013c7, err -22.

Error: The Kernel module installed correctly, but no devices were recognized.
FAILED

作者: 799959745 时间: 2022-6-27 14:32
提供相应的ko文件，更多的log帮助分析复现

作者: rudy_jenny 时间: 2022-6-27 15:21

799959745 发表于 2022-6-27 14:32
提供相应的ko文件，更多的log帮助分析复现

1、加载驱动命令
./load_driver.sh
打印如下：Loading driver...
Error: The Kernel module installed correctly, but no devices were recognized.
FAILED

2、列举PCIe设备信息命令
lspci
打印如下：0000:00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3588 (rev 01)
0000:01:00.0 Memory controller: Xilinx Corporation Device 7024
0002:20:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3588 (rev 01)
0002:21:00.0 Network controller: Broadcom Inc. and subsidiaries Device 449d (rev 02)

xdma.zip

251.17 KB, 下载次数: 14, 下载积分: 灯泡 -1 , 经验 -1

xdma驱动

Xilinx_Answer_65444_Linux_Files.zip

472.84 KB, 下载次数: 8, 下载积分: 灯泡 -1 , 经验 -1

xdma源码

作者: 799959745 时间: 2022-6-28 10:15

rudy_jenny 发表于 2022-6-27 15:21
1、加载驱动命令
./load_driver.sh
打印如下：Loading driver...

按照log来说驱动加载并没有问题。只是没有识别到pcie设备

作者: rudy_jenny 时间: 2022-6-28 10:43

799959745 发表于 2022-6-27 14:32
提供相应的ko文件，更多的log帮助分析复现

以下是dmesg中的pci信息，怀疑是不是能够给其他PCIe设备分配的内存不够导致的。
[ 2.447142] vcc3v3_pcie30: 3300 mV, disabled
[ 2.447188] reg-fixed-voltage vcc3v3-pcie30: Looking up vin-supply from device tree
[ 2.447191] vcc3v3_pcie30: supplied by vcc12v_dcin
[ 2.447225] reg-fixed-voltage vcc3v3-pcie30: vcc3v3_pcie30 supplying 3300000uV
[ 2.994201] rk-pcie fe150000.pcie: invalid prsnt-gpios property in node
[ 2.994225] rk-pcie fe150000.pcie: Looking up vpcie3v3-supply from device tree
[ 2.994245] rk-pcie fe170000.pcie: invalid prsnt-gpios property in node
[ 2.994253] rk-pcie fe170000.pcie: Looking up vpcie3v3-supply from device tree
[ 2.994262] rk-pcie fe170000.pcie: Looking up vpcie3v3-supply property in node /pcie@fe170000 failed
[ 2.994268] rk-pcie fe170000.pcie: no vpcie3v3 regulator found
[ 2.994451] rk-pcie fe170000.pcie: missing legacy IRQ resource
[ 2.994461] rk-pcie fe170000.pcie: IRQ msi not found
[ 2.994465] rk-pcie fe170000.pcie: use outband MSI support
[ 2.994469] rk-pcie fe170000.pcie: Missing *config* reg space
[ 2.994477] rk-pcie fe170000.pcie: host bridge /pcie@fe170000 ranges:
[ 2.994490] rk-pcie fe170000.pcie:    err 0x00f2000000..0x00f20fffff -> 0x00f2000000
[ 2.994499] rk-pcie fe170000.pcie:    IO 0x00f2100000..0x00f21fffff -> 0x00f2100000
[ 2.994508] rk-pcie fe170000.pcie:    MEM 0x00f2200000..0x00f2ffffff -> 0x00f2200000
[ 2.994516] rk-pcie fe170000.pcie:    MEM 0x0980000000..0x09bfffffff -> 0x0980000000
[ 2.994539] rk-pcie fe170000.pcie: Missing *config* reg space
[ 2.994564] rk-pcie fe170000.pcie: invalid resource
[ 3.010838] rk-pcie fe150000.pcie: missing legacy IRQ resource
[ 3.010858] rk-pcie fe150000.pcie: IRQ msi not found
[ 3.010868] rk-pcie fe150000.pcie: use outband MSI support
[ 3.010878] rk-pcie fe150000.pcie: Missing *config* reg space
[ 3.010895] rk-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
[ 3.010922] rk-pcie fe150000.pcie:    err 0x00f0000000..0x00f00fffff -> 0x00f0000000
[ 3.010949] rk-pcie fe150000.pcie:    IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
[ 3.010977] rk-pcie fe150000.pcie:    MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
[ 3.010995] rk-pcie fe150000.pcie:    MEM 0x0900000000..0x093fffffff -> 0x0900000000
[ 3.011033] rk-pcie fe150000.pcie: Missing *config* reg space
[ 3.011066] rk-pcie fe150000.pcie: invalid resource
[ 3.199139] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[ 3.224695] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[ 3.251359] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[ 3.271387] rk-pcie fe150000.pcie: PCIe Link up, LTSSM is 0x130011
[ 3.271579] rk-pcie fe150000.pcie: PCI host bridge to bus 0000:00
[ 3.271599] pci_bus 0000:00: root bus resource [bus 00-0f]
[ 3.271614] pci_bus 0000:00: root bus resource [??? 0xf0000000-0xf00fffff flags 0x0]
[ 3.271631] pci_bus 0000:00: root bus resource [io  0x100000-0x1fffff] (bus address [0xf0100000-0xf01fffff])
[ 3.271644] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
[ 3.271657] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
[ 3.271707] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
[ 3.271732] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
[ 3.271750] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
[ 3.271767] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[ 3.271848] pci 0000:00:00.0: supports D1 D2
[ 3.271862] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[ 3.278024] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[ 3.282362] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
[ 3.282473] pci 0000:01:00.0: [10ee:7024] type 00 class 0x058000
[ 3.282541] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x01ffffff]
[ 3.282578] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x0000ffff]
[ 3.282615] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x01ffffff]
[ 3.282744] pci 0000:01:00.0: enabling Extended Tags
[ 3.282950] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot
[ 3.293278] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
[ 3.293308] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
[ 3.293324] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
[ 3.293337] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
[ 3.293351] pci 0000:00:00.0: BAR 8: no space for [mem size 0x05000000]
[ 3.293364] pci 0000:00:00.0: BAR 8: failed to assign [mem size 0x05000000]
[ 3.293380] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0200000-0xf020ffff pref]
[ 3.293400] pci 0000:01:00.0: BAR 0: no space for [mem size 0x02000000]
[ 3.293413] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x02000000]
[ 3.293426] pci 0000:01:00.0: BAR 2: no space for [mem size 0x02000000]
[ 3.293438] pci 0000:01:00.0: BAR 2: failed to assign [mem size 0x02000000]
[ 3.293451] pci 0000:01:00.0: BAR 1: no space for [mem size 0x00010000]
[ 3.293463] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x00010000]
[ 3.293477] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[ 3.296775] pcieport 0000:00:00.0: PME: Signaling with IRQ 142
[ 3.304692] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[ 3.331358] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[ 3.358027] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[ 3.374942] ehci-pci: EHCI PCI platform driver
[ 3.384691] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[ 3.411358] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[ 3.438023] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[ 3.619781] reg-fixed-voltage pcie30-avdd1v8: Looking up vin-supply from device tree
[ 3.619863] pcie30_avdd1v8: supplied by avcc_1v8_s0
[ 3.620091] pcie30_avdd1v8: 1800 mV, enabled
[ 3.620264] reg-fixed-voltage pcie30-avdd1v8: pcie30_avdd1v8 supplying 1800000uV
[ 3.620383] reg-fixed-voltage pcie30-avdd0v75: Looking up vin-supply from device tree
[ 3.620435] pcie30_avdd0v75: supplied by avdd_0v75_s0
[ 3.620589] pcie30_avdd0v75: 750 mV, enabled
[ 3.620744] reg-fixed-voltage pcie30-avdd0v75: pcie30_avdd0v75 supplying 750000uV
[ 3.654754] rk-pcie fe170000.pcie: PCIe Link up, LTSSM is 0x30011
[ 3.654970] rk-pcie fe170000.pcie: PCI host bridge to bus 0002:20
[ 3.654990] pci_bus 0002:20: root bus resource [bus 20-2f]
[ 3.655005] pci_bus 0002:20: root bus resource [??? 0xf2000000-0xf20fffff flags 0x0]
[ 3.655021] pci_bus 0002:20: root bus resource [io  0x0000-0xfffff] (bus address [0xf2100000-0xf21fffff])
[ 3.655035] pci_bus 0002:20: root bus resource [mem 0xf2200000-0xf2ffffff]
[ 3.655051] pci_bus 0002:20: root bus resource [mem 0x980000000-0x9bfffffff pref]
[ 3.655100] pci 0002:20:00.0: [1d87:3588] type 01 class 0x060400
[ 3.655141] pci 0002:20:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[ 3.655335] pci 0002:20:00.0: supports D1 D2
[ 3.655351] pci 0002:20:00.0: PME# supported from D0 D1 D3hot
[ 3.658739] pci 0002:20:00.0: Primary bus is hard wired to 0
[ 3.658744] pci 0002:20:00.0: bridge configuration invalid ([bus 01-ff]), reconfiguring
[ 3.658884] pci 0002:21:00.0: [14e4:449d] type 00 class 0x028000
[ 3.658956] pci 0002:21:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 64bit]
[ 3.659002] pci 0002:21:00.0: reg 0x18: [mem 0x00000000-0x003fffff 64bit]
[ 3.659449] pci 0002:21:00.0: supports D1 D2
[ 3.659452] pci 0002:21:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 3.675040] pci_bus 0002:21: busn_res: [bus 21-2f] end is updated to 21
[ 3.675074] pci 0002:20:00.0: BAR 8: assigned [mem 0xf2200000-0xf27fffff]
[ 3.675086] pci 0002:20:00.0: BAR 6: assigned [mem 0xf2800000-0xf280ffff pref]
[ 3.675099] pci 0002:21:00.0: BAR 2: assigned [mem 0xf2400000-0xf27fffff 64bit]
[ 3.675148] pci 0002:21:00.0: BAR 0: assigned [mem 0xf2200000-0xf220ffff 64bit]
[ 3.675193] pci 0002:20:00.0: PCI bridge to [bus 21]
[ 3.675203] pci 0002:20:00.0: bridge window [mem 0xf2200000-0xf27fffff]
[ 3.677404] pcieport 0002:20:00.0: PME: Signaling with IRQ 159
[ 7.455652] [dhd] dhdpcie_pci_probe : no mutex held. set lock
[ 7.455656] [dhd] dhdpcie_init: found adapter info 'DHD generic adapter'
[ 7.455712] pcieh 0002:21:00.0: enabling device (0000 -> 0002)
[ 7.533366] [dhd] dhdpcie_bar1_window_switch_enab: bar1_switch_enab=0 ramstart=0x170000 ramend=0x2affff bar1_size=0x0
[ 7.580221] [dhd] dhdpcie_bus_attach: making DHD_BUS_DOWN
[ 7.580231] [dhd] dhdpcie_init: rc_dev from dev->bus->self (1d87:3588) is ffffff8005fa0000
[ 7.580339] [dhd] dhdpcie_init: rc_ep_aspm_cap: 1 rc_ep_l1ss_cap: 1
[ 7.580351] [dhd] dhdpcie_request_irq: INTx enabled
[ 7.580405] [dhd] dhdpcie_bar1_window_switch_enab: bar1_switch_enab=0 ramstart=0x170000 ramend=0x2affff bar1_size=0x400000
[ 7.581000] [dhd] dhdpcie_oob_intr_unregister: irq is not registered
[ 7.688040] [dhd] dhdpcie_pci_probe : the lock is released.
[ 8.971510] [dhd] dhd_bus_devreset: dhdpcie_bus_start_host_dev OK
[ 8.972807] [dhd] dhdpcie_bar1_window_switch_enab: bar1_switch_enab=0 ramstart=0x170000 ramend=0x2affff bar1_size=0x400000
[ 8.972828] [dhd] dhdpcie_request_irq: INTx enabled
[ 8.972868] [dhd] dhdpcie_dump_resource: BAR0(VA): 0x0000000000000000, BAR0(PA): 0x00000000f2200000, SIZE: 32768
[ 8.972873] [dhd] dhdpcie_dump_resource: BAR1(VA): 0x0000000000000000, BAR1(PA): 0x00000000f2400000, SIZE: 4194304
[ 8.974102] [dhd] dhd_conf_set_path_params : Final fw_path=/vendor/etc/firmware/fw_bcm43752a2_pcie_ag.bin
[ 8.974116] [dhd] dhd_conf_set_path_params : Final clm_path=/vendor/etc/firmware/clm_bcm43752a2_pcie_ag.blob
[ 8.974208] [dhd] dhdpcie_download_code_file: dhd_tcm_test_enable 0
[ 8.974213] [dhd] dhdpcie_download_code_file: download firmware /vendor/etc/firmware/fw_bcm43752a2_pcie_ag.bin
[ 8.974272] [dhd] dhd_os_open_image1: /vendor/etc/firmware/fw_bcm43752a2_pcie_ag.bin (857142 bytes) open success
[ 9.062195] [dhd] dhdpcie_download_nvram: dhd_get_download_buffer len 7335
[ 9.062221] [dhd] dhdpcie_download_nvram: process_nvram_vars len 5860
[ 9.079980] [dhd] dhdpcie_bus_write_vars: Download, Upload and compare of NVRAM succeeded.
[ 9.160382] [dhd] dhdpcie_readshared: PCIe shared addr (0x001f8974) read took 80000 usec before dongle is ready
[ 9.160837] [dhd] dhdpcie_readshared: max H2D queues 40
[ 9.160878] [dhd] dhdpcie_oob_intr_register OOB irq=161 flags=0x1
[ 9.160919] [dhd] dhdpcie_oob_intr_register: enable_irq_wake
[ 9.161035] [dhd] dhd_bus_dump_dar_registers: dar_errlog(0xa60:0x0) dar_erraddr(0xa64:0x0) dar_pcie_mbint(0xa68:0x0)
[ 9.190336] [dhd] dhd_os_open_image1: /vendor/etc/firmware/clm_bcm43752a2_pcie_ag.blob (29225 bytes) open success

作者: rudy_jenny 时间: 2022-6-29 08:49
问题已解决

作者: Liuth 时间: 2022-6-29 09:04

rudy_jenny 发表于 2022-6-29 08:49
问题已解决

解决方法可以说一下吗？

作者: rudy_jenny 时间: 2022-6-29 11:27

Liuth 发表于 2022-6-29 09:04
解决方法可以说一下吗？

pci 0000:01:00.0: BAR 2: no space for [mem size 0x02000000]，看这一句，主要是rk3588的pcie低端bar内存大小只有16MB，对端FPGA把内存配置减小就好了

作者: wx__BkXu5u 时间: 2022-8-2 10:41
请问您做的是itx 3588j和fpga板卡通过pcie互联嘛，我想咨询一下3588开发板的pcie的endpoint模式切换能不能实现，通过rk3588官方的修改dts就可以吗

作者: rudy_jenny 时间: 2022-8-2 11:02

wx__BkXu5u 发表于 2022-8-2 10:41
请问您做的是itx 3588j和fpga板卡通过pcie互联嘛，我想咨询一下3588开发板的pcie的endpoint模式切换能不能 ...

dts没有修改，主要是FPGA那边进行适配，你要看一下3588的dts文件，最大能支持多少的内存

作者: wx__BkXu5u 时间: 2022-8-5 16:27

rudy_jenny 发表于 2022-8-2 11:02
dts没有修改，主要是FPGA那边进行适配，你要看一下3588的dts文件，最大能支持多少的内存

好的，谢谢啦

作者: chaoloveai120 时间: 2023-1-11 14:23
谢谢分享解决经验

作者: xiaguangbo 时间: 2023-6-30 11:19
我也出现了同样的报错，lspci能看到设备，就是报 bar 空间不够。我用的是 riffa, 所以 fpga 的 pcie ip 用的是最基础那个，也有 bar 设置, 建议改哪个？
[attach]12662[/attach]

作者: xiaguangbo 时间: 2023-6-30 11:35
我看了下，我这里 fpga 所在的 pci 01:00.0 是分配了的，1K*32b 正好是 0x3ff，但是 pci 00:00.0 有问题，这各是rk3588自己的东西吧，影响吗？

debian@debian:~$ lspci
00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3588 (rev 01)
01:00.0 Memory controller: Xilinx Corporation Device 7024

复制代码

[ 3.041940] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
[ 3.041945] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
[ 3.041950] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
[ 3.041954] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
[ 3.041958] pci 0000:00:00.0: BAR 8: assigned [mem 0xf0200000-0xf02fffff]
[ 3.041963] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0300000-0xf030ffff pref
]
[ 3.041969] pci 0000:01:00.0: BAR 0: assigned [mem 0xf0200000-0xf02003ff]
[ 3.041982] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[ 3.041987] pci 0000:00:00.0: bridge window [mem 0xf0200000-0xf02fffff]

复制代码

image.png (42.34 KB, 下载次数: )

作者: wx__H6Z50O 时间: 2023-7-19 14:38

rudy_jenny 发表于 2022-6-29 11:27
pci 0000:01:00.0: BAR 2: no space for [mem size 0x02000000]，看这一句，主要是rk3588的pcie低端bar内 ...

麻烦问下具体如何去改呢。我尝试修改dma_pcie的pcie:Bars中的size为1或其他。还是会报no space for的错误

作者: Jack700 时间: 2024-3-27 16:33
好好学习一下，不知道后面如何XDMA

作者: mmry 时间: 3 天前
你好，请问我在使用3588编译xdma.ko的时候会报错误，如图mmiowb()这个函数找不到，请问是什么原因如何解决呢？谢谢！

微信图片_20240506193818.jpg (307.88 KB, 下载次数: 0)

作者: mmry 时间: 3 天前

rudy_jenny 发表于 2022-6-29 11:27
pci 0000:01:00.0: BAR 2: no space for [mem size 0x02000000]，看这一句，主要是rk3588的pcie低端bar内 ...

请问这个是改的FPGA的哪里呢？

欢迎光临 Firefly开源社区 (https://dev.t-firefly.com/)