Clarification on PCIe Port Driver Behavior when Removing a Device

Background

I would like to clarify the behavior and intention behind the pcieport driver actions during device removal.

First, I reviewed the PCI-PM and PCIe Gen6 Base Specifications, focusing on the sections related to power management (PCI-PM). According to the PCIe specifications:

  • Ch.5.3.2.1 Entry into the L1 State
  • Ch.5.4.1.3.1 ASPM Entry into the L1 State

I concluded two rules:

  • A downstream port cannot initiate L1.
  • A downstream port can only respond to an L1 entry request; it cannot initiate the request.

System Configuration:

  • PCIe Root Port at 00:03.0
  • PCIe Endpoint (EP) at 01:00.0
  • Linux Kernel Version: 6.3.0 (x86_64) on Ubuntu 22.04.3 LTS

I have added printk statements in dmesg for various files under drivers/pci/*.c and used my internal tools to trace the driver behavior (CfgRd/CfgWr) for PCIe Gen6 Type1 & Type0 devices.


QUESTION

When I run the command “echo 1 | sudo tee /sys/bus/pci/devices/0000:01:00.0/remove” to remove the endpoint, I observe the following behavior:

  • CfgWr to PMCSR of Downstream Port to D3_hot
  • The pcieport driver writes the PMCSR of the downstream port to D3_hot (config_write: bdf: 00:03.0 addr: 78 data: 10b) after the device 0000:01:00.0 is removed. Is there a specific reason for this action?
  • The downstream port keeps generating PME? as indicated by the following function calls. I wonder why does the driver exhibit this behavior?
pci_pme_list_scan
pci_pme_wakeup
pci_check_pme_status
pci_read_config_word
pci_pme_list_scan
pci_pme_wakeup
pci_check_pme_status
pci_read_config_word
pci_pme_list_scan
pci_pme_wakeup
pci_check_pme_status
pci_read_config_word
... (repeating until rescan pci)

When I run “echo 1 | sudo tee /sys/bus/pci/rescan”, the pcieport driver first writes the downstream port to D0, and then the sysfs adds the endpoint 0000:01:00.0 back. This process does not seem related to the LTSSM initiating L1.

It appears that the driver behavior is to simply change the downstream port’s D-state without involving an L1 request.

What is the reasoning behind this behavior?


Logs

Here is a summarized trace of the functions called during the “remove” process (I may not be 100% sure of the hierarchy sequence, but I am certain these functions are involved):


######## echo 1 | sudo tee /sys/bus/pci/devices/0000:01:00.0/remove ########
drivers/pci/pci-sysfs.c:485 (remove_store)
        drivers/pci/remove.c:123 (pci_stop_and_remove_bus_device_locked)
                drivers/pci/probe.c:3300 (pci_lock_rescan_remove)
                
                drivers/pci/remove.c:116 (pci_stop_and_remove_bus_device)
                        drivers/pci/remove.c:67 (pci_stop_bus_device)
                                drivers/pci/remove.c:18 (pci_stop_dev)
                                        drivers/pci/pci.c:2462 (pci_pme_active)
                                                drivers/pci/pci.c:2416 (__pci_pme_active)
                                                        drivers/pci/access.c:545 (pci_read_config_word)  # config_read: bdf: 01:00:0 addr: 48 data: 8
                                                        drivers/pci/access.c:574 (pci_write_config_word) # config_write: bdf: 01:00:0 addr: 48 data: 8008
                            
                            # ...
                            # drivers/base/dd.c:1280 (device_release_driver)
                            # drivers/pci/proc.c:448 (pci_proc_detach_device)
                            # drivers/pci/pci-sysfs.c:1510 (pci_remove_sysfs_dev_files)
                            # drivers/pci/pci-sysfs.c:1149 (pci_remove_resource_files)
                          # ...
                        
                        drivers/pci/remove.c:87 (pci_remove_bus_device)
                                drivers/pci/remove.c:32 (pci_destroy_dev)
                                
                                        drivers/pci/vgaarb.c:1505 (pci_notify)
                                                drivers/pci/vgaarb.c:844 (vga_arbiter_del_pci_device)
                                                # dmesg shows "bus: 'pci': remove device 0000:01:00.0"
                    
                    
                        # ...
                        # drivers/pci/pci-driver.c:1361 (pci_pm_runtime_idle)
                                # drivers/pci/pcie/portdrv.c:641 (pcie_port_runtime_idle)
                                # drivers/pci/pci-driver.c:1274 (pci_pm_runtime_suspend)
                                
                                # drivers/pci/pcie/portdrv.c:633 (pcie_port_runtime_suspend)
                                        # drivers/pci/pcie/portdrv.c:427 (pcie_port_device_runtime_suspend)
                                        
                                                #### drivers/pci/pci.c:1646 (pci_save_state) #### (do many config_read...)
                                                pci_save_pcie_state(dev);
                                                pci_save_pcix_state(dev);
                                                pci_save_ltr_state(dev);
                                                pci_save_dpc_state(dev);
                                                pci_save_aer_state(dev);
                                                pci_save_ptm_state(dev);
                                                return pci_save_vc_state(dev);
                                                
                                                #### drivers/pci/pci.c:1493 (pci_save_pcie_state) #### (do many config_read...)
                                                pcie_capability_read_word(dev, PCI_EXP_DEVCTL, &cap[i++]);
                                                pcie_capability_read_word(dev, PCI_EXP_LNKCTL, &cap[i++]);
                                                pcie_capability_read_word(dev, PCI_EXP_SLTCTL, &cap[i++]);
                                                pcie_capability_read_word(dev, PCI_EXP_RTCTL,  &cap[i++]);
                                                pcie_capability_read_word(dev, PCI_EXP_DEVCTL2, &cap[i++]);
                                                pcie_capability_read_word(dev, PCI_EXP_LNKCTL2, &cap[i++]);
                                                pcie_capability_read_word(dev, PCI_EXP_SLTCTL2, &cap[i++]);
                                                return 0;
                                                
                                # drivers/pci/pci.c:2746 (pci_finish_runtime_suspend)
                                        # drivers/pci/pci.c:2640 (pci_target_state)
                                        
                                # ...
                                # ...
                                # ...
                                
                                # drivers/pci/pci.c:2416 (__pci_pme_active)
                                        # drivers/pci/access.c:545 (pci_read_config_word)  # config_read: bdf: 00:03:0 addr: 78 data: 8 size: 2 bytes
                                        # drivers/pci/access.c:574 (pci_write_config_word) # config_write: bdf: 00:03:0 addr: 78 data: 8108 size: 2 bytes

                                # drivers/pci/pci.c:1412 (pci_set_power_state)
                                        # drivers/pci/pci.c:1339 (pci_set_low_power_state)
                                                # drivers/pci/access.c:545 (pci_read_config_word)  # config_read: bdf: 00:03:0 addr: 78 data: 108 size: 2 bytes
                                                # drivers/pci/access.c:574 (pci_write_config_word) # config_write: bdf: 00:03:0 addr: 78 data: 10b size: 2 bytes
                                                
                                        # drivers/pci/pci.c:68 (pci_dev_d3_sleep)
                
                # ...
                drivers/pci/probe.c:3306 (pci_unlock_rescan_remove)
                
        # REPEAT...
        [ 2812.018124] [JAY-DBG-PCI]drivers/pci/access.c:545 (pci_read_config_word) # config_read: bdf: 00:03:0 addr: 78 data: 10b size: 2 bytes
        [ 2812.018563] [JAY-DBG-PCI]drivers/pci/pci.c:1120 (pci_platform_power_transition)
        [ 2812.018565] [JAY-DBG-PCI]drivers/pci/pci.c:1018 (platform_pci_set_power_state)
        [ 2812.018566] [JAY-DBG-PCI]drivers/pci/pci-acpi.c:1047 (acpi_pci_set_power_state)
        [ 2813.034196] [JAY-DBG-PCI]drivers/pci/pci.c:2380 (pci_pme_list_scan)
        [ 2813.034238] [JAY-DBG-PCI]drivers/pci/pci.c:2343 (pci_pme_wakeup)
        [ 2813.034238] [JAY-DBG-PCI]drivers/pci/pci.c:2308 (pci_check_pme_status)
        [ 2813.034239] [JAY-DBG-PCI]drivers/pci/access.c:545 (pci_read_config_word) # config_read: bdf: 00:03:0 addr: 78 data: 10b size: 2 bytes
        [ 2814.058170] [JAY-DBG-PCI]drivers/pci/pci.c:2380 (pci_pme_list_scan)
        [ 2814.058177] [JAY-DBG-PCI]drivers/pci/pci.c:2343 (pci_pme_wakeup)
        [ 2814.058177] [JAY-DBG-PCI]drivers/pci/pci.c:2308 (pci_check_pme_status)
        [ 2814.058178] [JAY-DBG-PCI]drivers/pci/access.c:545 (pci_read_config_word) # config_read: bdf: 00:00:0 addr: 78 data: 10b size: 2 bytes
        [ 2815.082026] [JAY-DBG-PCI]drivers/pci/pci.c:2380 (pci_pme_list_scan)
        [ 2815.082031] [JAY-DBG-PCI]drivers/pci/pci.c:2343 (pci_pme_wakeup)
        [ 2815.082032] [JAY-DBG-PCI]drivers/pci/pci.c:2308 (pci_check_pme_status)
        [ 2815.082033] [JAY-DBG-PCI]drivers/pci/access.c:545 (pci_read_config_word) # config_read: bdf: 00:03:0 addr: 78 data: 10b size: 2 bytes
        [ 2816.106322] [JAY-DBG-PCI]drivers/pci/pci.c:2380 (pci_pme_list_scan)
        [ 2816.106335] [JAY-DBG-PCI]drivers/pci/pci.c:2343 (pci_pme_wakeup)
        [ 2816.106337] [JAY-DBG-PCI]drivers/pci/pci.c:2308 (pci_check_pme_status)
        [ 2816.106339] [JAY-DBG-PCI]drivers/pci/access.c:545 (pci_read_config_word) # config_read: bdf: 00:03:0 addr: 78 data: 10b size: 2 bytes
        [ 2817.130301] [JAY-DBG-PCI]drivers/pci/pci.c:2380 (pci_pme_list_scan)
        [ 2817.130306] [JAY-DBG-PCI]drivers/pci/pci.c:2343 (pci_pme_wakeup)
        [ 2817.130307] [JAY-DBG-PCI]drivers/pci/pci.c:2308 (pci_check_pme_status)
        [ 2817.130307] [JAY-DBG-PCI]drivers/pci/access.c:545 (pci_read_config_word) # config_read: bdf: 00:03:0 addr: 78 data: 10b size: 2 bytes
        [ 2818.154190] [JAY-DBG-PCI]drivers/pci/pci.c:2380 (pci_pme_list_scan)
        [ 2818.154196] [JAY-DBG-PCI]drivers/pci/pci.c:2343 (pci_pme_wakeup)
        [ 2818.154197] [JAY-DBG-PCI]drivers/pci/pci.c:2308 (pci_check_pme_status)
        [ 2818.154198] [JAY-DBG-PCI]drivers/pci/access.c:545 (pci_read_config_word) # config_read: bdf: 00:03:0 addr: 78 data: 10b size: 2 bytes
        [ 2819.178095] [JAY-DBG-PCI]drivers/pci/pci.c:2380 (pci_pme_list_scan)
        [ 2819.178099] [JAY-DBG-PCI]drivers/pci/pci.c:2343 (pci_pme_wakeup)
        [ 2819.178100] [JAY-DBG-PCI]drivers/pci/pci.c:2308 (pci_check_pme_status)
        [ 2819.178101] [JAY-DBG-PCI]drivers/pci/access.c:545 (pci_read_config_word) # config_read: bdf: 00:03:0 addr: 78 data: 10b size: 2 bytes
        [ 2820.202248] [JAY-DBG-PCI]drivers/pci/pci.c:2380 (pci_pme_list_scan)
        [ 2820.202252] [JAY-DBG-PCI]drivers/pci/pci.c:2343 (pci_pme_wakeup)
        [ 2820.202252] [JAY-DBG-PCI]drivers/pci/pci.c:2308 (pci_check_pme_status)
        [ 2820.202253] [JAY-DBG-PCI]drivers/pci/access.c:545 (pci_read_config_word) # config_read: bdf: 00:03:0 addr: 78 data: 10b size: 2 bytes
        
        
        
The following is the detail lspci result for my PCIe devices

qemu@qemu:~$ sudo lspci -kvvv
00:00.0 Host bridge: Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
    Subsystem: Red Hat, Inc. QEMU Virtual Machine
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0


00:03.0 PCI bridge: Vadatech Inc. Device 1234 (rev 17) (prog-if 00 [Normal decode])
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin A routed to IRQ 23
    Region 0: Memory at 800000000 (64-bit, prefetchable) [size=64K]
    Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
    I/O behind bridge: 0000f000-00000fff [disabled]
    Memory behind bridge: c0900000-c09fffff [size=1M]
    Prefetchable memory behind bridge: 00000000c0800000-00000000c08fffff [size=1M]
    Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
    Expansion ROM at 80040000 [disabled] [size=32K]
    BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
        PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
    Capabilities: [74] Power Management version 3
        Flags: PMEClk- DSI- D1- D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [98] MSI: Enable- Count=1/32 Maskable+ 64bit+
        Address: 00000000fee00000  Data: 00ef
        Masking: 00000000  Pending: 00000000
    Capabilities: [b8] Express (v2) Root Port (Slot+), MSI 1a
        DevCap: MaxPayload 2048 bytes, PhantFunc 0
            ExtTag+ RBE+
        DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
            RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
            MaxPayload 128 bytes, MaxReadReq 512 bytes
        DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
        LnkCap: Port #0, Speed unknown, Width x8, ASPM L1, Exit Latency L1 unlimited
            ClockPM- Surprise- LLActRep+ BwNot+ ASPMOptComp+
        LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed unknown (ok), Width x8 (ok)
            TrErr- Train- SlotClk- DLActive+ BWMgmt+ ABWMgmt-
        SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise-
            Slot #0, PowerLimit 0.000W; Interlock- NoCompl-
        SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg-
            Control: AttnInd Unknown, PwrInd Unknown, Power- Interlock-
        SltSta: Status: AttnBtn- PowerFlt- MRL- CmdCplt- PresDet- Interlock-
            Changed: MRL- PresDet- LinkState-
        RootCap: CRSVisible+
        RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible+
        RootSta: PME ReqID 0000, PMEStatus- PMEPending-
        DevCap2: Completion Timeout: Range BC, TimeoutDis+ NROPrPrP- LTR-
             10BitTagComp+ 10BitTagReq+ OBFF Via message/WAKE#, ExtFmt+ EETLPPrefix+, MaxEETLPPrefixes 3
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS- LN System CLS 128byte cachelines, TPHComp+ ExtTPHComp- ARIFwd+
             AtomicOpsCap: Routing+ 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled, ARIFwd-
             AtomicOpsCtl: ReqEn- EgressBlck-
        LnkCap2: Supported Link Speeds: RsvdP, Crosslink- Retimer+ 2Retimers+ DRS+
        LnkCtl2: Target Link Speed: Unknown, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
             EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported, DRS+
             DownstreamComp: Reserved
    Capabilities: [100 v1] Extended Capability ID 0x2f
    Capabilities: [154 v3] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
        RootCmd: CERptEn+ NFERptEn+ FERptEn+
        RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd-
             FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0
        ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000
    Capabilities: [270 v1] Virtual Channel
        Caps:   LPEVC=0 RefClk=100ns PATEntryBits=8
        Arb:    Fixed- WRR32- WRR64- WRR128-
        Ctrl:   ArbSelect=Fixed
        Status: InProgress-
        VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
            Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
            Status: NegoPending- InProgress-
    Capabilities: [4ec v1] Root Complex Link
        Desc:   PortNumber=00 ComponentID=01 EltType=Config
        Link0:  Desc:   TargetPort=00 TargetComponent=00 AssocRCRB- LinkType=Config LinkValid-
            Addr:   00:00.0  CfgSpace=0000000000000000
    Capabilities: [738 v1] Root Complex Internal Link <?>
    Capabilities: [75c v1] Power Budgeting <?>
    Capabilities: [770 v1] Secondary PCI Express
        LnkCtl3: LnkEquIntrruptEn- PerformEqu-
        LaneErrStat: 0
    Capabilities: [794 v1] L1 PM Substates
        L1SubCap: PCI-PM_L1.2- PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1- L1_PM_Substates+
              PortCommonModeRestoreTime=150us PortTPowerOnTime=10us
        L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
               T_CommonMode=150us LTR1.2_Threshold=0ns
        L1SubCtl2: T_PwrOn=10us
    Capabilities: [7b4 v1] Physical Layer 16.0 GT/s <?>
    Capabilities: [7dc v1] Lane Margining at the Receiver <?>
    Capabilities: [83c v1] Data Link Feature <?>
    Capabilities: [874 v1] Extended Capability ID 0x2a
    Capabilities: [8b0 v1] Extended Capability ID 0x31
    Capabilities: [8d0 v1] Extended Capability ID 0x32
    Capabilities: [924 v1] Extended Capability ID 0x33
    Capabilities: [9d0 v1] Extended Capability ID 0x34
    Kernel driver in use: pcieport


01:00.0 Mass storage controller: Device 18ef:1234 (rev 49)
    Subsystem: Seeq Technology, Inc. Device 58e0
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin A routed to IRQ 11
    Region 0: Memory at c0800000 (64-bit, prefetchable) [size=1M]
    Region 2: Memory at c0900000 (64-bit, non-prefetchable) [size=1M]
    Expansion ROM at <ignored> [disabled]
    Capabilities: [44] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold-)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [50] MSI: Enable- Count=1/2 Maskable- 64bit+
        Address: 0000000000000000  Data: 0000
    Capabilities: [60] MSI-X: Enable- Count=19 Masked-
        Vector table: BAR=0 offset=00000000
        PBA: BAR=0 offset=00000130
    Capabilities: [6c] Express (v2) Endpoint, MSI 0f
        DevCap: MaxPayload 1024 bytes, PhantFunc 7, Latency L0s unlimited, L1 <64us
            ExtTag+ AttnBtn- AttnInd+ PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W
        DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
            RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
            MaxPayload 128 bytes, MaxReadReq 512 bytes
        DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
        LnkCap: Port #0, Speed unknown, Width x8, ASPM L1, Exit Latency L1 unlimited
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta: Speed unknown (ok), Width x8 (ok)
            TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range B, TimeoutDis+ NROPrPrP- LTR+
             10BitTagComp+ 10BitTagReq- OBFF Via message/WAKE#, ExtFmt+ EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS- TPHComp+ ExtTPHComp+
             AtomicOpsCap: 32bit+ 64bit+ 128bitCAS+
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
             AtomicOpsCtl: ReqEn-
        LnkCap2: Supported Link Speeds: RsvdP, Crosslink- Retimer+ 2Retimers+ DRS+
        LnkCtl2: Target Link Speed: Unknown, EnterCompliance- SpeedDis-
             Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
             Compliance De-emphasis: -6dB
        LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
             EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [100 v1] Extended Capability ID 0x2f
    Capabilities: [1e8 v3] Advanced Error Reporting
        UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
        UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
        CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
        CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
        AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn- ECRCChkCap+ ECRCChkEn-
            MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
        HeaderLog: 00000000 00000000 00000000 00000000
    Capabilities: [240 v1] Virtual Channel
        Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
        Arb:    Fixed- WRR32- WRR64- WRR128-
        Ctrl:   ArbSelect=Fixed
        Status: InProgress-
        VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
            Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
            Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
            Status: NegoPending- InProgress-
    Capabilities: [31c v1] Device Serial Number ab-cd-ef-01-23-45-67-89
    Capabilities: [32c v1] Latency Tolerance Reporting
        Max snoop latency: 0ns
        Max no snoop latency: 0ns
    Capabilities: [344 v1] Secondary PCI Express
        LnkCtl3: LnkEquIntrruptEn- PerformEqu-
        LaneErrStat: 0
    Capabilities: [3e0 v1] Address Translation Service (ATS)
        ATSCap: Invalidate Queue Depth: 1b
        ATSCtl: Enable-, Smallest Translation Unit: 00
    Capabilities: [640 v1] Page Request Interface (PRI)
        PRICtl: Enable- Reset-
        PRISta: RF- UPRGI- Stopped+
        Page Request Capacity: 0000071f, Page Request Allocation: 00000000
    Capabilities: [684 v1] L1 PM Substates
        L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2- ASPM_L1.1- L1_PM_Substates+
              PortCommonModeRestoreTime=106us PortTPowerOnTime=10us
        L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
               T_CommonMode=0us
        L1SubCtl2: T_PwrOn=10us
    Capabilities: [a18 v1] Physical Layer 16.0 GT/s <?>
    Capabilities: [e40 v1] Lane Margining at the Receiver <?>
    Capabilities: [e70 v1] Data Link Feature <?>
    Capabilities: [e80 v1] Extended Capability ID 0x2a
    Capabilities: [ea8 v1] Extended Capability ID 0x31
    Capabilities: [ec0 v1] Extended Capability ID 0x32
    Capabilities: [f00 v1] Extended Capability ID 0x33
    Capabilities: [f2c v1] Extended Capability ID 0x34
    Capabilities: [fc8 v1] Extended Capability ID 0x2e

New contributor

Jay Chen is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa Dịch vụ tổ chức sự kiện 5 sao Thông tin về chúng tôi Dịch vụ sinh nhật bé trai Dịch vụ sinh nhật bé gái Sự kiện trọn gói Các tiết mục giải trí Dịch vụ bổ trợ Tiệc cưới sang trọng Dịch vụ khai trương Tư vấn tổ chức sự kiện Hình ảnh sự kiện Cập nhật tin tức Liên hệ ngay Thuê chú hề chuyên nghiệp Tiệc tất niên cho công ty Trang trí tiệc cuối năm Tiệc tất niên độc đáo Sinh nhật bé Hải Đăng Sinh nhật đáng yêu bé Khánh Vân Sinh nhật sang trọng Bích Ngân Tiệc sinh nhật bé Thanh Trang Dịch vụ ông già Noel Xiếc thú vui nhộn Biểu diễn xiếc quay đĩa Dịch vụ tổ chức tiệc uy tín Khám phá dịch vụ của chúng tôi Tiệc sinh nhật cho bé trai Trang trí tiệc cho bé gái Gói sự kiện chuyên nghiệp Chương trình giải trí hấp dẫn Dịch vụ hỗ trợ sự kiện Trang trí tiệc cưới đẹp Khởi đầu thành công với khai trương Chuyên gia tư vấn sự kiện Xem ảnh các sự kiện đẹp Tin mới về sự kiện Kết nối với đội ngũ chuyên gia Chú hề vui nhộn cho tiệc sinh nhật Ý tưởng tiệc cuối năm Tất niên độc đáo Trang trí tiệc hiện đại Tổ chức sinh nhật cho Hải Đăng Sinh nhật độc quyền Khánh Vân Phong cách tiệc Bích Ngân Trang trí tiệc bé Thanh Trang Thuê dịch vụ ông già Noel chuyên nghiệp Xem xiếc khỉ đặc sắc Xiếc quay đĩa thú vị
Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa
Thiết kế website Thiết kế website Thiết kế website Cách kháng tài khoản quảng cáo Mua bán Fanpage Facebook Dịch vụ SEO Tổ chức sinh nhật