8 Page request queue

Chapter 8 Page request queue The message format arriving on the PRI queue in response to an incoming PCIe PRI message closely follows the PCIe packet format. The field names have been generalized for consistency with terminology elsewhere in this specification. PRI messages are little-endian. The system must ensure that the mapping between StreamID and RequesterID or Root Complex is identical whether used in either incoming or outgoing directions for PRI, ATS or normal DMA traffic. Page Address [63:12] 127 96 Page Address [63:12] 95 76 RES0 75 73 PRGIndex 72 64 SSV 63 L 62 W 61 R 60 X 59 58 RES0 57 52 SubstreamID 51 32 Priv StreamID 31 0 • StreamID[31:0] – When used with ATS/PRI, StreamID[15:0] must equal PCIe RequesterID[15:0]. Bits StreamID[n:16] might indicate different Root Complex sources where more than 16 bits of StreamID are implemented. • SSV: Substream valid – When 1, the Page Request was issued with a PASID TLP prefix. – When 0, the Page Request was not issued with a PASID. This value is also 0 when Substreams are not supported by the SMMU. When SSV == 0, the X and Priv bits are both 0. ARM IHI 0070 H.a Copyright © 2016-2026 Arm Limited or its affiliates. All rights reserved. Non-confidential 972

Chapter 8. Page request queue • SubstreamID[19:0] – This value is UNKNOWN when SSV == 0. • Page address[63:12] • PRGIndex[8:0] • Last, W, R, X, Priv: Last/Write/Read/eXecute/Privileged: – These bits encode requested page permissions – ‘Last’ is set on the last request in a PRG – The PCIe ‘Stop PASID’ marker message delimits Page Request message groups as a request with LWR == 0b100. Stop Markers have a PASID, that is, SSV == 1. A message that arrives without a PASID and that has LWR == 0b100 is not a Stop Marker, has SSV == 0, and is treated as a PRI Page Request. Several related PRI Page Requests (PPRs) might arrive in batches, named Page Request Groups (PRGs). Several pages might be requested in a PRG and all arrive with the same PRGIndex value. The last page request in a PRG is marked by the Endpoint as Last == 1 and the PRI queue can be configured to send an interrupt when Last == 1 is received (see SMMU_PRIQ_IRQ_CFG2. Member requests of a PRG are not guaranteed to be contiguous in the PRI queue as requests of more than one PRG might be interleaved. PPRs that are members of the PRG are not guaranteed to appear in the PRI queue in the order that they were issued from an endpoint, with the exception of the Last == 1 entry which is not re-ordered with respect to prior PPRs. The SMMU is not required to track and verify that, for a given PRGIndex value, all entries received in a PRG have identical PASID TLP prefixes. The CMD_PRI_RESP command causes a PRI response to be sent to an endpoint and takes a success or failure parameter to inform the endpoint of the overall status of the PRG. When Substreams (PASIDs in this context) are supported and this command is used with a valid SubstreamID (SSV == 1), a PASID tag is present on the response. Note: Arm expects this command to be issued after all PPRs in a PRG have been processed, including the Last == 1 entry. Note: The PRI queue does not overflow with correct software usage and endpoint credit management, because software grants credits to each device (using PCI configuration space) which sum to the amount of space in the PRI queue minus an allowance for any ‘Stop PASID’ markers that are required. The device can only issue as many outstanding PPRs as it has credits for. The SMMU freely accepts incoming PRI messages and puts them in the PRI queue, if space is free and the queue is enabled and not in an error condition. If the queue is full a programming error (or protocol failure by PCIe endpoint) has occurred. Note: A device is intended to always receive a response to a PRG, whether positive or negative. Note: A CMD_PRI_RESP returns a credit to the Endpoint that could cause another PPR. This must be considered when calculating credit capability of a queue. If the number of credits granted to devices equals the queue capacity, a PPR must be consumed from the queue (freeing an entry) prior to issuing a CMD_PRI_RESP for that PPR. The SMMU supports a maximum PRI queue size of 219 entries. Note: When a guest VM manages a PRI queue for page request service from one or more devices assigned to that guest, it is up to the guest OS to allocate and size its local PRI queue. It will subsequently give credits to the devices to allow them to issue PPRs, using configuration space accesses. In this scenario, the hypervisor manages the actual SMMU PRI queue and the sum of the credits given to all clients of that queue must not be greater than the size of the queue. For safety, Arm expects that a hypervisor will trap the write of a credit value of a guest to a device, and substitute its own allocation if necessary. This would ensure that the total number of credits cannot exceed the PRI queue size, even if a guest OS requests a larger number of credits to be used. The hypervisor is also expected to choose an appropriate PRI queue size which will approximate the sum of credits reasonably expected to be used by all guest VMs using the PRI facilities of an SMMU. ARM IHI 0070 H.a Copyright © 2016-2026 Arm Limited or its affiliates. All rights reserved. Non-confidential 973

Chapter 8. Page request queue 8.1. PRI queue overflow 8.1 PRI queue overflow The PRI queue enters an overflow condition when the following conditions apply: • The PRI queue is enabled, as indicated by the effective value of SMMU_CR0.PRIQEN == 1. • One or more PRI messages are received by the SMMU while the PRI queue is full, as determined by the general queue semantics. The SMMU indicates that this has happened by toggling the SMMU_PRIQ_PROD.OVFLG flag, provided that an overflow condition is not already present. The overflow condition is acknowledged by writing the SMMU_PRIQ_CONS.OVACKFLG flag to the same value as OVFLG. An active overflow condition inhibits new entries from being written to the PRI queue. Note: This behavior differs from the active overflow condition on an Event queue. See also section 7.4 Event queue overflow. Note: On detection of overflow, Arm recommends that software processes the received PRI queue entries as quickly as possible, and follows this with a single write of SMMU_PRIQ_CONS to simultaneously move on the RD pointer and acknowledge receipt of the overflow condition by setting OVACKFLG to the value read from OVFLG. When an overflow condition is active, incoming PRI messages are discarded and: • An automatic PRG Response is generated for Page Request messages having Last == 1, including the PRGI of the PPR that caused the overflow condition to become active. This response is returned to the endpoint that originated the PPR. The ResponseCode is defined below. Note: This behavior occurs only for a Page Request message and does not occur when a Stop Marker is discarded in an overflow condition. • PPRs having Last == 0 are silently ignored. Note: The auto-response behavior can maintain safe operation when PCIe endpoints fail to fully adhere to the PRI credit mechanism. Note: Although in an overflow condition the system software will not have addressed the cause of the PPR, on receipt of the automatic successful response Arm expects that the device will gracefully attempt an ATS request and PRI request again. On subsequent requests, system software might have had time to process the PRI queue contents and free space for the retries. Note: Auto-responses might be sent because of an overflow condition, or for other reasons. See section 8.2 Miscellaneous. When the SMMU supports PASIDs, an automatic PRG Response that is sent in response to a PRI queue overflow condition has the following properties: • If the associated Page Request message did not have a PASID, the response has no PASID and the ResponseCode field of the response is Success (0b0000). SMMU_IDR3.PPS and STE.PPAR do not affect this case. • If the associated Page Request message had a PASID: – If SMMU_IDR3.PPS == 1, then the response has the same PASID and the ResponseCode is Success (0b0000). STE.PPAR is not checked and its value is ignored. – If SMMU_IDR3.PPS == 0, the STE.PPAR field associated with the StreamID of the PPR is checked. The PASID depends on the associated STE: * If the associated STE is valid, the response has the same PASID if STE.PPAR == 1 and it has no PASID if STE.PPAR == 0. In both cases, ResponseCode is Success (0b0000). * If the associated STE is inaccessible or invalid, such that STE.PPAR cannot be checked, no PASID is used and the ResponseCode is Failure (0b1111). This occurs if SMMU_CR0.SMMUEN == 0, or ARM IHI 0070 H.a Copyright © 2016-2026 Arm Limited or its affiliates. All rights reserved. Non-confidential 974

Chapter 8. Page request queue 8.1. PRI queue overflow the StreamID is out of range of the Stream table, or an External Abort was encountered during the STE fetch or VMS fetch, or the STE is ILLEGAL. Note: In SMMUv3.2 or later, an STE access is permitted to also access the VMS if the VMS is enabled, which means that VMS access is permitted to occur as part of an STE.PPAR check and that access might give rise to this response failure case. If SMMU_CR2.RECINVSID == 1 and SMMU_CR2.REC_CFG_ATS == 1, and the SMMU encounters C_BAD_STREAMID while attempting to check STE.PPAR, the event is recorded in the Event queue. If SMMU_CR2.REC_CFG_ATS == 1, and the SMMU encounters F_STE_FETCH, F_VMS_FETCH, C_BAD_STE or F_CFG_CONFLICT while attempting to check STE.PPAR, the event is recorded in the Event queue. Otherwise, no event is recorded. Note: For a normal transaction, the same conditions lead to C_BAD_STREAMID, F_STE_FETCH, and C_BAD_STE, respectively. When PASIDs are not supported, an automatic response does not have a PASID prefix and the STE.PPAR field is not used. In this case, an implementation is permitted, but not required, to attempt to locate a valid STE and return either ResponseCode == Success or Failure in the manner described in this chapter even though STE.PPAR is not required. If the implementation does not perform this STE check, the ResponseCode == Success. Note: A PRI-capable endpoint advertises whether it expects a PASID prefix to be present on a PRG response returned to it in response to a Page Request made with a PASID prefix, through the PRG Response PASID Required flag in the PRI Status register of the endpoint. Arm expects the STE.PPAR field associated with an endpoint to be programmed to the same value as this flag in the endpoint. Stop PASID markers that arrive in an overflow condition are discarded and no auto-response is generated. Note: The PCIe specification [1] does not consider PRI Stop PASID markers to be credited in the same way as PPRs. If endpoints disrespect Page Request allocation and credits, Stop PASID markers cannot be guaranteed to be visible to the system, given this rule. Software must use other means to determine PASID stop status in the presence of non-compliant PCIe endpoints. 8.1.1 Recovery procedure At the time that an overflow is detected, the PRI queue might contain several groups of page requests that are legitimate, but might contain one or more groups of requests that have been truncated so that their ‘Last’ event was among those lost when overflow occurred, and these groups have had a ‘Success’ auto-response sent when the ‘Last’ event was discarded. Note: To re-synchronize and regain normal processing of page requests, one possible method is: • Process the entire queue: Entries are read from the SMMU_PRIQ_CONS.RD index up to the SMMU_PRIQ_PROD.WR index, sending Page Request Group Response commands using the Command queue as appropriate, when ‘Last’ entries are encountered. • The PRI queue might contain the start of groups of page requests whose Last requests were lost, having been received and auto-responded to after overflow. A group must be ignored by software if no Last event has been found for it up to the point that the SMMU last wrote (the SMMU_PRIQ_PROD.WR index). • Update SMMU_PRIQ_CONS.RD, including OVACKFLG, to clear the overflow condition and mark the entire queue as empty or processed. Note: After an overflow condition has been resolved, a PRGIndex value might be used in PPRs that later become visible, associated with a semantically different PRG to that of a PRG with entries in the pre-overflow PRI queue. This would occur when the queue enters overflow state before the Last PPR of the initial PRG can be recorded. The Last PPR has a successful auto-response sent, meaning the endpoint is free to re-use the PRGIndex value. ARM IHI 0070 H.a Copyright © 2016-2026 Arm Limited or its affiliates. All rights reserved. Non-confidential 975

Chapter 8. Page request queue 8.2. Miscellaneous 8.2 Miscellaneous An external abort detected while accessing the PRI queue, for example when writing a record, activates the SMMU_GERROR.PRIQ_ABT_ERR Global Error. If PRIQ_ABT_ERR becomes active, one or more PRI records might have been lost. The behavior of PRIQ_ABT_ERR relates to entries written to the PRI queue as the behavior of EVENTQ_ABT_ERR relates to entries written to the Event queue, as described in section 7.2.2 Event queue access external abort. It is IMPLEMENTATION DEFINED whether an external abort that is triggered by a write to the PRI queue is synchronous or asynchronous. When the write of a PRI queue entry results in a synchronous external abort, an automatic PRG Response with ResponseCode == 0b1111 is generated for the associated PPR. When the write of a PRI queue entry results in an asynchronous external abort, one or more queue entries might have been lost and no automatic response is generated for the PPRs that are associated with these entries. An implementation is permitted to read any address within the bounds of the PRI queue when the queue is enabled (that is, when the effective value of SMMU_CR0.PRIQEN == 1). In addition to writes of new records to the queue, such reads might also lead to an external abort. A PPR received from a Secure stream is discarded, is not recorded into the PRI queue and results in an automatic Response Message having ResponseCode == 0b1111. If an active GERROR.PRIQ_ABT_ERR error condition exists, or if the effective value of SMMU_CR0.PRIQEN == 0 (SMMU_CR0.SMMUEN == 0 implies PRIQEN == 0) all of the following apply: • No entries are written to the PRI queue. • All incoming PPRs cause an automatic PRG Response having ResponseCode == 0b1111 (‘Response Failure’) and the messages are discarded. Note: Incoming PRI Page Requests are not affected by SMMU_CR0.ATSCHK or STE.EATS configuration. The SMMU does not generate responses to Stop Marker messages. Automatically-generated PRG responses have the following properties: • Same StreamID as the PPR that triggered the response so that the message is returned to the originating RequesterID. • Same Page Request Group Index as the PPR that triggered the response. • Response Code depends on the cause of auto-generated response, see this section and sections 8.1 PRI queue overflow and 8.3 PRG Response Message codes. • No PASID prefix is used on responses that were auto-generated because the effective value of PRIQEN == 0 or PRIQ_ABT_ERR occurred. • A PASID prefix is used if all of the following are true: – The response was auto-generated because of PRI queue overflow. – The SMMU supports substreams. – SMMU_IDR3.PPS == 1, or SMMU_IDR3.PPS == 0 and STE.PPAR == 1. – The response will not have ResponseCode == 0b1111. – The PPR that triggered the auto-response had a PASID prefix. • If a PASID prefix is used, its PASID value matches that of the PPR that triggered the response. ARM IHI 0070 H.a Copyright © 2016-2026 Arm Limited or its affiliates. All rights reserved. Non-confidential 976

Chapter 8. Page request queue 8.3. PRG Response Message codes 8.3 PRG Response Message codes ResponseCode Status Returned for 0b0000 Success CMD_PRI_RESP with Resp == Success • Software has paged in all pages successfully. This response is returned for an auto-response on overflow where no PASID was supplied on the associated request, or SMMU_IDR3.PPS == 1, or no error is encountered in checking STE.PPAR. 0b0001 Invalid Request CMD_PRI_RESP with Resp == InvalidRequest • Software failed to page-in all pages. It encountered one or more pages that do not exist or have insufficient privileges 0b1111 Response Failure CMD_PRI_RESP with Resp == ResponseFailure • Software has determined that ATS is disabled for the stream, or some other (non-mapping-related) permanent failure. This response is returned for an auto-response on overflow where SMMU_IDR3.PPS == 0 and the associated request was supplied with a PASID and the SMMU is unable to locate a valid STE when checking STE.PPAR. This response is also returned for all incoming PPRs when any of the following conditions exist: • The effective value of PRIQEN == 0 (which might be because SMMU_CR0.SMMUEN == 0), • A GERROR.PRIQ_ABT_ERR error is active. • The PPR is received from a Secure stream. ARM IHI 0070 H.a Copyright © 2016-2026 Arm Limited or its affiliates. All rights reserved. Non-confidential 977