Comments

Heart Transplantation Committee | 10/01/2021

The OPTN Heart Transplantation Committee appreciates the opportunity to comment on the Enhance Transplant Program Performance Monitoring proposal. Committee members expressed concern over how the addition of offer acceptance criteria may negatively impact programs. Instead of receiving more specified offers, as a result of transplant programs using less expansive filters, programs will likely see a reduction in offers as a whole thus impacting other performance metrics. Alternatively, some programs allow for broader acceptance criteria for their candidates and if they decline those offers it would negatively impact the programs performance therefore causing additional stress on the system. Members noted that these types of regulatory changes impose additional administrative burdens on transplant programs with regard to compliance and access to information that can negate the intended efficiency of the proposal. The Committee also stated that updates to performance metrics were requested by the community and this proposal aims to achieve that.

Anonymous | 10/01/2021

The Data Advisory Committee appreciates the opportunity to provide feedback on MPSC’s proposal Enhance Transplant Program Performance Monitoring System. The Committee overall supports moving away from using post-transplant metrics as the sole indicator of performance. However, the Committee questions the description in the proposal under “Lack of OPTN Authority” which states that the OPTN is not currently charged, by regulation or contract, to collect data regarding the pre-listing phase and recommends considering pre-waitlisting metrics such as patient referral and evaluation data, if collected by the OPTN in the future, as an “aspirational or experimental monitoring metric”. The Committee recommends the consideration of equity in these new metrics and providing additional context to programs (e.g., stratifying the data by race or socioeconomic status) when sharing metrics to promote its usability in guiding practices to reduce inequities that may not be as identifiable when the data is risk adjusted. The Committee also recommends developing educational material for transplant program staff in order to mitigate potential behavior changes that result from reporting waitlist mortality as well as establishing a monitoring plan to identify whether the implementation of these changes is having unintended consequences on access to transplantation.

Heart Failure Society of America | 10/01/2021

Thank you for allowing the Heart Failure Society of America (HFSA) the opportunity to provide comments on the Organ Procurement and Transplantation Network’s (OPTN) proposal to enhance the Transplant Program Performance Monitoring System. HFSA appreciates MPSC’s effort to incorporate metrics that evaluate multiple phases of transplant care to create a more holistic approach to evaluation of transplant program performance. However, we have some concerns about the proposed metrics, which are detailed in the attached letter. Thank you for your consideration of HFSA’s comments.

View attachment from Heart Failure Society of America

Anonymous | 09/30/2021

Per the current SRTR lung transplant data, the programs that have the lowest pre-transplant mortality HRs are the programs with the low waitlist volumes. We are concerned that inclusion of this measure will force programs to try to improve this metrics by changing their listing practices. this may possibly result in delayed listings, and missed opportunities for transplant. Please review the SRTR lung data for WL mortality HR.

Hospital of the University of Pennsylvania | 09/30/2021

The Penn Transplant Institute has concerns about this new policy proposal. While we applaud the OPTN’s efforts to adopt performance metrics that are fair and allow for centers to impact their outcomes, we believe that the emphasis on mortality after listing and organ offer acceptance rates is likely to limit access to patients, particularly those who are hard to match and may have comorbidities that are not captured within the risk model. The methodology of the risk model for mortality after listing makes it difficult for programs to monitor this metric in a timeframe that will allow centers to make improvements in their outcomes. The challenge of monitoring this outcome in real time may cause centers to become overly conservative in which patients they decide to list. The policy proposal incentivizes not listing patients until they are almost certain to be transplanted, which limits transplant opportunities for less critical, but still severely ill patients, who may be more likely to do well post-transplant. The organ offer acceptance metric is extremely challenging to monitor proactively and will encourage centers to restrict which offers they will consider. This is both detrimental to patients who may have a difficult time finding a match, and could discourage centers from developing innovative practices for ensuring patient access to donors. We believe that the limitations associated with these metrics will not produce the desired goal of increasing the number of transplants performed. Instead, these metrics will encourage centers to become overly conservative in both their listing practices and their organ offer acceptance rates, which will result in limiting patients’ access to a life-saving treatment.

Jewish Hospital, UofL | 09/30/2021

Proposed Transplant Performance Metrics: 1. Waiting list mortality rate ratio: Waitlist mortality is an important metric for determining if individuals who are registered to the waitlist are receiving life-saving organ transplantation within the timeframe of when they are competitive for organ offers yet robust enough to survive being transplanted. Specifically for liver transplantation, while individuals listed with non-competitive (lower) MELDs may receive highly protocolized care by remaining registered on the waitlist, smaller centers, such as ours, have a limited bandwidth for the number of listed patients who can be managed by pre-transplant coordinators. As such, our center trends towards listing patients with higher MELDs who may have higher mortality rates. This performance metric could be punitive to smaller centers such as ours. 2. Offer acceptance rate ratio: Offer acceptance ratio will be a useful metric for determining the risk that a transplant center is willing to take by accepting organs with extended donor criteria including organs from donors deceased after cardiac death (DCD). In general, DCD organs convey risk for early and late allograft failure as well as long term complications. The offer acceptance rate ratio may be a helpful metric if stratified by donor types, (i.e. brain death, extended donor criteria, DCD etc.) with the acceptance of brain-dead donors being a primary metric. 3. 90-day graft survival hazard ratio: The Trager Transplant Center at Jewish Hospital fully supports the 90-day graft survival hazard ratio as an additional performance for transplant. The first 90 days after transplant is a critical timeframe where a patient’s overall post-transplant trajectory is often determined. While our Center strives to provide quality, evidenced-based care for every post-transplant patient, this metric will allow for careful data collection during the most critical period of a patient’s post-transplant course, yielding quality improvement initiatives on a local and national level. 4. One-year graft survival conditional on 90-day graft survival hazard ratio: This proposed metric will further solidify the importance of the one-year survival metric, which is already in place.

Kidney Care Partners | 09/30/2021

Please see the attached comment letter explaining KCP's recommendations for modifying the current proposal to align performance measures with those that apply to other stakeholders in the kidney care community to improve outcomes for patients with kidney disease. On behalf of Kidney Care Partners (KCP), I want to thank you for the opportunity to provide comments on the OPTN Membership and Professional Standards Committee (MPSC) “Enhance Transplant Program Performance” proposal (Proposal). Improving the availability of and access to solid organs, especially kidneys, is a priority for KCP and our members. We strongly agree that the monitoring program should seek to increase the number of transplants. KCP also supports efforts to lessen the emphasis on transplant outcomes such that the metrics do not inadvertently disincentivize including sicker patients on the waitlist. We are concerned that the proposed waitlist mortality measure will not achieve these goals if implemented because it does not align with the transplant waitlist measures that apply to dialysis facilities and nephrologists in the Medicare program. In addition, without modification, the measure will likely incentivize transplant centers to adopt criteria that result in sicker individuals living with kidney failure not being able to qualify for the waitlist. KCP is an alliance of members of the kidney care community that serves as a forum for patient advocates, dialysis care professionals, providers, and manufacturers to advance policies that support the provision of high-quality care for individuals with chronic kidney disease (CKD), including End-Stage Renal Disease (ESRD). Since 2005, KCP has led the kidney care community in its efforts to shift to a patient-centered, quality-based approach to providing kidney care in America. Through the Kidney Care Quality Alliance (KCQA), our members have developed measures, navigated them through the National Quality Forum’s (NQF) endorsement and maintenance processes, and advocated for their inclusion in the Medicare ESRD QIP and other quality programs. In the Spring of 2021, KCQA renewed its measure development agenda. Led by Drs. George Aronoff and Keith Bellovich as the co-chairs of the KCQA Steering Committee, KCQA is developing additional measures, including in the domain of kidney transplant. KCP supports the Biden-Harris Administration decision to continue the previous administration’s prioritization of increasing access to kidney transplant. Part of these efforts have included increasing accountability for dialysis facilities and nephrologists to support patients with kidney disease and/or kidney failure being added to transplant waitlists. There are transplant waitlist measures in the ESRD Quality Incentive Program (the traditional Mr. Rich Formica September 30, 2021 Page 2 of 4 Medicare value-based purchasing program for dialysis facilities) and the ESRD Treatment Choices (ETC) Model (the mandatory Medicare model that initially applies to 30 percent of the dialysis facilities and nephrologists in the United States). With more than 21,000 kidney transplants performed in 2018 and nearly 95,000 patients on current waitlists, nephrologists and facilities face substantial limits in what they can do to increase the number of kidney transplants. It is, therefore, critically important that the kidney care community and the transplant community work together to ensure that kidney care measures and transplant centers measures align to help encourage more patients with kidney failure to be added to transplant waitlists. We support MPSC’s efforts to improve monitoring, including creating new metrics, to support expanding access to transplants. We are concerned that the proposed waitlist mortality ratio measure will incentivize transplant centers that do not provide daily direct care to dialysis patients to avoid including higher-risk, sicker patients on their waitlists to improve overall mortality. Yet, these are the very patients on which the health care sector needs to focus to improve access to transplant. This result would lead to the opposite outcome that MPSC is trying to achieve – increasing the number of transplants. Avoiding such a consequence is also essential to addressing the gap in transplants that exists between Black and White patients. According to the U.S. Renal Data Systems 2020 Annual Report, the median wait-time for dialysis patients initially listed in 2013 was 49.2 months. Median wait-time was five years for Black patients who were initially wait-listed in 2013, but only 3.4 for years for White patients, a difference of more than 1.5 years. At the same time, there has been a slight improvement in transplant centers adding Black and Hispanic patients to their waitlist. USRDS reports that between 2017 and 2018, the number of Black patients who were added to the waitlist increased by 8.5 percent, which was slightly more than the corresponding change in White patients which was 8.0 percent. During that same time period, the number of Hispanic or Latino patients who were added to the waitlist increased by 13.1 percent compared to the corresponding change in non-Hispanic patients of 7.1 percent. It is important that the waitlist mortality measure, which we understand seeks to encourage transplant centers to move more patients to transplant, not result in fewer patients being waitlisted or erode the small improvements that are underway. While we appreciate that MPSC seeks to address this concern with the risk adjustment, the experience of KCP members suggests that this approach which was tried and failed to work in the past with a CMS metric. We ask that MPSC engage with the kidney care community, including KCP and our measure development sister organization the Kidney Care Quality Alliance (KCQA), to find a way to balance the need for the mortality measure with the dialysis facility and nephrologist measures so the incentives are aligned to increase access to the transplant waitlists and to move more waitlisted patients to transplant.

View attachment from Kidney Care Partners

Massachusetts General Hospital Transplant Center | 09/30/2021

We applaud efforts to develop a more comprehensive assessment of transplant program performance. While we support broadening the analysis beyond solely one-year patient and graft survival, we have concerns about the unintended consequences and misaligned incentives that may result from the proposed metric additions. We disagree that wait list mortality is an indicator of transplant program performance. Depending on the organ, it is neither feasible nor appropriate for the transplant center to manage the end stage organ disease, and wait list mortality depends as much, if not more, on unaffiliated hospitals’ and physicians’ disease management as it does on transplant center practices. This is particularly true for organs in which rank on the wait list does not depend on disease severity, but primarily on time. For example, wait time for a kidney can be up to 7 years in some regions of the country, and it seems unreasonable to hold the listing center responsible for outcomes during that time period, which are the result of dialysis care and management of the many co-morbidities that can accompany renal failure by the patients’ primary physicians. Although the intent is to make appropriate adjustments for any confounding factors, it is well recognized that the data collected on those is incomplete and inconsistent, and even if complete and consistent, still fails to capture all confounders. Even with data capture, risk adjustments can only be made for factors present in a large enough percentage of patients for the difference in risk to be statistically detectable. Wide geographic variation in organ availability, an important variable influencing wait list mortality, is not clearly accounted for under a uniform risk-management assessment tool as described in this proposal. Anatomic or other surgical issues that can make patients difficult to transplant or to find organs for are not well tracked by currently collected data. Of greatest concern is the potential for monitoring wait list mortality to disincentivize the listing of high-risk patients, whether from biologic or social determinants, and lead to more conservative listing practices. Already, centers are modifying their listing practices by holding back patients from listing until they are more likely to receive an offer in an attempt to improve metric performance. Our most vulnerable populations will be disproportionately impacted by such practices, limiting access to care and widening the gap in transplant disparities. We currently have no mechanisms in place to monitor the referral, evaluation, and listing of patients with end stage organ disease, and the risks of further obscuring the outcomes of unlisted patients should be seriously weighed against any benefits of a policy that discourages broader listing practices. Similarly, offer acceptance rates are not clearly correlated with transplant center performance. It is appropriate and, in fact, desirable that a transplant center decline organs for a patient who may be at the top of the list but inappropriate for that particular organ, so that it can be used to better effect for a patient further down the list, even if that patient is at a different center. As organ quality is a combination of multiple factors, it is not possible to rule out all, or even the majority, of inappropriate or suboptimal organ offers by fiddling with the limited screening criteria available in the current “all or nothing” format, without also filtering out many appropriate organs. It is neither feasible nor in the patients’ best interest to adjust screening criteria or active/inactive status in real time in response to the candidates’ fluid physiologic status, especially when match runs are often done several hours before offers are sent out, and sometimes days before the organs are procured. Furthermore, the ability to adjust offer acceptance rates by using offer filters is also the greatest risk of monitoring this metric. It is not even an unintended consequence, as the proposal encourages the use of filters to decrease the denominator of organ offers that a center’s acceptances will be judged against. Discouraging centers from even considering organs for transplant seems inconsistent with the currently stated goal of increasing transplantation and the elsewhere stated goal of increasing organ utilization. The stated benefit of decreasing the time from the OPO sending out the offer to obtaining an offer acceptance seems unworthy of the risk of increasing organ discards and decreasing the number of transplants. Suzanne Algeri, Associate Chief Nurse, MGH Todd Astor, Medical Director, MGH Lung Transplant Emily Bethea, Associate Clinical Director, MGH Liver Transplant Ray Chung, Medical Director, MGH Liver Transplant David D’Alessandro, Surgical Director, MGH Heart Transplant Nahel Elias, Surgical Director, MGH Kidney Transplant Jay Fishman, Associate Director, MGH Transplant Center Nat Langer, Surgical Director, MGH Lung Transplant Joren Madsen, Director, MGH Transplant Center Jim Markmann, Clinical Operations Director, MGH Transplant Center Paul Myoung, Senior Administrative Director, MGH Transplant Center Elahna Paul, Medical Director, MGH Pediatric Kidney Transplant Dan Pratt, Clinical Director, MGH Liver Transplant Leo Riella, Medical Director, MGH Kidney Transplant Uzma Shah, Medical Director, MGH Pediatric Liver Transplant Win Williams, Director, MGH Center for Diversity and Inclusion Heidi Yeh, Surgical Director, MGH Pediatric Transplant

Hackensack University Medical Center | 09/30/2021

While HUMC is confident that the new metrics will not have a profound effect on our current standard of care and transplant success, we do have concerns that implementing these types of changes amid a global pandemic and so soon after the implementation of new kidney allocation rules, could lead to unintended consequences, including fewer transplants. Our specific concerns are outlined in more detail in the attached letter.

View attachment from Hackensack University Medical Center

Colleen O'Donnell Flores | 09/30/2021

We are members of the organ transplant team at Northwell Health, where we have active adult kidney, adult liver, pediatric kidney and adult heart transplant programs. We wish to provide feedback on the proposed Enhancement of the Transplant Program Performance Monitoring Systems. Overall, despite best intentions, when programs are flagged, it causes programs to restrict activities and often, decrease transplant rates. We hope the OPTN will take this opportunity to provide transplant programs with one-on-one counsel and advice. We would like to see a proactive approach to performance improvement. As a membership organization, we believe outreach to members to improve metrics, routinely and proactively, would be a good model. Dashboards • Programs need real time access to their own MPSC dashboards to see where they are and how they are modeling. If we are able to see our own dashboards, then we can incorporate these into robust transplant QAPI programs and align improvements. • We would like to know the timeframe for reporting. Will it be every 6 months like the PSRs? Or a rolling review? Long term post-transplant monitoring • We support long term post-transplant monitoring, however 5 years is too long. Programs change, rules and regulations change and technology changes. We do not see this as a meaningful metric. • We would like post-transplant monitoring to account for patient deaths with functioning grafts. • The proposed cohort is 7/2019-12/2021 – we believe these model need to account / isolate COVID deaths Tools resources needed to implement: • Organ offer acceptance rate – Programs need a way to say yes/no to turning off organ offers in the category as ‘hard to place’. Currently, in UNet, we may isolate organ offers by categories such as DCD, distance, etc. but not a way to limit offers from “hard to place”. • The proposal asks, “what do we need to describe these metrics to patients”. We believe the SRTR has information appropriate to the public. We believe that OPTN MPSCH dashboards should be offered as peer review initially and not made available to the public during a pilot or trial period. Thank you, Lewis Teperman, MD, Director of Organ Transplant, Northwell Health Colleen O'Donnell Flores, MHA, Organ Transplant Regulatory Affairs

Kidney Care Council | 09/30/2021

Thank you for the opportunity to provide public comments on the OPTN Membership and Professional Standards Committee (MPSC) “Enhance Transplant Program Performance” proposal (MPSC Proposal). Kidney Care Council (KCC) members collectively provide life-sustaining kidney care and dialysis treatment to 95 percent of individuals living with kidney failure, or End Stage Renal Disease (ESRD), in the United States. KCC members2 constitute a diverse coalition of small, medium, and large businesses, are organized as both for- and not-for-profit organizations, employ tens of thousands of dedicated health care practitioners, and deliver quality care in urban, suburban, and rural communities across the country. KCC members deliver dialysis treatments in more than 6,000 facilities; train, support, and manage the care of thousands of patients who have elected home hemodialysis or home peritoneal dialysis; and support patients waiting for a kidney transplant. Kidney care and dialysis providers are responding to COVID-19 by marshalling every possible clinical and operational solution we can develop during a dynamic and evolving public health emergency. KCC members are committed to help our patients be in the best position possible to be included on a transplant waitlist and to receive a transplant. We support efforts of the transplant center providers to create meaningful measures to establish transparency with the goal of increasing the number of transplants. Despite the good intent of the current proposed measures, it appears that the transplant waitlist measures will create the unintended consequence of encouraging transplant centers to avoid waitlisting the very patients who have had difficulty accessing transplant waitlists and receiving organ transplants. The perverse incentive comes from the fact that less than perfect patients will be counted against the transplant center. In addition to the negative impact on patients seeking a transplant, the proposed transplant waitlist measure is not aligned with the waitlist measures used in the Medicare program for nephrologists and dialysis facilities. In the ESRD Treatment Choices (ETC) Model, these providers are held accountable for increasing the number of dialysis patients on transplant waitlists. It also includes measures for pre-emptive transplants for nephrologists. Similarly, the ESRD Quality Incentive Program (QIP) for dialysis facilities includes the Percentage of Prevalent Patients Waitlisted (PPPW) measure. Both the ETC Model and the ESRD QIP programs apply payment cuts to nephrologists and providers that fail to meet the benchmarks. CMS has adopted these measures to incentivize these providers to help patients navigate the waitlisting process. It is important for transplant center measures to take these metrics in account so that the incentives are aligned to make sure the patient is receiving the care that they need. While the MPSC has included risk adjusters to try to address the potential problem KCC has highlighted, past experience demonstrates that the risk adjusters are unlikely to be robust enough to overcome the incentive to avoid including higher risk patients on the transplant waitlist. We believe that it is possible to appropriately address the issue and would welcome the chance to share our recommendations with MPSC. Finally, we ask that UNOS and the Department of Health and Human Services provide more information about the data collection envisioned for this measure to allow us to evaluate the impact of the measure. It is important that the measure developer engage actively with dialysis facilities as well to ensure that a measure like this one will accomplish the goal of improving waitlist options for underserved patient populations. Addressing the problem is particularly important to addressing inequities in transplant because dialysis patients are disproportionately from communities of color. Black and Hispanic dialysis patients continue to experience a gap in access to transplant. While socio-demographic factors play a role, it is also true that these patients face the challenge of multiple co-morbidities, lack of wrap-around insurance coverage, and other risk factors that can increase their risk of death as they wait for a transplant. When they are waitlisted, these patients experience longer wait-times. For example, the U.S. Renal Data System 2020 report indicates that the median waittime for Black patients who were first waitlisted in 2013 was five years compared to 3.4 years for White patients. Transplant centers, dialysis facilities, and nephrologists need to work together to address these gaps. It is important that UNOS monitoring measures help eliminate this problem and do not inadvertently make it worse. KCC greatly appreciates the opportunity to provide comments on these important issues. We would welcome the opportunity to with MPSC to make sure that the transplant waitlist measure aligns with the existing waitlist measures of other providers who care for these patients every day. Please contact me at ccepriano@kidneycarecouncil.org or (202) 744-2124 to discuss any of these issues further.

View attachment from Kidney Care Council

The National Kidney Foundation | 09/30/2021

The National Kidney Foundation (NKF) appreciates the opportunity to comment on the MPSC proposal, “Enhance Transplant Program Performance Monitoring System.” NKF is committed to leveraging the tools of measurement, transparency, and oversight to improve the patient-centricity of kidney transplant. This means centers increase access to transplant by both supporting and maximizing living donor opportunities and accepting organs that align with patient’s preferences, provide an efficient process that makes best use of patients’ time, communicate effectively, are not unduly risk averse, provide equitable care, and, within reason, help patients overcome financial, social, psychological, and clinical barriers to transplant. Summary Recommendations: • Ensure the MPSC performance monitoring strategy is driving positive behavior changes and having a tangible impact on patient access to and experience of receiving a transplant. • Provide a summary in the final proposal of the expected impact of the implementation of the measures on patients in clear, unambiguous language • Remove the proposed measure of waitlist mortality rate for kidney programs given long waiting times and the potential for risk aversion. • NKF supports the proposed measures of offer acceptance rate and perioperative (90 day) and postoperative outcomes. • Ensure that relevant results are transparent to all appropriate stakeholders, including patients, caregivers, and referring providers. • UNOS should enhance its data collection and reporting efforts, including of processes of care prior to listing, in order to develop and implement more patient-centered measures of transplant center performance via a meaningful performance monitoring and improvement strategy. • SRTR “Task Five” should develop patient-reported measures of the transplant process, a waitlist management process measure, a measure of shared decision-making regarding acceptance, a measure of five-year outcomes, and a quality-of-life metric While we acknowledge that transplant centers have different areas of specialization and levels of buy-in from hospital administrators, NKF’s goal is that all centers share an obligation to strive for the highest possible levels of performance on domains that matter to patients and that are relevant across all centers. Metrics are important levers to assess and incentivize changes to improve center performance and for the purpose of quality assurance. We strongly support replacing measures, specifically one-year patient and graft survival, that can increase the potential for risk aversion in recipient listing and donor acceptance. We further support the move to measures that holistically evaluate transplant program performance. In theory, if designed and operationalized properly, measures can drive positive programmatic changes that serve patients and increase organ utilization. In order for the proposed measures to drive meaningful change for patients, the MPSC must implement them in the context of a performance monitoring and improvement strategy that is purposeful and effective. We are concerned that the current performance monitoring strategy is not incentivizing performance improvement and that incorporating the new measures into the current strategy may not noticeably change either patient access to transplant or the patient experience of transplant. Our concern is based the impact of the one-year patient and graft survival measure, which is often cited as contributing to risk aversion. We hypothesize that the use of the measure by CMS in the transplant center Conditions of Participation had a greater impact on risk aversion than the MPSC performance monitoring strategy based on the measure. Conversely, CMS’ decision to remove the one-year patient and graft survival measure from the Conditions of Participation has had a much greater impact on diminishing risk aversion than changes to either the MPSC process or the measures used in that process would. We believe that MPSC and UNOS have an obligation to proactively improve performance nationally by aggressively closing gaps in performance. This will be challenging to achieve if, as the proposal notes, there will be no change in transplant centers’ obligations to participate in the performance review process. The proposal to create a “yellow zone” or programs that are identified as needing performance improvement should be accompanied by requirements for meaningful interaction with the OPTN to implement quality improvement programs. This should not be elective, even if it remains confidential. The outcomes of performance reviews should be evaluated and documented such that evidence that oversight is impactful can be evidence based. This should include measurement of both intended and unintended components of quality reviews. Generally, we applaud the MPSC for taking on the often-fraught effort to develop performance measures. We encourage the MPSC to carefully consider how this proposal can be operationalized to have a constructive impact on patients, both those who are waiting for a transplant and those who could be transplanted but have not gained access to the system. One simple example would be to ensure the proposed measures are appropriately transparent to key stakeholders, especially patients, who deserve access to data that allow them to make informed decisions about transplant programs. We recommend the MPSC include a section in the final proposal that describes how these measures in isolation and their implementation will benefit patients. This section could be included in “Bylaw Evaluation" Developing New Metrics We understand that this proposal reflects numerous constraints faced by the MPSC including limited data, elements of performance outside the scope of the OPTN contract, and the need to develop measures relevant across organ types. We strenuously recommend that UNOS enhance its data collection apparatus will the goal of developing more patient-centered measures of transplant. This can be achieved through innovative uses of EMRs and other tools, which could provide enhanced granularity without creating unreasonable burdens. UNOS should expand the collection of patient-reported data to inform the development of Patient Report Outcome Measures (PROMs). Patients and living donors care very deeply not just about the outcome of transplantation, but also the opportunity to enter into the process of receiving a transplant and the efficiency and effectiveness of the process itself. Enhancing the collection and reporting of data should be a preliminary step in the development of new metrics. Accordingly, we strongly support HRSA’s collection of data on the early steps to transplant, prior to waitlisting, at which point OPTN’s oversight of the process begins. We ask HRSA to ensure that these data (e.g., evaluation rate as a % of all referrals, time to listing from referral and proportion of referred patients that are listed withing a predefined time interval e.g., 1 year) are collected and reported. We would also note the value of collaborating with the ESRD Networks and USRDS to align data collection and reporting efforts. The pre-waitlisting steps in the process, such as transplant referrals, the time-to-evaluation for transplant candidates and donors, the length of the evaluation, and the efficiency of listing can and should inform the evaluation of transplant center performance. In addition, NKF will be advocating for enhanced data collection across all elements of the transplant process that we expect will be able to inform the development of future performance metrics. We recommend that the MPSC’s performance monitoring effort align with SRTR’s “Task Five.” Task Five should develop patient-reported measures of the transplant process, a measure of shared decision-making regarding acceptance, a measure of five-year outcomes to incentivize more intensive post-transplant care, and a quality-of-life metric to supplement the current and proposed outcome measures. Measures Selected 1. Waitlist Mortality Ratio NKF resolutely opposes the use of this measure for kidney transplant programs. We request that the MPSC remove it from consideration. Being managed on the waitlist is of the utmost importance to patients. Patients with whom we consulted with about this proposal compared the waitlist to a “black box.” Another said, “it was like being in a dark room. No staff talked about it to me.” Yet, access to the waitlist is phenomenally important to patients because it reflects that transplant is a possibility, if not a guarantee. We do not believe a heightened focus on waitlist mortality will incentivize the kinds of behaviors that patients perceive as important. In fact, we fear that this measure will result in unintended consequences with profound implications, such as denial of access to transplantation for patients perceived at higher risk of death on the waitlist and thus that are considered for susceptible to poor outcomes. Transplant centers already exhibit high levels of selectivity with regards to the patients they are willing to list, as evidenced by increased waitlist removals, increased survival after removal and decreasing mortality on the waitlist. Every effort must be made to ensure that patients are given an equal opportunity to be considered for transplantation. A recent analysis by Schold et al. found that access to the kidney transplant waitlist has not improved in the more than two decades since 1997 and has actually fallen among structurally disadvantaged populations. We are extremely concerned by the potential for a waitlist mortality measure to exacerbate disparities in access to the waitlist. There is compelling evidence that the number of patients that could benefit from transplantation is highly misrepresented by the patient cohorts who are placed on the waiting list. A measure of waitlist mortality is likely to exacerbate this critical problem. We also disagree that waitlist survival is a proxy for “the overall patient experience throughout the transplant process.” While kidney transplant centers are not the sole provider of direct patient care for waitlisted patients, transplant centers are responsible for communicating with patients, ensuring readiness for transplant, and providing empowering education about living donation. As patients report above, remaining active on the list, knowing activation status, and time on the waitlist are more relevant than waitlist survival and would be more patient-centered measures of the patient’s experience of the process. Alternative metrics to assess and improve waitlist management (such as % of patients active on the waiting list, or % of total time on the waiting list spent in active status) could be considered instead. These metrics would hopefully have a lower risk of adverse selection.). NKF recommends removing the proposed waitlist mortality ratio measure in its current form. There is clear evidence that metrics can change behavior in an adverse manner than deleteriously affects access to transplantation and patient outcomes. The complexity of this measure, its association with factors outside the transplant center’s control including social determinants of health, and the desire to increase access to the list exceed the value of the waitlist mortality as currently being considered as a regulatory metric. Along with other data that provide essential context, waitlist mortality should, however, be collected and reported in a patient-friendly format. We would emphasize again that waitlist mortality should not be used for regulatory purposes. If a waitlist mortality measure is considered essential by the MPSC, we propose that mortality is measured only for patients who have already received an organ offer. This would adjust for the highly variable waiting times across different regions of the country and also mitigate the unintended incentive to delay listing until a patient has accrued adequate waiting time to get an offer quickly. Finally, flagging programs for waitlist mortality is misaligned with other transplant policy goals including increasing transplant referral and waitlisting rates as are being incentivized with policies stemming from the Executive Order to Advance American Kidney Health. CMS has demanded that dialysis facilities and nephrologists refer centers and measures the proportion of patients who are placed on the waitlist. These are endeavors that NKF supports and are diametrically opposed to a measure that encourages centers to selectively list only the best candidates. Kidney care is already heavily siloed. All stakeholders (nephrologists, dialysis facilities, and transplant centers) must have aligned incentives in order for policies that aim to increase access to transplantation to be effective. Ultimately quality assurance should provide comprehensive assessment of the processes of care for all potential transplant patients across providers of care and incentivize optimal quality irrespective of patient’s status as a late-stage chronic kidney disease or end-stage kidney disease patient or a transplant candidate or recipient. 2. Offer/Acceptance Ratio NKF strongly supports inclusion of a transplant center organ offer/acceptance rate measure. We believe this measure, if properly constructed and implemented as part of a meaningful performance improvement strategy, is fundamental to improving the current system. A key premise underlying an organ offer/acceptance rate measure is that kidneys that are currently discarded may have clinical value to patients when compared to dialysis or further time on the waitlist. NKF is a proponent of shared decision-making between patients and transplant centers. Patients’ willingness to accept less than perfect organs may change numerous times during a patient’s transplant journey. An organ offer/acceptance rate measure will further incentivize transplant programs to have regular conversations about patients’ preferences. For example, ensuring that centers are reviewing and confirming each patient’s signed consent form on organ offer preferences during reevaluation as age, circumstance, and preferences shift. Finally, an offer/acceptance ratio measure is aligned with the goal of increasing overall transparency to patients, particularly around kidneys that are declined on their behalf. The organ offer/acceptance rate measure must be risk-adjusted for cold ischemia time and patient demographics. Cold ischemia times are currently unknowable but could be modelled for the purposes of risk-adjusting the measure. This is essential to prevent transplant centers from being held accountable for accepting organs with unacceptably high cold ischemia times, especially in light of broader sharing. While an organ offer acceptance rate measure will encourage the appropriate use of UNOS filters by centers, increase the efficiency of the system and reduce organ discard from poor logistical coordination, the measure may be of limited value to patients who are trying to select centers with the highest probability of transplant after waitisting. We recommend that the organ offer/acceptance rate be made publicly available both with and without the exclusion of the bypassed offers. 3. 90-Day Graft Survival Some patients raised concerns that 90-day graft survival is not a patient-centered measure because patients assume that survival of their transplant for 90 days is a given. We understand, however, that 90-day graft survival is an important measure for capturing safety concerns resulting from poor organ selection and patient management. However, it is important to account for whether differences between centers are statistically and clinically relevant to necessitate intervention. The thresholds for identifying centers with underperformance should reflect the inability to account for all patient and donor factors and the incidence of random events that are not attributable to quality of care. Accordingly, NKF supports this measure. 4. Conditional One-Year Graft Survival For immediate purposes, NKF supports conditional one-year graft survival. We understand that graft survival is a component of performance and relevant to OPTN’s mandate to “monitor and review OPTN member performance, including threats to patient health.” As metric development proceeds, however we encourage the MPSC and SRTR to look beyond one-year graft survival in two seemingly contradictory directions. Though transplant centers are familiar with one-year graft survival, it is well characterized that the focus on relatively short-term survival of the patient and graft has contributed to a culture of risk-aversion at transplant centers. We are also concerned that one-year graft survival may be misaligned with efforts to list and transplant more patients and to accept and transplant more less-than-perfect kidneys. Up to a point, transplant centers should not be penalized for giving patients the chance to receive a transplant. We support the maintenance of exclusion of particular high-risk groups (e.g., EPTS>85 transplanted with KDPI>85 organs) from this metric. This is consistent with the European “old to old” transplant program that encourages organ utilization. Patients opt into this system, so it is patient centric. Conversely, most patients want and expect their transplants to last longer than one year. UNOS should track and SRTR should report 5-year outcomes compared with anticipated survival if they remained on dialysis. Five-year outcomes can also be a measure of transplant center performance. A longer-term outcome measure would also serve to incentivize effective long-term management of patients. Any outcome measures should be adjusted for acuity and social determinants of health so as not to disincentivize transplantation of higher-risk patients and should be measured relative to medical management. Ultimately, outcome measurement is challenged by the fact that patients’ desired outcomes must be placed into the context of the patient’s life and thus are not the same patient to patient. It is our hope that the MPSC can strike the right balance between ensuring patient safety and giving transplant centers the flexibility to make the acceptance decision that align with each patient’s values and preferences. In conclusion, the MPSC should emphasize the need for continuous learning and improving. Static performance is not sufficient. Based on the principals of the Institute for Health Care Improvement, we believe that transplant care has the potential to further improve. While short term outcomes are excellent, long-term outcomes continue to lag. At the same time, more than 100,000 Americans are waiting for transplant and many more are not represented due to lack of systematic data capture. As outlined in multiple federal initiatives, the status quo of facility-based dialysis care is not acceptable. Transplant must be a key part of the solution. We appreciate the MPSC’s attention to our comments. We would welcome the opportunity to discuss our perspective further. Please contact Miriam Godwin, Health Policy Director, at miriam.godwin@kidney.org.

View attachment from The National Kidney Foundation

American Society of Transplant Surgeons | 09/30/2021

The American Society of Transplant Surgeons (ASTS) strongly opposes this policy proposal as written. ASTS applauds the OPTN for its initiative in its undertaking to revise the metrics used by the MPSC for performance review. The ASTS also believes that such efforts should be collaborative and in cooperation with UNOS and other transplant organizations with the goal of improving patient outcomes. Regulatory metrics are complex and have significant unintended consequences. For example, it is well recognized that the current MPSC one-year outcome metrics, along with the SRTR five-star public ratings, do not allow patients to achieve optimal transplant opportunities as they strongly disincentivize transplant programs from accepting organs at risk of discard and from transplanting older and medically complex recipients. The available clinical literature strongly supports that transplant programs that are flagged by the MPSC for performance review curtail transplantation. In light of the large number of potential transplant recipients who die awaiting a life-saving transplant, it is clear that the OPTN’s mission of enhancing patient safety is best served by eliminating disincentives to transplantation created in part by the transplant program monitoring processes, including disincentives created by the current MPSC outcomes-based triggers for performance review. In addition, the newly proposed MPSC performance review criteria do not further the cause of patient safety as both wait list mortality and organ acceptance rates depend on multiple geographic, clinical and organ distribution issues that are not captured by current data. Importantly, the proposed MPSC performance review criteria do not appear to improve quality, increase the number of transplants, or promote innovation. The proposed revised metrics have the potential to increase, rather than reduce, risk aversive patient and organ selection and to reduce, rather than increase, the number of clinically appropriate transplants performed. Specifically, the addition of waitlist mortality to the metrics that trigger performance review has the potential to incentivize exclusion of sicker patients and those with lower socio-economic status from transplant program waiting lists. The inclusion of an organ acceptance metric as a trigger for performance review has the potential to encourage transplant programs to narrow their organ acceptance criteria to avoid being flagged—that is, to narrow the organs that it will be offered in order to avoid review based on the organ acceptance metric. This will ultimately result in fewer transplants, more waitlist deaths, and increased rather than decreased organ discards. In our view, the current proposal does not address the critical flaw inherent in the current flagging criteria (i.e. the criteria’s reinforcement of risk averse recipient and organ selection) and applying these multiple regulatory metrics concomitantly has the potential to exacerbate current transplant program risk aversion decreasing transplants for patients in the U.S. We urge the OPTN to adopt a focused approach to transplant program oversight that has a single goal: To increase the availability of clinically appropriate transplantation while maintaining or improving quality. Our suggested approach is comprised of two components: • Reform the current outcomes metrics to establish a tiered risk-adjusted pass/fail system under which the “pass rate” is established in a manner that ensures a defined excellent outcome and a patient benefit of transplantation as compared with the available clinical alternatives (e.g. dialysis, in the case of renal transplants) rather than an “expected” survival rate which is unpredictable and has increased each year due to centers’ increasingly risk averse patient and graft selection from the prior years. • Implement Systems and Quality Improvement Projects that focus on improving access to transplantation through non-punitive pilot programs. A. Patient-Centered Outcomes Metric Any effort to reform the MPSC’s oversight of transplant programs should begin with reform of outcome metrics that are currently used to trigger performance review. For the reasons set forth in the ASTS Recommendations for the Optimization of Transplant Center Assessment (January 12, 2021 (See #1 in References), we urge the OPTN to establish an easily understood binary (pass/fail) metric under which a transplant program’s “pass rate” is established by reference to national annually predetermined excellent outcomes that are pinned to the available clinical alternatives. For example, based on SRTR data, the mean age of a renal transplant recipient at time of transplant is approximately 58 years old and that the mean renal transplant recipient has been on dialysis for 3.5 years at the time of transplant. Assume further that the one-year patient survival for patients in this age cohort who have just begun dialysis is 89% (USRDS data). Under these circumstances, a renal transplant program that exceeds this benchmark would not be flagged for MPSC review. This basic concept could be further refined to ensure that chosen survival rates are high enough to protect patient safety and justify the risk and cost of transplantation. In its deliberations and its policy proposal, the OPTN rejected this approach on the grounds that it is not risk adjusted and will increase, rather than decrease risk aversion. We believe that if the “pass/fail” rate is determined in the manner suggested—as a straightforward fixed level of performance—it will provide transplant programs with the predictability necessary for them to increase the level of risk that they will accept. We believe that, for example, establishing the pass/fail threshold at or above 90% graft/patient survival would significantly increase the level of risk that programs would be willing to accept. Moreover, risk adjustment could also be included in the system in a number of ways. Risk adjusted outcomes could be used, if necessary, in a secondary analysis. The initial pass-fail determination could be made without risk adjustment and full risk adjustment could be applied to those transplant centers that do not meet the organ or patient survival pass-fail threshold to determine whether that center’s performance would meet the risk-adjusted threshold. Centers would need to fail to meet both the unadjusted pass/fail threshold and the risk adjusted expected outcome threshold to undergo MPSC performance review. Thus, this system will establish easy to understand thresholds as well as provide risk adjustment analysis as needed to encourage transplantation in our country. We urge the OPTN to make this change to the outcomes metrics as a first step on a trial basis and in conjunction with the elimination of the SRTR created PSR “star ratings” to assess whether, and to what extent, transplant programs respond to improvements in more appropriately selective regulatory enforcement. We believe that increased transplantation resulting from the elimination of disincentives to transplantation may increase access to transplantation in a manner that modifies waitlist mortality and impacts organ acceptance practices, so that imposing additional metrics relating to these areas may be unnecessary. Any further modification of the MPSC flagging metrics that may be necessary to further encourage clinically appropriate transplantation could be implemented in a step-wise fashion to avoid a dramatic increase in risk-averse behavior and a subsequent restriction in access to transplant for patients. Systems and Quality Improvement Projects We believe that adoption of patient-centric outcomes metrics should be accompanied by the initiation of Systems and Quality Improvement Projects that likewise focus on increasing the availability of transplantation. Additional Systems and Quality Improvement Projects focused on increasing the availability of transplantation might include, for example: • A project that excludes from any outcomes-related MPSC metric (but not data submission requirements) transplants performed under research protocols approved by the MPSC, so long as outcomes data is submitted to the OPTN for study purposes. • A project that excludes from any outcomes-related MPSC metric (but not data submission requirements) transplants of certain organs at particularly significant risk of discard (as defined by the OPTN, so long as outcomes data is submitted to the OPTN for study purposes). • A project that increases the flexibility of transplant programs to accept organs at risk of discard (as defined by the OPTN based on current discard data) for pre-identified recipients on the match run, regardless of that potential recipient’s place on the waiting list. We look forward to working with the OPTN on these and other avenues to improve the availability while maintaining the quality of organ transplantation, and we urge the OPTN to adopt our proposed approach to transplant program oversight to further this goal. B. Direct Response to Current Proposed Metric Changes: Flagging Thresholds Regardless of whether or not the OPTN adopts the alternative approach to MPSC performance review described above, we strongly urge the OPTN to modify its flagging thresholds to appropriately reduce the number of transplant programs that needlessly undergo performance review. This change is critical to reduce the disincentive to transplantation created by the current performance review process. It is well documented in publications that transplant programs become considerably more risk averse after having been flagged. Reducing the number of programs flagged has the potential to reduce the risk aversion of transplant programs as a whole with no documented risk to patient safety. It appears clear that the current flagging criteria are over inclusive. Approximately one third of unique transplant programs are flagged every three years, and an average of 10-12% are flagged every six-month review cycle (see #2 in references). Yet very few programs flagged for outcome queries have ever been shown to have true patient safety concerns. This level of regulatory oversight has made the fear of flagging pervasive and has contributed significantly to risk averse patient and organ selection by transplant programs while delivering no well documented benefit to patient safety. For this reason, regardless of what metrics are used, we believe that no more than 2.5% of programs should be flagged as being in the Red Flag zone for any solid organ for each PSR. This approach more accurately targets true outliers and has the potential to substantially reduce risk averse behavior by transplant programs. The currently proposed Red Flag zone criteria were set with an eye on not exceeding the number of currently flagged programs in the U.S. While maintaining adequate oversight is important in attempting to maintain a certain number of flagged programs, it appears somewhat random and not scientific. It is our understanding that the purpose of MPSC performance review is to ensure patient safety. The ASTS believes that ensuring patient safety should be a priority to all those working in the field and that peer review and oversight are important elements of ensuring patient safety. In light of the positive safety profile of transplantation as a whole, MPSC performance reviews should be relatively infrequent. There is no body of published data that supports improved patient safety due to the current large number of U.S. transplant programs undergoing MPSC performance reviews. There is published data, however, showing the negative consequences of the large number of MPSC performance reviews – fewer transplants to U.S. patients and therefore harm to U.S. patients. Summary Summarizing, we would strongly urge the OPTN to adopt a simplified proposal which includes a fixed 1-year post transplant patient survival metric. The bar for expected survival should be determined on expected patient survival with the alternative therapy when available (dialysis, or LVAD survival or other best medical management) and should be high enough to justify the risk and cost of transplantation and protect patient safety. This will allow for more patients to be listed for transplant, more grafts to be accepted, more transplants to be performed, and will allow centers to innovate and develop new transplant options while remaining within the accepted standard. If centers fall below this first measure, a second risk adjusted analysis should be performed using very well-defined variables such as recipient/donor age, DCD, etc. Centers that meet the unadjusted pass/fail threshold or the risk adjusted analysis should not be flagged for performance review. No additional metrics should be added unless and until the impact of this change is evaluated. Regardless of whether or not this recommendation is adopted we strongly believe that Red Zone flagging criteria should be established at levels that ensure that no more than 2.5% of programs—the true outliers—are flagged for each review cycle. This change has the potential to increase U.S. patient transplant opportunities due to reduced risk aversion by transplant programs and to save lives. Finally, ASTS believes that statistical significance of self-prescribed thresholds are not clinically meaningful. The current OPTN/MPSC practice of deeming about 10% of all organ programs in need of performance review every 6 months is not well founded in the professional quality realm. ASTS thinks the OPTN proposal, while thoughtful and indeed sophisticated, will not result in substantial positive change in behavior. In addition, applying multiple metrics at the same time could negatively impact the transplant eco-system. The ASTS continues to be optimistic about transplantation and believes that we have one of the best and safest transplant systems in the world. We look forward to working with all involved in introducing more collaboration to the field. Our responses to questions raised in the OPTN proposal are included as an Attachment. References 1. https://asts.org/docs/default-source/regulatory/asts-white-paper-on-optimization-of-transplant-center-assessment-january-12-2021.pdf?sfvrsn=43a46d3_2 2. Schold, J. , Miller, C. , Henry, M. , Buccini, L. , Flechner, S. , Goldfarb, D. , Poggio, E. , Andreoni, K. & (2017). Evaluation of Flagging Criteria of United States Kidney Transplant Center Performance. Transplantation, 101 (6), 1373-1380. doi: 10.1097/TP.0000000000001373. Attachment The OPTN Membership & Professional Standards Committee requests feedback on: 1. Do you think transplant programs that fall within the performance improvement or “yellow” zone would take advantage of offered assistance and if so, what types of assistance would be most helpful? Interaction within the defined “yellow” zone is voluntary, and reaction to notification of “yellow” zone status will likely vary based on the unique situation, resources, and culture of each program. Access will range from use of educational material and best-practices information to significant interaction with the MPSC such as peer-peer engagement or other interactions. The MPSC should be prepared for the possibility that interaction with programs in the “yellow” zone (YZ) will be very labor intensive. Raising the hazard ratios for the YZ and RZ, thus decreasing the number of anticipated YZ and RZ flags would decrease the likelihood of unmanageable MPSC workloads emerging from this policy change. 2. Would you support the future addition or replacement of the 1-year post-transplant graft survival with a longer-term period-prevalent survival metric, such as a 5-year period prevalent post-transplant graft survival? The fundamental problem is that the longer-term the outcome measure, the more tenuous the link to the actual performance of the Transplant Center (TC) in the perioperative phase and the less control the TC has over those patients forming the long-term cohort. The assessment of the pros and cons would depend on the weighting given to a long-term metric and the hazard ratios or thresholds utilized to determine flagging for the metric. This would be more a measure of patient socio-economic status than transplant center performance as those with better insurance would have better community care over the five-year term. 3. One of the desired attributes of a good monitoring system is the monitored entity understands the measures being used. In order to ensure this understanding: What types of resources do you anticipate needing to respond to these new metrics? a. Are you comfortable with the concept of risk adjustment or do you think additional education on risk adjustment is needed? The problem with risk adjustment is that it results in a moving target for centers. What was an acceptable survival last year may not be accepted in the next cycle, based on changes to the model which cannot be predicted by programs. A second problem is that some factors that are currently in the model, such as peripheral vascular disease and diabetes have a very wide spectrum of disease morbidity, and centers that are more familiar with the PSR models are coding for these risks more liberally than others. A third problem with risk adjustment is that not all variables known to impact survival are included in the model and the process for including variables is opaque. Because of these and other factors, we do not believe that risk adjustment is trusted by transplant centers. The theory of risk adjustment makes obvious sense, but in reality, the granularity of the patient data necessary to achieve meaningful and fair risk adjustment is simply impossible to obtain. b. What education resources do you need to describe these metrics to your patients? We do not believe that the new metrics should be adopted, nor do we believe that metrics that require significant patient education are likely to be useful. Any metrics that are adopted should be sufficiently meaningful to patients and sufficiently straightforward to require little or no explanation from health care professionals.

View attachment from American Society of Transplant Surgeons

American Society of Nephrology | 09/30/2021

On behalf of the more than 37,000,000 Americans living with kidney diseases and the 21,000 nephrologists, scientists, and other kidney health care professionals who comprise the American Society of Nephrology (ASN), thank you for the opportunity to comment on the Organ Procurement and Transplantation Network’s (OPTN) recommendations developed by the OPTN Membership and Professional Standards Committee. Transplantation is the best available therapy for many individuals with kidney failure. However, Black Americans, Latinx Americans, Native Americans, and Native Hawaiian/Pacific Islander face disparities in nearly every step of transplant care. Black Americans are less likely than white Americans to be identified as transplant candidates, referred for evaluation, placed on the kidney transplant waitlist, receive kidney transplants especially pre-emptive transplants, and/or receive living donor kidney transplants, while also being more likely to receive lower quality kidneys, have organ offers declined for them and have poorer transplant graft survival. Many factors contribute to these inequities that are within the control of federal policy makers. In recognition of the OPTN’s role in setting standards, monitoring, and promoting high quality care in the transplant arena, ASN feels the organization has an important role in working with policymakers to address these inequities. Factors where OPTN’s leadership influence federal policy include: • Addressing transparency of confusing federal oversight structures of organ procurement, allocation, and transplantation • Eliminating policies that engender risk aversion and transplant center evaluation measures that are misaligned with patient interests • Removing patient evaluation criteria to gatekeep transplant access that have a limited association with patient outcomes (especially compared to patient outcomes on other kidney replacement therapies), such as income, race, requirement of dual insurance, substance use, and obesity. The new MPSC proposal has several measures of quality, and, while we recognize the need for quality measures to drive improvements in transplant, the current proposal raises significant concern as outlined below. Waitlist Mortality Ratio The national waitlist is already shrinking as a direct result of fewer additions and an increasing number of waitlisted candidates being removed. As a result, the proportion of patients with end stage kidney disease that are on the waitlist is at the lowest level in over two decades suggesting that the waitlist is no longer a reflection of the need for transplantation in this population. This measure will only further disincentivize waitlisting of candidates and will further obscure transplant center practices with regard to the referral, evaluation, and selection for waitlisting. Also, given that kidney transplant centers are not primarily responsible for the care of patients while they are on the waitlist, their ability to institute or change clinical practices that would influence the mortality of waitlisted candidates is limited. This proposal raises further questions about the relevance of this measure as a measure of the quality of care being provided by transplant centers. Offer Acceptance Ratio While we agree that this is an important measure that is highly variable across transplant centers and appears to have a direct impact on the probability of transplantation for candidates at that center, the implications of continuing to exclude bypass offers from the denominator needs to be better understood as does the benefit of excluding those organs that are discarded. In addition, this measure should be made easily accessible to patients along with information on how their center practices are likely to impact their ability to get a transplant. Alternatively, the rate at which transplantable organs are declined for some patients but are used successfully at other transplant centers may achieve the same goal while also creating a clear and unambiguous feedback look to transplant centers about the organ offer evaluation processes. 90-Day graft survival It is not clear that the relatively few events in this category are likely to allow for meaningful analysis or interpretation. The development of quality measures should focus either processes of care or outcome measures where there is an opportunity for clinically relevant improvement. Given that it is unlikely that 90-day survival rates can be lowered any further in a systematic manner and that adverse events are relatively rare events that preclude the identification of systemic problems by this measure. Conditional 1-year graft survival With the withdrawal of one year graft outcomes by CMS as a quality measure and the concurrence of Health Resources & Services Administration (HRSA) that longer term measures are preferred by patients, this measure would be at odds with these trends. Currently, the differences in the 1-year survival rates between transplant centers are small and not clinically meaningful. In addition, patients do not perceive this as a meaningful measure of quality of care, the ability of centers to make clinically significant improvements in their 1-year survival rates undercuts the value of this as a meaningful quality measure. Aspirational/Experimental Metrics Measures of quality of care should be objective, reliable, valid, valued by patients, provide insights where there is clinically significant variation across centers, and account for variations in the clinical acuity being encountered. Quality measures should provide actionable insights and be designed in a manner that the processes being assessed are under the influence of transplant centers. This is the only way to ensure that they have the ability to influence the quality measure and allow it to reflect changes in clinical practice. Measures should be designed to drive improvements in processes of care that result in improved patient outcomes that are meaningful to both the health care system (i.e., reduced mortality, reduced graft failure) and to patients (i.e., improved patient activation, improved adherence, improved quality of life). New measures of quality should also be evaluated for the risk of unintended consequences especially those that adversely impact access to transplantation. Access to transplantation is important to both patients and the health system and improved access to transplantation is a prerequisite to reducing disparities and increasing transplantation rates. As a result, measurement of process measures (such as the proportion of referrals that start evaluation or the time spent in evaluation) or outcome measures (such as the proportion of referred individuals who are eventually waitlisted) would be important ways that access to transplantation could be objectively measured with a focus on processes that are exclusively under the purview of the transplantation. Novel metrics are likely to require additional data capture. These additional data may be necessary because the process or outcome of interest is not currently being captured in the OPTN registry. An example of this would be data elements about the pretransplant evaluation process are not currently part of the OPTN registry. Data elements necessary for a more robust risk adjustment may also be necessary to reflect current clinical care complexities and to ensure that there is confidence in the community that the measures are reflective of the variations in patient population across centers. Alignment of metrics and other policies Subjecting transplant centers to different quality metrics that are not aligned or using different methodologies/ thresholds has been recognized as being part of the historical challenge in the transplant community. The resulting chaos contributed to the decision to withdraw CMS measures. Needing to track and respond to different quality measures that are not aligned creates logistical challenges for smaller programs. Such a situation will also result in the creation of competing priorities at transplant centers that would undermine the primary mission of increase transplants. Similarly, new payment models and the development of interventions such as the kidney transplant learning collaborative should be designed to align with one another. It is unclear if there was an attempt to ensure that that new metrics align with the recently implemented mandatory ESRD Treatment Choices (ETC) model and with the proposed voluntary, value-based payment model: the Kidney Care Choices (KCC) model. As you are aware, the ongoing “Task 5” effort, by HRSA and the Scientific Registry of Transplant Recipients (SRTR), to identify and implement new proposed metrics is ongoing. This proposal from the MPSC appears to be distinct and separate from those efforts. As a result, these proposals seem both premature and create a situation that will limit what the Task 5 effort can propose or risks being distinct and misaligned from their final recommendations. The more than 37,000,000 Americans living with kidney diseases and the 21,000 nephrologists, scientists, and other kidney health care professionals who are ASN members thank the Biden-Harris Administration for its commitment to providing a robust, transparent approach to transplantation critical to improving kidney care for all patients. To discuss this letter further, please contact David White, ASN Regulatory and Quality Officer, at dwhite@asn-online.org or (202) 640-4635.

View attachment from American Society of Nephrology

Anonymous | 09/30/2021

Region 11 sentiment: 1 strongly support, 13 support, 4 neutral/abstain, 0 oppose, 0 strongly oppose Comments: Region 11 supported this proposal and shared significant feedback. Several attendees were concerned about using waitlist mortality rate as a metric noting that this could cause centers to be more conservative in their listing practices which would disadvantage patients. Two attendees shared concern about socioeconomic disparities that aren’t always captured in the risk adjustment statistics. They went on to note that socioeconomic disparities will be worse when programs are held to a higher standard as it increases their costs. Another common concern surrounded offer acceptance rate. One attendee was concerned about kidney acceptance rate and how it relates to travel logistics. Another attendee noted that acceptance rate is measured on individual patients and not necessarily on organs and does not account for medical judgement. They went on to comment that this metric could incentivize programs to restrict OPTN acceptance criteria in UNET to avoid having to turn down marginal organs, adding that there should be a more efficient way for organs to go to patients at the top of the list. They added that the goal should be DonorNet projects to be able to more appropriately match those patients. Another attendee noted that there needed to be work on the pre-transplant metrics and would rather see some objective best practices/pilots shared and more research into the appropriate metrics. They also shared that the intent of the metric needs be better defined, so it is clear what defines a good program in the pre transplant phase. The attendee went on to note they supported the more liberal post-transplant metric but urged the committee to consider tracking a longer term outcome (3 years). An attendee suggested potentially looking at a more holistic approach rather than a performance metric, something like a certification based on a combination of quality, volume, innovation, etc. Several members supported updating the monitoring system and noted the importance of a meaningful change to oversight, they emphasized that this should not be done in a way that dissuades programs from ultimately transplanting more organs. Two attendees supported the changes and noted that all stakeholders need to be measured on continually advancing performance metrics in order to meet the needs of those awaiting lifesaving transplants. One attendee supported aligning CMS and OPTN metrics for transplant rate and graft survival rate. Finally, several attendees had concerns about how center metrics could be used by insurance companies.

HonorBridge | 09/30/2021

HonorBridge (formerly Carolina Donor Services) supports the MPSC’s proposed metrics that measure transplant program performance. Both OPOs and transplant programs should be held to metrics that aim to increase organ utilization and accelerate the number of transplants. HonorBridge is optimistic that the acceptance rate metric will incentivize centers to better utilize organ offer filters and provide a needed improvement in identifying an accepting center more efficiently and timely, potentially decreasing discard rates by getting offers from medically complex, DCD, and older donors directly to centers that are likely to accept.

Society for Transplant Social Workers | 09/30/2021

The Society for Transplant Social Workers appreciates with work of the MPSC and support the expansion of assessment metrics beyond just the 1 year patient and graft survival and including metrics along the continuum. We do have concerns about an unintended consequence of transplant centers being reluctant to list and transplant people who are at higher risk of complications and the acceptance of less than ideal organs for transplant. It is not clear how the proposed changes to the metrics will increase the number of transplants performed. Waitlist mortality can also be a challenging metric in that many of our transplant candidates are managed by outside providers. Ultimately, it is our experience that people with advanced organ failure want access to transplant services and for those organs to last a long time. Overtime, it may make sense to add metrics to would measure access to listing and longer term survival statistics, e.g. 5 or 10 years.

Anonymous | 09/30/2021

The Patient Affairs Committee (PAC) appreciates the work of the Membership and Professional Standards (MPSC) Committee in developing this proposal and for the opportunity to comment on it. The PAC feels passionately about having as much data available to patients as possible so they can make the best and most informed decision about their care. There was no consensus from the Committee on whether they felt this information should be shared with the public or remain an internal, peer-reviewed process. Members articulated compelling arguments for both sides, but there was no Committee wide stance on this. Members felt this proposal lacked readability and clarity to the public, and suggested including a list of metrics in the ‘specific questions for feedback’ section of the Executive Summary. Members also expressed concern over the proposal’s potential to unintentionally disincentive transplantation. Alternatively, the patient population would prefer centers to have the flexibility to make riskier decisions in transplanting a patient, when medically appropriate. Ultimately, the PAC wants transplant centers held to high standards that ensure patient safety and quality of care. Overall, the PAC is supportive of the MPSC Committee’s policy proposal and is appreciative of the opportunity to provide the unique patient perspective.

Hatem Amer | 09/29/2021

My personal opinion is that this proposal is a good step in moving beyond one metric, and only short-term outcomes. For that I am supportive of the proposal. I do think that there needs to be additional steps in the not to distant future to avoid the pitfalls that are likely to happen. Some expected unintended consequences such as some programs not listing too sick a patient for fear that the wait list mortality will be flagged, or tightening the organ acceptance criteria so that fewer organs are declined due to donor quality. The ideal system would track patients through the continuum of care from prevention to organ replacement with transplant, if transplant is the best option for a specific patient. For some patients that might be the case through their health plans looking to improve overall health, minimize health care costs, and improve efficiency of health care. I think where the OPTN and SRTR can help nationally and alleviate some of the concerns raised, is to develop new techniques for risk adjustment. The current system relies on the national cohort, and the limited variables that are collected to determine what variables confer risk and adjust for those variables. The goal is comparing apples to apples. That system may appear to work for common condition and items that are concrete: deceased donor vs. living donor, age, etc. It is however hampered by: the desire to lower the burden of data collection, resulting in less granular data, failure to adjudicate complex diagnoses, or adjust for rare conditions. As as nephrologist I find it hard to believe that all glomerular diseases are alike. Does IgA really carry the same risk as a PGNMID, C3 nephropathy etc.? Are all FSGS cases in the SRTR the same. So primary FSGS is just the same as secondary FSGS? Is all heart disease the same? Would a patient with CAD, revascularized and with a low EF fare better with a kidney transplant or dialysis? There are several publications addressing some of these rare disease clearly showing a greater risk for early graft loss. Others have refined the risk assessment of heart disease in kidney transplant patients. To balance the burden vs. benefit of data reporting, allow innovation, and improve access to transplantation to patients who would benefit but have unaccounted for risk, one can envision two avenues to be layered over the current risk adjustment model: 1. If data exists for a discrete disease or biomarker with peer reviewed publications, allow centers caring for those patients to provider strict documentation and grant the appropriate risk adjustment. Concerns will certainly be raised that this can be misused. In that case plan regular audits / paper reviews, not as a flag but an affirmation process. 2. Develop a list of certain conditions that would exempt the patient from the analysis cohort. This will likely work for rare conditions as noted above and will likely only be managed by a hand full of centers that would have the expertise to mange those cases. This will help innovation that will hopefully offer safe and effective therapy for these unfortunate patients who frequently are young and would benefit greatly from transplantation.

Dennis Lyu | 09/29/2021

I am supportive of moving away from 1 year survival and using a more comprehensive system in evaluating a transplant program such as the proposed model. I am concerned about how one of the metrics is calculated when determining acceptance rate. A transplant program that is in the same city as the donor hospital or organ procurement center is at a distinct advantage in accepting a questionable organ. If one center declines an organ that is far away requiring a flight, that is ultimately accepted by a local center, I do not think this should count against the acceptance rate of the center that declined.

NATCO | 09/29/2021

NATCO approves this proposal and would like to thank the MPSC and SRTR for coming together to apply four metrics that will be evaluated in the pre- and post-transplant phase. The review is that in which allows for support and collaboration from the MPSC to address performance improvement opportunities and is not meant to be punitive. These metrics also allow for more recent performance monitoring, rather than waiting for the SRTR data. This would allow for more timely recognition of any issues or concerns, allowing for more timely intervention for those programs that are not performing well. A concern with the performance metrics is that it may cause transplant centers to be risk adverse and limit access, although this is somewhat balanced by several measures instead of just graft and patient survival. Potential risks or unintended consequences for the proposed changes could be decreased access to listing, increased use of DD filters which could expedite the organ placement. The organ acceptance metric does not take into consideration barriers related to geographic location other than miles along, and availability of commercial flights can add hours to cold ischemic time for kidneys. Peer review assistance would be helpful, as MSPC proposes, but there would need to be a culture change at UNOS. To assure understanding of the monitoring system, better understanding of the organ acceptance metric needs to occur as there is increased confusion within the community, as well as more education as to risk adjustment would be helpful. We would further support the added long-term metric of 5-year graft survival. To monitor the metrics, tracking of listing trends as well as the overall number of transplants that have occurred should be done.

Society for Pediatric Liver Transplantation | 09/29/2021

The Society for Pediatric Liver Transplantation supports this proposal to add more metrics of both pre- and post-transplant outcomes to review of transplant programs. We appreciate that the MPSC has considered the impact of relatively small transplant volume for pediatric programs and adjusted their review criteria accordingly. These additional criteria are a step in the right direction, but we would also recommend considering longer-term outcomes for pediatric and other transplant programs, as the 1-year outcomes are a very small percentage of the expected total post-transplant survival time for the vast majority of pediatric transplant recipients. Although the goal is to incentivize behaviors that increase transplantation, we also recognize that the proposal may pose a challenge for centers considering high-risk candidates, especially in smaller transplant volume pediatric programs where few undesired outcomes will flag the program for review. Offering programs in the “yellow” zone assistance would be helpful, and likely acceptable, to many pediatric programs – in the interest of improving the care of their transplant candidates and recipients and avoiding a CMS review. The review process might justify more resource allocation to the transplant program. The review process may provide some root cause analysis and call to changes in practice through transplant protocol modification ( i.e. immunosuppression, anticoagulation, use of technical variant grafts, etc.) We also encourage the MPSC to develop metrics for assessment of use of technical variant grants (split and living donor) in the assessment of both pediatric and adult programs, as these are important opportunities to increase transplant numbers and transplant access – and are an important subset of offer acceptance. The youngest (

Anonymous | 09/29/2021

I am strongly in favor of a more comprehensive system that evaluates several parts of transplant program performance. However, the system should evaluate parts of a transplant program's performance that patients actually most care about! These include being placed on the waiting list, receiving a transplant (living or deceased donor) and short/long term outcomes. Therefore, I am strongly opposed to the addition of the waiting list mortality rate ratio. This measure discriminates against programs that aggressively list patients and discriminates against people more likely to die on the list such as older and diabetic individuals. Patients want to be put on the list for a chance! Do not penalize programs that give patients a chance. I am in support of monitoring the offer acceptance ratio since this speaks to patients wanting a deceased donor transplant and reports how aggressive or conservative a program is with regard to organ offer acceptance. I am strongly against monitoring 90 days outcomes. Patients care about short and long term outcomes after transplant. Why would you monitor short term and shorter term outcomes? I suggest one year and 5 year outcomes. This addition of 90 day survival adds very little meaningful information for patients. In fact, the US lags behind other countries in long term outcome in part because of its myopic look at one year outcomes only. A long term measure is needed, not two short term measures! I am strongly opposed to no measure of living donor transplant effectiveness. The best outcome is for a patient to receive a living donor transplant but the MPSC neglects this issue. Is a program doing well if it adds 200 patients to the list, does 100 deceased donor transplants and 5 living donor transplants? I would argue that this program needs review. The MPSC should monitor number of living donor transplants as a function of new waitlist additions. Nationally, this has been about 16-17%. If a program is significantly underperforming this metric the MPSC should look into it … to help the program achieve more living donor transplants which are best for patients!

Dell Children's Medical Center | 09/29/2021

We support separating adult from pediatric metrics.

Association of Organ Procurement Organizations | 09/29/2021

AOPO strongly supports the MPSC’s efforts to enhance the transplant program monitoring systems. The development of transplant programs metrics that better align with the new CMS performance metrics for OPOs will facilitate significant systemwide performance improvement. The timing of the OPTN’s work on this is critical. With the national focus on increasing transplantation through the Kidney Health Initiative and AOPO’s own initiative to achieve 50,000 transplants by 2026, it is important that metrics between all components of the system are developed in a manner that will synergistically generate the most growth possible to serve waiting patients. AOPO appreciates the attention the MPSC took to define the principles of a metric and agrees with the proposal that these components are essential for a valuable metric: (1) Measures aspects of care that are clearly within the authority of the OPTN; (2) Measures aspects of care that the transplant program can impact; (3) Has a clear desired outcome (3) Does not require collection of new data or development of a new metric; (4) Measures a discrete aspect of transplant care provided by transplant programs; (4) Is risk-adjusted; and (5) Incentivizes behaviors that will increase transplantation. AOPO anticipates future OPTN metrics that may be developed for other members like OPOS will follow these same principles. AOPO notes that while the transplant rate is not solely within the control of a transplant program, it is also not solely within the control of OPOs, but nonetheless, OPO are accountable to a transplant rate under both the CMS metric and the OPTN yield measures. Multiple factors impact an OPO’s transplant rate that are outside of the OPO’s control such as type of organ donor (DCD v DB), offer acceptance practices of programs, donor conditions unknown at time of organ placement, and ultimately the transplant itself. AOPO suggests that the MPSC reconsider a transplant rate for transplant programs. Transplant rate could potentially be a shared performance metric evaluated for both OPOs and Transplant Programs recognizing that the calculation of such a rate will measure different aspects of the process as between different member types but that nonetheless some sort of aligned and shared accountability is required to result in the desired performance (increased transplants). The metrics selected by the MPSC in the proposal are supported by AOPO as measuring important and distinct aspect of transplant program patient care: (1) waitlist mortality rate measuring waiting list patient care, (2) offer acceptance rate measuring offer acceptance practices, (3) 90-day post-transplant graft survival measuring peri-operative care, and (4) 1-year post-transplant graft survival conditional on 90-day graft survival measuring post-operative care. The measurement of pre-transplant care through waitlist mortality is particularly important as the system shifts to a patient-centric understanding of risk-benefit which must evaluate the risk to a candidate of receiving an offered organ versus the risk to a candidate of not receiving the organ offered. This paradigm is incredibly important to support through aligned performance metrics if the system is intent on better utilizing the available organ pool which is increasingly from older and medically complex donors. AOPO believes that this measure (waitlist mortality rate) alone has the potential for reducing discards, which at 20% for kidneys is widely recognized as an area the system needs to improve upon to better serve patients. AOPO is also very supportive of an offer acceptance metric to pair with the waitlist mortality rate to ensure that programs are not evaluating the risk of an offered organ versus the theoretical lower risk of a theoretic organ offer that may come in the future. The system cannot accelerate its growth in transplant without metrics that align with the goal of increasing organ utilization for both OPOs and transplant programs. AOPO also believes the offer acceptance rate metric will incentivize best practices regarding transplant program use of filters which will provide a needed efficiency improvement to organ placement efforts by OPOs. This is particularly urgent in the context of broader distribution and the increasing number of organs being recovered from older and medically complex donors. AOPO recognizes that development of the specific flagging boundaries and process through the MPSC to improve performance are a critical part of the proposal and defers to transplant program colleagues with statistical expertise to answer some of the specific questions posed in the proposal but stands ready to partner with all transplant programs to align performance to deliver the highest performing system possible.

American Society of Transplantation | 09/29/2021

The American Society of Transplantation is supportive of moving beyond the current one-year outcomes used to assess transplant programs but has considerable comment regarding this proposal as written. It is unclear whether this proposal will have a positive impact on access to transplantation or an increase in the total number of organ transplants. We do wish to note our support for the principles applied for creating an effective metric. We would be pleased to see the OPTN continue to apply these principles in the development of future policy language related to transplant center and adapted to apply to OPO performance metrics as well. Additionally, we recognize the critical timing of this effort. The donation and transplantation community has the opportunity to leverage these changes to improve system-wide performance by aligning with OPO metrics and addressing the potential competition with CMS metrics for dialysis programs. We suggest that it may be wise to consider explicitly adding the alignment with performance metrics for other components of the system as an additional principle for creating an effective metric. Although we are encouraged that there was emphasis that any one metric would not be a firm trigger, there was concern that the previous near two-decade experience where the lack of change of the O/E standards as the outcome SD became narrower and narrower resulted not just in stifling of innovation, but also decreased access to transplant, results in the group being somewhat apprehensive of hard cutoffs that don’t explicitly say that they should be adjusted with time. It is further acknowledged that the modelling for outcomes, while often reasonable, makes compromises that may inappropriately impact multiple risk situations in a way that may discourage transplant (e.g. CAD as a yes/no risk factor when it is a continuum of disease, the lack of interaction between DCD status and donor age, the lack of WIT in DCD risk calculation, etc.). While such issues may average out with large numbers, center volumes (even for the largest centers) are such that local impact can be profound if such models are overly rigorously interpreted by regulatory bodies or payors. There is a concern that audits and site visits may be viewed as punitive by hospital administration who may be less versed in the statistical modeling utilized for transplant, resulting in inappropriate pressure to do “less risky” transplants; this could be counterproductive. For this reason, educational outreach to both transplant centers and hospital administration is encouraged. The AST believes that there is value in emphasizing the role of the MPSC as an educational tool that helps programs improve their practice in a non-punitive, evidence-based manner. We believe that a strong educational effort by the OPTN to highlight this message will help to combat the negative narrative here and aid centers in using flagging as a tool in educating administrative staff and seeking resources to strengthen programs. There was significant concern that adding a pre-transplant mortality metric – which was meant to incentivize performing more transplants – may have the opposite effect of transplant centers not listing or activating high risk or difficult to transplant patients because their deaths or inability to be transplanted (which may be totally unrelated to organ acceptance patterns) will damage their wait list metrics. This problem could be particularly acute in areas of the country with short wait lists such that each wait list death would have a significant impact on their metrics. Less than 30% of end stage organ disease patients make it to the wait list and we are losing many more patients in the referral/evaluation step than after they make it to the wait list, and any policy changes that exacerbate barriers to care will worsen any inequities in access to transplantation. This caused quite a bit of concern for members of the Kidney and Pancreas Community of Practice, as kidney patients are most frequently managed by their local nephrologists and dialysis unit—unlike most other pre-transplant patients. It was acknowledged that the bar for this metric is set at a currently reasonable level but given that there is little that the transplant centers can actually do to impact this metric that may not either decrease access or game the algorithm, for kidney, this particular metric seems most prone to future misapplication, gamesmanship, and inappropriate loss of access to the waitlist for various groups of patients who may be considered higher risk for waitlist mortality. While we acknowledge the desire/need to find a metric for waitlist management, there was concern that at least for kidneys where pre-transplant kidney patient care is at the level of the community nephrologist and not the transplant center, it would be useful to help develop standards of waitlist management. Absent agreement on standards of waitlist management, what is the standard of de-listing and waitlist evaluation? We believe that better surveillance and care of individuals on the waitlist is crucial and appropriate resources must be allocated to optimally manage these needs. The second metric, offer acceptance rate ratio, was less controversial and was generally conceptually supported. There was concern that the approach of using any refusal as the same as a donor refusal would cause misapplication. For example, current refusal codes in the 810’s, which are recipient specific, may have different implications that those in the 830’s (donor specific). As an example, a kidney donor with significant AKI may not be a great match to a frail recipient with CHF, but the same center may happily use that same kidney for other candidate populations. That said, the fact that the new offer filter function in DonorNet exempts filtered patients from the calculation was considered supporting the intent of this metric in a way that made it much less controversial (while acknowledging that it may take a couple of years to reach a steady state of filter use). The third and fourth metrics, 90-day and conditional 1-year graft survival were not considered controversial and were embraced, with the caveat that care must be taken to avoid overemphasis of a “narrowed bell curve,” and that the actual cutoff should be adjusted over time. Similarly, with similar caveats, the separation of pediatric outcomes was not controversial. The Society shares the following thoughts for the MPSC’s consideration: • 90-day graft and 1-year graft survival conditional on 90-day survival seems to be overlapping, and perhaps a longer term (2 or 3-year graft survival should be considered) • Concerns were also raised about the risk adjustment being performed that weigh into evaluating organ acceptance offers and waitlist mortality. It was felt that not all relevant factors are accounted for, particularly in the case of patients with significant non-liver-related cardiopulmonary comorbidities whose actual risk may not be captured, but who still benefit from liver transplantation in experienced centers. • Same concern regarding elevated risk of waitlist mortality/dropout was made for centers who are aggressive in listing extended oncologic indication patients for transplant (HCC beyond Milan criteria who may be even beyond UNOS DS criteria but who may benefit from LT, unresectable cholangiocarcinoma patients – both hilar and intrahepatic- who are not typically considered at most centers. These patients all come at a higher risk of waitlist dropout and mortality- but this needs to be accounted for in the risk adjustment. • The development of ratios that are risk adjusted should incentivize the overall number of transplants and allow the identification of behaviors that can be adopted by other transplant centers to improve or modify the care provided to transplant patients. We suggest also that continuous monitoring and evaluation of the currently selected and new variables should be adopted to adjust the models in the future based on future observations. • The number of VCA transplants performed at any one center and in the nation is small. It will be necessary to develop metrics that address the unique circumstances of VCA transplants and at the same time carry out the duty of the OPTN in assessing program outcomes which would include VCA transplants. • One area of concern is the decision to adjust the thresholds in order to "flag" a comparable number of programs. We ask that the MPSC provide some evidence that the flagging process has actually improved outcomes and hasn't negatively impacted innovation in the field. Of importance, this is the first proposal where the Society’s transplant community representation shares its feedback. The AST values the patient voice in considering these changes. The thoughts below were submitted directly from the AST’s Transplant Community Advisory Committee, currently comprised of transplant recipients. The Transplant Community Advisory Council, a body of the American Society of Transplantation with membership comprised of organ transplant recipients, is generally supportive of this proposal and offers its thoughts from the perspective of the transplant patient for consideration. While we applaud the effort to enhance standardized patient safety and transplant quality, we believe that more can be done to improve the monitoring system and raise awareness and transparency for the “customers” that it serves. We remain very uncomfortable with the one-year outcome as a measure of success, and strongly believe that long-term outcomes be part of the discussion when assessing patient safety and quality of life. From the viewpoint of a recipient, we are grateful for every day of life we receive from a transplant, but we don’t go through the physical and emotional challenges of the awaiting and receiving a transplant with a goal of just 365 more days of life. We strongly believe that this element should be addressed in this proposal. We understand that success will look different for every patient and may differ by organ type, but we support continual and intentional discussion to ascertain the appropriate metric. We also recognize that this singular metric of one-year survival cannot be the focal point for transplant evaluation, as it impacts the center’s decision-making process related to listing candidates, acceptance of organs, and perhaps even post-transplant care. Moreover, it appears that this metric sets a low bar that nearly all transplant centers are able to meet reliably, thereby creating consistently favorable metric-based data that is made available to the public. We’d like to see more work to encourage performance growth and positive change to challenge the status quo in transplant survival. Comfortable goals or metrics create stagnancy and do not encourage the innovation that will ultimately lead to longer life post-transplant and perhaps fewer returns to the waiting list. This, in turn, could increase the number of available organs so that more patients can receive the gift of life. While we appreciate the specialty of transplantation and the concept of confidential medical peer review, we do share concerns regarding the closed process in general. We question whether this proposal, that may be a step forward for transplant professionals, is also a step forward for protecting and serving patients. While we trust and respect our individual providers, a better understanding of the transplant program around them would be valuable information. We do not believe that the SRTR data goes far enough to provide transplant patients with information that may be critical to their self-care, patient advocacy, and decision-making when cause for concern arises regarding the safety and fitness of particular transplant programs. Organ-refusal data, for instance, is one type of information that is of great importance to us as consumers within the transplant system. We recognize that information gathered by the MPSC can be sensitive or complex and may not be of interest to every organ transplant or recipient. However, we believe strongly that transparency in the healthcare arena is vitally important and can be achieved while maintaining sensitivity to privacy/HIPPA considerations as well. It troubles us that we are able to glean more information about restaurant food safety grades given by inspections (they are posted on the door) than we are privy to regarding the transplant programs that treat us. While some patients may not have the means to select or change their transplant program based on their review of this data, we argue that they should at least be aware of any serious weaknesses that may exist in their care—if only to try to avoid the consequent pitfalls. Patients are a big part of the success of their transplant care, after all; a paternalistic approach to sharing information, then, is not appropriate or acceptable. Overall, we appreciate the need for rigor, measurement, and accountability in this area and the attention being paid to this by the MPSC. However, as patients, we are not sure how these proposed changes will impact our individual care, treatment, and quality of life in a palpable, positive way. We believe that the MPSC should consider additional factors or metrics, including candidate and recipient quality and extension of life measurements that will improve the existing transplant monitoring system. We believe that the outcome of these changes will ultimately advance the field of organ transplantation, enhance patient safety, and increase the quality and longevity of post-transplant life. And, finally, we also firmly believe and wish to emphasize our view that increased transparency is critical for both patient safety and patient choice.

Gabriel Loor | 09/28/2021

In my opinion the current system does an appropriate job at balancing the use of organs with expectations for outcomes. If there is a scarcity of organs, then they should be reserved for patients and centers that are likely to achieve a reasonable benchmark of success. However, over time, innovations in organ utilization such as DCD, EVLP, extended criteria, HCV have led to a surplus in donor organs. But physicians remain too concerned with the one year outcomes to give certain donors a chance. Perhaps more importantly, there is an increasing number of patients in dire need of a donor organ. But again, given the heightened level of focus on one - year outcomes, many patients who could potentially benefit from a life saving organ are not given a chance. The current proposal seems to strike a better balance between proper organ utilization and recipient outcomes. I am in favor of this proposal.

Region 10 | 09/28/2021

Region 10 sentiment: 2 Strongly Support; 12 Support; 2 Neutral/Abstain; 2 Oppose; 0 Strongly Oppose Comments: Members of the region supported the proposal, but several concerns were raised about the proposed metrics. One member noted the following: waitlist mortality is an unreasonable marker for kidney transplant programs given the lack of ability to address medical issues handled by PCP/primary nephrologists regardless of collaborative relationships formed with referring providers. Second, the proposed outcome measures could lead to decision to not list patients living in remote areas who are deemed unable to seek access to healthcare at a transplant center or tertiary care center. Third, while the monitoring of organ acceptance rates is geared to minimize the delayed response times of centers that never intended to consider a particular organ/donor, it also inappropriately penalizes centers that keep criteria broad in order to assess possible viable organs and yet maintain their ability to rapidly Accept/Decline the organ offer. Lastly, ongoing request to consider "omitting" the penalization of declining an organ that is then accepted by another center further down the list if that declined organ ends up having graft failure in 90 days (while multiple factors do lead to graft loss, a poor quality graft is still a large contributory factor - especially in kidney transplant). In regards to pediatric metrics, organ offer acceptance rates will need to be reviewed and addressed in its entirety as pediatric hospitals have more stringent guidelines on organ quality including anatomy, donor diagnosis, size and functionality, which are not regularly used in the adult world. A simple acknowledgement in pediatric organ offer acceptance rates will be different than an adult hospital is all that is required. Additionally, due to small volumes in pediatric centers, one negative outcome will have a greater impact on the hazard ratio. If a hazard ratio is >1.60 for 90-day or 1-year graft survival, but the lower limit of the confidence interval for the hazard ratio is less than 1, will a program still be flagged? Another member recommends adding a 3 year post-transplant metric instead of a 5 year metric. In regards to kidney programs, another member voiced support for the longer term metric. Having the longer term metric would help programs identify the prevalence of re-listing patients who have already received a kidney. The MPSC could use that data to determine best practices to help kidney programs decrease the number of re-transplants. The member also voiced support of receiving regular feedback from the MPSC for those programs in the yellow zone to help a program avoid the red zone. Another member voiced support for kidney programs not being flagged for waitlist mortality. It is a worthwhile metric to collect in order to gauge its prevalence, but kidney programs should not be flagged for that metric. It was also noted that risk adjustment clearly will be important in this model (even more so than the current model). It would be a good idea before implementing this that the OPTN include a link to the risk adjustment model the MPSC will use. This is a dynamic risk adjustment model so every center should be aware that periodic review of the link to review the risk adjustment factors would be important. Another member voiced concern about binary factors, such as coronary artery disease, and if such factors are accurate in the risk adjustment model. Lastly, another member noted that since the MPSC will be monitoring each metric individually, and there are more metrics than what is currently monitored, this may mean more work for the MPSC going forward.

Daniel Stanton | 09/28/2021

The intent to include pre transplant metrics in the overall rating of transplant programs is a positive change. Including an offer acceptance ratio is a good addition to the modeling. This should encourage transplantation among all programs, and thus bring additional patient benefit. The MPSC states that the proposed changes are expected to increase the number of transplants performed. However, this is unlikely to occur based on 2 of the remaining metrics proposed. The direct focus on outcomes as a primary measure of program success will continue to limit transplantation for many patients. The “risk adjustment” for medically complex patients simply doesn’t provide enough adjustment for consideration when overall 1 year outcomes are expected to be at the mid to high 90th% for all organ types. Waitlist mortality is also a poor overall indicator for transplant program success. This measure will almost certainly lead to more conservative waitlisting practices, and again limit access to transplant for many medically complex patients. To increase access to transplantation, perhaps waiting list additions on a population based geography or alternative metric could be utilized. This would encourage evaluations/listings in lieu of penalizing programs for the reverse. I feel this proposal falls short of its desired intent to further promote / increase transplantation rates and volumes nationally.

Anonymous | 09/28/2021

The metrics that are being proposed do not, in their totality, meet the goals of increasing transplantation rates safely across the country. Specifically, following waiting list mortality will be especially punitive for transplant centers that do not typically have control over their patients prior to transplant. This will surely remove any incentives for programs to list patients (especially lower volume programs) that, over time, may not survive to transplant. Transplant programs should not be penalized for attempting to provide patients with the possibility of being transplanted should they meet the centers' standards for waitlisting. Also, I have significant concerns about the metric to monitor organ acceptance rates. This will be especially troublesome for pediatric programs who have specific criteria for organ acceptance. Lastly, for us as a transplant community to still be measuring one-year survival is more than slightly problematic. All of us who are active in the field of transplantation certainly hope that our patients will gain more than one year of life after transplantation. Expanding the metric may be what truly helps us provide and direct transplant resources to those patients that will truly benefit from it. One of the glaring omissions in these proposed metrics is the lack of "death with a functioning graft" as a measurement or adjustment for survival statistics. Programs cannot and should not be penalized for patient mortalities that (especially in this pandemic era) are truly beyond their control - for eg. CVA, COVID, MVA's, etc

Region 1 | 09/24/2021

Region 1 sentiment: 2 Strongly Support, 3 Support, 0 Neutral/Abstain, 2 Oppose, 1 Strongly Oppose. Comments: A comment was submitted stating that the principles for these metrics are well articulated and should be equally applied when developing OPO metrics. Two commenters suggested that "hard to place" organ outcomes should not be included in these metrics, as centers are concerned with the potential for MPSC review for a small series of poor outcomes that might be exacerbated by use of marginal organs. It was suggested that this could help provide an appropriate quality assessment that balances between optimization. A member expressed concern that anything that is overly punitive results in less transplants and more discarded organs.

OPTN Pediatric Transplantation Committee | 09/24/2021

The Pediatric Committee thanks the OPTN Membership & Professional Standards (MPSC) Committee for the opportunity to review their public comment proposal. The Committee provides the following feedback: The Committee noted that program improvements addressing graft failure and patient survival are requirement by Centers for Medicare & Medicaid Services (CMS) for approval of a transplant center based on mitigating factors, so it would be interesting to see how that might change with this proposal. The Committee also strongly encourages the incorporation of longer-term graft and patient outcome metrics in the future, as 90 days and even 1 year are very short compared to the expected decades of survival for most pediatric recipients. The Committee expressed appreciation of the collaborative work the MPSC has done to focus on transplant outcomes while also being respectful of program size.

Anonymous | 09/24/2021

The Pancreas Committee thanks the OPTN Membership & Professional Standards Committee (MPSC) for the opportunity to review their public comment proposal. The Committee provides the following feedback: A member mentioned that there’s a wide variety among what constitutes an acceptable candidate between programs. For example, some kidney programs do not accept patients who smoke or patients who are beyond a certain body mass index (BMI). The member inquired about a pre-transplant metric that may bring more conservative programs to a more reasonable stance. The member emphasized this would focus more on the candidate, as opposed to the outcomes and criteria set by the program. A member voiced opposition to longer monitoring, but strongly supported having good outcomes long term because it helps the patients, the program, and society. It was emphasized that asking programs to take control of patients beyond one year is unfair to the program. A member stated that, from the organ procurement organization (OPO) perspective, they think the organ offer acceptance metric is great and believe OPO’s will appreciate it.

Anonymous | 09/23/2021

Overall trying to assess programs via multiple metrics with different systems assessed by the various metrics is sensible and commendable. However I have four major concerns: 1. This proposal still provides no means of fairly assessing smaller programs. These programs will continue to be unfairly punished from single rare events especially if they happen in a short period of time. Since percentages and hazards are used a small program could easily be flagged due to the bad outcome of a single high risk patient coupled to an unfortunate outcome in a more standard risk patient within a short period of time. Now, by adding 4 metrics you are simply increasing the risk that a small program will be flagged based on having more metrics with which to flag them. This approach continues to push for larger programs and eventually both forces smaller programs to avoid high risk patients (in direct opposition to the goal of the changes) but also moves us closer to needing a regional transplant system -- which again moves against the final common rule goal of equal and fair access. For organs like heart and lung distance is critical as the candidate will often have to live near enough to the transplant hospital to be there in hours at the time of acceptance which creates enormous hardships and disincentive for patients to be listed. 2. This entire enterprise is predicated on the belief that the current risk models by SRTR utilized by UNOS are effective. As a pediatric heart transplant physician who cares for high risk children with congenital heart disease, we as a field have reported to SRTR repeatedly that their risk model is ineffective for CHD patients. This is not the fault of SRTR but is the truth of this highly heterogeneous population with poorly understood risk factors. The current SRTR model has a very small cadre of relatively unimportant risk factors when viewed from the point of view of someone caring for the patients. Physicians likely no better than statisticians which patients are at risk -- and the current fields and data collected by SRTR simply do not allow a good modeling. In fact attempts to model CHD risk with much more granular and complete data also fail to pass muster further reinforcing the inability of such risk modeling to be meaningful. 3. In that vein while described as an iterative process -- how is SRTR/MSPC testing that their current modeling is correct? How will they monitor that they are succeeding (those metrics were not discussed) and how will they test that their models are successfully identifying centers that actually need improvement not ones that are just managing challenging cohorts of patients, small patient numbers, etc. Again while SRTR insists their modeling works it does not seem to have a real world test to prove this -- nor has it attempted to work with small programs and pediatric programs to refine their analysis. 4. Finally, I remain concerned about the long standing policy of SRTR/MPSC to include all deaths in analyses. If a patient dies on the wait list of non organ related complications this should not count against a program whether on the wait list or post transplant. Similarly while I understand that removals will now be included in wait list statistics -- it seems counterintuitive. If you are listing sicker patients they are more likely to progress to become untransplantable and the responsible thing to do at that point is to delist them for becoming too ill. This is the desired active management discussed in the presentations. But as this is constructed that management will not change the negative ramifications of a patient getting more ill and dying. In heart transplant this would actually encourage a program to place the patient on high risk life support, accept an organ despite this increased risk of organ loss -- because now you have saved a wait list mortality AND increased your post-transplant risk assessment such that if the patient does not survive they are risk adjusted. There is no way this is the right approach to a shallow donor pool, and matches what has been seen with the new adult allocation policy where post-transplant survival has suffered due to pushing programs to take more inappropriate risks. While I support the project moving forward I think some real time needs to be spent evaluating if the metrics work, especially for pediatric and other small programs. If the metrics do not work, or the transplant programs do not have faith that they work based on their experience and lack of willingness for SRTR to engage with smaller and pediatric programs then this process will ultimately be for naught.

American Nephrology Nurses Association (ANNA | 09/23/2021

ANNA supports that transplant programs performance should be measured with more than one performance metric; however, we feel further discussion is needed. We have questions about how the proposed metrics will increase transplantation rates. Also, we oppose the waiting list mortality rate ratio metric, as transplant centers do not directly medically manage patients on the waiting list.

View attachment from American Nephrology Nurses Association (ANNA

Region 6 | 09/23/2021

Region 6 sentiment: 0 Strongly Support; 9 Support; 2 Neutral/Abstain; 0 Oppose; 0 Strongly Oppose. Comments: An attendee noted the OPTN should consider a project that would examine various factors related to risk adjustment. Another attendee asked the committee to consider excluding patients who have died due to COVID-19 from the data. Another commenter stated this proposed approach may lead to an increase in selection bias among programs as there may be a high acceptance rate for conservative programs when they are listing patients; and there may be a decrease in acceptance rates for programs that accept more marginal offers. The same attendee added centers that do not typically accept organs from DCD donors may not be flagged for review, since they are not listing their patients for these types of offers. A third attendee commented that for pediatric and other difficult-to-transplant recipients there are many reasons why an ideal organ may not be accepted for a specific patient, and there is no refusal code to capture these circumstances.

Region 8 | 09/22/2021

Region 8 sentiment: 9 strongly support, 12 support, 1 neutral/abstain, 0 oppose, 0 strongly oppose. Comments: Region 8 supports this proposal. A member commented that having a panel of metrics rather than a single static metric is a great idea and has been a goal for a number of years. The member also asked for consideration on two issues: The kidney allocation response: kidney acceptance is/can be influenced by program "rurality" and this is not completely taken into account by distance from the procurement location. The ability or inability to receive a flight greatly influences the member’s institutions ability to accept a kidney. And a kidney with 18 hours of cold time that cannot get here without an additional 14-16 hours because of flights (rather than 3-4) makes a big difference especially when that kidney is not pumped, and Longer term outcome metric: A conditional 1-year metric is fine, but real patient value comes from 3-year survival and longer. A 1-year survival with an eGFR of 22 is not a good value for the patient since the patient will probably need a re-transplant within the next year or two. A member stated that in general his institution supports this proposal. But notes that it would be helpful for the MPSC to articulate more clearly how they arrived at the assumption that the current number of programs that are "flagged" is the correct number. With the push to focus on QAPI partnerships, including those in the "yellow zone", rather than peer-review / punitive monitoring one would hope that the number of programs that need to be "flagged" will achieve the same overall system improvement would decrease. A member stated that he agreed with other comments regarding kidney programs and waitlist mortality concerns. He stated that the centers do not typically medically manage these patients while awaiting the transplant event. The member wanted to know how the persistence of COVID impacts all of the proposed criteria. A member asked that with regard to the question about resources required for informal MPSC assistance, it would be important to know what type of review types are available to transplant centers. The member asked that if a center chooses to not pursue assistance, would this have a negative impact on a program that transitions to the red area? A member asked that if a transplant program is evaluated on risk adjusted metrics/expected outcomes rather than "actual" it gives the transplant program an opportunity to accept offers that previously may have been declined due to their risk-adverse standards, thus increasing the number of transplants and increasing utilization of donor organs. The member suggested that a longer post-transplant (for example, five years) patient survival rate should be included in the metrics, for best use of the donor gift. The member suggested that this program should be rolled out on a small scale initially to confirm the intended outcomes of this system prior to national implementation. A member stated that he is in favor of reviewing this process, and would be interested in the approval to implementation period to know how this system would change the programs that are flagged/not-flagged, and what opportunities for improvement we may miss because of citing fewer programs.

Susan Bourgeois | 09/21/2021

Grateful to the MPSC and SRTR for this effort to broaden the scope of high performance to more than one aspect. This is a great step forward. Great job!

Marco Ricci, MD, MBA | 09/20/2021

I congratulate the MPSC committee for the proposed changes to the mechanism of transplant program performance evaluation. Of these, the rate of organ acceptance and 90-day survival are valuable. However, in my opinion, revisions to the SRTR model are needed for pediatric heart transplantation. The current SRTR model does not include powerful risk variables such as age at transplant (< 1 year), previous cardiac operations, single ventricle circulation, and allosensitization, among others. These risk variables have been shown in contemporary literature to adversely affect post-transplant survival. Also, other pre-operative risk variables such as renal dysfunction and liver dysfunction are inadequately weighted. Not using an accurate risk model precludes appropriate risk adjustment and evaluation of center specific performance.

Shahnawaz Amdani | 09/20/2021

While the intent is to improve performance, my biggest concern is about pediatric heart transplant candidates who are evaluated for heart transplant but not listed given concern for increased waitlist mortality. OPTN does not collect information for heart transplant candidates who are evaluated but not listed. This is extremely concerning as we may be limiting access to a critical group of children who have no other options. Biggest concern is that we may be picking candidates deemed to have the best outcomes, when what we want to achieve as a community is open access to all who are in need of a transplant.

FMCNA | 09/17/2021

FMNCA supports the proposal overall but we are concerned with competing incentives under the waitlist mortality measure as discussed in detail in our comment.

View attachment from FMCNA

Stephanie L | 09/17/2021

The noted concern of transplant centers denying patients due to not wanting to risk their numbers being affected (or the like) is a real issue presented by centers. By imposing strict monitoring, it does provide insight into individual centers, but it also provides punishment if numbers aren't met. As a result, patients suffer. Smaller centers don't want to take the risk, and often smaller centers are found in rural communities. Patient's end up having to travel state's away to see providers and be listed, which may not even be an option. Thus, patient's may lose the ability to even have transplant access. That being said, monitoring has to occur to ensure center's are being compliant, following regulations, ensuring patient safety... I just believe more consideration needs to be placed on the "why" of patient graft failure, instead of the "how many."

Anonymous | 09/15/2021

Region 7 sentiment: 3 strongly support, 7 support, 2 neutral/abstain, 1 oppose, 3 strongly oppose. Comments: One attendee stated they would like to see more information about how these changes would improve transplant rates. An attendee had concern that using waitlist mortality as a performance indicator may discourage programs from placing sicker patients on the waiting list. Some attendees expressed concern regarding using offer acceptance rate as a metric because it potentially disadvantages patients as programs will be forced to be much more rigid in setting organ filters. There was added concern that late turn-downs are actually much more disruptive than centers that would set their offer filters to be wider but still individually evaluate organs in a timely fashion. One attendee suggested that the granularity and specificity of organ parameters in DonorNet be improved to allow better filtering of organs before programs are graded on organ acceptance as a metric and added that there is often significant information that is not available at the time of the initial offer that impacts donor quality. One attendee commented that for difficult-to-transplant patients, setting strict opt-outs in organ filters may disadvantage patients, whereas evaluating each offer individually maximizes the patient’s opportunity for transplant. This might not represent the best balance between optimizing individual patient care and system efficiency. One attendee made a suggestion for a metric related to programs declining an organ offer when no additional data was provided. An attendee had concern that it is difficult to assess the quality of an organ based on multiple independent criteria. When reviewed with an experienced eye, the organ quality may look very different than just the variables available in the organ filters. One attendee commented that organ acceptance measures will be very different in different regions based on the quality of deceased donors. Another attendee asked if organ acceptance should be tied to transplant rate and should programs that are doing well on transplant rate be penalized for not screening out more organs based on filters at the initial offer. They asked what would be the expected improvement in system efficiency versus the cost in transplant rate. One attendee had concern that imperfect risk adjustment leads to difficulty in identifying programs that are truly under-performing. Not all factors that are clinically important are included in the risk models, in part because some are difficult to quantify. Further, incomplete understanding of risk adjustment models can lead programs to make decisions that are not in the program (or their patient’s) best interest. This attendee recommended that OPTN continue to make education about risk adjustment widely available. An attendee suggested that evaluating programs on patient rather than graft survival would decrease the disincentive for re-transplants and might reduce organ discards. One attendee had concern that since the proposal essentially maintains the same number of programs flagged, it will lead to more risk aversion by programs. Because there is no absolute value for the boundary between an adequately-performing and under-performing program, as programs become more risk-adverse, the overall survival increases but at the cost of doing fewer transplants especially with “riskier” organs or recipients. Although there may be statistical differences between programs, how do we know if these are clinically significant? Flagging could lead programs to deny risky patients and turn down more aggressive organs even though there is clinical benefit to recipients. Additional comments were made that overall this is a good concept and worthy of additional discussion and implementation.

Le Bonheur Children's Hospital | 09/14/2021

I support the development of separate pediatric and adult metrics

Region 9 | 09/14/2021

Region 9 sentiment:? 2 Strongly Support; 5 Support; 2 Neutral/Abstain; 0 Oppose; 0 Strongly Oppose. Comments:? Region 9 supported this proposal. A member suggested that transplant programs should have access to data and benchmark reports from the MPSC via a dashboard. A member suggested that the current system may discourage transplant programs from working with patients with specific and difficult issues, as outcomes for these patients may be poor. The member added that if the system encouraged transplant programs to take on difficult patients, more may do so and develop expertise in the treatment of these patients. A member commented that the MPSC has a number of good ideas of new ways to monitor performance. Also, a comment was made that while a member supported some of this proposal, they felt there was still too much that was unknown about it.

Caroline Rochon | 09/13/2021

I am in favor of these changes. It finally supports holistic quality of care that considers process, structure and outcomes rather than outcomes alone. I do have reservations about the offer acceptance rates however. It is easy to change a program's criteria to receive very selective offers. Programs who wish to be high organ utilizers ; improve access to transplant and decrease organ discard often open up the offer possibilities wide. Although we turn down many, we still use a lot of organs that many others overlooked. This being said, our offer acceptance rates may not always reflect this (because of the biased way it is measured). In the end, if offer acceptance rates will be maintained as a monitoring criteria, it should be based on a common denominator of offers (meaning include offers some centers decided to not even be notified of) or, may I suggest, a better metric would be rate of acceptance of offers ranked >100. My 2 cents.

Region 3 | 09/10/2021

Region 3 sentiment: 3 strongly support, 6 support, 2 neutral/abstain, 3 oppose, 0 strongly oppose. Region 3 supported this proposal. Many attendees had feedback for the committee and provided the following comments. Risk adjustment models in regards to listing do not adequately capture the risks programs see in person, such as poor physical function or frailty. These factors should be considered in risk adjustment models, if there are data to support them. The OPTN needs to encourage OPOs to perform more echos for heart offers. Often, OPOs turns down hospitals when they request another echo within a certain number of hours. Heart function may improve within a few hours, but still OPOs often won’t repeat echos. Transplant physicians are making every effort to improve the transplant rate and Waitlist mortality, and should be able to request repeat echos without being penalized. An attendee noted all metrics are risk-adjusted and at this time there are no outliers for the kidney program. However, the attendee has concerns about waitlist mortality being a component of the new transplant center performance metrics. Waitlist mortality is a factor that is largely out of a kidney program's control, as patients are actively managed by community nephrologists for many years on the list. This could change center practices to limit listing patients to avoid potential waitlist mortality. Co-morbidities change while dialysis patients are on the waitlist. How will this system account for these changes, which limit the predictive ability? There is concern this proposal will exacerbate racial disparity in listing, as minorities already have challenges getting listed. Early evaluation and listing should be promoted, not discouraged, as some patients do get early offers, and living donor discussions can start. The 90-day graft loss metric may flag poorly performing centers, but it can lead to centers shying away from higher risk patients.

Region 2 | 09/10/2021

Region 2 sentiment: 8 Strongly Support, 11 Support, 4 Neutral/Abstain, 2 Oppose, 0 Strongly Oppose Overall, the region was supportive of the proposal. One member voiced support in the goals of the proposal but noted that there is a risk for potential unintended consequences, such as risk aversion to listing high-risk patients or uncertainty about organ decline rate. Another member noted that the modeling data showed only a few programs that would be flagged, and they wonder if there is any utility for the proposal if so few programs are going to be flagged. There was a suggestion that the committee consider the use of MPSC workgroups to support those programs that are flagged. It was also noted that the metrics should reflect more on donor and recipient quality. Lastly, another member noted that it is important to align the goals of the OPOs and the transplant centers to transplant more organs, not just transplant the pristine organs. The responsibility of getting more organs transplanted should not solely be on the OPO, it is a shared responsibility of the OPOs, the transplant centers and the donor hospitals. Assess not only the outcomes of the transplant center, but the function of the centers, for example, what days of the week they accept organs, how many offers do they turn down and why.

Ronald Trammel | 09/10/2021

Ian Jamieson emphasized that MPSC performance monitoring data would not be made available to payers. However, is there anything that would prevent a payer from requesting this data directly from transplant program as a requirement for consideration for contracting, Centers of Excellence designation, etc. ?

Robert Goodman | 09/08/2021

As a heart transplant recipient and former member of the MPSC, I concur with the approach to simplify and make as unbiased as possible the proposed monitoring system. I believe it to be patient-centric, always a key consideration for me, and that it will provide good guidance for evaluating program performance to others. Patient safety, more transplants and better patient outcomes are of paramount importance here and elsewhere.

UAMS Solid Organ Transplant | 09/08/2021

The all-cause mortality index averaged for the five highest states is 926/100,000 and 624/100,000 people in the five lowest (CDC 2017). We are concerned that this disparity will be magnified and amplified in patients with end-organ disease and discourage listing of patients with limited financial and healthcare resources.

Anonymous | 09/07/2021

Further to a comment already posted, does OPTN have a minimum percentage of kidney transplant programs that must be flagged? If the criteria is a statistical comparison, then why not use a Z score of -1.96, which would be 2 standard deviations below the O-E/E? Within two standard deviations from the mean would cover 95% of programs.

Kenneth Andreoni | 09/07/2021

For unclear reasons, the MPSC and SRTR monitoring continue to define about 10% of kidney transplant programs as underperformers - 'flagged'. No other professional group deems 10% of its members as failures every six months. No matter how the MPSC/OPTN, SRTR, or HRSA want to frame this, transplant hospital leadership (CEOs) see this as a failure of their transplant teams, NOT AS PEER REVIEW. As has been PUBLISHED and relayed in other comments on this page, this ridiculous method of oversight continues to lead to decreased transplant opportunities for our patients and for donors and their families. When will we stop this insanity? The US short term kidney transplant - as well as extra-renal transplant - outcomes are excellent! We should have RARE 'flagging' of transplant centers. With the new Red Zone proposal, there will still be 10% of kidney programs flagged every six months. As has been shown throughout the last more than 10 years of MPSC outcome reviews, nearly all of these flagged programs did not have true patient safety issues. This is small random variation in six month patient events that lead to centers being deemed statistically 'flagged'. The FLAGGING OF FAR TOO MANY TRANSPLANT PROGRAMS is the EASIEST change to make to INCREASE TRANSPLANTATION to patients in the US with end stage organ disease. Let's please re-evaluate the ridiculously high number of programs that are flagged every six months. This should be a very small number, if any, of programs. The promise that 'no program may flag' is this proposed system is mathematically possible, but in reality impossible due to small random variation in outcomes that we all have as programs. I STRONGLY OPPOSE any proposal that continues to decrease transplant opportunities for patients due to the need to simply demonstrate regulatory oversight, not only without patient benefit, but with demonstrated patient harm. Ken Andreoni, MD Former OPTN/UNOS President Former Chair MPSC

kym watt | 09/06/2021

I have read the documents and viewed the videos. There are significant concerns with these that need to be voiced. Although everyone agrees we need checks and balances, the transplant community is held captive by overly restrictive metrics dictating ultimately who can get waitlisted and who can get transplanted. Yes, we do not want any patient getting suboptimal care. Survivals after transplant keep increasing partly due to improved care and management, but just as much if not more, due to patient selection, out of fear you do not meet the metrics. Every center is "expected" to have only a small number of deaths (that include, MVAs, suicide, murder, and even covid to name a few that are entirely unrelated to transplant itself). If you have even 1 or 2 of these occur, you cannot take any risk. If you are the patient and you may not have a 94% survival, you may not get to transplant. Every center feels this stress. These metrics have been become unsustainable if you truly want to save lives of patients with organ failure. There are many patients who have a very good chance of survival after transplant, but centers can only take so much risk. These post transplant metrics do not reduce this. Comparing to all other centers perpetuates this. Yes, there is tiny bit more wiggle room in the HR, but there needs to be a reset to the base. I realize your argument about not having a floor, somehow incentivizes bad behavior, but having this very high ceiling will continue to limit transplant to patients who will benefit from it, out of fear of not meeting metrics. Risk adjustment will only help if all factors are adjusted for. Patients with cholangiocarcinoma for example get no adjustment on either the waitlist or the post survival data despite having clear evidence demonstrating their increased risk. Since "only a few centers are affected" it is steadfastly refused to be adjusted for. Since pre transplant malignancy gets some adjustment - this means someone with skin cancer gets the same risk adjustment as CCA. And yes, this only affects a few centers, but it affects ALL CCA patients as those centers may not be able to absorb the risk. Adding in the graft survival component will have downstream impact on patients getting offered retransplant. it is possible a center cannot afford a second graft loss and not be able offer a reLT to an unexpected PNF or HAT. Yes, the metrics need to change. But since this is supposed to be about patient safety and our oath is "first, do no harm", these new metrics may only increase the harm. To be able to provide a safe transplant to patients who need it, We NEED to reset the ceiling.

Paul Morrissey | 09/03/2021

“Last poor outcome that you cannot get past.” – Matt Cooper, MD. President, UNOS. Quoted from Region 1 UNOS Meeting The UNOS Region 1 Fall Meeting was held last week (August 31, 2021). Dr. Cooper addressed the issue of organ utilization and described one’s “last poor outcome” as a barrier to future practice. Four months ago, I transplanted a kidney from a 60 y.o. donor who suffered CVA. DCD donor, terminal creatinine of 0.9 mg/dL and a favorable pre-transplant biopsy (see below). The recipient, a 69 y.o. with HTN and DM, was #32 on the allocation list. The kidney was turned down for 31 other patients, 30 of them coded “830” (donor age or quality). The recipient suffered dense DGF. His course was complicated by wound infection leading to bacteremia and an episode of urosepsis, again with bacteremia. He was briefly (3 weeks) off dialysis with a best creatinine of 3.7 mg/dL. The recipient of the mate kidney also required prolonged dialysis after transplant and the current creatinine is 3.6 mg/dL off dialysis. BIOPSY Donor Center Biopsy (frozen section): No arteriolar sclerosis, no hyalinosis, 1+ IFTA, 2-4% glomerulosclerosis. Recipient Center Biopsy (permanent section): Mild diffuse peritubular interstitial fibrosis. Moderate to severe arterial intimal fibrosis of likely donor origin. Earlier this year we lost a patient 7 weeks after renal transplant to a fatal PE and 10 months ago we lost a kidney to renal artery thrombosis after injuring a segmental renal artery during the transplant surgery and losing the kidney to retrograde thrombosis of the main renal artery on post-operative day #8. Overall, this led to 3 allograft losses in 34 DDRT performed in the past 12 months. The 12 months prior we had no graft losses. However, one-year graft survival in our program was flagged for two consecutive periods in the past 5 years by the SRTR (Five years includes ten data cycles of actual one-year patient and graft survival.) Aside from those two periods and that single metric, we have a solid kidney transplant program. SRTR reports for our program consistently show better than predicted 3-year allograft survival for deceased donor and living donor kidney transplants. We passed all our UNOS reviews with only minor administrative issues requiring correction. There are no glaring deficiencies in the listing, transplant, or post-transplant sectors of our program. How should we proceed presently after three recent graft losses? I’m not sure. How will we proceed? I’m more certain. For the next 12-18 months we will not accept marginal allografts: no ECD/DCD kidneys, for example. When we have candidate # 32 or #60 on the list we will likely turn-down the kidney. When we review marginal kidney offers in our monthly OPO meeting a surprising number of discarded kidneys are considered “transplantable” after the fact. Why were they not used? The answers are two-fold, and our recent graft loss perfectly illustrates them. 1. We cannot get past our “last poor outcome.” It’s heart-breaking dealing with a patient with the possibility of graft failure. It’s harder still when it becomes evident that the return to dialysis is permanent. The 15 encounters I’ve had with this patient: in-hospital, in clinic, in the hallways of our unit were increasingly challenging, discouraging and dispiriting. 2. We fear MPSC review. We are worried that outcomes that establish a hazard ratio of > 1.2 outside of our peers performance will open us up to punitive action and unfavorable publicity. Many transplant physicians in our region have commented on the conflicting messages we are receiving from UNOS: (1) transplant more kidneys and (2) maintain one-year allograft survival within 1.2 HR of other programs. We are all transplanting candidates who are number 60, 80 or 140 on the transplant list, but the use of these kidneys is not routine, and the kidneys transplanted are highly selected. When we transplant marginal allografts, we often have good outcomes, but often may only be 80% of the time. Coupled with another misadventure or two or an unpredicted fatal event, we can suddenly be on the radar of the MPSC. One’s practice then becomes difficult: troubled by oversight, feelings of inadequacy, and second-guessing of programmatic and personal decision-making. All this exacts an emotional toll on the transplant team. I hope that the wider set of criteria proposed by the MPSC will help alleviate some of these concerns that clearly influence organ utilization. Perhaps the adjusted HR of 1.75 will eliminate some of the burdens for marginally inferior outcomes. I doubt the new proposal will address the more difficult questions of which allograft to use for which patient and how to use marginal allografts when your peers are not. Presently, strict oversight and fear of MPSC reprisal (long-term review, extensive paperwork, lengthy explanations) leads to risk averse behavior and reduces organ utilization. Balancing these conflicting messages to transplant programs should be strongly considered as metrics are developed for measuring outcomes in kidney transplantation.

Dixon Kaufman | 09/02/2021

I appears that the same goals can be largely accomplished by evaluating patient survivals instead. This is what is essentially being measured for the vital organ transplants when monitoring graft survival (the outlier is kidney transplantation in this regard). Emphasis on patient survival offers several advantages. It is most relevant for patients and their families. Emphasizing patient survivals eliminates the dis-incentive for re-transplants, transplants in highly sensitized recipients, and other challenging recipient populations. In this context it minimizes the very difficult situation of trying to do what is in the best interests of the transplant candidate (and their family) versus prioritizing program outcomes. Emphasis on patient survival will reduce organ discards and will permit a more direct comparison to survival rates of alternative treatments to transplant. It also pertains to waiting list mortality, and transplant rates, important measures. The use of more organs does not mean patient safety will be compromised since all transplant programs have QAPI programs to self-monitor (backed-up by the MPSC) toward continued improvement, and as professionals, we do not aim to harm anyone with transplant surgery. The benefits would appear to outweigh the risks. Continuing discussion on this specific monitoring outcome is important.

Anonymous | 09/02/2021

Any metrics introduced should be patient based and represent the interest of patients in increasing access and number of transplants. Competitive comparisons of centers and fluctuating risk adjusted measures lead to defensive practices by programs and stifle innovation in the field. Any changes need to develop a proposal that would eliminate center-to-center competition and focus on serving patients, developing policies to allow surgeons, physicians, and the transplant community to best care for their patients, and create metrics to measure these policies.

IOWA DONOR NETWORK | 09/01/2021

As a former member of the MPSC, I applaud the work the committee has done to establish multi-dimensional transplant program metrics and the focus on improvement work. As a committee member, reviewing cases, I often felt that I did not have the whole picture or a complete understanding of all decision factors. I believe that this proposal is a strong step in the right direction and I encourage the committee to keep evaluating mitigating factors. For instance, when calculating waiting-list survival, what factor will be used. Is it Active Patients on the waiting list? Or will Inactive patients and their outcome be included? I push for evaluating only mortality among patients actively listed as this truly reflects transplant center involvement and awareness of candidate status. I applaud all efforts to look at organ offer turn downs as I believe this could be a very good tool for streamlining organ placement IF it is used as intended. Often centers have very broad donor criteria that include organ offers that they will never seriously consider. This practice just serves to, "gum up" the system and slow down the organ allocation process. Now that acceptable donor criteria can be established on a per candidate basis, I encourage all centers to take advantage of that feature. In that way, a transplant practitioner will only receive offers that he/she will seriously consider.

Anonymous | 08/30/2021

Region 5 sentiment: 7 strongly support, 16 support, 1 neutral/abstain, 0 oppose, 1 strongly oppose The majority of Region 5 supported the proposal to Enhance Transplant Program Performance Monitoring System with one neutral/abstention and one strong opposition. A member supports this proposal but points out two concerns: (1) that different programs can have different tolerance levels for how aggressive they are, and (2) waitlist mortality is a reflection of that aggression level, rather than a reflection of patient care. A member expressed a concern that the proposed changes to metrics may disincentives transplant. A member supports several parts of this proposal but does not support where the proposal allows governing and individual center’s decision making in accepting or declining offers because centers vary in how aggressive they have the capability of being in organ offer and acceptance. A member supports this proposal and suggests implementing it with a broad approach that considers all phases of transplant: pre-transplant, transplant, and post-transplant. A member supports an improvement in MPSC monitoring of program performance. However, the member suggests there shouldn’t be a “one size fits all” performance metric; rather, there should be different metrics and scales for each organ type. The member also asked the OPTN to help bridge the gap between OPOs and transplant centers. A member suggests that this monitoring system is imperative in order to achieve a consistent, safe, and ethical system.

James Wynn | 08/27/2021

Dr. Formica's presentation on this proposal included a slide that enumerated the criteria that the MPSC used to evaluate proposed performance metrics. The MPSC's first criterion required that the metrics measure aspects of care that: 1) are clearly within the authority of the OPTN, 2) are discrete and provided by transplant programs, and 3) can be sufficiently influenced by the transplant program. I believe that the MPSC showed excellent judgment in choosing this and the other four criteria. Unfortunately, the MPSC failed to consider the above criterion when it chose to include waiting list mortality as a transplant program performance metric, at least regarding kidney transplantation. Waiting list mortality measures an aspect of care that IS NOT provided by the transplant program and IS NOT substantially influenced by the program. Furthermore, oversight of ESRD care is a primary responsibility of CMS and the ESRD networks with which CMS contracts. I wholeheartedly support the other proposed metrics - but the MPSC clearly ignored its own criteria when choosing to include waiting list mortality as a performance measure. It should be removed from the policy proposal prior to its final consideration by the OPTN Board.

Anonymous | 08/27/2021

4 strongly support, 9 support, 1 neutral/abstain, 0 oppose, 2 strongly oppose. Many attendees had feedback for the committee and provided the following comments: Several attendees are concerned about organ offer acceptance ratio for pediatric patients. Adding that this cannot be applied to children, as there are many organs that sound great initially, but, based on anatomy, will not "fit" in children due to size, number of arteries/veins/etc. These organs can easily be transplanted in adults. This is a good thing, and should not be held against pediatric transplant programs who are using good judgement. A disincentive to reviewing a wide array of offers for a particular child could result in missed opportunities for transplantation, use of split livers, etc. It is important that members read and understand the risk adjustment model. This theme comes up in many different ways when we discuss monitoring. Members think they are incentivized to not transplant sicker patients on all organs or they are being incentivized to only list people who will have best possible outcomes. If a center only lists ideal patients, some will still have incidents (accidents) and the center will go past 1.0. If centers enter realistic screening and acceptance criteria for candidates, they will turn down fewer organs that are subsequently accepted by another center. This is vital, necessary work and I commend the MPSC on their approach. Expanding monitoring between 1 year survivals is something we've all agreed is long overdue and I believe the MPSC's approach is on track to significantly improve performance monitoring. Extensive organizational education will be necessary. In addition, a nonprofessional’s summary of the current system, the need for change, and the proposed changes should be prepared for patient education. A similar document will be helpful to transplant centers as they interact with patients. It seems pediatric programs are being held to a higher standard, without data to justify the thresholds. With regard to organ acceptance, organs suitable for teenagers are not suitable for infants, especially kidneys thus if we use narrow acceptance criteria so that we have a high graft acceptance rate, we will disadvantage our teenagers.

Anonymous | 08/13/2021

My biggest concern with this proposal was the seemingly primary intent to not increase the number of programs with difficulties. In my opinion, the primary intent should be to identify all programs that are operating at a sub-optimal level, where that is more or less than currently. We owe that to the patients on the lists in those centers. Period.

Anonymous | 08/09/2021

The metrics to assess transplant centers seem fair but they should be available to the public.