Skip to main content

Enhance Transplant Program Performance Monitoring System

eye iconAt a glance

Current policy

The Membership and Professional Standards Committee (MPSC) evaluates transplant program performance using one-year after transplant patient and organ outcomes.  Members have questioned whether the use of one measure sufficiently assesses program performance. Other members have suggested that relying only on after transplant outcomes may result in less transplants. The MPSC proposal wants to establish a more comprehensive system that evaluates several parts of transplant program performance.

Supporting media

Rich Formica, M.D., presents an overview of the proposal on behalf of the MPSC.

Watch MPSC Performance Monitoring Improvement Subcommittee chair Rich Formica's Aug. 4 webinar presentation that provides a more detailed overview of the MPSC's proposal.

Ian Jamieson, MPSC Chair, and Jon Snyder, Director of the SRTR, discuss the four evaluation metrics involved in the proposal, describe the models used to produce these metrics, and answer a number of common questions.

Presentation

View presentation

Proposed changes

  • Establishes four metrics that measure program performance before and after transplant
  • Sets separate adult and child criteria to identify potential patient safety issues
  • Reviews a transplant program if the program meets any new criteria
  • Provides an additional performance improvement zone to notify programs of concerning trends and offer assistance, if requested
  • Documents existing peer visit process in the bylaws

Anticipated impact

  • What it's expected to do
    • Increase number of transplants
    • Reduces emphasis on after transplant outcomes
    • Fulfills the MPSC’s obligation to monitor member performance and identify patient safety issues
    • Provides opportunities for the OPTN and the MPSC to support and work with transplant programs for better outcomes
  • What it won't do
    • Does not increase the number of transplant programs that have to work with the MPSC due to underperformance
    • Does not affect the metrics available or the 5 tier outcome assessments on the Scientific Registry of Transplant Recipients (SRTR) website

Themes

  • Holistic transplant program performance monitoring
  • Patient health and public safety
  • Performance improvement assistance

Terms to know

  • Waiting list mortality rate ratio: Describes risk of death once candidates are listed but before they are transplanted. The waiting list mortality rate ratio estimates the program’s waiting list mortality relative to the national expectations.
  • Offer acceptance rate ratio: Indicates whether a program is more or less likely to accept offers than the national average. If the offer acceptance ratio is greater than 1.0, then the program tends to accept more offers than average; if the offer acceptance ratio is less than 1.0, then the program tends to accept fewer offers than average.
  • 90-day graft survival hazard ratio: Provides an estimate whether the program has higher or lower than expected organ failure rates during the first 90-days after transplant as compared to transplant outcomes for all U.S. transplant programs. Organ failure numbers include organ failures, retransplants and patient deaths.
  • One-year graft survival conditional on 90-day graft survival hazard ratio: Provides an estimate whether the program has higher or lower than expected graft failure rates during the the first year after transplant.

Click here to search the OPTN glossary

Provide feedback

eye iconComments

Region 1 | 09/24/2021

Region 1 sentiment: 2 Strongly Support, 3 Support, 0 Neutral/Abstain, 2 Oppose, 1 Strongly Oppose. Comments: A comment was submitted stating that the principles for these metrics are well articulated and should be equally applied when developing OPO metrics. Two commenters suggested that "hard to place" organ outcomes should not be included in these metrics, as centers are concerned with the potential for MPSC review for a small series of poor outcomes that might be exacerbated by use of marginal organs. It was suggested that this could help provide an appropriate quality assessment that balances between optimization. A member expressed concern that anything that is overly punitive results in less transplants and more discarded organs.

OPTN Pediatric Transplantation Committee | 09/24/2021

The Pediatric Committee thanks the OPTN Membership & Professional Standards (MPSC) Committee for the opportunity to review their public comment proposal. The Committee provides the following feedback: The Committee noted that program improvements addressing graft failure and patient survival are requirement by Centers for Medicare & Medicaid Services (CMS) for approval of a transplant center based on mitigating factors, so it would be interesting to see how that might change with this proposal. The Committee also strongly encourages the incorporation of longer-term graft and patient outcome metrics in the future, as 90 days and even 1 year are very short compared to the expected decades of survival for most pediatric recipients. The Committee expressed appreciation of the collaborative work the MPSC has done to focus on transplant outcomes while also being respectful of program size.

OPTN Pancreas Transplantation Committee | 09/24/2021

The Pancreas Committee thanks the OPTN Membership & Professional Standards Committee (MPSC) for the opportunity to review their public comment proposal. The Committee provides the following feedback: A member mentioned that there’s a wide variety among what constitutes an acceptable candidate between programs. For example, some kidney programs do not accept patients who smoke or patients who are beyond a certain body mass index (BMI). The member inquired about a pre-transplant metric that may bring more conservative programs to a more reasonable stance. The member emphasized this would focus more on the candidate, as opposed to the outcomes and criteria set by the program. A member voiced opposition to longer monitoring, but strongly supported having good outcomes long term because it helps the patients, the program, and society. It was emphasized that asking programs to take control of patients beyond one year is unfair to the program. A member stated that, from the organ procurement organization (OPO) perspective, they think the organ offer acceptance metric is great and believe OPO’s will appreciate it.

Anonymous | 09/23/2021

Overall trying to assess programs via multiple metrics with different systems assessed by the various metrics is sensible and commendable. However I have four major concerns: 1. This proposal still provides no means of fairly assessing smaller programs. These programs will continue to be unfairly punished from single rare events especially if they happen in a short period of time. Since percentages and hazards are used a small program could easily be flagged due to the bad outcome of a single high risk patient coupled to an unfortunate outcome in a more standard risk patient within a short period of time. Now, by adding 4 metrics you are simply increasing the risk that a small program will be flagged based on having more metrics with which to flag them. This approach continues to push for larger programs and eventually both forces smaller programs to avoid high risk patients (in direct opposition to the goal of the changes) but also moves us closer to needing a regional transplant system -- which again moves against the final common rule goal of equal and fair access. For organs like heart and lung distance is critical as the candidate will often have to live near enough to the transplant hospital to be there in hours at the time of acceptance which creates enormous hardships and disincentive for patients to be listed. 2. This entire enterprise is predicated on the belief that the current risk models by SRTR utilized by UNOS are effective. As a pediatric heart transplant physician who cares for high risk children with congenital heart disease, we as a field have reported to SRTR repeatedly that their risk model is ineffective for CHD patients. This is not the fault of SRTR but is the truth of this highly heterogeneous population with poorly understood risk factors. The current SRTR model has a very small cadre of relatively unimportant risk factors when viewed from the point of view of someone caring for the patients. Physicians likely no better than statisticians which patients are at risk -- and the current fields and data collected by SRTR simply do not allow a good modeling. In fact attempts to model CHD risk with much more granular and complete data also fail to pass muster further reinforcing the inability of such risk modeling to be meaningful. 3. In that vein while described as an iterative process -- how is SRTR/MSPC testing that their current modeling is correct? How will they monitor that they are succeeding (those metrics were not discussed) and how will they test that their models are successfully identifying centers that actually need improvement not ones that are just managing challenging cohorts of patients, small patient numbers, etc. Again while SRTR insists their modeling works it does not seem to have a real world test to prove this -- nor has it attempted to work with small programs and pediatric programs to refine their analysis. 4. Finally, I remain concerned about the long standing policy of SRTR/MPSC to include all deaths in analyses. If a patient dies on the wait list of non organ related complications this should not count against a program whether on the wait list or post transplant. Similarly while I understand that removals will now be included in wait list statistics -- it seems counterintuitive. If you are listing sicker patients they are more likely to progress to become untransplantable and the responsible thing to do at that point is to delist them for becoming too ill. This is the desired active management discussed in the presentations. But as this is constructed that management will not change the negative ramifications of a patient getting more ill and dying. In heart transplant this would actually encourage a program to place the patient on high risk life support, accept an organ despite this increased risk of organ loss -- because now you have saved a wait list mortality AND increased your post-transplant risk assessment such that if the patient does not survive they are risk adjusted. There is no way this is the right approach to a shallow donor pool, and matches what has been seen with the new adult allocation policy where post-transplant survival has suffered due to pushing programs to take more inappropriate risks. While I support the project moving forward I think some real time needs to be spent evaluating if the metrics work, especially for pediatric and other small programs. If the metrics do not work, or the transplant programs do not have faith that they work based on their experience and lack of willingness for SRTR to engage with smaller and pediatric programs then this process will ultimately be for naught.

American Nephrology Nurses Association (ANNA | 09/23/2021

ANNA supports that transplant programs performance should be measured with more than one performance metric; however, we feel further discussion is needed. We have questions about how the proposed metrics will increase transplantation rates. Also, we oppose the waiting list mortality rate ratio metric, as transplant centers do not directly medically manage patients on the waiting list.

Attachment

Region 6 | 09/23/2021

Region 6 sentiment: 2 Strongly Support; 16 Support; 3 Neutral/Abstain; 3 Oppose; 0 Strongly Oppose. Comments: An attendee noted the OPTN should consider a project that would examine various factors related to risk adjustment. Another attendee asked the committee to consider excluding patients who have died due to COVID-19 from the data. Another commenter stated this proposed approach may lead to an increase in selection bias among programs as there may be a high acceptance rate for conservative programs when they are listing patients; and there may be a decrease in acceptance rates for programs that accept more marginal offers. The same attendee added centers that do not typically accept organs from DCD donors may not be flagged for review, since they are not listing their patients for these types of offers. A third attendee commented that for pediatric and other difficult-to-transplant recipients there are many reasons why an ideal organ may not be accepted for a specific patient, and there is no refusal code to capture these circumstances.

Region 8 | 09/22/2021

Region 8 sentiment: 9 strongly support, 12 support, 1 neutral/abstain, 0 oppose, 0 strongly oppose. Comments: Region 8 supports this proposal. A member commented that having a panel of metrics rather than a single static metric is a great idea and has been a goal for a number of years. The member also asked for consideration on two issues: The kidney allocation response: kidney acceptance is/can be influenced by program "rurality" and this is not completely taken into account by distance from the procurement location. The ability or inability to receive a flight greatly influences the member’s institutions ability to accept a kidney. And a kidney with 18 hours of cold time that cannot get here without an additional 14-16 hours because of flights (rather than 3-4) makes a big difference especially when that kidney is not pumped, and Longer term outcome metric: A conditional 1-year metric is fine, but real patient value comes from 3-year survival and longer. A 1-year survival with an eGFR of 22 is not a good value for the patient since the patient will probably need a re-transplant within the next year or two. A member stated that in general his institution supports this proposal. But notes that it would be helpful for the MPSC to articulate more clearly how they arrived at the assumption that the current number of programs that are "flagged" is the correct number. With the push to focus on QAPI partnerships, including those in the "yellow zone", rather than peer-review / punitive monitoring one would hope that the number of programs that need to be "flagged" will achieve the same overall system improvement would decrease. A member stated that he agreed with other comments regarding kidney programs and waitlist mortality concerns. He stated that the centers do not typically medically manage these patients while awaiting the transplant event. The member wanted to know how the persistence of COVID impacts all of the proposed criteria. A member asked that with regard to the question about resources required for informal MPSC assistance, it would be important to know what type of review types are available to transplant centers. The member asked that if a center chooses to not pursue assistance, would this have a negative impact on a program that transitions to the red area? A member asked that if a transplant program is evaluated on risk adjusted metrics/expected outcomes rather than "actual" it gives the transplant program an opportunity to accept offers that previously may have been declined due to their risk-adverse standards, thus increasing the number of transplants and increasing utilization of donor organs. The member suggested that a longer post-transplant (for example, five years) patient survival rate should be included in the metrics, for best use of the donor gift. The member suggested that this program should be rolled out on a small scale initially to confirm the intended outcomes of this system prior to national implementation. A member stated that he is in favor of reviewing this process, and would be interested in the approval to implementation period to know how this system would change the programs that are flagged/not-flagged, and what opportunities for improvement we may miss because of citing fewer programs.

Susan Bourgeois | 09/21/2021

Grateful to the MPSC and SRTR for this effort to broaden the scope of high performance to more than one aspect. This is a great step forward. Great job!

Marco Ricci, MD, MBA | 09/20/2021

I congratulate the MPSC committee for the proposed changes to the mechanism of transplant program performance evaluation. Of these, the rate of organ acceptance and 90-day survival are valuable. However, in my opinion, revisions to the SRTR model are needed for pediatric heart transplantation. The current SRTR model does not include powerful risk variables such as age at transplant (< 1 year), previous cardiac operations, single ventricle circulation, and allosensitization, among others. These risk variables have been shown in contemporary literature to adversely affect post-transplant survival. Also, other pre-operative risk variables such as renal dysfunction and liver dysfunction are inadequately weighted. Not using an accurate risk model precludes appropriate risk adjustment and evaluation of center specific performance.

Shahnawaz Amdani | 09/20/2021

While the intent is to improve performance, my biggest concern is about pediatric heart transplant candidates who are evaluated for heart transplant but not listed given concern for increased waitlist mortality. OPTN does not collect information for heart transplant candidates who are evaluated but not listed. This is extremely concerning as we may be limiting access to a critical group of children who have no other options. Biggest concern is that we may be picking candidates deemed to have the best outcomes, when what we want to achieve as a community is open access to all who are in need of a transplant.

FMCNA | 09/17/2021

FMNCA supports the proposal overall but we are concerned with competing incentives under the waitlist mortality measure as discussed in detail in our comment.

Attachment

Stephanie L | 09/17/2021

The noted concern of transplant centers denying patients due to not wanting to risk their numbers being affected (or the like) is a real issue presented by centers. By imposing strict monitoring, it does provide insight into individual centers, but it also provides punishment if numbers aren't met. As a result, patients suffer. Smaller centers don't want to take the risk, and often smaller centers are found in rural communities. Patient's end up having to travel state's away to see providers and be listed, which may not even be an option. Thus, patient's may lose the ability to even have transplant access. That being said, monitoring has to occur to ensure center's are being compliant, following regulations, ensuring patient safety... I just believe more consideration needs to be placed on the "why" of patient graft failure, instead of the "how many."

Region 7 | 09/15/2021

Region 7 sentiment: 3 strongly support, 7 support, 2 neutral/abstain, 1 oppose, 3 strongly oppose. Comments: One attendee stated they would like to see more information about how these changes would improve transplant rates. An attendee had concern that using waitlist mortality as a performance indicator may discourage programs from placing sicker patients on the waiting list. Some attendees expressed concern regarding using offer acceptance rate as a metric because it potentially disadvantages patients as programs will be forced to be much more rigid in setting organ filters. There was added concern that late turn-downs are actually much more disruptive than centers that would set their offer filters to be wider but still individually evaluate organs in a timely fashion. One attendee suggested that the granularity and specificity of organ parameters in DonorNet be improved to allow better filtering of organs before programs are graded on organ acceptance as a metric and added that there is often significant information that is not available at the time of the initial offer that impacts donor quality. One attendee commented that for difficult-to-transplant patients, setting strict opt-outs in organ filters may disadvantage patients, whereas evaluating each offer individually maximizes the patient’s opportunity for transplant. This might not represent the best balance between optimizing individual patient care and system efficiency. One attendee made a suggestion for a metric related to programs declining an organ offer when no additional data was provided. An attendee had concern that it is difficult to assess the quality of an organ based on multiple independent criteria. When reviewed with an experienced eye, the organ quality may look very different than just the variables available in the organ filters. One attendee commented that organ acceptance measures will be very different in different regions based on the quality of deceased donors. Another attendee asked if organ acceptance should be tied to transplant rate and should programs that are doing well on transplant rate be penalized for not screening out more organs based on filters at the initial offer. They asked what would be the expected improvement in system efficiency versus the cost in transplant rate. One attendee had concern that imperfect risk adjustment leads to difficulty in identifying programs that are truly under-performing. Not all factors that are clinically important are included in the risk models, in part because some are difficult to quantify. Further, incomplete understanding of risk adjustment models can lead programs to make decisions that are not in the program (or their patient’s) best interest. This attendee recommended that OPTN continue to make education about risk adjustment widely available. An attendee suggested that evaluating programs on patient rather than graft survival would decrease the disincentive for re-transplants and might reduce organ discards. One attendee had concern that since the proposal essentially maintains the same number of programs flagged, it will lead to more risk aversion by programs. Because there is no absolute value for the boundary between an adequately-performing and under-performing program, as programs become more risk-adverse, the overall survival increases but at the cost of doing fewer transplants especially with “riskier” organs or recipients. Although there may be statistical differences between programs, how do we know if these are clinically significant? Flagging could lead programs to deny risky patients and turn down more aggressive organs even though there is clinical benefit to recipients. Additional comments were made that overall this is a good concept and worthy of additional discussion and implementation.

Le Bonheur Children's Hospital | 09/14/2021

I support the development of separate pediatric and adult metrics

Region 9 | 09/14/2021

Region 9 sentiment:  2 Strongly Support; 5 Support; 2 Neutral/Abstain; 0 Oppose; 0 Strongly Oppose. Comments:  Region 9 supported this proposal. A member suggested that transplant programs should have access to data and benchmark reports from the MPSC via a dashboard. A member suggested that the current system may discourage transplant programs from working with patients with specific and difficult issues, as outcomes for these patients may be poor. The member added that if the system encouraged transplant programs to take on difficult patients, more may do so and develop expertise in the treatment of these patients. A member commented that the MPSC has a number of good ideas of new ways to monitor performance. Also, a comment was made that while a member supported some of this proposal, they felt there was still too much that was unknown about it.

Caroline Rochon | 09/13/2021

I am in favor of these changes. It finally supports holistic quality of care that considers process, structure and outcomes rather than outcomes alone. I do have reservations about the offer acceptance rates however. It is easy to change a program's criteria to receive very selective offers. Programs who wish to be high organ utilizers ; improve access to transplant and decrease organ discard often open up the offer possibilities wide. Although we turn down many, we still use a lot of organs that many others overlooked. This being said, our offer acceptance rates may not always reflect this (because of the biased way it is measured). In the end, if offer acceptance rates will be maintained as a monitoring criteria, it should be based on a common denominator of offers (meaning include offers some centers decided to not even be notified of) or, may I suggest, a better metric would be rate of acceptance of offers ranked >100. My 2 cents.

Region 3 | 09/10/2021

Region 3 sentiment: 3 strongly support, 6 support, 2 neutral/abstain, 3 oppose, 0 strongly oppose. Region 3 supported this proposal. Many attendees had feedback for the committee and provided the following comments. Risk adjustment models in regards to listing do not adequately capture the risks programs see in person, such as poor physical function or frailty. These factors should be considered in risk adjustment models, if there are data to support them. The OPTN needs to encourage OPOs to perform more echos for heart offers. Often, OPOs turns down hospitals when they request another echo within a certain number of hours. Heart function may improve within a few hours, but still OPOs often won’t repeat echos. Transplant physicians are making every effort to improve the transplant rate and Waitlist mortality, and should be able to request repeat echos without being penalized. An attendee noted all metrics are risk-adjusted and at this time there are no outliers for the kidney program. However, the attendee has concerns about waitlist mortality being a component of the new transplant center performance metrics. Waitlist mortality is a factor that is largely out of a kidney program's control, as patients are actively managed by community nephrologists for many years on the list. This could change center practices to limit listing patients to avoid potential waitlist mortality. Co-morbidities change while dialysis patients are on the waitlist. How will this system account for these changes, which limit the predictive ability? There is concern this proposal will exacerbate racial disparity in listing, as minorities already have challenges getting listed. Early evaluation and listing should be promoted, not discouraged, as some patients do get early offers, and living donor discussions can start. The 90-day graft loss metric may flag poorly performing centers, but it can lead to centers shying away from higher risk patients.

Region 2 | 09/10/2021

Region 2 sentiment: 8 Strongly Support, 11 Support, 4 Neutral/Abstain, 2 Oppose, 0 Strongly Oppose Overall, the region was supportive of the proposal. One member voiced support in the goals of the proposal but noted that there is a risk for potential unintended consequences, such as risk aversion to listing high-risk patients or uncertainty about organ decline rate. Another member noted that the modeling data showed only a few programs that would be flagged, and they wonder if there is any utility for the proposal if so few programs are going to be flagged. There was a suggestion that the committee consider the use of MPSC workgroups to support those programs that are flagged. It was also noted that the metrics should reflect more on donor and recipient quality. Lastly, another member noted that it is important to align the goals of the OPOs and the transplant centers to transplant more organs, not just transplant the pristine organs. The responsibility of getting more organs transplanted should not solely be on the OPO, it is a shared responsibility of the OPOs, the transplant centers and the donor hospitals. Assess not only the outcomes of the transplant center, but the function of the centers, for example, what days of the week they accept organs, how many offers do they turn down and why.

Ronald Trammel | 09/10/2021

Ian Jamieson emphasized that MPSC performance monitoring data would not be made available to payers. However, is there anything that would prevent a payer from requesting this data directly from transplant program as a requirement for consideration for contracting, Centers of Excellence designation, etc. ?

Robert Goodman | 09/08/2021

As a heart transplant recipient and former member of the MPSC, I concur with the approach to simplify and make as unbiased as possible the proposed monitoring system. I believe it to be patient-centric, always a key consideration for me, and that it will provide good guidance for evaluating program performance to others. Patient safety, more transplants and better patient outcomes are of paramount importance here and elsewhere.

UAMS Solid Organ Transplant | 09/08/2021

The all-cause mortality index averaged for the five highest states is 926/100,000 and 624/100,000 people in the five lowest (CDC 2017). We are concerned that this disparity will be magnified and amplified in patients with end-organ disease and discourage listing of patients with limited financial and healthcare resources.

Anonymous | 09/07/2021

Further to a comment already posted, does OPTN have a minimum percentage of kidney transplant programs that must be flagged? If the criteria is a statistical comparison, then why not use a Z score of -1.96, which would be 2 standard deviations below the O-E/E? Within two standard deviations from the mean would cover 95% of programs.

Kenneth Andreoni | 09/07/2021

For unclear reasons, the MPSC and SRTR monitoring continue to define about 10% of kidney transplant programs as underperformers - 'flagged'. No other professional group deems 10% of its members as failures every six months. No matter how the MPSC/OPTN, SRTR, or HRSA want to frame this, transplant hospital leadership (CEOs) see this as a failure of their transplant teams, NOT AS PEER REVIEW. As has been PUBLISHED and relayed in other comments on this page, this ridiculous method of oversight continues to lead to decreased transplant opportunities for our patients and for donors and their families. When will we stop this insanity? The US short term kidney transplant - as well as extra-renal transplant - outcomes are excellent! We should have RARE 'flagging' of transplant centers. With the new Red Zone proposal, there will still be 10% of kidney programs flagged every six months. As has been shown throughout the last more than 10 years of MPSC outcome reviews, nearly all of these flagged programs did not have true patient safety issues. This is small random variation in six month patient events that lead to centers being deemed statistically 'flagged'. The FLAGGING OF FAR TOO MANY TRANSPLANT PROGRAMS is the EASIEST change to make to INCREASE TRANSPLANTATION to patients in the US with end stage organ disease. Let's please re-evaluate the ridiculously high number of programs that are flagged every six months. This should be a very small number, if any, of programs. The promise that 'no program may flag' is this proposed system is mathematically possible, but in reality impossible due to small random variation in outcomes that we all have as programs. I STRONGLY OPPOSE any proposal that continues to decrease transplant opportunities for patients due to the need to simply demonstrate regulatory oversight, not only without patient benefit, but with demonstrated patient harm. Ken Andreoni, MD Former OPTN/UNOS President Former Chair MPSC

kym watt | 09/06/2021

I have read the documents and viewed the videos. There are significant concerns with these that need to be voiced. Although everyone agrees we need checks and balances, the transplant community is held captive by overly restrictive metrics dictating ultimately who can get waitlisted and who can get transplanted. Yes, we do not want any patient getting suboptimal care. Survivals after transplant keep increasing partly due to improved care and management, but just as much if not more, due to patient selection, out of fear you do not meet the metrics. Every center is "expected" to have only a small number of deaths (that include, MVAs, suicide, murder, and even covid to name a few that are entirely unrelated to transplant itself). If you have even 1 or 2 of these occur, you cannot take any risk. If you are the patient and you may not have a 94% survival, you may not get to transplant. Every center feels this stress. These metrics have been become unsustainable if you truly want to save lives of patients with organ failure. There are many patients who have a very good chance of survival after transplant, but centers can only take so much risk. These post transplant metrics do not reduce this. Comparing to all other centers perpetuates this. Yes, there is tiny bit more wiggle room in the HR, but there needs to be a reset to the base. I realize your argument about not having a floor, somehow incentivizes bad behavior, but having this very high ceiling will continue to limit transplant to patients who will benefit from it, out of fear of not meeting metrics. Risk adjustment will only help if all factors are adjusted for. Patients with cholangiocarcinoma for example get no adjustment on either the waitlist or the post survival data despite having clear evidence demonstrating their increased risk. Since "only a few centers are affected" it is steadfastly refused to be adjusted for. Since pre transplant malignancy gets some adjustment - this means someone with skin cancer gets the same risk adjustment as CCA. And yes, this only affects a few centers, but it affects ALL CCA patients as those centers may not be able to absorb the risk. Adding in the graft survival component will have downstream impact on patients getting offered retransplant. it is possible a center cannot afford a second graft loss and not be able offer a reLT to an unexpected PNF or HAT. Yes, the metrics need to change. But since this is supposed to be about patient safety and our oath is "first, do no harm", these new metrics may only increase the harm. To be able to provide a safe transplant to patients who need it, We NEED to reset the ceiling.

Paul Morrissey | 09/03/2021

“Last poor outcome that you cannot get past.” – Matt Cooper, MD. President, UNOS. Quoted from Region 1 UNOS Meeting The UNOS Region 1 Fall Meeting was held last week (August 31, 2021). Dr. Cooper addressed the issue of organ utilization and described one’s “last poor outcome” as a barrier to future practice. Four months ago, I transplanted a kidney from a 60 y.o. donor who suffered CVA. DCD donor, terminal creatinine of 0.9 mg/dL and a favorable pre-transplant biopsy (see below). The recipient, a 69 y.o. with HTN and DM, was #32 on the allocation list. The kidney was turned down for 31 other patients, 30 of them coded “830” (donor age or quality). The recipient suffered dense DGF. His course was complicated by wound infection leading to bacteremia and an episode of urosepsis, again with bacteremia. He was briefly (3 weeks) off dialysis with a best creatinine of 3.7 mg/dL. The recipient of the mate kidney also required prolonged dialysis after transplant and the current creatinine is 3.6 mg/dL off dialysis. BIOPSY Donor Center Biopsy (frozen section): No arteriolar sclerosis, no hyalinosis, 1+ IFTA, 2-4% glomerulosclerosis. Recipient Center Biopsy (permanent section): Mild diffuse peritubular interstitial fibrosis. Moderate to severe arterial intimal fibrosis of likely donor origin. Earlier this year we lost a patient 7 weeks after renal transplant to a fatal PE and 10 months ago we lost a kidney to renal artery thrombosis after injuring a segmental renal artery during the transplant surgery and losing the kidney to retrograde thrombosis of the main renal artery on post-operative day #8. Overall, this led to 3 allograft losses in 34 DDRT performed in the past 12 months. The 12 months prior we had no graft losses. However, one-year graft survival in our program was flagged for two consecutive periods in the past 5 years by the SRTR (Five years includes ten data cycles of actual one-year patient and graft survival.) Aside from those two periods and that single metric, we have a solid kidney transplant program. SRTR reports for our program consistently show better than predicted 3-year allograft survival for deceased donor and living donor kidney transplants. We passed all our UNOS reviews with only minor administrative issues requiring correction. There are no glaring deficiencies in the listing, transplant, or post-transplant sectors of our program. How should we proceed presently after three recent graft losses? I’m not sure. How will we proceed? I’m more certain. For the next 12-18 months we will not accept marginal allografts: no ECD/DCD kidneys, for example. When we have candidate # 32 or #60 on the list we will likely turn-down the kidney. When we review marginal kidney offers in our monthly OPO meeting a surprising number of discarded kidneys are considered “transplantable” after the fact. Why were they not used? The answers are two-fold, and our recent graft loss perfectly illustrates them. 1. We cannot get past our “last poor outcome.” It’s heart-breaking dealing with a patient with the possibility of graft failure. It’s harder still when it becomes evident that the return to dialysis is permanent. The 15 encounters I’ve had with this patient: in-hospital, in clinic, in the hallways of our unit were increasingly challenging, discouraging and dispiriting. 2. We fear MPSC review. We are worried that outcomes that establish a hazard ratio of > 1.2 outside of our peers performance will open us up to punitive action and unfavorable publicity. Many transplant physicians in our region have commented on the conflicting messages we are receiving from UNOS: (1) transplant more kidneys and (2) maintain one-year allograft survival within 1.2 HR of other programs. We are all transplanting candidates who are number 60, 80 or 140 on the transplant list, but the use of these kidneys is not routine, and the kidneys transplanted are highly selected. When we transplant marginal allografts, we often have good outcomes, but often may only be 80% of the time. Coupled with another misadventure or two or an unpredicted fatal event, we can suddenly be on the radar of the MPSC. One’s practice then becomes difficult: troubled by oversight, feelings of inadequacy, and second-guessing of programmatic and personal decision-making. All this exacts an emotional toll on the transplant team. I hope that the wider set of criteria proposed by the MPSC will help alleviate some of these concerns that clearly influence organ utilization. Perhaps the adjusted HR of 1.75 will eliminate some of the burdens for marginally inferior outcomes. I doubt the new proposal will address the more difficult questions of which allograft to use for which patient and how to use marginal allografts when your peers are not. Presently, strict oversight and fear of MPSC reprisal (long-term review, extensive paperwork, lengthy explanations) leads to risk averse behavior and reduces organ utilization. Balancing these conflicting messages to transplant programs should be strongly considered as metrics are developed for measuring outcomes in kidney transplantation.

Dixon Kaufman | 09/02/2021

I appears that the same goals can be largely accomplished by evaluating patient survivals instead. This is what is essentially being measured for the vital organ transplants when monitoring graft survival (the outlier is kidney transplantation in this regard). Emphasis on patient survival offers several advantages. It is most relevant for patients and their families. Emphasizing patient survivals eliminates the dis-incentive for re-transplants, transplants in highly sensitized recipients, and other challenging recipient populations. In this context it minimizes the very difficult situation of trying to do what is in the best interests of the transplant candidate (and their family) versus prioritizing program outcomes. Emphasis on patient survival will reduce organ discards and will permit a more direct comparison to survival rates of alternative treatments to transplant. It also pertains to waiting list mortality, and transplant rates, important measures. The use of more organs does not mean patient safety will be compromised since all transplant programs have QAPI programs to self-monitor (backed-up by the MPSC) toward continued improvement, and as professionals, we do not aim to harm anyone with transplant surgery. The benefits would appear to outweigh the risks. Continuing discussion on this specific monitoring outcome is important.

Anonymous | 09/02/2021

Any metrics introduced should be patient based and represent the interest of patients in increasing access and number of transplants. Competitive comparisons of centers and fluctuating risk adjusted measures lead to defensive practices by programs and stifle innovation in the field. Any changes need to develop a proposal that would eliminate center-to-center competition and focus on serving patients, developing policies to allow surgeons, physicians, and the transplant community to best care for their patients, and create metrics to measure these policies.

IOWA DONOR NETWORK | 09/01/2021

As a former member of the MPSC, I applaud the work the committee has done to establish multi-dimensional transplant program metrics and the focus on improvement work. As a committee member, reviewing cases, I often felt that I did not have the whole picture or a complete understanding of all decision factors. I believe that this proposal is a strong step in the right direction and I encourage the committee to keep evaluating mitigating factors. For instance, when calculating waiting-list survival, what factor will be used. Is it Active Patients on the waiting list? Or will Inactive patients and their outcome be included? I push for evaluating only mortality among patients actively listed as this truly reflects transplant center involvement and awareness of candidate status. I applaud all efforts to look at organ offer turn downs as I believe this could be a very good tool for streamlining organ placement IF it is used as intended. Often centers have very broad donor criteria that include organ offers that they will never seriously consider. This practice just serves to, "gum up" the system and slow down the organ allocation process. Now that acceptable donor criteria can be established on a per candidate basis, I encourage all centers to take advantage of that feature. In that way, a transplant practitioner will only receive offers that he/she will seriously consider.

Region 5 | 08/30/2021

Region 5 sentiment: 7 strongly support, 16 support, 1 neutral/abstain, 0 oppose, 1 strongly oppose The majority of Region 5 supported the proposal to Enhance Transplant Program Performance Monitoring System with one neutral/abstention and one strong opposition. A member supports this proposal but points out two concerns: (1) that different programs can have different tolerance levels for how aggressive they are, and (2) waitlist mortality is a reflection of that aggression level, rather than a reflection of patient care. A member expressed a concern that the proposed changes to metrics may disincentives transplant. A member supports several parts of this proposal but does not support where the proposal allows governing and individual center’s decision making in accepting or declining offers because centers vary in how aggressive they have the capability of being in organ offer and acceptance. A member supports this proposal and suggests implementing it with a broad approach that considers all phases of transplant: pre-transplant, transplant, and post-transplant. A member supports an improvement in MPSC monitoring of program performance. However, the member suggests there shouldn’t be a “one size fits all” performance metric; rather, there should be different metrics and scales for each organ type. The member also asked the OPTN to help bridge the gap between OPOs and transplant centers. A member suggests that this monitoring system is imperative in order to achieve a consistent, safe, and ethical system.

James Wynn | 08/27/2021

Dr. Formica's presentation on this proposal included a slide that enumerated the criteria that the MPSC used to evaluate proposed performance metrics. The MPSC's first criterion required that the metrics measure aspects of care that: 1) are clearly within the authority of the OPTN, 2) are discrete and provided by transplant programs, and 3) can be sufficiently influenced by the transplant program. I believe that the MPSC showed excellent judgment in choosing this and the other four criteria. Unfortunately, the MPSC failed to consider the above criterion when it chose to include waiting list mortality as a transplant program performance metric, at least regarding kidney transplantation. Waiting list mortality measures an aspect of care that IS NOT provided by the transplant program and IS NOT substantially influenced by the program. Furthermore, oversight of ESRD care is a primary responsibility of CMS and the ESRD networks with which CMS contracts. I wholeheartedly support the other proposed metrics - but the MPSC clearly ignored its own criteria when choosing to include waiting list mortality as a performance measure. It should be removed from the policy proposal prior to its final consideration by the OPTN Board.

Region 4 | 08/27/2021

4 strongly support, 9 support, 1 neutral/abstain, 0 oppose, 2 strongly oppose. Many attendees had feedback for the committee and provided the following comments: Several attendees are concerned about organ offer acceptance ratio for pediatric patients. Adding that this cannot be applied to children, as there are many organs that sound great initially, but, based on anatomy, will not "fit" in children due to size, number of arteries/veins/etc. These organs can easily be transplanted in adults. This is a good thing, and should not be held against pediatric transplant programs who are using good judgement. A disincentive to reviewing a wide array of offers for a particular child could result in missed opportunities for transplantation, use of split livers, etc. It is important that members read and understand the risk adjustment model. This theme comes up in many different ways when we discuss monitoring. Members think they are incentivized to not transplant sicker patients on all organs or they are being incentivized to only list people who will have best possible outcomes. If a center only lists ideal patients, some will still have incidents (accidents) and the center will go past 1.0. If centers enter realistic screening and acceptance criteria for candidates, they will turn down fewer organs that are subsequently accepted by another center. This is vital, necessary work and I commend the MPSC on their approach. Expanding monitoring between 1 year survivals is something we've all agreed is long overdue and I believe the MPSC's approach is on track to significantly improve performance monitoring. Extensive organizational education will be necessary. In addition, a nonprofessional’s summary of the current system, the need for change, and the proposed changes should be prepared for patient education. A similar document will be helpful to transplant centers as they interact with patients. It seems pediatric programs are being held to a higher standard, without data to justify the thresholds. With regard to organ acceptance, organs suitable for teenagers are not suitable for infants, especially kidneys thus if we use narrow acceptance criteria so that we have a high graft acceptance rate, we will disadvantage our teenagers.

Anonymous | 08/13/2021

My biggest concern with this proposal was the seemingly primary intent to not increase the number of programs with difficulties. In my opinion, the primary intent should be to identify all programs that are operating at a sub-optimal level, where that is more or less than currently. We owe that to the patients on the lists in those centers. Period.

Anonymous | 08/09/2021

The metrics to assess transplant centers seem fair but they should be available to the public.