Asylum Outcome Increasingly Depends on Judge Assigned
The outcome for asylum seekers has become increasingly dependent upon the identity of the immigration judge assigned to hear their case. While judge-to-judge decision disparities have long existed, a detailed comparison of asylum decisions handed down by judges sitting on the same Immigration Court bench showed that differences in judge denial rates have significantly increased during the last six years. Nationally, the average decision disparity in asylum cases worsened by 27 percent.
The median level of asylum decision disparity that asylum seekers face is now over 56 percentage points. That is, the assignment of the judge for the typical asylum seeker could alter the odds of receiving asylum by this magnitude. For example, while the specific ranges differed by court, the typical asylum seeker might have only a 15 percent chance of being granted asylum all the way up to a 71 percent chance depending on the particular judge to whom their case is assigned.
Figure 1. Range in Judge Asylum Denial Rates
in the Ten Immigration Courts with Largest Disparities
During this same period, the court has become increasingly challenged by a rising backlog of cases, along with administrative pressure to expedite proceedings. While the evidence does not establish a definitive link between production pressures and increasing judge-to-judge decision disparities, as discussed further below, other administrative courts facing management pressures to reduce a backlog have brought about increases in decision disparity.
These latest Immigration Court numbers are based on case-by-case records on each asylum decision that were obtained and analyzed by the Transactional Records Access Clearinghouse (TRAC) at Syracuse University. These court records were provided by the Department of Justice's Executive Office for Immigration Review (EOIR) as a result of a series of Freedom of Information Act requests by TRAC.
Measuring Asylum Decision Disparity
It is natural for asylum decisions to vary since the merits of individual cases differ. Achieving the goal of "equal justice" under the law does not mean assuring the same outcome for every case. Decisions should reflect the individual merits of each situation, applying the standards that the law sets. Some individuals merit asylum, while others do not.
Large differences in the decisions of individual judges that are not the result of differences in the nature of the incoming cases, however, are not desirable. These differences are what TRAC means when it uses the term, decision disparity.
Ten years ago, in July 2006, TRAC published its first report on the handling of asylum cases by the Immigration Courts. After examining all recorded cases in which immigration judges decided asylum cases from FY 1994 through the first few months of fiscal year 2005, TRAC found wide disparities in asylum denial rates had persisted from judge to judge throughout this period.
In August 2006, shortly after TRAC's report was published, then U.S. Attorney General Alberto Gonzales directed EOIR "to review [TRAC's] study and provide an analysis and, if appropriate, recommendations to the Deputy AG with respect to this issue." This specific directive was part of wider reform measures the Attorney General also announced.
EOIR's review mandated by the Attorney General confirmed that a problem did exist (although it viewed the problem as "attributable to a relatively small number of judges"). The agency's internal report on its investigations concluded: "EOIR is committed to addressing this issue through an enhanced training, mentoring, and supervisory program that will not improperly impinge upon the adjudicatory role of the immigration judges, but that will ensure that all immigration decisions are based on the law and facts in each specific case." It then took steps to implement these to address the problem.
The reforms related to asylum decision disparity are described in TRAC's June 22, 2009 report. A broader assessment of all of the reforms initiated by Attorney General Gonzales is given in TRAC's June 30, 2009 report.
How does TRAC measure decision disparity in asylum cases? We do this by taking advantage of court practices that assign incoming asylum applications to judges within a court on a random basis.
These random assignment procedures parallel what happens in a scientific experiment where individuals are assigned randomly to different treatments, as in a drug trial. Here, however, instead of being assigned to different drug treatments, asylum seekers are assigned to different judges. When individual judges handle a sufficient number of asylum requests, random case assignment will result in each judge being assigned an equivalent mix of asylum seekers.
Or put another way: to the extent judges' grant or denial rates depart from one another within the same court, if cases were randomly assigned, these differences cannot be explained on the basis of the worthiness of the cases handled by individual judges because each is handling an equivalent mix of cases. One or more other factors - such as the pre-existing predilections of the judge assigned - must then be the source for these disparities[1].
Many Asylum Seekers Now Face Extreme Levels of Decision Disparity
Based upon a detailed analysis of immigration court records covering the last six years, TRAC compared the denial rates for individual judges within each court.
Table 1 lists asylum decision disparity for each of the forty-eight Immigration Courts that had two or more judges with a sufficient number of asylum decisions for reliable comparison[2]. Decision disparity is measured by the difference between the asylum denial rate for the judge with the lowest and highest denial rates on each court.
Considering these court-by-court decision disparities, we found that the median magnitude of asylum decision disparity that asylum seekers now face was 56.8 percent. That is, the particular judge assigned the asylum seeker changed the odds of receiving asylum by over 56 percentage points. Half of asylum seekers faced disparity this large or larger, while the remaining half confronted disparity that was this level or smaller[3]. For example, while figures varied by court, a typical asylum seeker might have only a 15 percent chance of being granted asylum all the way up to a 71 percent chance depending on the particular judge assigned to hear the case.
Ten courts with the highest decision disparities account for the vast majority (63%) of asylum decisions. These ten are shown in Figure 1.
Asylum seekers in the Newark, San Francisco and the Los Angeles Immigration Courts confronted the highest level of decision disparity. In Newark, the 6 judges serving on that court had denial rates that varied between a low of 15.7 percent and a high of 98.6 percent. Their grant rates accordingly varied from only 1.4 percent all the way up to 84.3 percent.
The outcomes handed down for the 17 judges on the San Francisco Immigration Court were similarly extreme. In that court, asylum denial rates varied from a low of 15.3 percent to a high of 97.7 percent depending on the judge. (Thus their grants rates varied from 2.3 percent to 84.7 percent.) The 33 judges on the Los Angeles Immigration Court had asylum denial rates ranging almost as widely. There the denial rates varied from a low of 21.8 percent to a high of 97.1 percent. (Grant rates that thus varied between 2.9 percent and 78.2 percent.)
(click title to open in a new window)
| Immigration Court* | Asylum Decisions | Number of Judges* | Judge Denial Rate | ||
|---|---|---|---|---|---|
| Lowest | Highest | Range | |||
| New York | 37,970 | 31 | 2.2 | 59.0 | 56.8 |
| Los Angeles | 13,052 | 33 | 21.8 | 97.1 | 75.3 |
| San Francisco | 6,524 | 17 | 15.3 | 97.7 | 82.4 |
| Arlington | 3,717 | 7 | 17.4 | 88.5 | 71.1 |
| Baltimore | 3,468 | 6 | 2.6 | 58.2 | 55.6 |
| Seattle | 3,064 | 5 | 52.6 | 70.0 | 17.4 |
| Miami | 2,935 | 12 | 50.9 | 84.4 | 33.5 |
| Orlando | 2,802 | 6 | 56.5 | 75.2 | 18.7 |
| Houston | 2,674 | 7 | 66.7 | 96.0 | 29.3 |
| Boston | 2,244 | 8 | 19.4 | 56.9 | 37.5 |
| Chicago | 2,243 | 7 | 40.9 | 95.7 | 54.8 |
| Atlanta | 1,738 | 5 | 70.8 | 97.7 | 26.9 |
| San Diego | 1,720 | 8 | 37.9 | 93.0 | 55.1 |
| Miami - Krome | 1,680 | 6 | 90.5 | 97.8 | 7.3 |
| Newark | 1,557 | 6 | 15.7 | 98.6 | 82.9 |
| San Antonio | 1,495 | 6 | 23.9 | 96.6 | 72.7 |
| Adelanto | 1,420 | 6 | 75.2 | 91.8 | 16.6 |
| Cleveland | 1,382 | 3 | 62.5 | 89.9 | 27.4 |
| Memphis | 1,382 | 3 | 52.5 | 84.5 | 32.0 |
| Los Fresnos | 1,371 | 3 | 74.5 | 78.0 | 3.5 |
| Dallas | 1,211 | 4 | 69.3 | 93.2 | 23.9 |
| Charlotte | 1,191 | 3 | 71.6 | 84.5 | 12.9 |
| Philadelphia | 1,182 | 4 | 22.4 | 50.5 | 28.1 |
| Tacoma | 1,180 | 3 | 71.3 | 73.2 | 1.9 |
| Bloomington | 1,053 | 3 | 71.8 | 97.2 | 25.4 |
| El Paso | 1,042 | 5 | 93.7 | 99.4 | 5.7 |
| Omaha | 1,012 | 3 | 70.4 | 88.2 | 17.8 |
| Portland | 1,012 | 2 | 48.7 | 57.6 | 8.9 |
| Detroit | 952 | 4 | 71.0 | 81.7 | 10.7 |
| Hartford | 892 | 2 | 44.7 | 72.6 | 27.9 |
| Honolulu | 889 | 2 | 26.4 | 36.5 | 10.1 |
| Elizabeth | 880 | 2 | 44.8 | 72.2 | 27.4 |
| Denver | 879 | 4 | 35.3 | 63.9 | 28.6 |
| Eloy | 751 | 4 | 90.4 | 97.4 | 7.0 |
| Las Vegas | 638 | 2 | 83.4 | 88.6 | 5.2 |
| Houston - Detained | 581 | 4 | 87.1 | 100.0 | 12.9 |
| York | 572 | 3 | 75.6 | 90.4 | 14.8 |
| Kansas City | 562 | 2 | 63.4 | 69.4 | 6.0 |
| Buffalo | 557 | 3 | 76.4 | 87.1 | 10.7 |
| Pearsall | 524 | 3 | 79.4 | 85.0 | 5.6 |
| Oakdale | 495 | 3 | 93.1 | 100.0 | 6.9 |
| Lumpkin | 479 | 2 | 95.8 | 96.3 | 0.5 |
| Florence | 407 | 2 | 90.2 | 91.8 | 1.6 |
| Harlingen | 393 | 2 | 47.8 | 58.8 | 11.0 |
| Tucson | 325 | 2 | 53.4 | 63.6 | 10.2 |
| New Orleans | 304 | 2 | 55.8 | 72.0 | 16.2 |
| Phoenix | 268 | 2 | 14.3 | 63.6 | 49.3 |
| Aurora | 233 | 2 | 77.6 | 96.3 | 18.7 |
Nor, did this range result from simply an individual "outlier" judge. Denial rates varied widely across the judges in each of these three courts. Little consensus was evident. The individual judge-by-judge denial rates on each court are given in the Appendix Table to this report. Clearly, the outcome of asylum cases in these courts appears to be highly dependent upon the identity of the judge assigned to hear the asylum seeker's case.
While significant decision disparity was the general pattern found in most courts, this was not true in all courts. As shown in Table 1, there were in fact twelve out of the 48 immigration courts with decision disparity under 10 percentage points. Judges on these courts were closely aligned in their asylum grant and denial rates. As a group, these twelve tended to have fewer judges and often heard only detained cases. Because they had generally fewer cases, these courts accounted for only a relatively small percentage (8.8%) of total asylum decisions during the study period.
However, having a small number of judges, or serving on a court that just heard detained cases, did not necessarily lead to high inter-judge agreement. For example, while the Immigration Court that heard cases in a detention facility in Elizabeth New Jersey had only two judges, their denial rates were quite different at 44.8 percent and 72.2 percent.
Is Decision Disparity Increasing or Decreasing?
TRAC previously found that immigration judge asylum decision disparity—although still substantial—had declined during the FY 2008 - FY 2010 period[4]. This decline followed the implementation of a number of administrative changes by the Bush Administration designed to help reduce asylum decision disparities. See side bar on "Prior Efforts at Reducing Asylum Decision Disparity."
With the latest results in Table 1 covering the subsequent FY 2011 - FY 2016 period, it is now possible to compare these more current figures, court by court, with the levels of decision disparity that existed during the FY 2008 - FY 2010 period. Based on this comparison, we can then assess whether the magnitude of asylum decision disparity has stabilized or continued to decline over the past six years. Or, alternatively, determine whether the problem has worsened.
The comparisons for the sixteen courts covered in TRAC's previous studies are shown in Figure 2 and Table 2. These courts accounted for three out of every four asylum decisions that were made during the past six years. As is apparent from the graph, asylum decision disparity levels have generally increased—sometimes dramatically. Increases were seen in 12 out of the 16 courts previously studied[5]. Decreases were seen in only 4 out of these 16 courts.
Figure 2. Asylum Decision Disparity Has Increased in Most Immigration Courts
Overall, results indicate that there has been on average a 27 percent increase in decision disparity over the past six years in these sixteen courts. Substantial increases in decision disparity in particular were seen in the Newark, San Francisco, Los Angeles, San Antonio, Arlington, Baltimore, San Diego, Chicago and Boston courts.
| Immigration Court | FY 2008 to FY 2010** | FY 2011 to FY 2016 | Change |
|---|---|---|---|
| Newark | 32.2 | 82.9 | up |
| San Francisco | 60.1 | 82.4 | up |
| Los Angeles | 47.2 | 75.3 | up |
| San Antonio | 48.8 | 72.7 | up |
| Arlington | 38.9 | 71.1 | up |
| New York | 63.7 | 56.8 | down |
| Baltimore | 25.1 | 55.6 | up |
| San Diego | 33.2 | 55.1 | up |
| Chicago | 38.7 | 54.8 | up |
| Boston | 11.5 | 37.5 | up |
| Miami | 28.2 | 33.5 | up |
| Houston | 32.8 | 29.3 | down |
| Philadelphia | 26.3 | 28.1 | up |
| Cleveland | 49.8 | 27.4 | down |
| Atlanta | 20.2 | 26.9 | up |
| Orlando | 33.1 | 18.7 | down |
These are very troubling trends.
Conclusion
The recent spate of hiring of new immigration judges, when combined with the significant number of judges now leaving the bench, present a new opportunity to re-open the dialogue on how to achieve greater inter-judge agreement in immigration court decisions, including those granting or denying asylum.
However, the growing backlog of cases has increased pressures to expedite proceedings. Other administrative courts facing management pressure to quicken case processing have had the unintended consequence of actually increasing decision disparity. It would not be surprising if pressures on immigration judges to speed decision making, along with the reduced time available for training and discussion on the practical challenges in deciding asylum cases, may also have been a source of the increasing asylum decision disparities that has occurred[6].
What is clear, is that the extreme level of disparity now present in asylum outcomes is inconsistent with our country's commitment to equal justice and the rule of law.
Footnotes
[1] The General Accountability Office uses a different methodology, based on statistical controls, in examining asylum decisions in its November 2016 report (GAO-17-72) finding "Variation Exists in Outcomes of [Asylum] Applications Across Immigration Courts and Judges." Because the study did not have measures of the "underlying facts and merits of individual asylum applications," the GAO properly noted that it was not able to rule out the possibility that observed differences in decision outcomes may have arisen from differences in the facts and merits of individual cases. See p. 38.
[2] Only courts with two or more judges who had decided at least 100 asylum cases during the FY 2010 - FY 2016 period are listed. These reflect the judge-by-judge grant and denial decisions for this period taken from the individual report on each judge found at: http://trac.syr.edu/immigration/reports/judgereports. To facilitate comparison, the denial rate from the individual report on each judge is listed in the Appendix Table to this report.
[3] This figure was computed by sorting courts by their respective disparity levels, and then selecting the value that reflected where the median asylum case was located along this range.
[4] TRAC compared trends in asylum decision disparity in its 2009 and 2010 reports.
[5] TRAC's 2010 study covered immigration courts that had 4 or more judges who had decided at least 75 decisions in the FY 2008 - FY 2010 post-reform period - fourteen in all. Two additional courts, Philadelphia and Atlanta, only met this criteria in the FY 2007 - FY 2009 period reported in TRAC's earlier 2009 study. To provide the broadest coverage these two additional courts are also included in Figure 2.
[6] To better understand the challenges of rendering asylum decisions and the problem of combatting implicit bias, see "Who, Me? Am I Guilty of Implicit Bias?" by Judge Dana Leigh Marks in Vol. 54, No. 4 of The Judges' Journal, 2015 available at: http://www.americanbar.org/publications/judges_journal/2015/fall/who_me_am_i_guilty_of_implicit_bias.html.