On July 4, 2011, TRAC posted a special report about the extent of the disparities in how the administrative law judges within the Social Security Administration's separate hearing offices decide their cases. It also described how these disparities have grown in the last few years. On the same day, the SSA released a statement sharply criticizing TRAC's study on several different grounds.
This very critical SSA press release was similar to the agency's criticism of an earlier TRAC report, posted on June 20, 2011, concerning the recent growth in the backlog of individuals awaiting a hearing on their claims for disability benefits. TRAC subsequently released a response to that earlier SSA criticism.
A point-by-point comparison of the SSA's July 4 criticism and TRAC's report of the same day indicates that the agency's objections are unjustified in light of what our report actually says. Furthermore, the agency's statement ignores the body of evidence that TRAC presented and the accompanying series of tables laying out very detailed evidence concerning every hearing office's disparity results and the underlying details on each judge's grant rate within that office.
Here is our response to the central thrust of SSA's criticisms:
There is absolutely no evidence that the variation in judges' grant rates is the result of decisional independence. In fact, such independence is the goal of all judicial proceedings because it is an essential component of "due process." In addition, history tells us that independence is far superior to the alternatives — hearings where the decisions are dictated by backroom deals, the pressures of powerful political or financial interests, or the short-term desires of managers needing to meet a target.
As our report noted, the goal here is not to assure that each judge in the nation is making the same decisions but to determine the extent to which their decisions are being reached in a reasonable and principled fashion. Judging after all is a challenging profession, and continual efforts at improving quality are an essential component in any well-run judicial system. This task is made infinitely more difficult if judges are not given decisional independence. And ALJs further need the tools and support required in order to focus needed attention on improving decision quality.
To the contrary, TRAC compared the decisions in each of the hearing offices both before and after the agency adopted initiatives to speed the processing of cases. The report laid out evidence documenting that, whether examined by the disparity levels among all judges in an office or the disparity levels found among the middle half of judges in an office, decisions today are more variable than they were before programs were changed to reduce the backlog and speed the hearing process. [The measure of disparity among the middle half of judges statisticians refer to as the "IQR" or interquartile range. Like the standard deviation, the IQR measures variability and is used in contexts where, as here, the presence of outliers badly biases results based upon standard deviation comparisons.]
There is no basis for this claim. A very basic mathematical proof — called the "law of large numbers" — demonstrates that, however diverse the cases a single hearing office may handle, the same fundamental law applies. When a large number of cases are assigned to judges at random — that is, "in rotation" — then the composition of the cases each judge receives will on average have the same "worthiness." As our report discussed, this basic principle means that however diverse the office's caseload, differences in the grant rates among judges in the same hearing office cannot be attributed to any difference in the facts and circumstances of the cases each judge was assigned to decide so long as the cases were assigned in rotation.
TRAC's sources of funding for this study were actually published in TRAC's reports. In regard to the July 4 study see last paragraph of the sidebar titled "About the Data." Our sources of support (published on TRAC's website) for the research center's operations more generally demonstrate that we do not accept funding from federal sources in order to preserve our independence to publish our research findings on the performance of federal government programs.
Related links:
TRAC's June 20, 2011 report, SSA statement, and TRAC response to SSA's claims:
http://trac.syr.edu/tracreports/ssa/253/
http://ssa.gov/pressoffice/pr/trac-report.htm
http://trac.syr.edu/tracreports/ssa/253/include/side_2.html
TRAC's July 4, 2011 report and SSA statement
http://trac.syr.edu/tracreports/ssa/254/
http://ssa.gov/pressoffice/pr/trac-report-2.htm
About the research center (includes sources of funding):
http://trac.syr.edu/aboutTRACgeneral.html