Academic

Professors Jame Stribopoulos and Moin Yahya recently published an article in the Osgoode Hall Law Journal entitled, Does a Judge's Party of Appointment or Gender Matter to Case Outcomes? An Empirical Study of the Court of Appeal for Ontario.  The abstract explains:

This study reveals that at least in certain categories of cases, both party of appointment and gender are statistically significant in explaining case outcomes.  Between these two variables, gender actually appears to be the stronger determinant of outcome in certain types of cases.  While these findings are cause for concern, this study also points toward a simple solution.  Diversity in the composition of appeal panels both from the standpoint of gender and party of appointment dampened the statistical influence of either variable.  In other words, in the case of gender, a single judge on a panel who is of the opposite sex from the others, or in the case of political party, a single judge appointed by a different political party, is sufficient to eliminate the potential distorting influence of either variable.  This finding suggests a need to reform how appeal panels are currently assembled in order to ensure political and gender diversity and minimize concerns about the potential for bias.

The methodology adopted by the authors attempted to isolate the influence on appeal outcomes of the gender and party of appointing Prime Minister composition of panels sitting on each case.  Their dataset included all reported judgments of the Court of Appeal from 1990 to 2003, a total of 4,906 cases.  The authors found that in several types of appeals that gender and party of appointment appear to have a significant effect on whether an appeal is likely to succeed.

One of the unanswered questions arising out of this study is the degree of variation in the policy preferences of the individual justices.  A methodology that is well suited to estimating these preferences is one developed by two American political scientists, Andrew Martin and Kevin Quinn.  It involves an item response theory model that is estimated using a Markov Chain Monte Carlo approach.  Stribopoulos and Yahya were kind enough to share their data so that I could estimate the policy preferences of the 40 justices who served on the Court of Appeal during this period with this different approach. 

Here is a summary of the findings:

There are a number of interesting aspects to these results.  First, the gap between the average ideal points of Conservative and Liberal appointees is 0.368.  The gap between male and female judges is considerably smaller, at 0.234.  In other words, the gap in average ideal point differences between Conservative and Liberal appointees is about 50% larger than the gender gap.

Second, the gap for party of appointment is driven entirely by male justices.  The gap between average ideal points of justices who are male and Conservative appointees and those who are male and Liberal appointees is much larger than the gap including both genders at 0.482.  For females, the gap is of the opposite direction and is a negligible -0.031.  In other words, party of appointment appears to have a polarizing effect only on male judges and not at all on female judges.

There are several weaknesses of the Martin-Quinn method in this context that I should briefly acknowledge.  One is that the method looks only to cases in which there is a dissent.  The Court of Appeal has an incredible rate of unanimity in its judgments--around 95%--meaning that there were only 277 appeals to analyze.  The second weakness is that there is no control for different types of cases.  Contracts cases are lumped in with criminal and constitutional cases with reckless abandon.  Third, there is no way to control the fact that the cases were heard over a number of years.  The method treats them as if they were decided contemporaneously.  Finally, because the Court of Appeal is ostensibly bound by judgments of the Supreme Court of Canada, the judges may not feel as if they have complete freedom to vote consistently with their policy preferences.

In light of all these weaknesses, the point of this analysis is not  to cast doubt on the results reached by Stribopoulos and Yahya.  Instead the goal is to point out the benefits of deploying multiple methodologies on the same data to check the robustness of the results and to rule out alternative explanations for the results reached.  The curious result that party of appointment seems only to matter for males on the Court of Appeal during this period is an interesting one that ought to be examined further.  The analysis here cannot be conclusive because there were just four female judges appointed by Liberals and five female judges appointed by Conservatives yielding this result, which cannot really be considered to be generalizable due to the small numbers involved.

Additional insights may well flow from an examination of the full results for each of the individual judges, which are reproduced below for those who are interested.  I invite comments on what can be made of them.