How the California Bar Actually Grades the First Year Law Students’ Exam

I’ve made a few posts discussing California’s First Year Law Students’ Exam (the “FYLSX”), noting that I failed my first attempt by a fraction of a percent when the Bar applied a grading formula different from that which was advertised, and then, using the knowledge gained of their altered grading scheme, passed on my second try so spectacularly that the Bar published one of my essays as an example of how to write their exam.  I also briefly mentioned that I — as I’m known to do — filed suit to ask a judge to effectively require them to grade like they say they will and be more transparent about the exam and how they grade it.

The particular dispute was that they advertised that the multiple-choice section and the essay section of the exam would be “converted to the same 400-point scale” in order to “give[] these sections equal weight.”  When I received my score report from the first exam, I noticed that the scaling formula used by the bar resulted in it being impossible to score more than about 362 points even if every question was answered correctly, and likewise, it was possible to score far more than 400 points on the essays.

The attorney for the Bar assigned to the case disclosed additional documents, formerly considered to be secret and until now never disclosed to law students, to me yesterday that all but flatly admit that not only were the sections not weighted equally, but they also didn’t use a 400-point scale!  See if you can follow this tortured grading system they describe, because it took me several reads to figure it out:

Multiple-choice raw scores (i.e., number of items answered correctly) were equated to the June 1998, 2011, and October 2013 exams using 21 items that were common to each of these exams.  The equating formula was as follows:

Multiple Choice Scale = (3.4092 x raw multiple-choice score) + 21.6267

…

The candidates’ raw total essay scores were scaled to a score distribution that had the same mean and standard deviation as their multiple choice scores using the following formula:

Essay Scale = (2.3536 x raw essay total score) + -442.389

A candidate’s total score was the sum of that candidates’ multiple choice and essay scores.

What I gather from this is that 21 of the 100 multiple choice questions were repeats of previous years, and based on how well students did on those 21 questions, their grade on the entire 100 questions was curved.  Then, they calculated the average multiple choice score and the standard deviation, and curved the essay scores such that the average student gets the same score on that section as the multiple choice, and the score distribution was normalized to the same standard deviation.

From this it is clear:

1. This has nothing to do with a 400-point scale per section.  The number 400 does not even appear in this internal document regarding their grading scheme.  The scale is created simply by comparing this group’s scores to previous scores and trying to curve it accordingly.  You’ll notice that if you plug in “100” as your raw multiple-choice score (a perfect 100 out of 100 questions) into the formula provided, the maximum score attainable is 362.5467.  Not 400.
2. The sections do not have “equal weight.”  Saying that the average test taker got the same score on both their multiple-choice and their essay questions is not the same thing as saying “half your grade comes from multiple-choice, and half from the essays.”  Whether the essays count more or less than the multiple-choice section depends entirely on how well the other students do on their exams.  This is not equality, it is normalization.

So, for those taking the June 2017 exam, know this: on both exams I sat for, the essays counted far more than the multiple-choice.  You should therefore be spending far more time studying how to write a good essay than how to select A, B, C, or D correctly, unless your multiple-choice practice exams are turning out abysmally (for more study tips, see my previous post).  Hopefully by the October 2017 exam, a judge will have “persuaded” them to abandon this system.