Search

Professional Troublemaker

 Jonathan Corbett, Civil Rights Advocate

Search results

"fylsx"

State Bar of California Agrees to Re-Write FYLSX Grading Description

Earlier this year I sued the State Bar of California for incorrectly stating that their “First Year Law Students Exam” consists of two sections — a multiple choice and a written section — that were graded equally.  In fact, the scaling methodology used to attempt to normalize the scores against previous sittings of the exam effectively made the written worth 37% more than the multiple-choice on the exam I sat for in June 2016.

Today I’ve reached a settlement agreement with the State Bar to re-write their grading policy to more accurately describe how the exam is actually graded.  The updated policy description will disclose to all preparing for the exam that:

Through a process known as “equating”, the multiple-choice raw scores will be converted to a scale with a theoretical maximum of 400 points. Because there are multiple forms of the examination, this process adjusts for the possible differences between forms and administrations of the examination in the average difficulty of the particular version of the examination that the applicant takes.
. . .
The raw essay scores will be converted, or “scaled” to a distribution that has the same mean and standard deviation of multiple-choice scale scores.

Although the process of normalizing to a mean and standard deviation is far from standard (math described here) and every mathematician I’ve spoken with has told me the accepted practice for grade scaling is to take the highest score earned, create a formula that makes it the highest score possible, and scale all other scores in line with that formula, the State Bar is free to use whatever formula they want so long as they don’t mislead exam takers.

Anyway, I’m pleased with the outcome and hope that future law students benefit from knowing that they better hit the sample essays much harder than the practice MCQ’s!  If you’re currently preparing, I also made a detailed post of study tips.

Passing the California First Year Law Students’ Exam, a/k/a The Baby Bar, FYLSX, FYLSE, Your Second Year Nightmare

[Edit 12/8/2017Welcome October 2017 FYLSX test takers!  It seems CalBar status pages have been updated.  If yours wasn’t, you probably didn’t pass.  It’s Friday — go out, have a drink or four, and start studying again in a week or two.  Tips are above, and, push that Follow button on the right side of the page!  I’m finishing my 4th year and regularly blog about issues relating to law school that you’ll want to read…]

Last week I got confirmation that I passed (indeed, annihilated) the October 2016 California First Year Law Students’ Exam!

The Exam

For those not in the know, this test, cutely dubbed the “Baby Bar,” is required for all California law students who are not in an ABA-accredited law school. As the ABA, being the grumpy old people that they are, refuses to accredit any online law schools, those like me and want to do law school while having a job (or traveling the world, raising a family, or whatever else makes you happy) are stuck taking the Baby Bar.  The exam is taken after you’ve finished your first year. It is offered every June and October, and you must pass within 3 sittings for the exam (e.g., I finished my first year in November 2015, and therefore was required to pass either June 2016, October 2016, or June 2017). If you don’t pass, your law school studies are over until you do, and when you eventually pass, you can receive law school credit only for your first year.

The exam is 100 multiple choice questions and 4 essay questions, allegedly each representing half your grade, at the same difficulty level as the real Bar Exam. The only difference is that the real Bar Exam tests 12 subjects instead of the 3 on the Baby Bar: criminal law, tort law, and contract law. The nice thing about this is that you can use actual past Bar Exam questions as prep for the Baby Bar. The bad thing about this is the pass rate for the Baby Bar is typically in the range of 20% – 25% — the odds are against you from the start. And, the cost to take the exam including all fees is $777, plus travel to Los Angeles or San Francisco if you’re not local.

My First Attempt

In the spring, I was getting ready for the test and taking practice exams, and from what I could see, my scores were in the passing range. Confident, in June I went to Oakland and sat for the Baby Bar exam, which felt pretty good upon taking it. The exam takes 45 days to grade, and in early August I got my results: failed by 1 percentage point.

I was disappointed and a bit surprised (although with a pass rate of 20% – 25%, not passing the first time is not exactly unexpected), since all the practice exams said that I was doing great on the multiple choice and ok on the essays. When I reviewed the written score report, the answer became clear: the Bar’s curving of the exam effectively made the essays count significantly more than the multiple choice.  My multiple choice received nearly a perfect score, but the bar examiners did not like my essays. This was frustrating since the State Bar specifically published that the sections would be given “equal weight:”

An applicant’s total score on the examination is the sum of an applicant’s converted scores on the multiple-choice and essay sections.  This step gives these sections equal weight in determining the total score.

The State Bar lies!

I tried appealing my grade to the State Bar since it would clearly have been passing had they weighted equally as they had promised, but they wouldn’t hear it.  So, it probably won’t surprise you, if you’re a long-time reader, that I’ll be filing a small lawsuit to get my exam fees back since the exam I was given was not the exam advertised. 🙂  But, I digress…

My Second Attempt

To fix my essay writing, I did two things:

First, during the previous sitting, I hand-wrote the exam rather than typed it.  The Bar charges an extra fee for the “privilege” of typing your exam, and I figured it was better to rely on pen-and-paper and avoid any technical issues that day.  But, quite simply, I can type more words per minute than I can hand-write, and on this exam, you need to be working as fast as possible.  I think if I had typed the June sitting, I probably would have passed (being 1% away from a passing score, pretty much any improvement at all would have made the difference!).  Seriously, do not hand-write this exam!

Second, I spent time going through several past exams, writing out the answers, and comparing them to the model answers that the State Bar published to figure out what I had missed.  This also gets you a feel for how the State Bar likes to see answers presented to them.  There’s a pattern, and it seems that following it may be a good idea. 🙂

The end result was that my multiple choice score stayed about the same, but each essay increased in score by, on average, almost 15 percentage points.  I could have actually written my 4th essay on “why it’s important to advertise the correct scoring methodology when you’re giving exams to law students” and taken a 0 and still passed.  (Actual essay questions for October 2016 FYLSX).

Study Tips

Here’s what I recommend for practice (these products are what I actually used and I receive no compensation for listing them):

1) PMBR Audio Lectures. PMBR is a company that, before they were bought by Kaplan, produced awesome bar review lectures. If it’s not on the Bar Exam (and thus not on the Baby Bar exam), it’s not covered, allowing you to focus on only the material that you need to know. Their criminal, tort, and contract lectures total about 14 hours and are by far the best use of 14 hours spent on reviewing substantive law. The easiest place to find their CDs is on eBay or Amazon.com (search “pmbr contracts audio,” or torts, criminal), as they don’t seem to offer the CDs as a stand-alone product anymore.

2) Fleming’s Baby Bar Review. Prof. Fleming runs one of the very few courses that targets the Baby Bar specifically. He has take-home materials, but for about $300 you can take a 3-day, in-person workshop where you’ll review substantive law, practice writing essays in exam-day conditions, and are given the opportunity to take home 6 practice essays which Fleming’s will grade for you, along with detailed feedback, for no additional charge.

3) AdaptiBar. Adaptibar licenses official Multistate Bar Exam questions from the National Committee of Bar Examiners (NCBE) and allows you to practice the multiple choice section on their Web site. The NCBE doesn’t actually write the questions for the Baby Bar, but they are substantially similar. Great for both getting an idea of how you’re doing and for learning what you need to study more of.

4) The Famed 1980 Exam. While the California Bar releases every Baby Bar’s essay questions and answers, they have only released one multiple choice exam ever — from 36 years ago! To be perfectly honest, this is because they are lazy and re-use questions from year to year, so if they released every exam, they’d have to re-write them. But, if you take the 1980 exam as practice and then take the next Baby Bar, it will astonish you how many questions they still re-use with only minor variations. You’d think for $777 per test taker they’d be able to afford to come up with fresh material.

5) Past Essay Exams. Take the questions, write them in an hour each, and compare your answers to the practice answers. They’re showing you exactly how they want it done. Do it that way.

…and finally, make sure you pay the laptop fee and type the exam, and if your typing is slow, learning how to type faster should be a part of your studies (you’ll need that as a lawyer anyway!).

How the California Bar Actually Grades the First Year Law Students’ Exam

fylsx-real-gradingI’ve made a few posts discussing California’s First Year Law Students’ Exam (the “FYLSX”), noting that I failed my first attempt by a fraction of a percent when the Bar applied a grading formula different from that which was advertised, and then, using the knowledge gained of their altered grading scheme, passed on my second try so spectacularly that the Bar published one of my essays as an example of how to write their exam.  I also briefly mentioned that I — as I’m known to do — filed suit to ask a judge to effectively require them to grade like they say they will and be more transparent about the exam and how they grade it.

The particular dispute was that they advertised that the multiple-choice section and the essay section of the exam would be “converted to the same 400-point scale” in order to “give[] these sections equal weight.”  When I received my score report from the first exam, I noticed that the scaling formula used by the bar resulted in it being impossible to score more than about 362 points even if every question was answered correctly, and likewise, it was possible to score far more than 400 points on the essays.

The attorney for the Bar assigned to the case disclosed additional documents, formerly considered to be secret and until now never disclosed to law students, to me yesterday that all but flatly admit that not only were the sections not weighted equally, but they also didn’t use a 400-point scale!  See if you can follow this tortured grading system they describe, because it took me several reads to figure it out:

Multiple-choice raw scores (i.e., number of items answered correctly) were equated to the June 1998, 2011, and October 2013 exams using 21 items that were common to each of these exams.  The equating formula was as follows:

Multiple Choice Scale = (3.4092 x raw multiple-choice score) + 21.6267

The candidates’ raw total essay scores were scaled to a score distribution that had the same mean and standard deviation as their multiple choice scores using the following formula:

Essay Scale = (2.3536 x raw essay total score) + -442.389

A candidate’s total score was the sum of that candidates’ multiple choice and essay scores.

What I gather from this is that 21 of the 100 multiple choice questions were repeats of previous years, and based on how well students did on those 21 questions, their grade on the entire 100 questions was curved.  Then, they calculated the average multiple choice score and the standard deviation, and curved the essay scores such that the average student gets the same score on that section as the multiple choice, and the score distribution was normalized to the same standard deviation.

From this it is clear:

  1. This has nothing to do with a 400-point scale per section.  The number 400 does not even appear in this internal document regarding their grading scheme.  The scale is created simply by comparing this group’s scores to previous scores and trying to curve it accordingly.  You’ll notice that if you plug in “100” as your raw multiple-choice score (a perfect 100 out of 100 questions) into the formula provided, the maximum score attainable is 362.5467.  Not 400.
  2. The sections do not have “equal weight.”  Saying that the average test taker got the same score on both their multiple-choice and their essay questions is not the same thing as saying “half your grade comes from multiple-choice, and half from the essays.”  Whether the essays count more or less than the multiple-choice section depends entirely on how well the other students do on their exams.  This is not equality, it is normalization.

So, for those taking the June 2017 exam, know this: on both exams I sat for, the essays counted far more than the multiple-choice.  You should therefore be spending far more time studying how to write a good essay than how to select A, B, C, or D correctly, unless your multiple-choice practice exams are turning out abysmally (for more study tips, see my previous post).  Hopefully by the October 2017 exam, a judge will have “persuaded” them to abandon this system.

California Bar Posts My Essay as Example of How to Write Exam Answer

fylsx_q2I was pretty excited to be tipped off by one of my law school professors that the California Bar’s “Selected Answers” for the last administration of the First Year Law Students’ Exam were posted and that one of them looked familiar.

Each sitting of the exam, every student writes 4 essays, and out of the thousands written they picked mine for “Selected Answer A” to Question 2 of the October 2016 exam.

Pretty cool considering I’m presently suing the bar over the way they arbitrarily weight the essay sections and multiple-choice sections of the exam differently, despite promising that they don’t, and then refused to provide documents when confronted with a public records request (original post, new amended complaint).  It’s also amazing how 85 out of 100 was apparently the best score on this exam question, further highlighting the absurdity of their grading.  Hopefully I’ll be able to bring some transparency to their grading process so that law students can get an accurate idea of what to study and how their knowledge will be measured.

Nonetheless, I’m glad they like my writing! 🙂

Blog at WordPress.com.

Up ↑