Gauging the level of question difficulty (FRM L2)

[[email protected]]

Hello,

I took FRM Level 1 last year and I recall very vividly the difference between expected and actual level of question difficulty. I did a course with a specialised provider, their sample questions were easy, basically around CFA Level 1, so, having passed CFA Level 1, I felt quite confident. Then I had a look at GARP's (awfully compiled) textbooks and my blood froze - now that was seriously difficult stuff. I was a subscriber to BT at that time, and then, just like today, I found BT probably most out of touch with curriculum (or reality...) - I mean, what's the point of even publishing "this won't be tested in the exam" questions? (that's one), but two, many of the questions you write, probably with best intentions, are absurdly difficult and non-computable in exam conditions (you would probably like me to get specific here - let's get specific: Malz p23 q4 - am I expected to know the B-S formula by heart in the exam?; Malz p36 q5 - seriously? that is what GARP can ask me in the exam? what does part of the formula mean?; Malz p46 q5 - it requires calculating CFD (N), which, as David explained in his own post dated Jan 29 2012, permitted TA calculators cannot compute; pls note i referred to this one reading simply because it's one I have at hand).

This isn't to trash BT btw, I'm just thinking out loud here - on the other side of the distribution so to speak we have widely popular Schweser materials, whose questions are at most times ridiculously simple, e.g.

"With which of the following structured products would the investor most likely receive cash flows based on the performance of the underlying assets?
A) Covered bonds
B) Mortgage pass-through securities (MBSs)
C) Collateralized mortgage obligations (CMOs)
D) Structured credit products

With regard to a three-tiered waterfall securitization, how would the excess spread most likely be distributed to the equity tranche?
A) A maximum amount based on an overcollateralization trigger.
B) Any residual amount after the accumulation trust is satisfied.
C) A coupon amount based on Libor, usually less than the mezzanine tranche.
D) A coupon amount based on Libor, usually less than the senior tranche."

I recall the actual L1 exam questions were quite difficult.

The limited number of available practice questions doesn't make this any easier - there are only a handful of past exams around, and that's that.

Suzanne, David, and fellow BT'ers - could you share you thoughts on that?

Kind regards,


Wojtek

 

[ShaktiRathore]

Hi,
I would add my opinions to your question whats the point of even publishing Qs may be BT would like add extra knowledge and concepts that would led you help better understand other exam related concepts and topics. It might not be seen obvious at first sight but yes it does help reinforce our understanding as a whole.
Regarding difficult and consumable Qs logic goes as that if you have mastered this then it would not be difficult for you to master the lesser difficult and less time consuming ones that appears in the exams and at the same time better grasp the concepts. It really helps i tell you from my experience after giving the frm. Its very easy to go from difficult to simple Qs rather than the other way around.
Yes if you know cfd() and such stuff its great whats the harm and BTW your preparation for the exam will not get harmed it will just be augmented in a rather unusual way.
thanks

 

[David Harper CFA FRM]

Hi Wojtek,

I love the example question you selected (I put many hours into the questions, and I'm especially proud of the one you cite).

First, I write the in-sequence practice questions with a dual intent to (1) address the AIMs and (2) support deeper comprehension on the idea this is preferable to (better help than) trying to anticipate an exam which is highly unpredictable, especially for the newest material (it is key to distinguish between seasoned material from unseasoned material like Malz that you cite). While the global Topic Review questions are more titled toward actual exams (simulation), the sequential AIM-by-AIM questions (like the one you cite) are organized by following the syllabus AIMs.

Second, here are the AIMs that I was writing for (in sequence; this question comes from T6.308, see http://forum.bionicturtle.com/threa...swap-cds-spread-curves-malz-section-7-3.6944/),

• "Construct a hazard rate curve from a CDS spread curve"
• "Construct a default distribution curve from a hazard rate curve."

These are two examples, among the new 2013 (in this case the new Malz and Gregory), several of which are "reckless." ... I maybe need a more politic adjective in my feedback to GARP, maybe I will want to characterize them as "unrealistic" or "untestable" or "puzzling" or "not actionable from an exam viewpoint" ... you get my point? Knowing that actual construction is a time-consuming skillset and, on the exam, the FRM candidate won't be asked to "construct a hazard rate curve," what is an FRM candidate, or a training provider, for that matter, to do? (btw, the Gregory chapters contains even more examples of AIMs which are "unrealistic as literally written")

What I decided to do is write a question that, I hoped, would help a candidate comprehend the most salient principle in the section; i.e., as I wrote in the answer "The equation solves for a hazard rate that equalizes, on the left-hand side, the payments made by the protection buyer with, on the right-hand side, the contingent payoff made by the protection seller."

The question has zero relation to an actual exam question. In fact, as it's hard to imagine what could be asked, on the exam, to query these AIMs. In the face of uncertainty (including game theory uncertainty: my annual feedback to GARP will very directly seek to alter/delete these unactionable AIMs), I decide to write a question that I like because I think it's a nice way of supporting a review of the chapter. Also, btw, why did i take even more time to write an EXCEPT-for question? so I could basically articulate each of the component meanings of the formula to the reader. This formula resembles Hull's method to price the CDS, so the concept it tries to reinforce (""The equation solves for a hazard rate that equalizes, on the left-hand side, the payments made by the protection buyer with, on the right-hand side, the contingent payoff made by the protection seller.") is, in my opinion, actually more durable than the flimsy AIMs, which do not deserve to survive. I find your criticism ironic because I actually spent a lot of time on this one question, and finally came up with something that utilized the reading contents to indirectly support a principle which is actually testable (if you missed it, again, it's that a CDS is priced by equating the payment to the contingent payoff leg; GARP has tested this!).

In summary, the question does not simulate an exam question. Lord only knows what an actual exam questions against the new Malz and Counterparty will look like. Given that, the question hopes to help with comprehension of underlying principle.

 

[David Harper CFA FRM]

... in regard to the broader reason for difficult practice questions, I agree with ShaktiRathore who basically articulates my philosophy behind the questions. My primary motivation (aside from the Topic Reviews, which do have more of an exam simulation tilt) is to help somebody who has just read the chapter, for example, grasp the concepts with hands-on exercises. Most of the questions are more like end-of-chapter questions, than attempts to simulate/predict an actual exam question. Thanks,