Review: ExamGuru Shelf Exam Question Bank

Updated 5/16/2016 to reflect new prices, new discount code, and some additional changes. Updated since then to reflect new discount codes. 

ExamGuru is the brand new and currently only question bank geared specifically for the third year NBME shelf exams. While the product itself is new, the company is not: it’s a new brand of the COMQUEST family, one of the two big players in the osteopathic question bank market (for the COMLEX exams, which are analogous to the USMLEs). It was released to the general public this week, but I had early access in order to write this review. I was also able to secure a discount for readers, so if you sign up using the code BWBW2022 , you’ll get 15% off whatever package you get, and I’ll get a few bucks.

Before we get started with the actual review, full disclosure: I wrote a small number of questions for this question bank as a freelancer several years ago. These were sold on a per-question basis; they are no longer my intellectual property, and I have no financial stake in the company or its success outside of the time-limited coupon above.

Size and Cost

The ExamGuru question bank is divided into separate shelf exam products, each with a goodly number of questions (as of the time of this post): family medicine (375), internal medicine (412), ob/gyn (369), pediatrics (406), psychiatry (395), and surgery (399). Each question comes with the detailed explanations we’ve come to expect from medical school question banks: 1) Concept/question explanation 2) Detailed answer choice explanations, including explanations of the incorrect options, and 3) Take home point.

You can buy a subscription for a single shelf product at a time: 1 month for $49 $37 and 3 months for $99 $79. Alternatively, you can buy all six products for a length of time ($129 for 1 month up to $379 for the year). Given that buying a month of each product would add up to almost $300, it would seem that the product is priced to encourage you to either buy a few products for a month each or just shell out for the whole year, which would allow you to the use the bank both as a shelf study resource and as an alternative/secondary Step 2 CK qbank.

Software

The website imitates the FRED software you’re intimately familiar with (and also has an option to change the layout to the one used on Osteopathic examinations for DO students). Everything is accessed via the website itself (no downloads or creepy UW spying/tracking), and the site is responsive: it works appropriately on your computer or your smartphone.

Peer percentage correct for each question is provided a la other competitors. Questions are also rated by difficulty, though I’m not sure how this was calculated; oddly, it seemed like most of questions I did were graded as “hard” in their software. I wasn’t sure if this was simply chance or reflects how they’ve self-graded the difficulty and the relative proportions of each within the qbank. Additionally, EG does provide good actionable data about your performance, including a breakdown by question task (establishing a diagnosis, management, etc), which may be a nice way to pick apart areas of weakness you didn’t know you had. If you’re getting the diagnosis questions but missing the management ones, then presumably it’s time to focus on the “next best step” and drugs of choice.

Question Quality

The question quality is good for its first iteration, but it’s not yet at the UW level of polish. This isn’t surprising: I remember using USMLERx back when it was a newer product, and it was awful and a total waste of time and money. The EG house style in particular a bit spotty and could use more homogenization: Multiple question writers and their particular quirks remain surfaced, and occasional aspects, particularly when it comes to the final stem and answer choices, sometimes stray a bit from what you’d actually see on game day. Buzzwords are over-employed and are even sometimes “in quotation marks” whereas nowadays these terms are more likely to described rather than simply called out. Explanations range from relatively short to long & fluffy, sometimes casual “Don’t forget XYZ on test day” and sometimes stiff.

Topics and narratives are fine overall, but some of the questions slipped through without matching the official question writing guide (which I’ve discussed before). A random example: a question about cirrhosis with blatant over the top SBP contained unbalanced answer choices (1 antibiotic choice versus multiple diuretics). That’s probably too easy and not reflective of the standards. I’d argue the question should have been a bunch of antibiotics asking you to know which type is used to treat SBP, or an even combination of both. One answer choice that stands out from the list is to be avoided. On the flip side, the family medicine section has some really great rash/skin questions, which are high yield and not well represented elsewhere.

EG also still needs a copy editor. Shelf questions are often long but almost never because of fluffy prose (only extraneous details!), and comma errors remain (inappropriate comma use before a coordinating conjunction used in a phrase will always be my pet peeve). Again, not necessarily substantive criticism but certainly one that signifies a lack of polish in its first iteration. The material is there, but these little differences do detract a little from question experience, which otherwise is well designed to approximate the real deal. As above, the software is solid.

Update: The EG CEO informs me that they brought on a copy-editor to deal the issues I raised in this review. He also tells me they’ve updated a lot of questions from user feedback. I haven’t personally taken a look again, but if nothing else, most question-banks generally get better over time, not worse.

On the whole, these are mostly nitpicks. But to me, the level of polish of a product is really important if you’re going to spend a lot of time with it. Errors and inconsistently can detract from the experience and distract from your education. 1If you harp on that kind of stuff like I do. That said, ExamGuru is probably one of the best things to come around for the Shelf exams for a while and breathes some new life into the static review lineup. The mistakes I found during my review were nearly all ones of style, consistency, and grammar. These are the things that are easiest to fix gradually. The core content I saw was just fine.

So my overall impression is that this a supplemental product, not a UWorld replacement. While the important topics are covered and the explanations are generally thorough (sometimes a bit lengthy, I’d argue), the overall quality is not yet up to the consistent quality of UW. The main benefit of EG is that it adds some meat to the UW bones, which are nearly ideal for Step 2 CK but a bit thin for several shelf exams. UW is still I think a critical component of shelf review, but there’s definitely space for another question source. And on the whole, I think ExamGuru is a better question source than the usual alternatives (e.g. PreTest) in terms of depth, ease of use, powers of software, etc. I’d much rather do dedicated shelf questions in a simulated USMLE environment on my computer or phone (yes, EG is mobile-enabled) than thumbing through a book or shelling out for another Step 2 qbank.

Conclusion

So should someone use or buy this product? Depends. If I were a third year again, I would for family (maybe peds and ob/gyn as well). Certainly not internal medicine, UW had plenty of that for me to chug through. Ultimately, as readers of this site are well aware, I believe strongly in doing questions, even as a core study method. UW just doesn’t have a very satisfying number of questions for several of the shelf exams (it’s well-pruned for Step 2 CK). Financially, it either makes sense to buy 1-3 of the shelves for the one-month period to use during a dedicated review push or, if you want more, just get the whole set for the year (ouch). I don’t think the quality or consistency is at the UW level but it’s a tailored source of questions in a friendly NBME style package that you can use on your computer or on your phone. And that’s a great start.

If you do end up using the EG product, shoot me an email or comment and let me know how it holds up to thorough use.