Please keep the forum protocol in mind when posting.

Judging Technology » Post: DCIX exam problems

DCIX exam problems

July 10, 2016 02:29:56 PM

Yuval Tzur
Judge (Level 2 (International Judge Program)), Regional Coordinator (Europe - East), TLC

Europe - East

DCIX exam problems

There are two problems I want to discuss.

The first one is a simple bug that could be easily fixed: mana symbols don't appear on in exams. Probably broken image path.

The second one is more conceptual and will require some work to fix (if we decide to fix it).
The problem is practice tests. I use the easy practice test to assess a candidate's basic level of knowledge as a first step of a candidate's way towards L1, and to measure progress. The problem with practice tests is that they include questions about the IPG, which L1 candidates are not required to know.
The only test (other than L1) that includes only the knowledge necessary for a L1 judge is an L1 practice test, but with its retest wait time, it's not a good tool to assess a candidate's progress.

Any ideas?

July 10, 2016 04:16:25 PM

Brian Schenck
Judge (Uncertified)

USA - Midatlantic

DCIX exam problems

Originally posted by Yuval Tzur:

The first one is a simple bug that could be easily fixed: mana symbols don't appear on in exams. Probably broken image path.

The developers have been advised of this problem, and I have been told that a fix is not immediately forthcoming. It is unfortunate, and I have no real answer to offer as to the why.

Originally posted by Yuval Tzur:

The second one is more conceptual and will require some work to fix (if we decide to fix it).
The problem is practice tests. I use the easy practice test to assess a candidate's basic level of knowledge as a first step of a candidate's way towards L1, and to measure progress. The problem with practice tests is that they include questions about the IPG, which L1 candidates are not required to know.
The only test (other than L1) that includes only the knowledge necessary for a L1 judge is an L1 practice test, but with its retest wait time, it's not a good tool to assess a candidate's progress.

Any ideas?

I would respond with a pair of conceptual questions…

(a) Why do you presume that a Level 1 candidate should have no familiarity at all with the MIPG? Even at a basic level?
(b) Why do you presume that the Easy Practice exam serves only the needs of Level 1 candidates?

The reason I recommend the Easy Practice exam for Level 1 candidates is because it contains content that most people will grasp or otherwise understand without a detailed knowledge of documents like the CompRules or Magic Tournament Rules. The exam serves as one path towards helping a person learn more about those documents and about both game rules and basic policies/procedures. In some cases, the question is simply enough that someone could look up the answer via the appropriate document or even in Gatherer. In other cases, they might suggest a document the person could look up.

Thus, the questions make rules or policies a little bit more accessible for someone who might not have a mentor. Or, if the person does have a mentor, something they could privately discuss. It's material that may not be overly complicated, or if a little challenging, contains a straight-forward enough explanation to get the person to the right document in a relatively easy manner. And we have a large number of people who use the Judge Center, many who are just interested in learning the rules or policies that might come up at various events.

I really can't go into a more detailed public conversation on the content featured in the exam pools, since that content is confidential. But I would stress that the composition of the Easy Practice exam pool is such that the majority of policy-related questions are geared towards Regular REL stuff and basic tournament procedures. The relatively few number of MIPG oriented questions are very basic in nature and comprise a small percentage of the pool. (EDIT: In some respects, they serve more to illustrate the difference between Regular REL and Competitive REL, or at least make people are that there is such a thing as different RELs.)

Outside of that, if there are specific concerns or questions about why a question is assigned to a particular exam… Aside from the “There is great care and deliberation about the exact nature of the question and where it is best suited” process that I go through every time, I am happy to explain a particular assignment if there is a question.

Edited Brian Schenck (July 10, 2016 04:24:04 PM)

July 10, 2016 05:06:04 PM

Yuval Tzur
Judge (Level 2 (International Judge Program)), Regional Coordinator (Europe - East), TLC

Europe - East

DCIX exam problems

One thing I want to clarify: I'm not against having the IPG as part of practice tests.
What I meant was that it might be a good idea to have a test that's designed for L1 candidates, but without the wait time (could be a new category).

Maybe it's a bad idea and I'm missing something. That's why I opened a discussion.

July 10, 2016 06:00:21 PM

Brian Schenck
Judge (Uncertified)

USA - Midatlantic

DCIX exam problems

Originally posted by Yuval Tzur:

One thing I want to clarify: I'm not against having the IPG as part of practice tests.
What I meant was that it might be a good idea to have a test that's designed for L1 candidates, but without the wait time (could be a new category).

Maybe it's a bad idea and I'm missing something. That's why I opened a discussion.

Well, I guess what I'm missing is: What is the problem with the resources L1 candidates and their mentors currently have in terms of Judge Center exams? Are the existing resources insufficient in some way? If there is a problem with prepping for the Level 1 exam (which doesn't appear to be the case based on the exams pass rate), is another exam the best solution?

July 11, 2016 01:04:29 AM

Yuval Tzur
Judge (Level 2 (International Judge Program)), Regional Coordinator (Europe - East), TLC

Europe - East

DCIX exam problems

I wouldn't say “insufficient”. It's more of a “nice to have”.