Ok, lets address some questions:
What are these undesireable behaviors mentioned in the blog post?
I realize that people really want to know what they are, but thats going to stay within the team. Exemplar is supposed to be highlighting the good in judges, coming out and going, “here's a list of all the crap were seeing” takes the focus away from that.

I'm upset about the loss of Proxy nominations. Im a remote judge and this makes it harder for me to get a nomination.
Only superficially. Others can still see you do great things and tell an RC, or an L2+. Issues with remote judges didnt change. Yes. It is harder to be seen doing stuff in your LGS when you are the only one there. But Exemplar doesnt stop at the walls of your store. We care about so much more, and there is so much more you can do online. We want you to interact and contribute to the larger Judge program.

Im upset about the loss of Proxy nominations. I work a lot with judges. I use all my recognition slots and still want to recognize more and used all my slots.
The number of slots given to the levels already account for this. You have approximately double of what is considered “normal” use. We expect L2s to use 2-3 nominations a wave. 5 slots is for when you are exceptionally active in a wave, so you have the slots if needed.
If you run out of slots, you can suggest to your RC that he/she nominate the judge. They have slots for that. And Im sure they would love to hear about it before 2 hours before the wave end deadline.

Why cant L1s nominate people?

So originally, and this predates me, L1s werent included for a several reasons.
-Exemplar was set up to push nominations down the level structure. L1s don't have a level below them to push down to.
-The quantity of L1s is such that even 1 slot each doubles/triples (and at the time quadrupled) the number of nominations.
-Creates a lot of potenial for abuse, as an L2 can certify a “circle of bros”. A lot of TOs are also L1s, and nominating your employees is a pretty clear conflict of interest.
-A brand new L1 doesn't have the experience or the scope to realize what is and isnt exceptional. Everything is new.
-It would actually encourage the creation of substandard L1s just for the ability to nominate people.

Now most of the large problems go away if you just say “no foils for those nominations”. I wasn't a fan of that solution because it systematically says that all recognitions from L1s have less value than nominations from others.

Now that we're introducing the more random component, that might be something to look into again after a few waves.

I have an idea on how you can fix Exemplar!

Great. Im genuinely interested in hearing new ideas. When Rob came to me with the idea of Tokens for everyone, it was a no-brainer. Deferred nominations was another idea. However, there is a little bit of fatique around improvements that boil down to: I want a guaranteed way to get foils.
For that, my answer is: go to a conference, and if one isn't nearby, offer to organize one.

Judges deserve to get paid!
Sure do. Talk to your TO about that.

I used the money from Exemplar packs to pay for travel to GPs because GPs dont pay enough
At a fundamental level, I see that as more of a problem with the current GP compensation model than with Exemplar.

So foils are nearly completely gone from Exemplar?
Not even remotely true. But exact numbers wont be public.

This will cause people to game the system more.
If the concern is that people who were farming nominations will try to farm even more nominations to get past the randomization…I can see that. But at the same time, the people inclined to do that were already doing that. Also, the more people they have to talk to, the more likely we are to catch them.

Id like to make comments anonymously because I am concerned about retaliation.
We dont really have a google form or anything like the JCC does. Mainly because I hadn't been asked before. Right now, I would say the best thing to do is just use a different email account to send an email to exemplarprogram@gmail.com.
However, I want to assure you, that as long as the comments aren't things like physical threats, I don't think you have to worry about retaliation. If you want to tell me of an abuse vector that you are aware of because you pulled it off 3 waves ago. Im not going to suspend you from Exemplar. I want to know about the holes so I can figure out a way to patch them.


“Every time I nominate someone, I recrudesce my chances to be randomly selected. Once people will nominate someone who got “lucky”, but get nothing themselves, those people will be less inclined to nominate their competition”. ”

The randomization setup we have doesn't support that conclusion.

Why wouldn't you just do the randomized removal of packets AFTER the first? If you get an exemplar, you should always get some foils.

Because when you say “Foils aren't guaranteed”, it doesnt make a lot of sense to then guarantee foils.

“we've got two review systems”

Exemplar nominations and reviews dont serve the same purpose and are very different animals. A real review, written by someone interested in your improvement and growth as a judge is an extremely valuable tool. Judges dont get detailed reviews very often because they are hard to write and take a lot of time and a commitment to the judge in question. Flash Feedback, is designed, really, to remove some of that daunting setup, and just get Judges to think about their fellow judges at an event, and to get reviews happening.



Are you permitted/willing to say whether the process is intended to remain largely the same wave to wave (i.e. restricted to updating/improving), or change deliberately from wave to wave?

So one of the interesting aspects of Exemplar is that any change you make, you cant tell if it had the desired result for 2-3 waves. I suspect this wave will be smaller than previous as more judges will abstain out of protest. Ok…does that become a trend? Is that a bad thing? Don't know.

I tend to like to look at results before determining if something needs to be changed.

So, short answer is: right now it will be largely the same, until it needs to deliberately change from wave to wave.

- If the process remains largely the same wave to wave, do you believe that it will be sufficiently complex yet not too specific so that zealous judges will be unable to “figure out the puzzle”?

Some will probably try. And if I say “no you can't”, that will just encourage a few more to try and do it, just because I said they can't.

- If the process intentionally changes from wave to wave, will the process for each wave be revealed after it closes?

No.

Also, does the amount of criticism/concern you are currently seeing from judges approach what was anticipated?

Yes and no. So Im going to be real human here for a moment. I knew that making this announcement people were going to be upset and I was going to need to take the brunt of it. That's my job, my responsibility. This announcement was about 2 weeks late because I kept finding other things to do instead of posting it.
However, I feel that this is the right call and best for the health of the project. Dealing with frustrated Judges isn't worth it otherwise.
Now, that said, Ive avoided a few FB threads of individuals who I know wont be happy regardless what happens, and WOW, what a difference.
The thing that surprised me though is the number of PMs ive gotten from people saying “I understand the changes, thank you for what you are doing.” I didnt expect those at all. That's meant a lot to me and has come from some unexpected places. Also, I've read some really cool theories about why these changes were made.

Originally posted by Bryan Prillaman:

“we've got two review systems”

Exemplar nominations and reviews dont serve the same purpose and are very different animals. A real review, written by someone interested in your improvement and growth as a judge is an extremely valuable tool. Judges dont get detailed reviews very often because they are hard to write and take a lot of time and a commitment to the judge in question. Flash Feedback, is designed, really, to remove some of that daunting setup, and just get Judges to think about their fellow judges at an event, and to get reviews happening.

This is only partially true.
Yes, writing exemplar nominations is easier than writing reviews, but a big reason (I believe the main reason) that exemplar is more widely used than reviews, is that they have a reward attached to them, which makes the effort of actually writing them more worthwhile. When you reach the tipping point where you feel the effort is greater than the reward, people will stop doing it.

Originally posted by Bryan Prillaman:

“Every time I nominate someone, I recrudesce my chances to be randomly selected. Once people will nominate someone who got “lucky”, but get nothing themselves, those people will be less inclined to nominate their competition”. ”

The randomization setup we have doesn't support that conclusion.

“Some people get and some people don't, randomly chosen” feels bad, no matter how you spin it.

Originally posted by Yuval Tzur:

Judges deserve to get paid!
Sure do. Talk to your TO about that.

Fair point.
In the past year, I've spent dozens of hours writing and reviewing articles, writing exercises for L2 mentoring, created tools for conference organization, mentored and certified judges, acted as AC, managed judge-player communication in my local community and more, yet my local TOs refuse to pay me for it. Can you please give some tips on how I can convince them to pay me more? Thanks.

In a less sarcastic tone, getting a pack of high value foils reinforces the feeling that the personal time I invested was worthwhile, so I was encouraged to do it again. I didn't do it for the foils, but they serve as a reinforcement to do it again. A lesser reward won't cause people to stop contributing to something they love, but a change in priorities is enough for things to be delayed or not get done at all.

On the other hand, the fuzzy feeling of getting a bunch of personalized tokens when someone else gets a pack of foils just doesn't cut it, and it's exactly the kind of thing that can push me over the line and saying “F*ck this sh*t” and stop wasting my time.

Finally, I feel much less excited about showing people all my custom-art zombie tokens than my playset of Phyrexian Elesh Norns.

Hi Yuval,

I believe your way of thinking is something WotC and high-level judges are sincerely afraid of. No one from this “inner circle” of judge management would want to open this discussion into public.

Historically, the Judge program has been a group of volunteers who had a genuine interest in making Magic communities and events as best as possible. The game grew through the years and it requires thousands of such volunteers worldwide.

WotC need to have this staff (!unofficial! so they are not directly connected as employers-employees) to keep the events associated with their game run under some standards. Judge foils are a great tool to keep the group of game ambassadors happy and working and inventing how the events can be better.

However, WotC is also pushed to hinder the employer-employee link even more. Why do you think there is that original wording in Exemplar Program's articulated mission: “Again, to be clear, not every recognition will equate to Wizards of the Coast sending foils.” That is just one security valve implemented by WotC lawyers.

The main issue is that, in time, foils will be less and less available to judges as a “compensation” for their work outside of events (the compensation for running events has already been moved to TOs only - which is a correct thing to have a healthy event environment). And unless something radical happens, this will also mean that some judges will just lose the “selling point”. For example, I personally love to judge and do all the stuff related to judge program. However, every time I do something in my free time for the program, I have to also evaluate that I am taking that time from my family. Honestly, my wife hates Magic and the only reason why she clenches her teeth and says nothing is that I bring some decent money to the family budget with it.

On the other hand, I believe that in long term such changes will purge the program. Either it will be destroyed or made better, cleaner, more honest and transparent.

Originally posted by Milan Majerčík:

I believe your way of thinking is something WotC and high-level judges are sincerely afraid of. No one from this “inner circle” of judge management would want to open this discussion into public.

I know. Everybody knows. That's why everybody resents these changes so much.
We all know what things are and what things aren't, and the way people in charge plug their ears and hope that if they chant the “exemplar's goal is the pat on the back, not foils” mantra long enough, we'll all believe it, is what makes people so angry.

Foils drive the exemplar program because we're a hard-working, under-compensated group. Cutting our pay while hiding behind the PC explanation of “give us your time and energy because we'll think you're awesome, and that's what really counts” encourages people to give up.

Originally posted by Yuval Tzur:

For most of my judges, one exemplar pack is worth more than a year's compensation.
As far as we're concerned, you just cut our lifeline.

From what you write it seems that it is normal for the judges of your region to nominate each other in order to receive foils? If so, I totally understand the changes that are being done.

Originally posted by Yuval Tzur:

In the past year, I've spent dozens of hours writing and reviewing articles, writing exercises for L2 mentoring, created tools for conference organization, mentored and certified judges, acted as AC, managed judge-player communication in my local community and more, yet my local TOs refuse to pay me for it. Can you please give some tips on how I can convince them to pay me more? Thanks.

Your local TO should never be required to pay for something you do out of your own compassion for the Judge community. If you don't want to do these things for the community, then don't do them.

I have got a fair share of Exemplar Nominations for doing community-related things, and I must say that the foils have been my driving factor for putting the extra energy in doing them. With a less guarantee to receive anything else than a “thank you”-token, I will probably be less eager to do the things I do. I will still do Judge-community things, but with less frequency than before.

Does this mean that you or I am invaluable to the community? Probably not. Someone else that is more than happy to do these things for nothing else than a thank you, will probably step up and do them. If not, we'll probably see a change back to how EP was in the waves prior to #12.

Thanks Bryan for your answers!

I have another question I hope you can answer: what was the feedback you've gotten on the not-guaranteed system that was used for the first few waves?

Anecdotally, I know of some judges who received recognitions but not foils in those early waves, and by both those judges and other judges who did receive both this was seen as extremely disappointing and unfair. It turned multiple people away from the exemplar program altogether.

Originally posted by Lars Harald Nordli:

Originally posted by Yuval Tzur:

For most of my judges, one exemplar pack is worth more than a year's compensation.
As far as we're concerned, you just cut our lifeline.

From what you write it seems that it is normal for the judges of your region to nominate each other in order to receive foils? If so, I totally understand the changes that are being done.

I think my point was misunderstood.
Just because the packs contain most of the value for judges in my country, doesn't mean they get nominated. People get nominated for their exemplary behavior.
Since we don't have much work in actual events, most judges here find other ways to be productive (writing articles, participate in projects for the community, translating documents and so on), which are the kind of things that usually awards you an exemplar nomination.

I wrote this as a direct response to the notion that exemplar is complementary to GPs, and wanted to show that our perception of the exemplar program is completely independent from GPs.
Reading my post out of that context distorts its meaning.

Originally posted by Lars Harald Nordli:

Originally posted by Yuval Tzur:

In the past year, I've spent dozens of hours writing and reviewing articles, writing exercises for L2 mentoring, created tools for conference organization, mentored and certified judges, acted as AC, managed judge-player communication in my local community and more, yet my local TOs refuse to pay me for it. Can you please give some tips on how I can convince them to pay me more? Thanks.

Your local TO should never be required to pay for something you do out of your own compassion for the Judge community.

It was a comment to point out that “Talk to your TO about that” is an absurd and irrelevant answer to me pointing that it's a source of income for judges who don't work many events.
I believed the sarcasm was evident by the opening of the paragraph that followed:

Originally posted by Yuval Tzur:

In a less sarcastic tone…

Evidently, I was wrong.

Originally posted by Lars Harald Nordli:

I have got a fair share of Exemplar Nominations for doing community-related things, and I must say that the foils have been my driving factor for putting the extra energy in doing them. With a less guarantee to receive anything else than a “thank you”-token, I will probably be less eager to do the things I do. I will still do Judge-community things, but with less frequency than before.

This is true for a lot of judges.

Originally posted by Yuval Tzur:

Originally posted by Lars Harald Nordli:

I have got a fair share of Exemplar Nominations for doing community-related things, and I must say that the foils have been my driving factor for putting the extra energy in doing them. With a less guarantee to receive anything else than a “thank you”-token, I will probably be less eager to do the things I do. I will still do Judge-community things, but with less frequency than before.

This is true for a lot of judges.

And it is not bad by itself. It is just a matter of supply and demand. There will be less foils, i.e. less compensation for judges who do things outside of tournaments. So less work will be done. And that is probably where judge management and WotC want to shape the things.

After all, we need to understand that the future of Competitive Magic is in digital Magic events. WotC/Hasbro management want to push into direction to compete with Hearthstone and other games. It is understandable - with digital Magic, you will not need so many rules enforcers, online coverage is vastly better, you can do changes to the Standard environment much easily. And look, even such things as cheating at high level event will be almost eradicated by digital Magic. No more situations as DQs at PTs and Worlds (see to the current weekend - another stupid DQ).

You will need just IT specialists and no judges to run such events.

I think that a lot of the variance in this thread comes from the fundamental difference:
“I judge because I love it and I don't care what I get in return”
“I judge because I love it and if the time isn't justified, I need to do other things”

I'm in the second category. Both of these types of people love judging. Conferences and GPs are some of my favourite parts of the year. This year I have organized multiple confs, attended as many others and created community games.

And frankly, this change is quite possibly going to make me stop. Not because I love organizing stuff any less, but because my income is not high enough. There, I've said it. I can't spend hours and hours on a project to get a token. I am doing all these things because they're fun and awesome, but without a reward, I will not. Make fun of me being dramatic or whatever, but thank you's can't be cashed out.

Is this an undesireable behaviour? Am I gaming the system, or working in a way that is unintended? Is my absence from the exemplar program or the judge program going to make this community better?

Please come forward with some reassurances. My actions, of going the extra mile, as well as others, directly or indirectly provides a gain for this company and this program. For some of us, whether the reward exists or not, makes all the difference.

Edited Alexander Papageorgiou (Dec. 3, 2017 01:17:40 PM)

> Bryan Prillaman
> “Every time I nominate someone, I recrudesce my chances to be randomly selected. Once people will nominate someone who got “lucky”, but get nothing themselves, those people will be less inclined to nominate their competition”. ”
>
> The randomization setup we have doesn't support that conclusion.

Tuval Tzur
> “Some people get and some people don't, randomly chosen” feels bad, no matter how you spin it.

We are discussing different things here. In one post, a judge made the statement that if he nominated other judges, that increases the likelihood of him not getting foils if he receives a nomination.
I want to assure judges that might believe that: no, that is not the case.

Originally posted by Alexander Papageorgiou:

I think that a lot of the variance in this thread comes from the fundamental difference:
“I judge because I love it and I don't care what I get in return”
“I judge because I love it and if the time isn't justified, I need to do other things”

I think that there are undertone of that, but I don't think these are the only point of views.

I personaly care very much about what I get in return. To speak colorpie-wise, I'm a black-aligned judge, and I'm proud of it.
However, I prefer randomly getting something than never getting something. So if there need to be a push to further separate exemplar nominations from foil, I'll support changes that are an alternative to the suppression of foils (which would quite radically create that separation, and, if there are too many push back for alternatives, will become the easiest solution to implement), even if I totally understand the feel bad that happen when random elements are introduced.

Also, these tokens are cool. I used to vainly display on my wall my RC letters listing my recognitions. Now I'll be able to play with them.

- Emilien

> On Dec 3, 2017, at 7:57 AM, Toby Hazes <forum-40079-14e7@apps.magicjudges.org> wrote:
>
> I have another question I hope you can answer: what was the feedback you've gotten on the not-guaranteed system that was used for the first few waves?
>
> Anecdotally, I know of some judges who received recognitions but not foils in those early waves, and by both those judges and other judges who did receive both this was seen as extremely disappointing and unfair. It turned multiple people away from the exemplar program altogether.

The first three waves were WoTC controlled.
Wave 1 was, to be blunt horribly mismanaged and miscommunication.
It was approximately 6 months late. Since this was supposed to be the first time foils were distributed after they left GPs, there was a lot of uncertainty if they would actually be sent at all.

When the mailings did go out, they weren’t just for Wave 1. It combined a proto-test run of a Wave 0, and the last of L4/L5 recognition foils. The different sources of nominations resulted in different cards being sent. (I think. This is what I’ve been able to reverse engineer, but there’s no documentation). However they were all advertised as Wave 1 Exemplar promos.
So you had people posting “I got a FoW and an Elesh Norn” and others posting “I got a swords to plowshare”
What made things even worse: Wave 1 nominations were published, but not the others. From the outside, it looked like someone with 1 (or zero!) visible nominations got a FoW/Elesh and someone with 7 nominations got a swords to plowshares and a greater good.
And the only explanation was a shrug emoji.
The Exemplar lead at the time put out a statement, but it wasn’t very helpful because WoTC wasn’t being communicative with what happened. So it was all guesswork from the Exemplar side.

Wave 2 was similarly late but the lack of communication was even more pronounce, the foil stuff was a big mess as well.

Wave 3 was when we started to get a handle on thing. There was a lot more communication between Hamer (then JM) and me, and we started fixing the major issues.

So it was a huge mess all over the place. Massive delays, no communication, unexplainable foil allocations, at a time that people were convinced that all this mayhem was an orchestrated attempt to remove foils. (I was even one of them)

Why is this different? Random before was awful. Why are you doing it again?

It’s different now because the allocation issues were amplified within the context of other problems.
Exemplar now doesn’t have the massive delays it had before. It’s very regular and scheduled and reliable. There’s much more communication and explanation of things as opposed to radio silence on even the most basic stuff. We communicate changes before they impact you as opposed to waiting until after they do.

For people that are concerned that foils are going away, I can easily point to the new Rules Lawyer Promo. It’s not part of their regular cycle, as hinted at by its collector number of 009/008. It’s an extra thing that WoTC wanted judges to have because it’s cool and it screams “Judge!”

Is the wave(s) with the Rules Lawyer promo going to be the ones in the new “Not every nominated judge will get them” system? Because an extra thing WOTC wants judges to have because it screams judges better than every previous judge reward, but not all judges, and not even all nominated judges will get one feels even more bad than losing out on the higher priced promos because the odds rolled against you.

> On Dec 3, 2017, at 2:46 PM, Daniel Ruffolo <forum-40079-14e7@apps.magicjudges.org> wrote:
>
> Is the wave(s) with the Rules Lawyer promo going to be the ones in the new “Not every nominated judge will get them” system? Because an extra thing WOTC wants judges to have because it screams judges better than every previous judge reward, but not all judges, and not even all nominated judges will get one feels even more bad than losing out on the higher priced promos because the odds rolled against you

Rules Lawyer will not be distributed in Exemplar packs.

It will be distributed as an additional card in conference packs starting soon, and another TBA method a bit later on.

Please check Judgeapps for information on attending a regional conference or mini-conference. You can also volunteer to organize a conference by contacting your RC.

-bryan