Originally posted by Isaac King:

That isn't correct at all. Those aren't 46 random people, those are 46 people who chose to respond to your survey. It's actually quite likely that the other 60% would have given different answers about some things had they been forced to respond.

Since you're the third one to mention this, I wonder if I indeed missed something, or was the article not clear enough?

From the entire population of uncertified judges on JudgeApps, I randomly chose 108 people. The only exception I did was making sure to include 4 from each region, so that might distort their world wide distribution, but it's still random (per region).
They were all contacted in the same way, given the same questioneer, and in the end 46 unrelated people replied.

If there's a sampling bias here please tell me where and what kind it is.

Perhaps you mean a nonresponse bias?
If you're suggesting that the reason why they chose to reply has something to do with the reason why they lapsed, please elaborate. While the idea is not impossible, it's not very likely either.
Moreover, if that's the case then we'll never know, as they didn't reply, and there's nothing really that can be done about it (I'm not about to force anyone to reply).

If that still doesn't sattle your mind, take from the article the reasons, and ignore the amounts.

Originally posted by Zohar Finkel:

If there's a sampling bias here please tell me where and what kind it is.

Perhaps you mean a nonresponse bias?

That's the one, yes. Your initial 108-person sample may have been randomly selected, but the group of people within that selection who responded was not. We don't know that the reason they didn't respond is related to why they lapsed, but we don't know that it isn't either, so we don't know if we can draw broad conclusions about the group “all lapsed judges” when we only have data from the group “lapsed judges who respond to surveys”.

This doesn't make the article any less useful, we just have to keep in mind that it may not tell the whole story–as you said, we'll never know.

I'm going to stir the pot a bit :)

The way I read it, the article focuses on the number of judges leaving the program, what are the reasons, and how can we solve the problem. The article is based on an inherent assumption that “judges leaving the program” is a problem, and that it needs to be resolved.

The way I see it, it's not a problem. This happens in other areas in life as well. A lot of people start something and halfway they realize they don't want to follow through. It's natural.
By giving everyone a chance, and letting “natural selection” to weed out people who's goals or needs don't coincide with the program, we increase the chances of a few to find their place in the program and become long-term contributors.
The article also fails to take into account judges who lapse and return (there aren't many of those, but I am one, so I have a soft spot in that regard :) ).

I have a long history with Zohar (the article's author), and his stance was always to avoid judges lapsing, even at the cost of not certifying them in the first place.
As far as the article (and Zohar's opinion) goes, a judge who lapses is a failure. I disagree.

Originally posted by Yuval Tzur:

A lot of people start something and halfway they realize they don't want to follow through. It's natural.

There's a distinction to be made here:
Let's take “John” and “Jake”, both former judges who lapsed after only one year.
John didn't like pushing chairs in the store, and found out he prefers to play with his friends. The store owner told him that it's fine, he can just be a back up for when their other judge can't make it, but that time never comes, and John who doesn't care about maintenance requirements at that point simply lapses.
Jake on the other hand, had to move a few weeks after certification because of his new job. There are two magic stores in the new town, both who already have their own judges. Jake wants to judge, but in the few times the tournaments don't overlap with his working hours, he's not needed. He too fails to meet his maintenance requirements and lapses.
So while most people assume that we're dealing with a John, maybe we're dealing with a Jake, who actually wants to follow through, but find it hard to do so.
Some of the reasons given in the article describe people like that, those who are forced to lapse because of circumstances beyond their control, and that's where I hope that by knowing those reasons, the program might be able to find ways and retain them longer.

Originally posted by Yuval Tzur:

{Zohar's} stance was always to avoid judges lapsing, even at the cost of not certifying them in the first place … a judge who lapses is a failure.

Not entirely accurate.
I've been told before that people losing interest and dropping fast is not a problem, and I realize I'm in the minority who still thinks it is. I agree there's no real harm in certifying anyone who qualifies and letting “natural selection” run it's course, but it's a waste of time and effort that people can do without.
I also don't think I ever denied someone certification for that reason.
My bigger concern is that the open gates sometimes let the wrong kind of people in, but luckly there are safeguards to deal with those as well.
Anyhow, if the program already thought it through and decided it's not a problem, then that's fine.

Originally posted by Yuval Tzur:

The article also fails to take into account judges who lapse and return

Yes it does, since going by the available data I have, those people are impossible to identify.

Originally posted by Zohar Finkel:

So while most people assume that we're dealing with a John, maybe we're dealing with a Jake, who actually wants to follow through, but find it hard to do so.

That's what RCs and ACs are for.

There will always be people who have personal difficulties due to lack of judging opportunities or timing restrictions, and those people should contact their program representatives (i.e. the RC or AC), and these representatives can handle these issues on a case-by-case basis. People who wish to work are usually vocal about it.

Judges lapsing en-mass isn't a problem in my opinion. Only judges who lapse because the system failed them (and they are, AFAIK, a small minority).

Originally posted by Zohar Finkel:

I also don't think I ever denied someone certification for that reason.

You told me several times that you believe we shouldn't certify new judges because our area doesn't have enough work to allow everybody to maintain their judge level. I might be wrong, but my understanding is that the possibility of lapsing due lack of judging opportunities was the reason not to certify them (and several judges who were refused certification for that reason, proved themselves and are either good L1s or L2s in our area).

Originally posted by Yuval Tzur:

You told me several times that you believe we shouldn't certify new judges because our area doesn't have enough work

1. That's a different reason. I was talking about people I think will lose interest and drop quickly.
2. I still believe what I told you, but again, never refused anyone for that reason (that is if I believed he or she would make a good judge regardless).

Anyway thanks for bringing this up since I believe it's important for people who read an article to know what the author believes are, so that they can be more critical about the content of the article, but I get the feeling we're starting to digress here a little too much.

Edited Zohar Finkel (April 16, 2018 08:35:35 AM)

Originally posted by Zohar Finkel:

I get the feeling we're starting to digress here a little too much.

Agreed.

My point is that we shouldn't look for a solution for people lapsing. Not because we don't need a solution, but because it isn't a real problem.
The rate of decay, as you call it, in the judge program resembles the decay rate of people involvement in other areas of life, such as job retention or higher education (this is based on personal experience, and not on proven numbers).

Sure, some people fall through the cracks, and feel forced to leave the program even though they truly wish to continue judging, and we should improve in finding those people and help them, but as whole, I don't see lapsing as some kind of epidemic, or a problem that is plaguing the judge program any more than other communities in which people's involvement is voluntary.

I wouldn't worry too much about the specifics of whether lapsing is a good or bad thing. This sort of conversation is mostly useful in helping us better develop the acquisition, development and retention of those that want to stay within the program and help us work out how to better filter candidates at all levels to ensure we are suitably identifying talent and finding other avenues for those that feel they want to be part of something but maybe the Judge Program isn't the best fit.

This article and associated research is a great first step in that larger piece of work.

ad “we aren't good at retention”:
That depends on how long we expect/hope people to stay certified. For example in our company about 25% of our employees quit per year. Is this a problem? On the first glance yes. But we are a consulting company and in this business people stay on average for about 2,5 years. With us they stay on average 4 years and we are *extremely* happy that they stay so long.

So with judging what can we reasonably expect? How long would we like them to stay active or certified?
For me personally it's one and a half years (on average) for L1s, three years for L2s and five for L3s, but this might vary a lot.

Of course we should keep an eye on the most common issues for people quitting and try to prevent them, but as long as we are above the values mentioned we should be fine.

Christian

Originally posted by Yuval Tzur:

That's what RCs and ACs are for.

There will always be people who have personal difficulties due to lack of judging opportunities or timing restrictions, and those people should contact their program representatives (i.e. the RC or AC), and these representatives can handle these issues on a case-by-case basis. People who wish to work are usually vocal about it.

Judges lapsing en-mass isn't a problem in my opinion. Only judges who lapse because the system failed them (and they are, AFAIK, a small minority).

Forgive me for reviving this after a week, but this assertion piqued my thoughts and I wanted to speak to it. While I agree that we don't necessarily have to see all lapsing as bad, I don't agree that the sheer magnitude of lapsing isn't indicative of a systemic problem; the assertion you've made here that we have systems in place to prevent people from falling through the cracks already presupposes both that those systems are not only working as intended but that they have already solved the problems faced by people who want to be engaged but find it difficult to do so.

I can guarantee you personally on several accounts that simply being vocal about a desire to be engaged does not equate to retention and access. The system is wholly dependent on the people above you being willing to pull you upward, which can be indicative of both a problem of sustainability/retention AND aggressive acquisition.

You admit that judges who lapse because the system failed them are a problem, but if that's true, be careful about quickly dismissing information that can suggest where those problems may lie. People's lives are complex, and data won't necessarily cure all ills, but it can make suggestions to put us on the right path.

Edited Rebecca Lawrence (April 25, 2018 02:28:07 PM)