Monday, March 17, 2008

A Touchy Subject: Discrepancy Judging

Happy St. Patrick’s Day! I hope you’re wearing green today, or I may have to pinch you. :)

I wanted to talk about something today that may be a touchy subject. Discrepancy judging in contests. And the reason I’ve been thinking about it is that I served as a discrepancy judge a while back.

I know that different contests do it differently. Usually, if there is too large a variation in scores, then they bring in a discrepancy judge. Some contests look at the new score and throw out the lowest one. Others will average the scores. (Those of you who have coordinated contests can tell us more about the specifics.)

What I wanted to talk about today is what I learned from the experience. First, let me say that as an entrant, I've have had a discrepancy judge a few times. So I’m very appreciative of the opportunity! But as a judge, it was a hard, hard job--a lot harder than regular judging for me. What I found was that every single time, the scores I gave were closer to the lower scoring judge. And I was really surprised by that, because I tend to be very generous with my scores.

So what I decided (and this is totally my opinion, so take it for what it’s worth :)) is that the reason the coordinators ended up having to call in another judge is because one of the judges was too afraid to be totally honest about the manuscript (and was maybe trying to be nice and encouraging). I don’t think it was just a difference in taste. I think the high scores may have been inflated. Yep, I'm saying this even though whenever I've had a discrpancy judge on my entries, I wanted to say it was the low judge that was off. :)

Now I know there are cases where a judge gets bent out of shape because an entry may hit her hot button. I had a judge once that I swore must have hated men! My hero could do nothing right. And I was really upset when I got the results of that contest, because she kept me from finaling. But now that I’ve seen similar situations from the other side, it makes me want to go back and dig up that entry to look at it again. Maybe the lower-scoring judge was being more honest and helpful than I thought at the time. (My experience as a contestant was also that the discrepancy judge scores ended up closer to the lower ones--which didn't help my finaling status at all but seems to be proving my point.)

It’s hard to set aside the ego and to look at an entry objectively. That mean ol’ judge didn’t like my darling manuscript. She rejected my baby! (Yes, I've had similar thoughts.) But that’s what we have to learn to do. An editor will look at our manuscript very objectively with readers and the marketing department in mind. We have to learn to do the same.

So I'd like to make a suggestion. If you’ve had a discrepancy judge lately, then set your bruised feelings aside and look at the low-scoring judge’s comments. If she’s a man-hater and is spouting derogatory comments, well, okay. Just ignore it and move on. But if the scoring is halfway rational, then truly consider what the judge has to say. It may be that the judge who gave the nearly perfect score didn’t take the time (or didn’t have the guts) to give some constructive feedback.

Take that feedback and be honest with yourself. Is there something there you could work on? Is there some nugget of truth that could help take your writing to the next level? (And I’m talking to myself here, too. Readers and booksellers will be giving feedback in published contests soon.)

We all have room to grow. And that low-scoring judge may be the key to finding a gem of advice.


So, what do y'all think? What has been your experience as a judge or as a contestant?

Missy

36 comments :

  1. Wow i learnt something new again today about discrepancy judging. thanks for the post Missy. i do enjoy learning what Authors go through in contests.

    oh im not wearing green but if you want to come dump a bucket of cold icy water over me for punishment please do it quick! the change is coming and after 2 weeks of heatwave conditions it will be cooler! like under 80 and i cant wait. But my rooms like an oven so the ice water is welcome

    ReplyDelete
  2. Yeee-ouch, but I'd have to agree. I've only entered three contests, and as much as I love the kind comments, it seems like the "mean" judges usually have very valid points. It hurts when a judge doesn't say even one nice thing, but it doesn't mean that the harsher things aren't worth looking at. I've learned that they definitely are.

    ReplyDelete
  3. Missy, I'm Protastant not Catholic so I'm wearing orange today instead of green. Well, actually I'm wearing yesterday's clothes. Yesterday was Sunday, the faux-Sabbath for the Christian church, and Sunday for me is black and white with a occasional interjection of beige or pink. All right. I'm wearing workout clothes, because I walked to the car to take my son to school. Whew, I'm tired. I blame Janet for all her multi-tasking talk last week.

    (BTW, Janet, I did do my quiet time this morning.)

    Anyhoo...

    I was a discrepancy judge last year for the inspy category of a RWA contest. The coordinator of the inspy category was a friend and desperately needed a inspy-reading judge, so she asked moi. Call me a pack rat, but I think I still have copies of those entries on my hard-drive.

    Since I didn't know what the first round judges gave the entries, I didn't think too much about what I scored. If I had, I'd have tried to make my scores closer to the high-scoring judge because I'm a sickenly nice person (sometimes). Okay, all the time. Sheesh, I was trying for humility there.

    I don't remember much about those entries, except that one had a heroine way too snarky for me. I can tolerate a level of snarkiness in a non-Christian, but in a born-again Believer in Christ, I really struggle.

    Don't get me wrong. I certainly don't think once you get saved, you never say anything snarky again. If it were true, I'd seriously question my salavation. But in this entry, the heroine was a Believer. Yet she was down-right nasty.

    I really don't know if either of my discrepancy entries finaled. But I have to say I really like having a discrepancy judge OR having a minimum of three first round judges.

    In FHL's TBL last year, one of the Long Contemps had a 40-something point spread. Since the TBL didn't do discrepancy judging, the entry didn't final. I read the entry and the scoresheets (there's always a bonus to being a coordinator). With this particular entry, if I'd been the discrepancy judge, the entry would have finaled because the low-scoring judge was WAY OFF in some of her comments.

    On a side note....

    Not all contests can do judges' training. Sometimes a contest is soooooo desperate for judges, it'll take even untrained ones (not that I have a problem with that because every judge has to have a contest be her first). That's why it's VITALLY important for contests to include a judge's orientation letter with an explanation of the scoresheet questions. The Genesis had one that really helped me this year when judging, and I'm an RWA-trained judge.

    Anyhoo, anyhoo....

    Missy, great post! Discrepancy judging isn't one of those hot topics so I appreciate you bringing it into the light.

    ReplyDelete
  4. I'm with Ausjenny - I've learned something new! I think it's great that there is a discrepancy judge default in contests!

    The group we homeschool with has an academic fair where I often serve as a judge for the Creative Writing entries. It's not exactly the same, obviously, but done in the same spirit. Great idea!

    Kim

    ReplyDelete
  5. I got discrepancy judged two to my knowledge.

    One in the Golden Heart when someone gave Petticoat Ranch a ONE!
    ONE! A ONE! For the love of ...

    Well, anyway, I'm all better now. Completely over that stinking ONE!!!

    I ended up finishing in the top 25% which is the best I could ever manage as far as bragging rights to the GH.

    The other discrepancy judge!!!!

    RUTHY!!!!!!!

    You can come out now Irish Girl.

    She just bombed me so bad on my lovely Christian romantic suspense Ten Plagues.

    About a serial killer.

    Body floating in a blood filled fountain...naked...mutilated.

    Ruthy, the big brat, didn't think it'd sell. Not quite right for the Christian fiction reader, she said, a little edgy. (well it hasn't sold that that's not the point)

    So I got a 98, 99 and a ... what was it? 56? From Ruthy.

    The point is, I WON. Discrepancy judges over ruled her. I also met Ruthy when she emailed to 'splain herself. So I won twice. :)

    And I'm now prepared to remain calm and let the body floating the in the fountain of blood (I had to figure that out you know, not that easy to figure out where to find a fountain's worth of blood--ghastly research) have clothes on.

    Much more acceptable. Maybe

    Hey, I was unpublished a long time and I've written everything.

    ReplyDelete
  6. Excellent points about discrepancy judges, Missy. I haven't been in that position for a few years so I can't be specific about what scores I was given. But as long as I get feedback so I understand the score, I've paid attention to what s/he had to say. Maybe I've been blessed or maybe I just had sooo much to learn, but I've always gotten something of value from low scoring judges.

    My gripe is when a contest factors in all the scores. Even if the discrepancy judge scored the entry high, it can't final if the lower score isn't dropped.

    Gina, Proud of you. I got my quiet time in, too, and I really needed it this morning. What a blessing to talk to God.

    Mary, I see Ruthy's point. Not that I'm taking sides. :-) Clothed bodies floating in blood-tinted water with minimal mutilation is probably more apt to fly in the CBA. I'm fascinated by your diversity.

    Janet

    ReplyDelete
  7. Low scores hurt, no matter where they come from. I am just getting started at contests, so nobody is asking me to judge anything. I hope they never do. The only thing harder for me than receiving a negative critique is giving one.

    ReplyDelete
  8. Katherine, I highly recommend judging because I firmly believe you learn more about the craft of writing from evaluating another person's writing than you do from evaluating another person's view of your writing.

    Whew! That's a long sentence.

    Now why could a dead bloody (as in literal not in the profane use) be acceptable in the CBA yet a dead naked bloody body couldn't? How titilatting can a dead naked bloody body be?

    I've read a CBA fiction or two where a hero/ine was bathing. The reader clearly knows the character was naked. Or what about when the hero takes off his shirt? Seems to me both those situations are more sexually-focused than a dead naked bloody body.

    Sometimes I just don't understand the CBA market standards.

    ReplyDelete
  9. Jenny, I'm jelaous of the hot weather! I'm so ready for it. :)

    Jessica, you made a really good point. There needs to be something positive in a critique! I think that's VERY important. I try to use the sandwich method--something positive, then the negative (constructive feedback), then then the positive again. Or I at least put something positive after each negative to end on a positive note. There's nothing worse than totally negative feedback

    Gina, I agree that all judges should be trained! Or at least have some explanation about what's expected. I took the RWA training and learned a lot.

    Also, I'm Protestant too, but have Irish ancestors on all sides so wear green and celebrate. :))

    Kim, that's cool that you get to serve as a judge for the homeschoolers! I think kids who love to write can be encouraged by the feedback.

    Mary, Mary. Dead, "nekkid" (that's southern speak), mutilated bodies?? I'm glad Ruthy set you straight. But Ruthy, your score got booted! Next time don't make it so extreme! LOL

    Janet, I remember being frustrated by the same thing. My scores all got averaged so even with a discrepancy judge I couldn't final. Of course, like I said, the extra judge sided with the low score.

    Katerine, Gina's right. You can learn sooo much from judging because you can be more objective than you can be with your own work. But I so agree about how hard it is to give critical feedback! I spent forever on a Genesis entry yesterday because it was really good in that it's creative and unique. But it had a couple of pretty major problems. I wanted to help but not discourage, so it was really difficult. It's a tough balance.

    Gina, I had to take out a shower scene in my book. I was just using it as a time of reflection and to show that she was primping for the hero. But I guess it was too provocative. So she ended up doing her reflecting with wet hair after the shower while snuggly dressed in sweats. :)

    I'll check in later. We have Holy Week services at lunchtime each day this week.

    Missy

    ReplyDelete
  10. Another thought after reflecting on what Mary said...

    Do you think that sometimes the high-scoring judges give the higher scores despite the problems because they see the great writing talent and know good story-telling when they see it? The great story helps them overlook plot problems or character problems that could be easily fixed.

    Whadaya think??

    Missy

    ReplyDelete
  11. Missy, thanks for your great comments today! I've often heard it said that if scores are at both ends of the spectrum, it's because the writer has a unique voice. So that's a good thing. When a writer is breaking out from the mold, some will love that fresh and original work, others won't.

    No matter the scores, critique feedback needs to be looked at with an open mind. Hard as it may be, we need to remember that this is a business and our manuscripts are our products. Just as any entrepreneur has to rework their product to ensure it meets the market standard, so too should writers rewrite their stories until they're reader/editor/contest judge ready. The negative feedback is never an attack on who we are as a person. Rather, it reflects a product that needs more work.

    ReplyDelete
  12. Someone at my local group meeting made an interesting observation this week: she thinks that unpublished authors tend to be harder judges than published authors. I'd be interested to know what y'all think.

    I served as a discrepancy judge last year in the Genesis. I didn't know ahead of time what the other scores were--but knew that they had to be pretty diverse to necessitate a third pair of eyes. But I tried to set that apart as I read them and treat them just like I had the regular entries I'd looked at. I didn't let the fact that I knew someone had already given it a high score and someone else had given it a low score affect how I critiqued it. In most of them, though, I was able to see why someone might have scored them low--and a lot of it had to do with subject matter, not quality of writing--as in, is it appropriate for the audience/for the CBA market. That's a really hard thing to judge, because it is somewhat objective. But judges who've been around the industry for years are asked to make that call, so we have to be confident in our understanding of what the majority of houses are looking for.

    ReplyDelete
  13. I've been a discrepency judge before but never privy to what the other judges scored.

    In a perfect judging world there are no negative comments only constructive ones.

    I always learn more from the lower scores and I am grateful to those judges who take the time to actually dig into my msc instead of giving me gloss.

    Gloss I can get from my cat. And she doesn't charge me 25 bucks.

    If you get what you consider are low scores the rule of thumb since contest judging began in 1410 at the First Council on Judging Contest Standards is to set it aside. When your mind is capable of objective thought consider the critique.

    If two or more judges have similar comments..DUDE YOU ARE NAILED.

    If no comments accompany the low scores consider the judge a useless eating utensil.

    If comments and references are included then go with your gut but, if you are going to toss away every opportunity to improve your msc. then I suggest you enroll in the DIVAS SCHOOL OF WRITING immediatley. They never sell but they have lots of fun whining and sharing tiaras.

    If you ever get a chance to judge a contest do it.

    It is a teeth gnashing, annoying, experience. It is never, ever easy. As I told someone in Seekerville recently I can remember giving out only a handful of perfect scores. Judging will make you a better writer, a more empathetic writer, and a real bad ass. Can I say that? I just did.

    Growth comes from pain. Pain comes from low scoring judges and from being a contest judge. If you are not suffering please email me.

    ReplyDelete
  14. Kaye, I think unpubbed judges are harsher because they see things according to "the rules" that they're often told they have to limit their stories too.

    Do you think that sometimes the high-scoring judges give the higher scores despite the problems because they see the great writing talent and know good story-telling when they see it? The great story helps them overlook plot problems or character problems that could be easily fixed.

    I know if I come across a well written entry, I'll score it high enough to final. BUT if there is a significant problem, then I most certainly tell the writer "Hey, I think your entry deserves to final so I'm scoring you a little high on xxx and xxx becuase I don't want my score to keep you from finaling. But I do think xxx isn't as strong as it could be."

    For an example, one of my Genesis entries is fabulously written. I absolutely love the plot and lead characters.

    But...

    The heroine has no story goal as far as I can see in the entry. She's happily going along in life when something bad happens. Her reaction to the bad event causes something bad to happen and she reacts to it.

    She's completely a reactionary character.

    Let's say she wanted to move out of her apartment and buy a house (Goal) because she finally feels ready to plant roots instead of running from plact to place (motivation). Then xxx happens which causes her to lose her job (conflict). Then on her way to apply for a new job someone tries to kill her (conflict) so now she's on the run (obstacle) for her life as she also tries to figure out who's trying to kill her and why, while also working to either reclaim her job or find a new one.

    If she doesn't have that individual story goal to begin with, then when the killer arrives, everythign she does is reactionary. The movie STEALTH is a perfect example of reactionary characters who have no individual story goals. No wonder it bombed.

    The movie INDEPENDENCE DAY has 3 goal-oriented heroes who still seek their individual goals while dealing with the conflict around them. I really ought to blog about the story and why it's a great example of romantic action-adventure. BOURNE IDENTITY is great example of a romantic suspense.

    ReplyDelete
  15. I remember one year it was Touched By Love contest, I believe, I knew the coordinator really well, none of you, and she probably got back to me ... yikes...four times, 'can you judge a few more, my judges are failing me'
    I judged so, so, so too many from that contest. We both knew it and she was judging as fast as she could, too, to fill in.
    Then I discrepancy judged, then I finalist judged.
    That year they should have called it the 'Touched by Mary Connealy' Contest. Heaven only knows how badly I led all those poor contestants astray.

    ReplyDelete
  16. Thanks for that insight Mary.

    Do you think it would be construed as insider trading if I take that information and based on what Pzifer stock was trading last year during the contest and calculate how much Zoloft was prescribed thanks to you, consider my stock options during the period of this year's contest?

    I thought I'd ask before they give me the Martha Stewart cell.

    ReplyDelete
  17. The First Council on Judging Contest Standards, 1410 edition

    Drat. And to think I've been foolishly judging for six years without the original manual. Guess I better head on over to Amazon.com.

    Umm, no, Mary, I'm sure you advised them well. I think you were one of the TBL's LC final round judges. I coordinated that category. In fact, I probably still have your scoresheet.

    For the right price, I could share.

    ReplyDelete
  18. Then again, I'd hate for people to say I'm greedy.

    Make me an offer.

    ReplyDelete
  19. Great! Now people are going to think I'm easy.

    *sigh*

    Being unethical is so difficult these days.

    ReplyDelete
  20. Tina, Mary bought Pfizer with her last advance.

    Just thought you'd like to know.

    And Missy, it wouldn't have mattered what I gave Mary in that contest, 'cause it was an automatic discrepancy (which means lowest score gets dumped) so I would have gotten dumped for anything under a 96...

    What on earth were those women thinking?????

    :)

    And Mary fails to mention the very descriptive mutilation scenes...

    The real 'inside' look at the killer's thought processes....

    And blood EVERYWHERE!

    But the good side is that her story was chilling to say the least, and I couldn't see it going into anything like inspirational. I was too hard on her, though, but I remember asking her to validate what she was showing by asking herself the famous "WHY?"

    And then I promptly wrote to her, confessed it all even though we'd never met, because I really respected her writing and talent.

    And so did Barbour two years later and the rest is h-i-s-t-o-r-y.

    Missy, great post. As a coordinator, discrepancy judging is a two-edged sword. Necessary evil. Since judging is and always was (or should be) subjective, I think it works most of the time, and quite often like you suggested, to the lower score's favor.

    I've been the object of discrepancy judging, and the reason one was asked in at least once or twice, but honestly, I think my assessment of the entries in question was closer to reality than the other judges. (Including Mary's)

    But since it is opinion, that's okay.

    And I'm totally with Tina, that if you want a popularity contest, join a high school cheerleading squad.

    Rah, rah you!

    If you want to get better, suck it up, pull up your big girl panties, and stay in the game.

    Persistence pays off.

    And a tougher hide is exactly what we need for facing a reading public.

    And Gina:

    St. Patrick will be fine with you wearing green, darling, and I promise not to confirm you or anything like that.

    ;)

    You can remain a Protestant and still wear green as a symbol of...

    Irish Catholics like me, LOL!

    No matter what color ye don, lass, we'll still be lovin' ye here in Seekerville.

    Ruthy

    ReplyDelete
  21. Ruthy, I think for the sake of consistency I'll stick with black and white. I wouldn't want people to think anything else but that I'm color blind. Wait a sec. I don't think that came out like I meant it. Usually I match. But how can one go wrong with black and white.

    Note to self: interject color in wardrobe and not buy hanging up a landscape in the closet.

    off to do laundry....

    ReplyDelete
  22. Well, begora, Ruthy, what is the point of a mutilation scene if it's not descriptive???

    Hello, Master of the Obvious

    ReplyDelete
  23. I will confess here that in one contest, a judge RETURNED Ten Plagues to the contest coordinator and refused to judge it.
    TOO ICKY

    I was just testing my wings, ladies. I will say this, I still think Ten Plagues is one of my most fiercely Christian books. Very strong Christian message and characters throughout. None of them the bad guy however. :)

    ReplyDelete
  24. I like what you said, Debby: Our mauscripts are a product that need more work. We can't get too attached to them, and we can't protect them. Not if we want them to sell someday.

    Kaye, very interesting points. I've actually found that for my contest entries, the published judges usually gave me lower scores. And it often had to do with the fact that they thought it wasn't going to be picked up by an inspy publisher (this was for my women's fiction written in two first person pov's that had to do with infertility and a woman leaving her husband). The other thing you said that was interesting was that you could see why someone might score low based on the subject matter. I have a hard time deciding how to judge on that. I tend not to score down for that but do try to tell the person that in my opinion, it would be hard to sell.

    Tina, what can I say. You had me sitting her howling laughing!!!!!! Why don't you tell us what you really think! LOL! Very good points.

    Gina, I appreciate your point about a reactionary character. I'm working on that right now on the book I just finished. Lindi nailed me on that one. (Thank you, Lindi!):)

    Okay, all the Martha Stewart/insider contest trading/Zoloft stuff is cracking me up! Thanks for brightening my day. :)

    Missy

    ReplyDelete
  25. Well, I'm glad to hear you're cheered up, Missy.
    Seekerville = Cyber Zoloft

    ReplyDelete
  26. Take two Seekervilles and call me in the morning...

    New Day Prescription Service

    ReplyDelete
  27. I did wear green today, and it was a good thing, because when my kids discovered it was St. Patrick's Day, they ran around pinching each other :)
    speaking of contests, please enter me to win Jesus Saves! Take-Home Mini-Books. it sounds great!

    ReplyDelete
  28. Stampedwithgrace,

    I bet that's a lot of pinching going on in your household! I just saw your family photo. Wow. Seven kids! I bet it's lively around there. :)

    Thanks for stopping by!

    Missy

    ReplyDelete
  29. Hey, Seekers!

    I got a discrepancy judge on both my Genesis entries last years. My low scores were so incredibly low, it ain't even funny. The discrepancy judges gave me an in-between score, but since they averaged all the scores together, I didn't stand a chance. The discrepancy judges did give me some good comments I could use, but the low-scoring judges didn't. Sorry. They were just negative and their comments didn't make much sense and/or contradicted what someone else had said. Sometimes my lowest-scoring judge has been the most helpful. Those two times, they were not.

    Frankly, I like contests where there are three or even four judges and the lowest score is dropped. To me, that's the most fair way to do it.

    ReplyDelete
  30. Melanie, I like the idea of having 3 or 4 judges, then throwing out the lowest, too.

    I'm sorry you had the low scores with no feedback. It's what's bad about the Golden Heart, too. Of course, with the prestige of the GH, it's worth the entry fee even if you don't get feedback, but it sure would be nice to know why the low scores!

    Missy

    ReplyDelete
  31. Just to perk Melanie up here, I have to say that I think, sometimes, a writer with a really strong fresh voice can have a strong affect on judges. When you're seeing widely diverse scores, I'd go ahead, Mel, and assume it's because I was a genius.

    ReplyDelete
  32. Mary, I can go along with that. :) Seriously, I think sometimes a judge doesn't know what to do with an entry--or she may focus on one little problem. Like Debby said, sometimes a strong voice causes a big gap in scores.

    It's interesting that Melanie's discrepancy judges fell in the middle somewhere. It definitely balances out that lower score (which of course would be nice if it had been dropped!). :) Yes, I still think if myself more as a contest entrant than a judge. I'll be getting back in the contest fray next year!

    ReplyDelete
  33. I have learned so much on Seekerville, not the least of which is that on April 15th when my entry is returned, I should buy the biggest chuck of dark chocolate I can find and start eating it BEFORE I open the attachment with my scores.

    Pre-medicate, like when you had infants and the drs. would advise you to give Tylenol before you took the baby for shots.

    This is not a disparaging remark on the judges, but on my writing. Just so we're clear on that! :)

    ReplyDelete
  34. Yes, Mary, that's what I'd like to believe, too. I'm a genius.

    Seriously, I know it had some flaws, though minor ones, according to what the other judges said. But I also know that that judge just didn't like it. She made that very clear.

    ReplyDelete
  35. LOL, Gwen!! I hadn't thought of pre-medicating with chocolate! What a great idea! :)

    Missy

    ReplyDelete
  36. And all this time I had it backwards, eating chocolate afterwards!

    ReplyDelete