[Tlhingan-hol] Certification Test Woes

[Elizabeth Lawrence]

I don't think anyone believes we should simplify the level one test.
 However, I do think that many people would like a .5 level test, so that
there is a benchmark at which people can say "hey, I know something" before
being able to clear the level one bar.

This need not be as large an undertaking as the extant KLCP tests, though
it would be nice to get it to a similar level.  I would say that it should
use perhaps 100 words max, and that they should be specifically chosen to
be words a newbie would encounter or try to use.  Certainly, when I first
started actually studying for level 1, there were a lot of words I knew,
and a lot more I had never heard.  (SuvwI', vut, and bIQ should be on the
list, for example.  'Ip likely should not.)

We don't have to change the numbering of the tests, simply calling it a
beginner level test would do.

I also don't think that this thread was meant to call for a reorganization
of the testing system.  As far as I can tell, Qov merely wanted input on
what caused the most difficulty on the beginner test so she could better
instruct people who want to pass the test as it is.

As far as the suffix questions go, I think the answer to that problem is to
alter the questions to ask for more information, not less.  If the question
asked for the definition and classification type as well as classification
number, not knowing the number would only lose people a point or two on the
question, and that would be a truly minor revision to the test.

be''etlh



On Sun, Mar 30, 2014 at 6:47 PM, d'Armond Speers, Ph.D. <
speersd at georgetown.edu> wrote:

> I have tried to be consistent, fair and permissive in grading of KLCP
> tests.  The goal is not to make someone discouraged by grading
> hyper-critically, and to allow room for creativity (language is not a 1:1
> thing).
>
> For each question in the test bank there is an explanation of the purpose
> of the question, what is being tested by the question.  These are printed
> in the answer keys with the expected answer(s), which the proctor uses when
> grading.  When I'm grading tests, I am specifically looking for mistakes
> that pertain to the purpose of the question.  If someone answers the
> question correctly but is not strictly adhering to the vocabulary or
> grammar of their test level, I do not count that as an error.  (I would not
> count off if someone used {-'e'} in an answer in the level 1 test.)  If
> there are vocabulary or grammatical errors that do not pertain directly to
> the purpose of the question I count off a single point rather than marking
> the whole question wrong.  If they completely miss the point of the
> question, or the number of errors exceeds the available points per question
> (5 points per question for levels 1 & 2), then I just count the question
> wrong.  (If someone is making that many errors per question, then they
> really aren't at that level yet.)
>
> Whenever I distribute tests to someone else to administer and grade, I
> provide these instructions.  With multiple test administrators over a
> period of many years, it's easy to see how these guidelines may not be
> followed consistently.  I have not personally attended a qep'a' or qepHom
> in several years, or administered tests, though I do still regularly
> generate new tests from the test banks for qep'a'mey and qepHommey.  It
> sounds like we may need to get some written guidelines for test
> administrators and strive for greater consistency in grading.
>
> Creation of the KLCP was a group effort.  I designed the structure with
> Lawrence (3 levels with 3 pins, because of the cultural importance of the
> numeral 3).  There are 100 questions for each level in the test bank (plus
> the reading comprehension questions for Level 3), and generating tests
> means selecting questions from the test bank at random.  Level 1 is a
> sub-set of TKD; level 2 is all of TKD; and level 3 is open-season on all
> available materials (mainly including additional materials from KGT).  We
> produced written guidelines for each level, describing the vocabulary and
> grammar that was in scope.  Questions were written to be distributed evenly
> across the topics for each level.  Each question in the test bank was
> reviewed by multiple highly-skilled speakers when we were creating the
> program, following the written guidelines for the level.  I have all of
> this in a MS Access database, which allows me to manage the randomization
> and generation of tests and keys quickly and easily.  We put a ton of
> thought and effort into it, to make it as fair, correct and relevant as we
> could.  And ultimately, it is intended to encourage people to achieve
> certification; it is supposed to be a positive experience.
>
> All of that being said, we now have the benefit of many years of
> experience, which we didn't have when we were creating the program.  And if
> it's not achieving its goals of encouraging learning and rewarding
> progress, then we should be open to that feedback and willing to make
> adjustments.  The challenge is that the more significant the adjustments
> (severely limiting the vocabulary for level 1, for example), the more
> effort will be involved in re-developing the test bank.  If we make Level 1
> too simple, then the gap between Level 1 and Level 2 becomes huge.  And to
> be honest, I would not be able to do this re-development work, so someone
> else would have to take it over completely.  Taking out the questions about
> the suffix number would be easier, and I agree that it's probably too
> pedantic for the level 1 test.  These questions would have to be replaced
> with something else, maintaining balance across the topics for the level.
>  I'm not sure how many questions there are like this in Level 1, but if
> anyone has interest in writing some new replacement questions, I'd be happy
> to work with them on improving the test bank.
>
> Another thing to consider is that if we simplify level 1, then people who
> worked hard to accomplish that achievement might feel slighted because we
> lowered the bar.  Not sure if that's a real thing, it just occurred to me.
>
> Sorry for the wall of text, just lots ideas bouncing around.
>
> -- Holtej
>
>
> _______________________________________________
> Tlhingan-hol mailing list
> Tlhingan-hol at kli.org
> http://mail.kli.org/mailman/listinfo/tlhingan-hol
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.kli.org/pipermail/tlhingan-hol/attachments/20140330/4cf1bfd4/attachment-0001.html>