LAST UPDATE:4/3/07

ASSESSMENT DEBRIEFINGS
Under construction!  Will be updated throughout the semester.

1. INTRODUCTION
2. PREASSESSMENT DEBRIEFINGS
3. ASSESSMENT DEBRIEFINGS

     This is the page where feedback will be given to the class as a whole. (Feedback to individuals will be sent via e-mail or by adding comments to their Web pages.)  Debriefings will normally be focused on the current Lab assignment, PreAssessment, or Assessment.

PREASSESSMENT  DEBRIEFINGS:

These debriefings are published on the PreAssessment Hub.

The following comments are addressed to the class as a whole.  They pertain to those questions with which the class had the most difficulty.

• Assessment 1:
1. The class average was 59 (which is a little below average).  Anyway, since I normalize all assessments to 78, the normalization factor is 19.  (See section 5.7 of the Checklists.)  After this is published late reworks will not receive the full benefits of the normalization factor.  This, of course, will mean that the final class average on Assessment 1 will not be 78.  Consequently, those of you who did the rework will benefit even more, at the end of the course (if they fulfilled the Promise to Fulfill Minimum Expectations), when I normalize the final average.
1. The average on the preassessment questions (41-45) was 54 VERY DISAPPOINTING! (Two people did get 100%; CONGRATULATIONS!)
   
2. The average on the proficiency evaluation questions (46-50) was 57; again, VERY DISAPPOINTING., ESPECIALLY since these are among the best questions I can ask.  Let's see if we can't bring this average way up on assessment 2!
3. The average on questions 41-50 (questions that are public) was 56 (This says that our class can't "pass" questions that have been in front of them all semester).  Obviously this is unacceptable and will NOT be acceptable on Assessment 2; see item 2.3, below!!!
4. The average on the rework was 91 which, I guess, is fine, but I would like for it to be 100.
   
2. The fact that every year, every class SHOULD have "mastered" the Proficiency Evaluation means that ALL students should be comfortable with the best "test" questions that I can ask on my learning material.  As I have stressed, the assessments themselves are not of the highest quality because, since students keep them, I have to create new assessments for every class.  Consequently different assessments will be of different quality and different difficulty.  Varying "quality" is something I really can't prevent, but the fact that I normalize all grades to a class average of 78 guarantees that all assessment grades ARE fair, i.e. based on assessments of the same difficulty.  The combination of Proficiency Evaluations and normalization are the best way to guarantee optimum feedback for correcting misunderstanding and guarantee fair grades. Review tee the checklists (section 7.1 of PREPARING FOR AND TAKING AN ASSESSMENT):
1. The essence of my policy of normalization is the belief that students, after they correct their mistakes, are "better" than their original grade, and I want to give them credit for this improvement.  Part of this is based on the "fact (?)" that EVERYONE "should" have 100% understanding of 80 questions (40 from the Preassessment and 40 from the Proficiency Evaluation) even before they take an assessment; after the rework I should be able to say this about 120 questions!   HOWEVER, when students miss some of questions 41-50 this emphasizes that my assumption is NOT correct!!  So those who miss too many of the questions 41-50, especially 46-50, do NOT deserve the normalization!!  If that includes you, be sure to go back and restudy the Proficiency Evaluation and make my assumption correct - PLEASE!
   
2. If you "memorize" the answers to the preassessment and proficiency evaluation questions, you will probably forget them, perhaps by the time that the assessment rolls around.  However, if you take the time to understand the answer and understand why other answers are not the "best" answer to a particular question, you should be able to "come up" with that best answer when you see the same question again - on the assessment!  So, if you are not "comfortable" with all the answers on the preassessment and proficiency evaluation, PLEASE get comfortable by discussing the questions/answers with classmates, in class, or on the forum!
3. I record the % of questions 41-50 that you get correct along with the overall score.  That % is an indication of your effort in this course, i.e. how much effort you put into using the Q&A&F (questions and answers and feedback) I build into the course.    If a student can't pass the most important questions in the course - when they have been available since the beginning of the course and whose answers are publically checkable, then that student DOES NOT DESERVE ANY GRADE NORMALIZATION; therefore, if, on Assessments 2 and 3, you do not get 60% (6/10) on questions 41-50, you will NOT get the full normalization factor and you will NOT qualify for the final normalization at the end of the course - this are SEVERE penalties, so please avoid them!
1. On the other hand for those who DO master the preassessments and (especially) the proficiency evaluations, if, at the end of the course, you are on a grade borderline, a good % on questions 41-50 will help you receive the higher grade, but a poor % will almost certainly result in me leaving your grade at the lower level - this is a "swing" of a whole letter grade!
4. I really don't understand why people miss questions on the rework.  Perhaps it is partially because they have yet to recognize the power of the Find... commands found under the Edit menu of Navigator; see section 3.7 of the COSC 120 Checklists.  (If you are not viewing a page that has frames, the procedure may be different; however, in Navigator, the  Find in this Page.... works for all frames, but is a little more tedious if you want to search a specific frame.)
3. Analysis and Feedback on Assessment 1:
1. The most frequently missed questions were (boldface indicates questions missed most frequently on the rework as well): 1, 2, 8, 9, 16, 17, 19, 22, 26, 27, 30, 31, 39, 40, 41, 43, and 47.  Since most of you corrected your own mistakes on the others, only the boldface questions are discussed below.  (Note that the grader does not give me the wrong answers chosen, so I'll just address what I think is probably the problem.)
• Q2:   This is an example of the most difficult kind of multiple choice question; see the first item under 3.3, below.     The answer comes from understanding Figure OOC-2.
• Q26: I was surprised that people missed this question; I hope no one fell for the nonsense answer I put in.  The answer  is a "fancy" word for earthly.
• Q31: I was not surprised that this question proved difficult.  Bandwidth classifications do seem strange, at first.  The answer comes from understanding Figure OOC-2.
• Remember that the client-server concept is fundamental to all distributed computing (networked computer systems), e.g. Web pages are accessed from a Web server, e-mail is accessed via an e-mail server, files may be downloaded from or uploaded to an FTP server, the various teleconferencing facilities (chat, audio chat, video, desktop sharing, etc.) are accessed via their associated server, etc.  Users access server "services" via clients, e.g. browsers, e-mail programs, etc.  This whole idea is based on the client-server model of computing.
2. Most of the answers on this assessment are associated with fundamentally important concepts.  Specifically, the most fundamentally important  questions associated with cyberspace (as far as learning the essentials of COSC 120) include:
1. Important questions whose content will NOT be covered further during the course are 1-5, 25- 27, 29-33, 35, 45, 46, 49, and 50.  If you are unclear about any them, post your queries on the Class Forum.
2. Questions that will be discussed in more detail later include  6, 9-14, 16, 17, 28, 37, 38, and 43. (Important ones are in bold.)  If you are "uncomfortable" with any of these questions/answers, perhaps that will change when we cover these in more depth later.
   
Note that all answers, in that they are part of the vocabulary of computer science, are important; if we misunderstand, or, worse yet, misuse any of these, confusion will result!  We MUST "call a spade a spade"!  However, the questions listed in this section are the ones that are most critical to the correct communication of ideas associated with cyberspace.)
3. Questions worth additional comment:
• Q2, and 12:  These are examples of the most difficult kind of multiple choice question, one whose answer is "none of these".  They requires "recall" not just "recognition", i.e. you have to know the answer, not just choose from those written out in front of you.    The answer comes from understanding Figure C&N-5.
  
• Hint for the future: I will only use such questions for VERY important concepts.  I use these more frequently on public Proficiency Evaluations, where the most important questions are asked, but I use them on assessments for critically important concepts, i.e. those whose answers should be well understood.
• Q8, and 10 are examples of another difficult type of multiple choice question, a "best answer" question.
  
• Best answer questions are great questions to make you think and assess your "UNDERSTANDING" (as opposed to "Knowledge", i.e. memorized info.)  In fact, I try to make all questions best answer questions, especially on the Proficiency Evaluations.  The best of the best answer questions would have all answers acceptable, each with a different priority - this would really probe your understanding.  On the other hand, this type of question is more difficult than typical multiple choice questions, so the more of these in an assessment, the lower the original class average.
• What are the other acceptable answers to these questions?
• Q7: The question implies that the answer is the largest category of networks.
  
• Q11 and Q12 are fundamental; we will discuss this thoroughly in LM IV.  (Note the title of that LM!)
• Don't confuse modem protocols, e.g. PPP and SLIP, with Internet protocols, e.g. POP and SMTP.
  
• Q12:  Remember that "resource access" is one of the three categories of Internet "services", the others being communications and information retrieval.
• Q15: Netmeeting, "our" teleconferencing application, has this feature.
  
• Q17: The answer, when published, is a Web page.
• Q18: Using a browser, access each of these URLs to see what you get.
  
• Q27:  "Multimedia", in this questions, means we are referring to the most sophisticated type of network communications.
• Q23:  Digital radio is available, but, by far, the most common type of radio broadcast is ___________.
  
• Q30: Actually the answer refers to any type of connection between different kinds of devices.
• Q37:  The answer is the most sophisticated type of network node; it can do everything that the other answers can do.
• Q40: The answer is a fancy word for "first come, first served".
  
• Q41-Q45 are from Preassessment 1 and have been available all year.  PLEASE be "perfect" on all the questions there, especially on assessments 2 and 3!
• Q46-Q50 are from Proficiency Evaluatio 1, THE most important questions I can think of that are associated with what is covered. PLEASE, PLEASE be "perfect" on all the questions there, especially on assessments 2 and 3!
4. There are nonsense answers (that come from my imagination) in the following questions 3,  5(2), 10, 14, 15, 17-20, 23, 26, 28(2),29(2), 31-33, 36, and 38-40.  It is just as important (perhaps more important) to recognize these nonsense answers as to recognize the best answer!
5. I am assuming that once you have eliminated two wrong answers, that you will have the right answer to every question if you study your rework, so I do not ask you to rework the rework!   However, please make that assumption accurate, by correcting your mistakes yourself.  If there are any remain confusion about any question, post them on the forum.
6. In a "perfect world", EVERYONE would, after digesting this debrief, have 100% understanding of 120 questions (40 from the Preassessment, 40 new ones from the Proficiency Evaluation, and 40 more new ones from Assessment 1).   That's a lot of questions!  Therefore, everyone should have a pretty good understanding of the material covered in Assessment 1. This should give you confidence when preparing for the final exam.  PLEASE make these assumptions correct; let's try for that "perfect world"!
7. Finally, I now assume that every student understands how Tony's assessment strategy works, i.e. how the "diagnostic tools" (SAQs/TPQs/FIBs, preassessments, proficiency evaluations, and assessments) and the grade normalization policy are an integral part of the learning experience of all my courses.  If you have ANY questions about any of these diagnostic tools, now is the time to ask them - before we start preparing for Assessment 2 and the rest of the course.
• Assessment 2: To be published after the assessment is reworked and graded.
• Assessment 3: To be published after the assessment is reworked and graded.