Skip to content

Thoughts on ‘Test Better, Teach Better’ Chapter 5: An Introduction to Test Building

Ok, so Popham gives us in this chapter the beginnings to building our own test items.  But he does so with a few, actually five, caveats.  This is a fairly short chapter, easily read, and quick with information.  I have summarized quickly below:

– Two types of possible test questions: 1) selected response – in which you choose the answer from a list (multiple choice or true/false) and; 2) constructed response – in which you provide the answer to the question via essay, short answer, free response, etc.

– Both types of possible questions have their pro’s and both have their con’s…we have heard them all, even prior to reading Popham’s list of +/-

– 5 big roadblocks in tests

1.  unclear directions

2. ambiguous statements

3. unintentional clues

4. complex phrasing

5. difficult vocabulary

* “(Y)ou’ll hopefully recall that the central mission of all such assessment is (1) to help you make valid inferences about your student so you can then (2) make better decisions about how to instruct those students.” (p. 69)

* “Never, never create a test just to be creating a test.” (p. 69)

* “Beyond the mission of making inferences about your students based on their test performances, there is no other reason to test those students.” (p. 70)

* “For every test that you create, review all items and directions from the perspective of your students.” (p. 70)

A good chapter, a quick read…now, hopefully you are up through Chapter 5 and ready for our meeting on Thursday.

 

Keep on reading,

 

Darrin

Published inUncategorized

13 Comments

  1. Annie

    I had a conversion with Kristi and Matt just the other day that relates this chapter to the ACT test situation we find ourselves in. I wonder if anyone has guidance for me. To prepare our students for the ACT it would make sense to model our classroom assessment, at least to some extent, after the ACT format. Yet here is the catch I find myself in, by creating these passages with complex vocabulary, am really now assessing just my content? Am I assessing reading?

  2. Chelsea

    I focused on the the roadblocks to building tests because I tend to do many of the things that the book lists. The one that I do the most is #3, giving unintentional clues. I struggle to find a balance between to hard and to easy. This has been an issue that I have been working on all year and slowly but surely I believe that I am beginning to do better at making my assessments more rigorous without being impossible.

  3. Tania

    One helpful note that I learned in my curriculum class that the author did not mention is to have a coworker review the test. Even better would be a coworker in a different department. When creating an essay or paper we always have a peer review and make comments and adjustments as needed, so why not tests? The author mentions that we should create the test in the perspective of the student; nice idea, but most likely unrealistic. We know the information, we have taught it multiple times and enjoy the subject, which makes it difficult for us to see things from their point of view.

  4. Tonya

    With the new reading program adopted in our SpEd classes, the task of teachers creating tests has been eliminated. All tests are built in to the curriculum and contain both selected-response and constructed-response questions. The Cluster Assessments (yes, that is the official term) are open-book. At first I was unsure about this type of testing format. However, after further evaluation by our PLC, we realized the reasoning behind it. The questions are designed for each student to ‘show what they know’ and not just find and copy the answer. Of course, the students were excited about the idea of having the answers ‘right there,’ until they figured out that they had to apply their knowledge. It has become easier to ‘spot’ the students who are truly grasping the content. It is more difficult to spot the ones who aren’t. Is the student truly ‘not grasping’ the content, or is he/she just unmotivated to demonstrate the knowledge? One other detail I pulled from this chapter is regarding unclear directions. Since we don’t create the test, we don’t write the directions. Even though the Clusters are similar in design, I have to review the directions for each assessment.

  5. Matt Mikkelsen

    I agree with Annie. I believe we need to start preparing our test to look like and feel like the ACT tests. In order for our students to be successful on the ACT I think our classroom assessments should be practice runs. The one thing that I believe will hurt our students is time management on the test.

  6. Doporto

    Never, never create a test just to be creating a test. I think this is something that goes on a lot. large amounts of home work and tests. I feel a teacher should be able to see what a student knows during class time. Most students wont get help at home. I feel short home work assignments and more classroom participation will up test scores. that is if the teacher assesses and actually interacts with the students. tests then need to be to the point not a trick. Straight forward what do you or don’t you know.

  7. Tony

    The one thing that resonated with me was at the end of the chapter. “For every test that a teacher creates, review all items and directions from the perspective of your students.” If you put yourself in the students’ shoes, chances are you will improve your future test writing.

  8. Thomas Jassman

    I would like to compliment on what Tania states with regard to having a coworker review your assessment and quite possibly from someone who is not involved within the same content area. This will definitely bring a new perspective to the analysis of the test and could really help take care of the five caveats towards bad test making. Also, one our tests having somewhat of a structure to the ACT format is essential as well, especially with respect to it being timed. Bottom line, a more thorough review of our tests and it’s relationship to the subject matter taught and the ACT needs to be established.

  9. Jenny

    In reading chapter 5, I noted several things. I think that we all build questions that seem perfectly clear to us but only find out after the fact that they were not clear to students. Even if we proofread it numerous times (trying to use students’ eyes), that will happen. I think that it becomes very important that we have enough time to modify immediately after the test is given so as not to have the same problem if we give the same test the following year. But as extra time is so rare in our days, sometimes we put it off and it doesn’t get done and we find the same problem next year.

    Also, when the book makes the point that certain things should be clarified in the directions, I think that sometimes we given oral directions to clear up some of the ambiguity within the written directions. I may not state the time limit in the directions on the page but I do make sure to state it when I hand out the papers. Honestly, the wordier the directions on the paper, the less likely students are to read them anyway. I find that short, concise directions are best and then I explain the more intricate pieces orally.

  10. Bruce Metz

    I have always had a difficult time in seeing clear validity in selected response exams. I felt that during my schooling these type of tests lead to more anxiety and self doubt due to ambiguity within the questions and responses (ambiguous statements and complex phrasing). Many times students have to spend a large amount of time in reading the question and choosing between a,b and c, but not d,f… I feel that students can have a clear understanding of material and simply choose incorrect answers based on confusion within the test item. I also see students with no clear understanding of the material and choose correct responses with the old take a guess method. I do understand that these test items are much easier to grade and time definitely is a consideration, however, I don’t always feel that data collected within selected response is totally valid. I feel that a test should be a learning tool, not just a measurement of knowledge gained at the end of a chapter or unit. Without using the data for correction and remediation all we have is an “autopsy”and we have all heard that before.

  11. Julie

    The thing that jumped out at me the most in this chapter was that it is so important to make sure you know what you are testing for before you write a test. For me this is a very obvious thing. In math, I teach a concept and then test the students on that concept. It’s very important to make sure that they get it because the next unit will build on the previous information. The building of tests is not the difficult part, it’s making sure that the students get the information before I give them a test. Assessment for Learning is a very important step to take before ever getting to the test taking piece. The chapter talks about knowing if you’re testing student knowledge or if your testing on your teaching style. This is what AFL is for, you give this assessment to make sure that you are teaching in a style that they get and then you give the chapter/unit test to make sure they get the concept.

  12. Darcy Bath

    I agree with the fact that you should review the test that you create through your “students, eyes.” If the ultimate goal of the test is to “uncover students’ covert cognitive and affective status” is seems to me that the test must be created in student friendly language that has been used over the course of study.

  13. Donna Plemel

    What I have noticed in assessments is the fact that the true content of material can be lost in the wording. There is so many students that that “wording” at face value. In the development of the instruction, one needs to be aware of the outcome and then the manner in which the students are expected to reach the outcome.

Leave a Reply to Matt Mikkelsen Cancel reply

Your email address will not be published. Required fields are marked *

Subscribe By Email

Get every new post delivered right to your inbox.

This form is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Skip to toolbar