Analyzing observational data
Use these analysis methods for formative evaluation:
Example: Checklist
To better organize her lectures, Professor Richards developed a checklist, which she revised after discussion with a colleague who agreed to observe her class on three occasions. These were her colleague's observations after the first occasion:
The colleague explained the ratings and written comments, and he and Professor Richards discussed their impressions of the class's organization. The colleague completed the checklist during two more observations and discussed his impressions with Professor Richards after each. After completing all three observations, he wrote a summary, noting improvements and areas for continued work.
Example: Ratings
To increase student involvement during lectures, Dr. Smith decided to ask more questions, leave more time for discussion, and include interactive exercises. He explained his goals and gave a copy of the syllabus and a lecture outline to a colleague, who completed ratings after each observation:
1. The instructor's questioning skills
2. The instructor's skill in responding to questions
3. How engaged students are in interactive exercises
4. The instructor's skill in facilitating group work
Dr. Smith totaled the ratings (4 + 4 + 2 + 3 = 13) for the observation to compare it to later observations. After the second observation, Dr. Smith met with his colleague to discuss the ratings and their impressions of strengths and weaknesses. The colleague observed two more class sessions after Dr. Smith made changes and discussed the results with Dr. Smith.
Example: Narrative
Dr. Paderas, who taught morning and evening sections of the same course, wondered why she had fallen far behind the syllabus in her evening section. She asked a colleague to observe both sections and complete narrative logs. Like videotaping, a narrative log is a record of verbal and nonverbal behaviors but is less intrusive and provides another person's perspective. A portion of the logs for the first observation pair follows:
Morning lecture
8:50:
Dr. Paderas arrives ten minutes before the start of class, sets up a PowerPoint presentation, and organizes her lecture notes. Two students talk with her about the exam next Thursday, and she explains that she will hold a review session early next week.
9:00:
Class begins promptly with all but five students present. Dr. Paderas reviews the last lecture and presents the objectives for this class: understanding the possible causes and treatments for schizophrenia. She asks if there are questions about the previous lecture, and one student asks about the difference between dissociative identity disorder and schizophrenia. Dr. Paderas briefly describes dissociative identity disorder and explains key differences from schizophrenia, saying that she will provide more detail in today's lecture.
Evening lecture
4:59:
Dr. Paderas arrives one minute before the start of class. She has some difficulty organizing lecture notes because she is carrying more papers and books than during the morning lecture. Two students approach her to discuss their grades from the last exam. Dr. Paderas talks briefly with them but explains that she has to review her grade book to be certain about their grades. Two other students explain that they come directly from work to class and that recent job demands kept them from attending the last lecture. One of the students asks if this will affect his participation grade and Dr. Paderas explains that it will not as long as he does not miss more than four lectures.
5:06:
Class begins six minutes late with 10-15 students absent. Dr. Paderas reviews the last lecture and presents objectives for today's lecture about schizophrenia. She asks if there are questions about the previous lecture, and one student says she was confused about dissociative fugue. Dr. Paderas explains the concept and follows up to make sure the student is no longer confused. During the explanation, three students arrive and, as they sit, exchange comments with friends, which makes it difficult to hear Dr. Paderas. One student who arrives late asks Dr. Paderas to review the objectives for the lecture.
Dr. Paderas and her colleague discussed time management issues after the first set of observations. Dr. Paderas asked the colleague to observe a lecture in both sections two weeks later to evaluate the extent of improvements.
Additional Information:
Center for Teaching Excellence. (undated) Preparing for Peer Observation in, TX: The University of Texas at Austin.
Chism, N.V.N. (1999). Peer Review of Teng: A Sourcebook. Bolton, MA: Anker Publishing Co., Inc.