Where We Are Now; Or, Why We Have Never Been in Phase II

Because I was positioned as the DIS participant who would defend Phase II, I'll start there, and then work my way around to discussing the convergence of assessment, reflection, and transfer (which, by the way, is exactly where my interests are situated). 

Let me tell you how I became acquainted with Phase II. It was the spring of 2012, and I was conducting research on my program's portfolio assessment process, particularly in regards to its clearly flawed approach to assessing the reflective component of student portfolios. I won't rehash the whole experience because Joe and Bruce where in Writing Assessment, where I wrote frequently about that work, and KY read a draft of my study. Suffice it to say that the way Ed White discusses portfolio scoring at the bottom of p 584 aligns almost precisely with the way my program approached the matter: standard rubric, calibration sessions, benchmark essays, etc. Now, as a soon-to-be rhetoric and composition student/researcher, the role that reflection played in this whole portfolio process just didn't sit right with me. None of the faculty seemed to agree on why we assigned reflection, what it was good for, what it should look like, how we should integrate it into our classrooms, and so on. I set about attempting to change this, and as I was making sense of the data I collected on students' reflective introductions or reflective essays--depending on who you asked--my faculty mentor suggested I take a look at Ed White's Phase II. 

In retrospect I can say that I played Elbow's believing game a little too hard. I understood the practical difficulties a program faces in conducting portfolio assessment--time, money, faculty engagement, student and faculty resistance, and so on. I have been that uncertain reader trapped in a room with other uncertain readers over the course of three days, and let me tell you, it's pretty frustrating. So, my practical experience combined with my interest in reflection led me to focus (naively) only on those aspects of White's article. Asking faculty to care about reflection? Yes! Actually taking time to read and consider student reflection? Yes! It sounded like a great approach, and I played a big part in revising our assessment documents and practices to focus more heavily (though not exclusively) on what I understood to be reflection. 

Now, revisiting White's essay, I might be playing the doubting game a little too hard (if that's possible to do in this case?). Let me just run through a laundry list of the problems I see: 1) White is assuming that there was such a thing as "Phase I"--that all people in all places using portfolios used them the same way. Though he does admit that portfolios have different uses and purposes depending on the context (583-584), he doesn't seem to adhere to that understanding in this piece. 2) "Students" are all the same; there is only one class of student, and s/he seems to be a lazy trickster who indulges too heavily in "personal experience" and wallows in the "difficulty of writing and revising" (591). And when this student encounters Phase II, s/he is overjoyed and apparently says things like, "I never realized how much I have learned, since I was just interested in passing courses" (591). Ed White has three decades of experience that prove Phase II to be "the only assessments...that students genuinely find interesting, useful, and worth doing" (591). Which brings me to 3) To whom does White look for information about potential problems with portfolio scoring? Ed White (582). Also, this "means for scoring portfolios that has developed over the last few years..." seems to have no origin point. Who developed these means? Where? What's the context? Was there a study? Where's the data? 4) And, perhaps most problematically, the meaning of reflection in this piece is really difficult to pin down. For instance, in this passage White writes, "One particular strength of portfolio assessment is its capacity to include reflection about the portfolio contents by the students submitting portfolios. Most such assessments require the portfolio to open with a "reflective letter" or "cover letter" in which the owner of the portfolio comments about the products or the processes shown in it" (582; italics mine). Putting aside the issue of referring to the student as the "owner" of the portfolio, I have trouble understanding how what White is calling reflection actually is reflection in the sense/s that we've been discussing in this DIS. Here, reflection is about; later, "students should be involved with reflection about and assessment of their own work" (583; italics mine). Seemingly, the student is not the agent; s/he is not reflecting, but is merely participating in some kind of disembodied process that we're calling "reflection." But I think White's movement in this sentence from reflection to assessment makes sense: he is ultimately just describing institutionally prescribed self-assessment. Is that the same thing as reflection? I think the other readings from the past weeks can help us answer that question in the negative.

Leaker and Ostman really seem to describe a similar sort of process in writing about their experiences with PLAs: students are to reflect on their experiences and to render what they've learned institutionally recognizable. The similarities stop here, though. Leaker and Ostman are suspicious of ways that the institution co-opts student experiences and learning, of the way that student encounters with institutional assessment can harm them,  and of how this assessment process fails to respond appropriately to diverse student populations. What Leaker and Ostman come to recognize is that they must help students develop a vocabulary and a repertoire of rhetorical strategies with which they can make sense of and articulate their experiences in ways that the institution will recognize. Student learning is there already, and it can be carried forward through reflective exercises within the PLA context. The danger lies in what Dorothy Smith calls "institutional capture," articulated here by Leaker and Ostman: "Crudely put, the approach to alterity in this model is to encourage students to reflectively reframe it so that it becomes visible as academic credit" (15). Now, Ed White seems to view the institutionally-captured experience and learning as what we're going for when we ask students to reflect--we (Ed White) want students to articulate what they've learned using the language and values of the institution, and if they don't, perhaps they should retake FYC or leave the institution altogether. 

So, I have to say that I agree with Joe when he writes, "I think reflection is necessarily an articulation: in order to explore the self, to learn about ourselves as writers, and to develop our writing, we need to be able to articulate those ideas in order to manipulate, remediate, and revise those concepts." Reflection does seem to be about articulation, for ourselves and for others, i.e. Yancey's reflection-in-presentation. Yancey, Robertson, and Taczak (YRT, from this point forward) write about reflection and transfer in this way when they review transfer literature and emphasize in their own research the importance of students' developing a vocabulary for the discipline. They also write about the three-part role of reflection in their TFT course: 

(1) students learn reflective theory by reading about it; (2) students complete successive reflective assignments, including one accompanying every major assignment in which students theorize about key terms, writing processes and practices, and their identity as a writer, and (3) students engage in other reflective activities connecting reading, key terms, and assignments. (58)

Vocabulary clearly plays a key role in each of these components as students (re)articulate their theories of writing; however, students' experiences, learning, and identities are not forced into an framework of institutional outcomes. Students are not merely involved with "reflection"--they are reflecting, which ultimately leads to synthesis when they produce their theories. Joe asks what forms reflection-as-articulation can take, and I think my response would be, many. Gallagher gives us the example of the interface, which he does not directly connect with reflection, though it seems to me to be an example, especially in the context of KY, Stephen, and Elizabeth's article on eportfolios. Leaker and Ostman, and YRT give us another example: verbal or written reflections spurred by reflective assignments. And yet another: visual mapping in the TFT course of how key terms relate to one another, to other terms, experiences, contexts, and so on. Despite Bruce's skepticism on the matter, I would side with YRT and say that the literature demonstrates that reflection has much to do with transfer, that it does promote transfer, and that, as Kara has also argued elsewhere, it's necessary for transfer to occur. And, it seems that reflection in many forms can only aid in this process, though I would agree with Bruce that more research in multiple sites would help to support this claim.

In sum, here's where I am: reflection is crucial for transfer and important for ethical assessment. I believe that Wardle and Roozen's ecological model of writing assessment can help us to bring these three areas together, and hopefully we'll hear more from them soon on how the implementation is going at UCF. In the meantime, at the classroom level, assessment within the TFT course clearly depends upon reflection-as-synthesis, with this synthesis having been enabled through semester-long (re)articulations of a theory of writing. In these syntheses and in ePortfolios, we can see what students have concurrently transferred to and from other contexts. What we can't see is what happens after the TFT classroom over the course of a student's academic and professional/post-uni career/life. I believe the longitudinal portfolios that Wardle and Roozen describe would provide one way to trace this kind of transfer. We also don't yet know much about what happens with TFT in multiple sites (though that research is coming!), and we don't yet seem to know much about the roles that various "forms" of reflection play in the learning process. How might visual mapping interface with synthesis? with reflective assignments? How does this change when different materials and technologies are introduced? How do students of different racial, cultural, and economic backgrounds experience the TFT course, and does it privilege a particular way of knowing that troubles certain populations (as Leaker and Ostman describe)? We've already started drawing the map, but we have a lot more territory to chart in the coming years.