E-Portfolios and Jumping Through Hoops

“I Just Did it to Get it Done" -- The Transformation of Intentions in Portfolio Assessment in Teacher Education

This paper is already three years old, and although it sounds very narrow in its scope, it's worth looking at if you're at all interested in any type of assigned reflection (blogging, e-portfolios, learning journals) for students of any age. If you read the students' feedback on pages 10-14, you'll probably skip back and read most of the rest.

I thought that four themes emerged. One is the conflict between documenting change, mistakes, struggles (often essential parts of learning) and focusing on presenting competencies, confidence and completed learning outcomes. This is acknowledged in the process vs. product debate, but extends it further into discussion of how to present an online identity.

The paper also delves into the politics of portfolios in teacher education, touching on the issue of using portfolios for high-stakes assessment:

"The purpose of the portfolio was thus transformed from the individualistic, developmental, constructivist vision in the Design Document to a policy tool designed to address external program and state requirements."
...and "When people in power (i.e., the state, a teacher education faculty) impose a cultural tool, less powerful agents (preservice teachers) may master the tool but use it with reluctance or in subversive ways, or resist its use altogether."

It covers important issues surrounding faculty disengagement, lack of professional development, assessment difficulties, and uneven implementation, but the key point is revealed in the title of the paper: most of the students are jumping through hoops:

"Most said they produced these portfolios just to 'get them done,' because the program required them – not because they found them personally meaningful."

If most students aren't internalizing the process (or product?), they aren't experiencing many of the benefits of reflective learning. This would apply to assigned blogging and e-portfolios, and almost any type of assignment for that matter. Without engagement, the result of the process is a set of artifacts and writing that the student doesn't identify with. When the authors asked one whether she thought her portfolio represented her, she responded:

"No, no, I don’t think so. It’s accurate in what I’ve done, but not in, I
don’t know, in who I am, I guess, maybe, um, I don’t know, the whole experience was just so, I don’t know, it’s like this big, dark cloud hanging over it."

Lots of other issues in there as well, all important if we're planning to build and implement something that is actually useful and meaningful. Via David Tosh and Michelle Lamberson.

Serge Ravet on European E-Portfolios and the Future

Serge Ravet gave an excellent short presentation (PDF) related to e-portfolio development in Europe. He seemed to have the clearest vision of the future of e-portfolios, rather than focusing on the present storage/reflection functions. He had a couple of slides (page 10 and 11) with some points that really jumped out at me, asking us to think of e-portfolios as a suite of services leading to specific, tangible benefits, rather than as tools or products for their own sake:

• "My Dashboard” might give me an informative display of the current state of my skills and knowledge
  
• “My Planner” takes into account what I know (and can do), matches it up with what I want to accomplish, and helps me plan my learning
  
• “My IPR management assistant” could be seen as a tool to value, share and exploit my personal assets, intellectual property or artifacts
  
• The “socialite portfolio” is one way of viewing the intersection between your online identity, connections to other people's identities, and the content/artifacts/comments that define learning and knowledge.
  
• Like all good consultants, his vision has a handy acronym: TOUCH -- Transparency (show me yours, I’ll show you mine), Opacity (access, privacy), Ubiquitous (online, mobile), Celebration of achievements (value individual assets), Holistic (maintain a global view of the individual: reunites my digital self).

I thought that most of it was really strong stuff, and sent me back into Stephen's semantic social network and my subsequent musings on the topic.

BC's Portfolio Graduation Requirement

Probably because of the location and timing of the e-portfolio conference, many of the participants were educators and administrators from BC hoping to figure out how they're going to implement the province's new graduation requirements. The Ministry's goal is to have every Grade 12 student complete a high school portfolio, but they've only given one class in Grade 10 to introduce the process to students (along with everything else in the now-defunct Career and Personal Planning curriculum). Schools aren't necessarily opposed to the spirit of this new requirement, but they're mystified by the timeline and lack of an implementation plan. Recently the Ministry published some guidelines and a basic overview of the portfolio process (PDF), but they've essentially created a new program without funding or curriculum time and materials allotted.

Some of the comments and concerns I heard from educators:

• “Maybe if we close our eyes, it will go away.”
  
• “Districts have a deer-in-the-headlights feeling about this.”
  
• Several teachers talked about the need for heavy-duty professional development and implementation help.
  
• Others talked about a general sense of resistance to Planning 10 and the portfolio requirement, mostly focused on the hassle of assessment.
  
• Staffing and logistics seem like major hurdles. If a district goes with traditional portfolios, just keeping track of the status and location of all that paper could be a nightmare. Schools considering e-portfolios don't know what kinds of tools to build or buy, and many don't feel they have the technical infrastructure to support this type of wide-spread implementation.
  
• There was the general sense that even schools that are able to implement the program will just be jumping through the hoops, and the students will be instructed to do the bare minimum to meet the requirements.

One of the discussion groups included participants from eastern Canada -- apparently they're planning a similar program in Nova Scotia in 2006 while Quebec's QEP (PDF) has already implemented portfolios in elementary schools and plan to roll them out to middle and high schools. It started to feel like there was a movement afoot.

There was some talk about the tools that will be used or built if districts decide to go with e-portfolios to meet the requirements. One of the most interesting was from a group of undergraduate students at UBC who have already built a working version that will be free and open source. Their presentation was enthusiastic and the web application looks really solid -- since it was custom-made, it's already aligned with BC's 24 learning outcomes. The project leader, Brittney, admitted that she had hated doing portfolios in high school and in her first year of university -- she advocated getting more students involved in developing the tools and instruction so fewer students will just be jumping through hoops when it's time for them to start building their portfolios. She also mentioned that some departments at UBC were already requesting student portfolios to be used for admissions acceptance.

Some of this stuff reminded me that I had written a paper on this topic for my mostly useless masters course in program design early last year. In it, I proposed a professional-development program to help British Columbia's high schools integrate career- and college-preparation information into their courses and administrative processes, approaching the issue as if I was an educational consultant hired to study the new British Columbia graduation requirements. Here's the short version.

UBC E-Portfolio Update at Lifia

I enjoyed the presentation on UBC's portfolio initiative (PDF) at the Lifia conference. It also led me to this excellent list e-portfolio resources covering much of what they're doing.

It sounded like their pilot project to introduce blog-based (Movable Type, I think) portfolios in several courses has been mostly successful. It made sense that they had to completely revise instructional design of the courses to accommodate the portfolio focus. A few key points that I found interesting:

• Instructors reported that the most challenging part of the experience was to teach students how to reflect. Students just weren't used to reflecting in any meaningful way about their coursework.
  
• 70% of new students at UBC in these courses were classified as ESL (English as a Second Language). Assuming that they were expected to update their portfolios in English, that could be a significant challenge for most...and an opportunity for them to practice their emerging language skills.
  
• Some frustration about technology integration when students were in WebCT for the course and had to go elsewhere for their portfolio. But heaven forbid they lock the portfolios away in WebCT -- that would be the worst-case scenario. I'd rather see them use Movable Type as their LMS.
  
• Workload concerns from faculty, especially in assessment. Some of it was fear of the unfamiliar -- one faculty member who’s been doing it for years emphasized that it’s much less of a chore than traditional assessment because it’s so personal, and so applied.
  
• Professional develpment was essential. Faculty had to learn how to assess reflection and new types of artifacts. They used a great panel-based approach in training, where group members would individually assess the same sample portfolio. Then they'd all compare notes and come to some kind of consensus for the marking rubric.
  
• Student buy-in is adequate for sections that will assessed and marked, but not for self-directed components. If it's not for marks, they won't bother.

David Tosh at LIFIA E-Portfolio Conference

The LIFIA e-portfolio conference in Vancouver had my head spinning on the weekend. There are so many potential applications and permutations of e-portfolios that my pages of notes look like a swimming-pool-sized bowl of tangled spaghetti. At some point I hope to pull a few of those most important strands together into some sort of coherent summary. For now, I'm happy that the conference organizers have posted all of the presentations, mostly as PDFs converted from the presenters' PowerPoint slides.

Much of the conference was a lesson in terminology. Although we heard "e-portfolios" so many times that I started getting sick of the term, they were also described as personal archives, glorified resume/CVs, personal development plans, educational development plans, and others. I was most interested in definitions and phrases that included a planning component or future focus. Most of the applications discussed were focused on keeping a record of past process and present products (artifacts, presentations), without much reference to the value of using past and present experience in envisioning the potential of future paths (careers, learning, lifestyles, relationships).

David Tosh is writing his PhD at the University of Edinburgh on e-portfolios and the effect they have on students' perception of their learning. He's a smart, classy guy with a realistic view of the practical challenges of implementing e-portfolios -- his presentation even included screenshots of the e-portfolio application he's been working on. I was struck by the parallels between his work and some of the interaction I've been working on in the last few weeks. His presentation is available in PDF and PowerPoint.

I think David was the only presenter to talk about parallels and cross-pollination between blogs and e-portfolios, although some of the initiatives covered briefly do use blogging platforms (UBC and Institut St. Joseph) for e-portfolio development. He used the example of Dan Saffer's course blogging to outline some of the main benefits of this kind of reflective learning. I've found the same in using this site as a sort of e-portfolio while I'm taking courses, although you probably wouldn't be able to tell when it's formal learning (while I'm in a course) versus the informal learning I'm doing all the time.

When people at the conference described the benefits of e-portfolios, they're thinking about those top-notch examples from highly motivated students. Nobody would question the value of the learning experiences reflected in those portfolios, but I still feel like it's a real stretch to assume that most students in a class will get the same benefits as the keeners who really engage in the process. Even at the adult level, I had to be skeptical of the high-minded rhetoric implying that all or most people should and will have e-portfolios in the next decade.

My best instructor assigned e-portfolios for our class of motivated graduate-level educators last year and the results were largely disappointing. Even though she gave us full creative freedom (choose your tools, layouts, formats, artifacts, type of reflection), most students did the bare minimum, as "defined" by the very loose constructivist rubric. It was common for students to simply copy and paste a few of their comments from the discussion forum as evidence of reflection. A few really engaged in the process of designing their site but didn't bother with decent content, and even fewer really embraced the whole process and produced excellent portfolios. I suspect this ratio would hold true in most classes, which doesn't mean that e-portfolios shouldn't be assigned, but it seems misleading to assume that everyone will enjoy the same benefits by participating.

The e-portfolio project that David has been working on is focused on adult learners, particularly in reference to EUCEBS, which is a process of basic-skills certification related to prior-learning assessments (PLA). Several of the presentations focused on workplace competencies and basic skills, using e-portfolios to document and recognize prior learning. David's program lets users choose whether they want to use the more regimented prescribed testing or e-portfolios. He described the confidence-building process of portfolios that helps people get over the sense that they don't know much.

The interaction of the portfolio system David showed was clean and smart. Students set privacy levels, submit work and artifacts for assessment and approval, and the infrastructure supports easy searching for employers or administrators. This last point came up in other presentations as well -- the primary objection is that employers won't embrace e-portfolios because they're even more dense and complex than the resumes they're already overwhelmed by. It was acknowledged that if they're structured properly and include the ability to search on multiple facets, it could actually make recruitment and hiring much easier. This would require a paradigm shift for current HR departments, but it wouldn't be difficult to illustrate the benefits of a smart system.

One of the practical concerns that David brought up was the issue of buy-in from faculty and facilitators, particularly the perception that e-portfolios are a nightmare to assess. There was the suggestion that at some point students may demand this type of authentic assessment, but that's probably wishful thinking. More likely, new instructors will have to be converted by hearing the positive experiences of their peers. This tied into one thread of discussion that emerged several times: should e-portfolios be used as a form of (or platform for) high-stakes, standards-driven testing?

G-Mail

There's been lots of attention paid to Gmail, Google's new e-mail service. I shared the privacy concerns of many observers when I heard that they were planning to data-mine e-mails so they could target their ads. Seems kind of creepy, but I signed up anyway, somewhat out of curiosity and partly because I'm wowed by their 1GB of storage. I've had a Hotmail account for years and I'm always having to turf things because I'm out of space.

Anyway, I tried sending and receiving a few e-mails and sure enough, the ads are very targeted. They appear to be the usual text ads that always appear alongside their search results. If my e-mail talks about mountain biking, I get ads about mountain bikes (see screenshot). It's true that they already know a LOT about me and will know even more with this service -- should I trust them?

Rockman

Gwen sent me this link to an educational technology consultancy called ROCKMAN ET AL. They've done some interesting work, including this older report on the educational benefits for kids watching Bill Nye the Science Guy. I'd like to see similar studies and summaries of some of the online learning modules available now to see how they'd compare.

Don't Tell the Kids

It's Time To Tell the Kids: If You Don't Do Well in High School, You Won't Do Well in College (or on the Job)
This article was making the rounds last week, probably because it pulls together all kinds of interesting statistics about the college-for-all fallacy. Unfortunately the title, premise and main conclusion all end up sounding rather bizarre to me:

"Students must realize that high school grades are important: grades strongly predict future careers. There are strong incentives for school effort and students can improve their adult attainments by improving their high school grades."

I just don't believe this. I know too many smart people who found high school irrelevant (didn't care about grades at all), then hit their stride and did wonderful things after graduation. And conversely, we all know people who were stars in high school who found out at some point (usually in college) that they didn't have the foggiest clue why they were on the academic fast track.

The general prescription here is that if only students would get better grades in high school, they'd be more successful after graduation, regardless of what they do. This theory is based on the fact that so many middle-of-the-road students are now going to college and then finding that they are ill-prepared for the experience. Not surprisingly, students with the best grades (who also believe the process is important, are more motivated to do well, and perhaps have a clearer sense of why they're going to college later) tend to do better in college when you average things out. But to infer that having all of those middle-of-the-road kids raise their grades would ensure their future success is ridiculous.

Probably the only really valuable conclusion in the article is that high school counsellors have their hands tied to a certain extent:

"A recent study in eight diverse urban and suburban high schools found that even if students had poor grades, school counselors did not dissuade them from attending college, nor did they warn students when they had poor chances of college success."

Counsellors want every kid to believe that they can do anything, just like those same kids' parents do. If parents believe that their kid is going to college (even if the student has bad marks and no interest), and the counsellor guides the student into some kind of vocational training, the parents freak out.

But the primary problem with the article is that it places all the emphasis on something nearly irrelevant for their own sake: high school grades. Most students figure out pretty quickly that if you're not applying to the top-notch universities or hoping for scholarships, grades just don't matter. The teachers wish they did, which is why this was published by the teachers' federation. What actually does matter is real learning, which is closely tied to motivation and the students' interests, but generally, the education system still hasn't figured out how to measure and encourage these crucial things.

Lifia E-Portfolio Canada

We're driving down to Vancouver this weekend to take in the Lifia E-Portfolio Canada conference. I'm looking forward to hearing about the successes, opportunities and difficulties the participants are finding as they implement e-portfolios.

K-16 Pipeline

Joanne Jacobs finds some interesting holes in the K-16 pipeline, casting more doubt on the idea that all (or even most) kids should be going to college/university after high school. She's got some scary drop-out stats in there as well. As I've been saying lately to anyone who will listen, it just doesn't make sense from a labour perspective or for an individual's self-actualization. If I was a career counsellor right now, I'd be telling kids to take short-term training or apprenticeships for skilled trades and technician work -- aircraft and automotive mechanics, medical sonographers, dental hygienists, etc.

Outsourcing and Education

David Wessel on Outsourcing takes the education/employment discussion from my last post out into the future, including the difficulty in trying to project employment trends for individual occupations. I liked this simple distinction between jobs that will be easy or difficult to outsource:

"In trying to discern persistent trends, Mr. Levy distinguishes between jobs that require workers to follow rules and those that require them to recognize patterns. The first -- whether in manufacturing or services -- are vulnerable to technology and outsourcing. The second are less vulnerable."

Much of my objection to current trends in K-12 education toward standardization stems from this fact -- it isn't preparing students to think. They might be great at following rules and meeting standards, but that type of "knowledge" sets them up to be worker drones or pursuing careers that will be done cheaply offshore. The follow-up to the WSJ article on the same site is even better, although it's quite long and a bit dense: Thinking About Outsourcing.

Their argument is one of those compelling left-right splits that seem confusing at first. They're not advocating protectionism or a movement to stop outsourcing -- it's almost as if they believe that free global markets are inevitable anyway -- but instead say that governments in the west should be investing heavily in retraining and social services to ease the transition to the next type of economy once many today's "good jobs" have moved offshore:

"For we have not, we do not as a country make the investments in retraining and rebuilding needed to transfer some of the gains from the winners to the losers, and so make the process of economic change truly a win-win one."

I would tend to agree with the spirit of the idea, but retraining for what?! Haven't most retraining efforts (I'm thinking specifically of successive waves of inititatives in the Maritimes that seemed to fail miserably) been terrible at anticipating the needs of emerging markets and preparing workers from disappearing sectors for new types of work? Just putting everyone on the dole for a while and hoping that they'll land on their feet can't be the answer either...

Who Cares About Blogging for its Own Sake?

Sebastian Fiedler contributes to an interesting discussion: Are Students Really Blogging?. Many of the edubloggers seem to be wondering how we're going to get teachers and kids to embrace blogging for its own sake, mostly because it's been good for us and we think it's so very important. These ideas all have value and I share many of the concerns, but I think we've lost sight of the reality in the trenches.

Last week I experienced my first open mockery of the fact that I had "jumped on the trendy blogging bandwagon". My critic took the standard tack: "what would make you think that anyone would care about what you're writing?" My lame response (on the fly and feeling defensive) was that a blog isn't really any different from any web site -- people have been building sites on their own or with Tripod or Geocities for close to ten years already. He scoffed and announced, "well, some web sites are actually useful." Ouch. I think his view represents quite a large number of people out there who don't (and won't) do a lot of reading and writing online.

Around the same time, I got an e-mail from Darren, who's thinking about doing a blogging conference in Vancouver in fall. With my blogger hat on, I initially thought it was a great idea...I thought I might even consider going. But then I realized that I don't really care about blogging per se. No more than I care about my toaster oven, even though I really enjoy toast. I like the benefits of reading and writing online -- new learning, connections to interesting people, reflecting on what I've learned -- but as a writer/reader, it's sort of irrelevant to me how all those words got onto all of those web pages.

We know that the power of blogs lies in how easy it is to post stuff online. Most people won't ever create a site, blog or portfolio no matter how easy it is. And many of the passionate bloggers we see today would be maintaining sites full of their writing even if blogs had never been invented. But for the people in the middle -- ones who do want to have a say or an audience, but won't learn FrontPage or html and ftp -- blogs are their best conduit to the web. I may not care about what 99.9% of them have to say, but I'm pretty glad that the "technology" helped get some of my favourite thinkers and writers online. That said, as a user (as opposed to educational technologist) I still don't care about the technology itself...I'm just happy for the knowledge and connections it enables within my very narrow slice of interests.

Since one of my interests is educational technology, I see why the potential of blogs could be important. But I also remember being a student in the faculty of education when the web was really taking off in the mid-'90s, and the initial focus was to make sure that kids were being exposed to using it. Very little thought was given to what they should actually be using it for -- the purpose was just to get them online at all costs. Perhaps we're just emerging from that stage with blogging...we're realizing that the process must serve some specific need.

I can't find it now, but I saw a book about creating e-portfolios using Microsoft PowerPoint. An educator was enthused about the possibilities in the review I read because "we already have PowerPoint on all of our machines and know how to use it." I detest PowerPoint, and couldn't believe that someone would actually force kids to use it for their portfolios, especially if the goal was to put them online.

Compared to FrontPage or Dreamweaver, PowerPoint is easier to learn, but this is one example where the "technology" of blogs would really blow everything else away. By freeing students from an archaic interface and meaningless decisions about fonts and layouts and connecting pages, blogs would let them create attractive collections of their work, reflections and related learning. The focus would be on the content itself and the thoughts reflected in the writing. Good tools fade into the background immediately and force you into the real substance of what you're trying to create.

Along with using blogging tools for portfolio-style reflection and displaying work (think art or photography), I don't think it's any coincidence that the early successes in integrating blogs into classrooms have come in journalism and writing classes. These activities were already all about the writing, and the tools simply enable new types of connections, reflection and publishing. An ambitious teacher could get every student in a journalism class to learn Dreamweaver and have them all publish their work online, but why not use all of that teaching time to focus on the writing itself?

These types of examples seem more meaningful to me -- finding ways to apply blogging tools to situations that would really benefit from them, rather than getting excited about "getting more students blogging". Like most of the things we call education, we believe that the blogging process would be for their own good...but if they aren't motivated by the topics, it doesn't matter how easy the publishing is -- they're not learning. Conversely, some small percentage will embrace blogs because they're looking for a venue to write about what interests them, but we know that it won't be related to school in any way.

More from:

• James Farmer
  
• Aaron Campbell
  
• Will Richardson and again here

College Grads and Unemployment

George linked to this report, called unemployment level of college grads surpasses that of high-school dropouts, and finishes with a good question:

"The graph shown in figure two should be more alarming, though its trends be not so steep - it depicts unemployed as a percentage of those two populations, and actually shows a decrease in unemployment for high school drop outs, but a steady increase for college graduates. What, if anything, do these statistics tell us about the needs of learners today?"

It mostly tells us about supply and demand, Doug would say. Too many people with degrees and not enough good work for them. This is a topic dear to my heart, so in a fit of narcissism I googled around my site and pulled out the relevant posts. Probably the most interesting link was this illuminating table showing where the most job openings are showing up in Wisconsin -- low-wage positions requiring no post-secondary education. Similar ideas:

• North Dakota educating kids to leave
  
• college for everyone?
  
• comparing two educational paths
  
• college and employment, linking to a degree is often worth $6.75 an hour

When the employment demand for arts and general science grads declines past a certain point, eventually there has to be a correction in the supply. It's not hard to envision an emerging split after most people realize that the standard four-year degree just isn't cutting it anymore. More people will join the educational arms race and continue on to graduate degrees, potentially making them as irrelevant as bachelor's degrees.

Worker shortages in the skilled trades and technical vocations should drive up wages to the point where more people will be choosing to become electricians instead of wasting four years on university history and biology. Not that money is everything, but the problem is that most students believe they're following the dollars by getting into college in the first place. Self-employment numbers also continue to rise, and everyone's figured out that you don't need a degree to start a business. But nobody wants to be on the outside looking in (on prosperity), so until we stop believing that degrees are the only way to get good jobs, we'll keep increasing the supply of grads until the demand drops to zero.