Tuesday, December 20, 2005
43things Feeds
I ran across this Bloglines account that shows most of the interesting global feeds coming out of 43Things, including goals with each of the most popular tags. Fascinating to see what people plan to do with their lives.
Friday, December 16, 2005
Elgg as Digital Lifestyle Aggregator (With Learning Goals!)
Writing this post reminded me how well Elgg has integrated many of these ideas already. When they launched the "resources" section, I was thinking of it as a very lightweight aggregator for pulling in outside newsfeeds like I do in Bloglines. But then last week it hit me that the most obvious use for it in the context of the "personal learning landscape" was to put my own feeds in there -- using it as a way to collect the content I have distributed all over the web in Flickr, Webjay, 43 Things, my blogs, comments on other blogs, etc -- and suddenly my Elgg account is feeling like a more integrated representation of my online life.
It's not very flexible because I can't yet control how to display all the stuff that gets pulled in there. So you get photos of my kids and songs I'm listening to mixed in with my learning goals and musings on ed.tech, all glued together in one big chronological sequence (broken into chunks of 25 entries per page). Still, tons of potential for new ways of connecting learning and learners in a system with that kind of functionality.
The other thing the Elggmeisters added a while back was shared goals as part of user profiles. So if you look at my profile, you'll see that one of my goals is to visit Japan. Clicking the link from my profile takes me to a page listing all of the other users who share that same goal. I think the integration of learning (and other) goals with social software and e-portfolios could be huge. Looks like lots of exciting things happening with Elgg these days, which isn't surprising.
It's not very flexible because I can't yet control how to display all the stuff that gets pulled in there. So you get photos of my kids and songs I'm listening to mixed in with my learning goals and musings on ed.tech, all glued together in one big chronological sequence (broken into chunks of 25 entries per page). Still, tons of potential for new ways of connecting learning and learners in a system with that kind of functionality.
The other thing the Elggmeisters added a while back was shared goals as part of user profiles. So if you look at my profile, you'll see that one of my goals is to visit Japan. Clicking the link from my profile takes me to a page listing all of the other users who share that same goal. I think the integration of learning (and other) goals with social software and e-portfolios could be huge. Looks like lots of exciting things happening with Elgg these days, which isn't surprising.
Thursday, December 15, 2005
Situated Learning
Kids' Informal Learning with Digital Media: Frameworks and Methodology
Unfortunately, nobody is offering to give me $3 million to study digital kids, but perhaps I can benefit from some of their work to get my thesis done. First of all, their methodology sounds interesting, and I could probably duplicate the approach in a simpler, more focused way. Again, it sounds a bit like connectivism:
Unfortunately, nobody is offering to give me $3 million to study digital kids, but perhaps I can benefit from some of their work to get my thesis done. First of all, their methodology sounds interesting, and I could probably duplicate the approach in a simpler, more focused way. Again, it sounds a bit like connectivism:
"Our ethnographic approach draws from theories of situated learning and the social construction of technological systems. Situated learning theory is grounded in the recognition that learning is both a social process and a cognitive one."They've also done a literature review in the area of technology and informal learning, and they're finding mostly gaps, which is probably why I've had a hard time digging good stuff up:
"Yet as our literature review demonstrated, there has been little sustained study of learning in informal settings, and almost no foundational theoretical work on the properties of unorganized or non-institutionalized learning. These emergent practices challenge our cultural and institutional definitions of learning and social participation in ways that will likely reframe our theories of education and development."And finally, I love this paragraph describing how we are redefining what constitutes learning. It's about individuals integrating learning in an incidental way into the flow of their lives, without any dependency on institutions to transmit information or dictate how and when education should occur:
"For example, our focus on digital ecologies pushes us into learning domains outside what we traditionally consider educational settings, challenging cultural definitions of learning and knowledge. In contrast to most cognitive and psychological approaches to learning, situated learning approaches are generally agnostic as to whether content is explicitly educational or not, and recognizes learning as deepening engagement with any kind of cultural content and social group, even those that our society has labeled 'entertainment.'
In fact, this model is probably most illuminating in understanding contexts where learning is a by-product of participation in an activity, rather than a formalized process of knowledge transfer, for example, apprenticeship learning, sports, or everyday learning in the home. As digital technologies enable kids to gain knowledge and cultural competency in domains that are not framed by explicit educational agendas, we begin to see changes in how kids construct identities and reputation, and how they relate to school and academic knowledge."
Enhancing Learning with Social Networks
Can personal digital knowledge artefacts' management and social networks enhance learning? (pdf)
Of course they can! This paper provides a nice overview of the intersection between emerging e-portfolio models (like Elgg, which probably should have warranted more coverage in this context) and social software/networks with learning applications. From the intro:
Of course they can! This paper provides a nice overview of the intersection between emerging e-portfolio models (like Elgg, which probably should have warranted more coverage in this context) and social software/networks with learning applications. From the intro:
"It will be argued that through sharing one's digital knowledge artefacts with other learners one not only brings on-line learning in a social context that it is sometimes missing, but also allows new paths of learning with peers to emerge."All the usual suspects are represented here -- Flickr, Furl, del.icio.us, technorati, etc -- in an accessible format. I thought the profile of 43 Things was spot-on and it shows the approach of the paper:
"Short description: A site that provides an area where people can write their goals, become inspired by others and share their process as well as learn from others how to achieve goals.
Social Content: Lists of life goals, desired things to achieve and places that people plan and wish to visit. Also 'have done this' and user profile.
Social Context: Connections are build between people who have listed similar aims or desires in order to have a peer group to support one another. Connections can also be made between people who want to achieve some goal and the ones who have already done that in order to give guidance and support.
Possible Educational implementation: In an educational context writing down educational goals is an important part of planning one's learning path. Peer-support from people with the same goals can help achieve the setting of targets and sharing the experience. The feature 'have done this' would allow learners who have gained certain experiences to act as tutors or help in the scaffolding process.
Deciding What to Learn
Decision-Making and Problem-Solving
The learning goals I'm looking at in 43 Things are pointing me down the path of investigating how people use the site to decide what to learn (or not), which gets me thinking about decision-making and goal-setting for self-directed learners. This may be a rabbit trail leading nowhere, but I wanted to keep track of it anyway.
The learning goals I'm looking at in 43 Things are pointing me down the path of investigating how people use the site to decide what to learn (or not), which gets me thinking about decision-making and goal-setting for self-directed learners. This may be a rabbit trail leading nowhere, but I wanted to keep track of it anyway.
Monday, December 12, 2005
Self-Organized Learning and Slashdot
New Educational Methods for Organizational and Enterprise Learning Derived from Self-Organizing Systems Theory and Practice (PDF)
This one is from one of the authors of the PerlMonk study I wrote about here earlier. I must admit that much of this talk about emerging and self-organizing systems goes right over my head (slime molds and crystallization?), but I did glean some value. The middle section describes the Slashdot community in interesting ways from an educational perspective, and the author's analysis of the kinds of learning taking place there is worth pulling a few quotes from:
This one is from one of the authors of the PerlMonk study I wrote about here earlier. I must admit that much of this talk about emerging and self-organizing systems goes right over my head (slime molds and crystallization?), but I did glean some value. The middle section describes the Slashdot community in interesting ways from an educational perspective, and the author's analysis of the kinds of learning taking place there is worth pulling a few quotes from:
"Rather, Slashdot content is complex and ill-structured (just like life), which requires participants to engage in higher-order thinking and in case–based, problem–based learning that is so essential for transfer of learning to new contexts."
"The subscriber base of Slashdot contains broad and deep expertise and curiosity. Peer review of ideas offered in Slashdot postings occurs almost in real time."
"Also, because expertise is distributed and varied, not central and uniform, the multitude of Slashdot participants are more intelligent than one or two content experts ever could be."
"Social interaction is essential in 'knowing how' to perform because 'learning how' is a social–dialogical process of negotiating tacit knowledge, through dialogues and conversation."In his last main point, he focuses on the ease of induction into the Slashdot community, and I might have extended that further to say that you don't even have to become part of the community to learn from it, because the community is situated in the web. The sharing of news, opinions, advice and resources on Slashdot can benefit lurkers and searchers arriving from elsewhere, without them having to join up.
43 Things Meetup
Interesting to see the overlap between virtual communities in 43 Things and real people getting together in real places: first ever 43T meet up: Atlanta rules!. And it's cool to be able to see how they set it up and followed through on getting together -- would have been interesting to be a fly on the wall: were they talking about their shared goals, discussing 43 Things in general, or just hanging out for fun?
Nothing Feels Good
This short article about teen-focused blogging communities like MySpace contains an excellent book excerpt that supports the reminder in the conclusion:
"We shouldn't disregard what social and emotional roles online communities play in the lives of our students."
Thursday, December 08, 2005
Goal Setting
Self-Regulation through Goal Setting
If I'm going to look at learning goals, digging into some research on goal setting probably isn't a bad idea. This ERIC Digest had me thinking hard tonight about the value of 43 Things in choosing and pursuing goals. It covers research into effective goal setting in learning and summarizes strategies:
If I'm going to look at learning goals, digging into some research on goal setting probably isn't a bad idea. This ERIC Digest had me thinking hard tonight about the value of 43 Things in choosing and pursuing goals. It covers research into effective goal setting in learning and summarizes strategies:
* Subdivide a long-term goal into proximal sub-goals. Help learners determine what sub-goals must be accomplished to attain their long-term goals.I can see some evidence of all of these strategies in the entries people write about their goals in 43 Things. It makes me think a bit about different types of benefits in making goals explicit on the site:
* View the goals as reasonable and commit to attempt to attain them. Provide verbal encouragement (e.g., "You can do this.") to learners to help motivate them to accomplish their goals.
* Self-monitor progress. Students must learn how to gauge progress in learning or performance. Provide progress feedback on tasks where it is difficult for learners to gauge progress on their own.
* Use strategies for coping with difficulties. When progress is minimal students might seek help, attempt to determine a more effective strategy, or re-evaluate the goal and timelines.
* Self-evaluate capabilities. The perception of progress will strengthen self-efficacy, which is critical for continued motivation and self-regulation.
- help in deciding which learning goals are worth pursuing (including seeing goals in other lists that you hadn't considered)
- support from other people pursuing the same goal -- general (not directed at you) and personal (comments on your posts and cheers)
- support from people who have completed the goal (general and personal, both negative and positive)
- inviting others to join you in pursuing the goal as a specific team
- connections to learning resources for actual learning the skill/information
Learning Goals
I just did a search for learning goals on Google. Almost none of the hundred or so I skimmed through had anything to do with people setting goals for their own learning. Almost all of them were institutional or program goals that instructors, curriculum directors and instructional designers have created for educational experiences with the expectation that those goals will be adopted as learning goals by students who get stuck in the course.
One of the few links aimed at people wanting to learn stuff for themselves was this about.com page devoted to "Adult Learning" and "Continuing Education", which sound rather stuffy. Don't we all want to learn new things all the time, without it needing a label or a course? Interestingly enough, one of the other non-institutional result was the first page of 43 Things goals tagged with 'learning', which I've been paying close attention to.
One of the few links aimed at people wanting to learn stuff for themselves was this about.com page devoted to "Adult Learning" and "Continuing Education", which sound rather stuffy. Don't we all want to learn new things all the time, without it needing a label or a course? Interestingly enough, one of the other non-institutional result was the first page of 43 Things goals tagged with 'learning', which I've been paying close attention to.
Tuesday, December 06, 2005
Learning as Becoming
From A Nomad’s Guide to Learning and Social Software by Ulises Ali Mejias:
My friend Brian has been exploring these concepts for years, and I've always loved his tagline: "How do we learn the things we value most?". In this context, I'm thinking about how social software might help learn to figure out which things we value most. As an aside, I've noticed that it seems like of the people in 43 Things who have completed learning goals, a very low percentage tend to caution people against pursuing them. I wonder if that's partly a reflection of a shared perception that pursuing all learning is valuable.
All this talk of becoming reminds me of Paul Graham's undelivered commencement speech. His general advice relating to high school and self-actualization it to treat school as a day job and spend as much time as you can pursuing interesting questions and projects (outside of school, mostly):
"At a more fundamental level, models of learning based on social software can facilitate the shift from what Brown and Duguid (2000) call learning about to learning to be, or to give a more Deleuzian connotation, to learning as becoming. Learning about implies a passive consumption of knowledge in the form of facts. Learning to be implies the application of knowledge in the development of skills that allows us to fulfill a particular (professional or non-professional) role in society. But to highlight the fact that being is not static, I’m using learning as becoming to signify an ongoing process."There's something very compelling about this vision of learning as becoming. I wrote an article a while back about how blogging helps in the process of figuring out who you are becoming:
"This gap analysis shows the limitations of fixating on who I am – it’s only a first step in using reflection to understand the progression of identity. The next step is look for indications of who I am becoming and determine who I really want to be. This vector into the future creates an opportunity to design a style of living that is aligned with my values. In order to do that, I’ll need good models, support, and a fair bit of learning. If you’re still with me, you’ve probably already realized that the process of blogging helps achieve these goals, even if they are mostly subconscious."At some root level, this introduces the role of motivation in learning. There are probably people who are motivated by learning for its own sake, for the love of language or knowledge or information, but most of us need to know how we might apply what we are being asked to learn...or better yet, we might even get to choose what we learn based on what we want or need to know (and be able to do) to achieve our goals and get things done. You could argue that the industrial-age education system, with its focus on pre-defined curriculum and age-specific pacing, removes opportunities to develop the skill of determining what is most important to learn, finding the people and information required to learn it, and being able to apply what you've learned to something that matters to you.
My friend Brian has been exploring these concepts for years, and I've always loved his tagline: "How do we learn the things we value most?". In this context, I'm thinking about how social software might help learn to figure out which things we value most. As an aside, I've noticed that it seems like of the people in 43 Things who have completed learning goals, a very low percentage tend to caution people against pursuing them. I wonder if that's partly a reflection of a shared perception that pursuing all learning is valuable.
All this talk of becoming reminds me of Paul Graham's undelivered commencement speech. His general advice relating to high school and self-actualization it to treat school as a day job and spend as much time as you can pursuing interesting questions and projects (outside of school, mostly):
"And what's your real job supposed to be? Unless you're Mozart, your first task is to figure that out. What are the great things to work on? Where are the imaginative people? And most importantly, what are you interested in? The word 'aptitude' is misleading, because it implies something innate. The most powerful sort of aptitude is a consuming interest in some question, and such interests are often acquired tastes."Perhaps this is hinting at the real learning value of 43 Things -- helping people determine which goals are most worth pursuing. Which goals are most likely to help me become who I want to be? Who else shares my goals and what interesting things are they pursuing? Who has already pursued this goal and what was their experience? Once those questions are answered, the web and our networks make it much easier to actually achieve those learning goals, connecting us to the required resources, people and information when we need them.
Sunday, December 04, 2005
Online Learning Communities: PerlMonk
I had linked to this paper the other night, but didn't get to dig into it until tonight -- Self-Organizing Social Learning Through the Monastery Gates. It's an in-depth analysis of the informal learning taking place in PerlMonks, an online community formed around the programming language Perl. From the conclusion:
The paper connects the type of learning to constructivism:
If you type "Learn Perl" into Google, the related 43 Things goal page is one of the top results, and PerlMonk shows up at about result 350. Isn't that odd? The coverage of the topic on 43 Things is minimal, without much reference to the actual programming language, while PerlMonk has been around at least five years longer, has an active tech-savvy community, and covers the topic in unbelievable depth. Perhaps this says more about the Google ranking system than anything substantive about the relative value of the two sites, but it's still interesting to me.
There's something ephemeral about goals -- they can be whims or or indications of an interest that may never be followed up. The type of learning that takes place on 43 Things may be less focused teaching the content of the learning goal (like "Learning Perl"), and more focused on the process of learning about and deciding which goals you should actually pursue. Instead of answering the question, "How do I learn Perl?", it's helping answer questions like:
"PerlMonks is an example of a emergent, self-organizing system that promotes learning without overt management in a large, distributed community mediated by the global Internet. In what is essentially a self-directed learning effort, PerlMonks participants organize their own learning around authentic, real-world problems which they choose."I poked around on the PerlMonks site and indeed there is ample evidence of an incredible learning environment for techies who are immersed in Perl and related technologies. It appears to be total gibberish to anyone who is not -- this is a very focused and active community with a well-defined purpose. It does offer tutorials for beginners, but the level seems more advanced overall.
The paper connects the type of learning to constructivism:
"Seen from the perspective of learning system design, PerlMonks encompasses some useful constructivist conditions for learning. A few conditions evident within PerlMonks (many more are embedded) include: (a) authenticity, (b) real-time quality, (c) distributed knowledge creation, (d) social construction of knowledge, and (e) induction into a learning community through staged participation."This all got me thinking about how the same general topic is treated it 43 Things, where I found that 109 people want to learn Perl and 12 of them have written entries about the goal. Most of those are quick statements of intent, but others are more interesting. 50 other users have reported completing the goal and 18 of those have written posts either encouraging or discouraging the pursuit of the goal. Only one "expert" offered a link to an online resource -- their class outline and notes from an introductory course. Almost nobody received any comments on their entries except one pretty girl who got five in a group that seems to be mostly guys (coincidence?).
If you type "Learn Perl" into Google, the related 43 Things goal page is one of the top results, and PerlMonk shows up at about result 350. Isn't that odd? The coverage of the topic on 43 Things is minimal, without much reference to the actual programming language, while PerlMonk has been around at least five years longer, has an active tech-savvy community, and covers the topic in unbelievable depth. Perhaps this says more about the Google ranking system than anything substantive about the relative value of the two sites, but it's still interesting to me.
There's something ephemeral about goals -- they can be whims or or indications of an interest that may never be followed up. The type of learning that takes place on 43 Things may be less focused teaching the content of the learning goal (like "Learning Perl"), and more focused on the process of learning about and deciding which goals you should actually pursue. Instead of answering the question, "How do I learn Perl?", it's helping answer questions like:
- Why should I learn Perl?
- Who else is learning Pearl and what are they saying about it?
- What related programming languages might or should I be learning instead (or together with it)?
- If I was going to learn Perl, where would I start?
- Would others recommend that I pursue this goal?
Thursday, December 01, 2005
Model of Collaboration
One more before my brain explodes from reading. Recognising and promoting collaboration in an online asynchronous discussion sounds like the others I looked at tonight, but this one covers the creation of an analytical tool to classify the types of posts in an online discussion -- she calls it a model of collaboration. It's more complex than what I was envisioning for my analysis, but provides a great starting point for thinking about what exactly I'm trying to find out about how people are learning in 43 Things.
After looking at the six stages in Dr. Murphy's model and thinking about the types of interaction I've seen on 43 Things learning goals so far, I'd say that most of the interactions would fall under the "social presence" and "articulating individual perspectives" levels on the pyramid. I've seen evidence of collaboration that could be placed in the upper levels, and those are certainly the threads I'm most interested in...but they're not the norm. It sounds like they found the same trends in the online discussions they analyzed in the context of courses.
After looking at the six stages in Dr. Murphy's model and thinking about the types of interaction I've seen on 43 Things learning goals so far, I'd say that most of the interactions would fall under the "social presence" and "articulating individual perspectives" levels on the pyramid. I've seen evidence of collaboration that could be placed in the upper levels, and those are certainly the threads I'm most interested in...but they're not the norm. It sounds like they found the same trends in the online discussions they analyzed in the context of courses.
Assessing Discussion Contributions
Assessing Students' Contributions to Online Asynchronous Discussions in University-Level Courses
I found this paper most interesting because of the points of divergence from the types of learning and collaborating I'll be looking at. In 43 Things, there's no agenda, no curriculum, no requirements -- people just post what they're interested in doing and learning, or helping other people pursue goals they've already achieved themselves. In a university course, students have to basically be coerced into interacting in very specific ways and are assessed on their performance. I wanted to keep this list of requirements for posting in an online class, which are fairly typical for the courses I've taken:
This second article appears to be referencing the same study from a slightly different perspective: Students' self analysis of contributions to online asynchronous discussions. A student in the course goes through a very reflective process of analyzing her own participation.
And finally, this third one seems to be part of the same study, with a focus on challenges faced by participants in the study: Graduate Students' Experiences of Challenges in Online Asynchronous Discussions. The introduction has a great list of the benefits of online asynchronous discussions in the context of teaching and learning. Later, the most interesting section focuses on students reacting poorly to mandatory posting guidelines:
I found this paper most interesting because of the points of divergence from the types of learning and collaborating I'll be looking at. In 43 Things, there's no agenda, no curriculum, no requirements -- people just post what they're interested in doing and learning, or helping other people pursue goals they've already achieved themselves. In a university course, students have to basically be coerced into interacting in very specific ways and are assessed on their performance. I wanted to keep this list of requirements for posting in an online class, which are fairly typical for the courses I've taken:
- "The purpose of the discussion which counted for 20% of the students’ total course grade was to construct and share knowledge."
- "Each student had to make a minimum of two postings for each of the course’s four modules except in module one where they were required to make three postings."
- "Students were expected to limit the length of their postings to approximately 100 words."
- "In relation to the types of postings, for each module, students were required to make at least one original posting and one reply or follow-up to a posting made by another student."
This second article appears to be referencing the same study from a slightly different perspective: Students' self analysis of contributions to online asynchronous discussions. A student in the course goes through a very reflective process of analyzing her own participation.
And finally, this third one seems to be part of the same study, with a focus on challenges faced by participants in the study: Graduate Students' Experiences of Challenges in Online Asynchronous Discussions. The introduction has a great list of the benefits of online asynchronous discussions in the context of teaching and learning. Later, the most interesting section focuses on students reacting poorly to mandatory posting guidelines:
"Oliver and Shaw's (2003) investigation of strategies for encouraging student participation in discussions identified and explored factors that encouraged and inhibited student participation. In their analysis of the patterns of posting, the authors found that: 'students were "playing the game" of assessment, making the posting that earned them marks but rarely contributing otherwise' (p.64). The authors concluded as well that "contributions were not strongly interactive' (p.56)."
Manifest Content Analyis
My thesis supervisor has done some great work in the area of asynchronous learning discussions, so I'm digging into a few papers on the topic tonight.
Reading Between the Lines: Understanding the role of latent content in the analysis of online asynchronous discussions
This was my trigger for taking a content-analysis approach to the learning discussions surrounding goals in 43 Things, but as of right now, I'm not planning to study latent content. As interesting as it would be to survey participants about their motivations, impressions and experiences in using 43 Things, this may be outside of the scope of the project. I'm thinking that studying the manifest content (text, photos and links posted to the site) will be more than enough to chew on. That said, the design of this case-study research seems really solid. A few chunks I'd like to save for later:
Apparently there are lots of issues surrounding the reliability of the coding done in content analysis, many of which are covered in these articles that I only skimmed tonight...Sources of Difference in Reliability: Identifying Sources of Difference in Reliability in Content Analysis of Online Asynchronous Discussions and A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability.
Reading Between the Lines: Understanding the role of latent content in the analysis of online asynchronous discussions
This was my trigger for taking a content-analysis approach to the learning discussions surrounding goals in 43 Things, but as of right now, I'm not planning to study latent content. As interesting as it would be to survey participants about their motivations, impressions and experiences in using 43 Things, this may be outside of the scope of the project. I'm thinking that studying the manifest content (text, photos and links posted to the site) will be more than enough to chew on. That said, the design of this case-study research seems really solid. A few chunks I'd like to save for later:
"This distinction between manifest and latent content was highlighted in a general context of content analysis prior to the existence of online discussions. Berelson (1952) argued that content analysis should be limited to analysis of manifest content. Consistent with this perspective, he described content analysis as "a research technique for the objective, systematic, and quantitative description of the manifest content of communication" (p. 18). Content analysis proceeds in terms of "what-is-said", and not in terms of "why-the-content-is-like-that (e.g., 'motives') or how-people-react (e.g. 'appeals' or 'responses')" (p. 16). Hair, Anderson, Tatham, Ronald and Black (1995) argue, like Berelson, that content analysis should focus only on manifest content."These folks seem to think that content analysis on the manifest content alone is the way to go, which is encouraging. What I found a little disheartening was the rigour required to do the actual analysis, with four people each reading all of the content in the study and coding each unit using a classification instrument developed by Dr. Murphy:
"The transcripts of the discussion were grouped by a participant and coded by two independent coders against the nineteen indicators of behavior associated with PFR in the instrument using the paragraph as the unit of analysis. The transcripts were also coded a third time jointly by the two coders and the creator of the instrument and principal investigator. This third coding is used in this study to report aggregate results of engagement in PFR in the online discussion. Cohen's Kappa was used to calculate interrater reliability."And here I thought I was going to be doing some nice artsty-fartsy qualitative work...this looks more like hardcore statistics work, and who else (besides me) is going to go through thousands of posts on 43 Things to do this kind of coding? Ugh.
Apparently there are lots of issues surrounding the reliability of the coding done in content analysis, many of which are covered in these articles that I only skimmed tonight...Sources of Difference in Reliability: Identifying Sources of Difference in Reliability in Content Analysis of Online Asynchronous Discussions and A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability.
Subscribe to:
Posts (Atom)