Software for mobile assessment, in and out of the classroom.

Archive for the ‘Assessment’ Category

Why are Badges for School-Age Learners Different?

LearningBeyondLetterGrades_Canvas

I just finished watching the video of the fantastic second live session for the MOOC Learning Beyond Letter Grades.  The panelists Sheryl GrantJonathan Finkelstein and Sunny Lee present an excellent introduction to Open Badges and the reasons that we at Forall Systems are so enthusiastic about the Open Badge movement.  The panel got me thinking about some of the ways that badges are different for school-age learners.

For most learners, the main focus of Open Badges is to “Get recognition for skills you learn anywhere.”  Inherent in the idea of getting recognition is the assumption that badges can open up opportunities and the viewer/evaluator of a badge will be a potential employer, a university or colleagues.  In developing ForAllBadges, we have been interested in thinking about badges for kids as a tool for supporting learning.  In a perfect world, learning opportunities are not a limited resource and children don’t have to compete for access.

During the live session, I jotted down some of the ways that thinking about badges as a tool for learning changes how badges are designed, issued and displayed.

  • The most important evaluator of badges for learning is the learner themselves.  Badges can provide the framework for learners to reflect on their learning experiences and plan for future learning activities. Teachers and parents can use the badges to help support learning, but ideally the learner takes ownership of their own learning process.
  • Authentic assessment and the evidence associated with the badge is often more important than the badge itself.  A video of a science fair presentation or an essay often has more value than the badge or evaluation rubric associated with it.
  • Badges for learning are inherently lifelong.  For a seven year old, their third place finish in the 2nd grade science fair at their elementary school is an accomplishment to be proud of.  Ten years later, when they’re applying to university, the badge itself isn’t worth much.  However the evidence associated with the badge provides an opportunity for the young adult, in writing their application essay, to reflect on their lifelong interest in ecology and include a quote or video from their 7 year old self.  In many ways, these authentic assessments become more valuable as life goes on.
  • Badges can provide the structure for effectively reflecting on progress.  Fundamental skills are often a lifelong pursuit.  For example, writing well is something that all of us can always improve on.  By presenting a timeline view of badges that are aligned with the Common Core Anchor Standard CCSS.ELA-Literacy.CCRA.W.1, learners can reflect on their progress over time in learning to write strong arguments.  The badge and the standards alignment provide the structure to display the authentic writing samples in a meaningful way.
  • Badges can provide the structure for discovering learning opportunities.  A badge displayer such as Badgopolis can provide a view of learning opportunities that relate to the learners previous interests and also present information about how to access those opportunities.

Of course this approach of designing badges for learning is not only applicable to school-aged learners and is a perspective that can also be valuable in other circumstances. If you’ve gotten this far, thanks for reading my thoughts!  Please share your you ideas too.  This is a  fascinating topic and is only just starting to be explored.

Badges How to: Using Your Classroom Rubrics to Design a Badge System

Recently teachers and administrators have been asking us for ideas on how to go about implementing badges in their own schools.  In this post, I’m going to describe two approaches for how to take existing classroom rubrics and design complimentary badge systems.  This approach has several advantages:

  • a significant part of the work necessary for designing the badge system has already been done in developing the rubric,
  • teachers can continue to assess students with a proven rubric and
  • the badge system integrates well with the existing learning activities.

Approach 1:  Recognize Skills and Achievements

A primary motivation for Mozilla’s development of Open Badges was in order to recognize skills and achievements that aren’t traditionally recognized.

openbadges

Teachers in areas such as Physical Education, Technology, Special Education and Art are already seeing the value of badges for recognizing their students’ achievements.  This approach is to creating a badge system is demonstrated by Warren Grieve in his ICT Ladders for Badges.  Below is an example rubric he previously developed for evaluating skills with MIT’s Scratch programming language.

Scratch-Rubric

Below are the complementary badges that he designed for ICT Ladders for Badges that correspond to each of the levels of achievement in the rubric.  The criteria for earning a badge is the same as the criteria specified in the original rubric.

basic_scratch_skills_badgeproficient_scratch_skills_badgeadvanced_scratch_skills_badgeexpert_scratch_skills_badge

Like with girl scout badges, this type of badge system provides students a way to:

  • set goals by identifying badges to work towards,
  • reflect on the goals they’ve already reached and
  • share their accomplishments with others.

Approach 2:  Scaffold Learning

A common use for rubrics in the classroom is for supporting student learning and development.  This kind of instructional rubric is typically written in student-friendly language to help students evaluate their own work and improve overtime.  This use of rubrics is well demonstrated by the set of rubrics provided by the Illinois State Board of Education (ISBE) to support student learning for effectively solving math word problems.  This set of rubrics includes a scoring rubric for use by the teacher and three age appropriate versions of the rubric for use by students.

The  rubric below is the scoring rubric for use by elementary school teachers to inform instruction.  (click on the rubric for a larger image)

Scoring-Rubric

The  rubric below is the instructional rubric for use by 3rd and 4th grade students to support learning.  (click on the rubric for a larger image)

Student-Rubric

The process of learning how to solve math word problems is an ongoing learning activity and requires a significant amount of practice and feedback.  The student friendly rubric is effective at making it clear to students what the expectations are for this learning activity.

In our design for a set of badges that correspond to ISBE’s student friendly rubric, the first change we made was to convert the “Items for Evaluation”  in the original rubric to “Roles to Achieve” in the badge system as follows:

  • Mathematical Knowledge (Do you know it?) —> Math Wizard
  • Strategic Knowledge (How do you plan?) —> Master Detective
  • Explanation (Can you explain it?) —> Expert Explainer

The second change we made was to eliminate the score of zero.  By its nature, badge criteria is positive.  Although it makes sense to award a score of zero if the student doesn’t attempt an answer.  It doesn’t make as much sense to award a badge for doing nothing.

Below is what our initial badges look like.

Student_Badges

The last change we made was to convert the scoring guidelines to badge criteria by removing negative scoring guidelines, since the negative guidelines are implicitly included in the leveled badges.  Below is the the criteria for Math Wizard badges including the struck out text from the original Grade 3-4 Student Rubric for Mathematical Knowledge.

Level 4 Badge Criteria (the same as for scoring rubric)

  • I get the right answer.
  • I label my answer correctly.
  • I use the right math words to show I understand how math works. (Example: I know when to add or subtract.)
  • I work it out with no mistakes.

Level 3 Badge Criteria (almost the same as for scoring rubric)

  • I do the problem, but I and only make small mistakes.

Level 2 Badge Criteria 

  • I understand a little, but I make a lot of big mistakes.
  • I only give part of the answer.

Level 1 Badge Criteria

  • I try to do the problem, but I don’t understand it.

Level 0 – Badge Criteria (no such badge)

  • I don’t try to answer the problem.

With the development of our badge system, we now have three different views of the assessment that all work well together:

  • the teacher scoring rubric for assessing the student’s progress,
  • the student instructional rubric for understanding the expectations and
  • the student badge system for visualizing the learning path.

ForAllRubrics + Badges

We’re in the process of developing a new version of ForAllRubrics that supports both Rubrics and Badges.  This new version will support a badge design process like what has been described in this post.  It will let teachers complete scoring rubrics and the system will automatically generate the appropriate badges for the students.  The students (and their parents) will be able to view the student friendly rubric to understand the scoring on a particular assignment.  The teacher, the student and their parents can view the student’s awarded badges to reflect on the student’s accomplishments to date and view potential badges for setting goals for the future.

If you’re interested helping us test out our new “ForAllRubrics + Badges” and giving us feedback, please send us an email to info@forallsystems.com and we’ll get you set up.  We greatly appreciate comments and suggestions.

ForAllBadges for Hackasaurus

Hackasaurus Supergirl

As a pilot test of ForAllBadges we’ve implemented support for issuing Hackasaurus badges. For folks not already familiar with Hackasaurus, it is a fantastic open source project developed by Mozilla that is part of their effort to build a generation of webmakers. A central component of their project are events called Hack jams that “make hacking and digital literacy accessible, social and fun.” It is a really nice first pilot for us, since it has a small and well-defined badge system included as part of their Hacktivity kit (much thanks to Daniel Hickey for directing me to Hackasaurus as a great resource for experimenting with badges).

As part of a Hack jam participants are able to collect paper badges. As described in the Hacktivity Kit:

At the end of a module, participants will “pledge” or apply for badges to gain specific super powers. They must successfully complete the pledged task in front of a peer mentor or facilitator who will award them the badge to show they’ve earned their superpower.

Our goal is to implement digital badges in a way that enhances the Hack jam experience and also integrates with Mozilla’s Open Badge Infrastructure (OBI). Integrating technology into a live interactive experience so that it doesn’t interfere with the fun, is a lot harder than you might think at first. So we’re hoping to get feedback on our design and improve the implementation over time.

Our implementation of ForAllBadges for Hackasaurus has two components:

  • A mobile app for use by facilitators and peer mentors at the Hack jam while wandering around and interacting with participants. We’re thinking that a BYOT (“Bring-your-own-technology”) approach will be most practical. Currently we support iOS devices (iPhones, iPod Touches and iPads) but we are planning to add support for more platforms.
  • A browser based application for use by participants, facilitators, peer mentors and organizers for managing badges and participants, including publishing badges to the OBI Backpack.

Screenshots of Mobile App

View Participants

Add Participant with Photo

Choose Badge to Award

Add Evidence and Comments

Issue a Custom Badge with Custom Badge Details

Screenshots of Website

For now we have defined three roles for the website:

Participants can view the “Badge Board” with the badges that have been issued to all the participants. They can view the details of badges and send their own badges to their Mozilla Badge Backpack.

Facilitators can view and issue participants’ badges. They can also administer participant information.

Administrators can do every thing a facilitator can do and also can manage information about events, facilitators and participants.

Our Implementation
Both the website and the mobile app are written using HTML5. The mobile app uses the open source PhoneGap platform. The website uses Python/Django/Javascript. For now the mobile app only works online and you need internet access. With our ForAllSchools platform we can provide offline access too, but for a Hack jam it seems better to keep all the information synchronized and internet access has to be available for the Hack jam anyhow.

Feedback

We’re posting this information because we’re interested in getting feedback. Please let us know your thoughts! Also if you’re organizing a Hackasaurus event and want to try out our software, just let us know and we’ll get you set up.

On Badges: Why Evidence Is More Important Than Credentials

As John Seely Brown pointed out in his keynote at DML 2012, the lifetime of skills is getting shorter all the time. Rather than just recognizing skills, digital badges create new opportunities to come up with creative ways to support learners in reflecting on their learning experiences and planning new ones. Skills will become obsolete, but learning experiences will continue to be valuable. Digital badges can provide an open resource for collecting and sharing authentic work samples that demonstrate to employers the experiences and qualities that make an applicant a great hire. An ePortfolio for an engineering job that includes a video of the applicant at 14 years old presenting their science fair project could say much more than 1,000 words.

In the current version of the Mozilla Open Badge Infrastructure (OBI), the badge issuer is given full responsibility for hosting and managing: (1) the description of the badge and criteria for earning the badge and (2) the evidence with information about how the specific user earned the badge. I am concerned that this approach is going to weaken the usefulness of badges for the following reasons:

  • Ownership. Depending on the technology the issuer chooses to display the requirements and evidence, learners will not be able to save their assessment information for themselves. Learners should have the ability to retrieve and use their awarded badges along with the supporting information.
  • Privacy. At present, the evidence for an awarded badge is referenced by a URL and by default not secured. The release of learner assessment information should be under the control of the learner.
  • Persistence. If badges are going to be useful for supporting lifelong learning, then badge information needs to be available for a lifetime. Websites, companies and funding come and go. It is essential that all the information for awarded badges be managed by the learner and not by the issuer.
  • Validity. Over time, badge definitions will be updated and requirements could change. The URL in an awarded badge may not point to the same information that it did when the badge was issued. Similarly the evidence link might not always point to correct information.
  • Expiration. Once an awarded badge expires, how long is an issuer expected to keep badge information available? That video of the 14 year old presenting their science fair project is useful, even if the awarded badge has expired.
  • Open Display. Depending on how the requirements and evidence are displayed by the issuer, it may not be retrievable by displayers. The criteria and evidence for an awarded badge are important for display tools such as ePortfolios. The badge ecosystem should allow the software for displaying learner’s badges to be creative and invent new ways of sharing badge information that haven’t even been previously imagined.
  • Obsolete Data Formats. Supported data formats for digital media such as images and videos change over time and most all data formats will eventually become obsolete. The learner should be able to convert the requirements and evidence to well supported data formats.

Here are a some possible approaches to address these problems that I would like to offer to the discussion about the implementation of Open Badges:

  • In addition to issuers, learners and displayers, add a new role of “hosts”. Badge hosting services can play an important role in allowing Learners to choose where they want to host their badges. They shouldn’t be required to host with the issuer of the award. It also makes it less burdensome for issuers to award badges.
  • Define a standard export/import format for badges so that users can download and transfer their badges information between issuers and other badge hosting services. It seems like the technical issues are closely related to the issue of transferring blogs between blog hosting services.
  • Require or strongly suggest that badge requirements and evidence be tagged with Schema.org or LRMI metadata so that they can be accessed by displayers for creating ePortfolios and other reports.
  • Let the badge issuers be responsible for the authentication of the credentials awarded to a learner. The badge itself should just contain the URL where the credentials can be authenticated. With this approach, if for example it is discovered that the learner cheated or didn’t properly fulfill the requirements for the badge, then the issuer has the power to revoke the award.

The purpose of badges should be to support learners. As much as is reasonable, the technology infrastructure should assist in realizing that goal.

Using iPads for Meaningful Assessment at Burley Elementary


Great blog post by Carolyn Skiba at Burley Elementary on classroom assessment with iPads. Lots of useful resources and practical suggestions.

Follow

Get every new post delivered to your Inbox.