Featured Member

mackzoore sb
mackzoore sb
(Santa Barbara - United States)

I currently teach middle school science and math at the Laguna Blanca school in scenic Santa Barbara. I previously taught at the American Nicaraguan School in Managua, where I developed curriculum and taught physics, conceptual physics and earth ...

Tips for Resources

Tip: Join Curriki so you can edit, build up, organize, copy, comment on, rate, flag, and nominate resources.

Curriki Review System Collection

Overview of the Curriki Review System

Educational content on Curriki receives ongoing review, with the goals of:

  1. Helping members find high quality learning resources within the Curriki database; and
  2. Helping contributors improve the learning resources that they are building
To achieve these goals, the site features and utilizes several different forms of review:

  • File Check: A Curriki staff member is responsible for reviewing the site and deleting any content that is patently offensive, harmful (viruses), not educationally focused, or otherwise objectionable (vandalism).
  • Member Comments: Curriki members may post comments on any public learning resource they access on the site.
  • Curriki Review System (CRS): Learning resources are reviewed and rated by expert staff members and trained volunteers on grounds of completeness, accuracy, and pedagogy. The remainder of this section details the Curriki Review System.
Ratings in the Curriki Review System

The Curriki Review System assigns resources one of five possible overall scores, as shown in the table below. Small resources that form part of complex resources often cannot be evaluated in isolation and are considered "Not Rateable."

Content that displays "Partner" has been provided by Curriki's partners and has already been field-tested or shown to rest on a strong research base.

All other resources will be rated on a scale from 1-3, as below.

IconTerm
Definition
 Not Rated/Not Rateable
A learning resource that has not yet been rated or that cannot be rated (e.g. an image that is part of a larger, composite resource and is too small to be rated by itself)
Overview/crs1.gif" />BasicA resource that requires a significant amount of cleanup or expansion to be used for teaching and learning
 GoodA resource that will be immediately useful to teachers and learners, but which has gaps or lacks polish
Overview/crs3.gif" />ExemplaryA complete and highly polished resource of publishable quality that uses best practices in instructional design
 PartnerHigh-quality content provided by partner organizations
The ratings above are displayed prominently when viewing a resource in the main site. A reviewed resource’s Comments tab displays details of the review that explain the rating given. Ratings are also displayed in search results, and users can filter or sort search results by CRS rating.
 

Rating Sub-Scores

For every resource that has been reviewed, the date of its last review is displayed. If the review is recent, it should be very trustworthy. If the review is old, you may want to investigate further. Check the Revision History from the Information tab for that resource. Has the resource been edited since the last review? How many times? For a resource that has changed significantly, the review will be a clue to quality, but you should also examine the resource itself. You may even want to nominate it for a new review.

A resource’s overall CRS rating 1-3 is derived from its subscores in three areas:

  • Technical Completeness – Are all necessary parts of the resource provided? Are activities fully and clearly explained? Has adequate metadata been applied (educational level, subject, ICT, etc.)?
  • Content Accuracy – Is the subject matter presented consistent with expert consensus in the field? Does a spot-check reveal any errors in assessment items or answer keys?
  • Appropriate Pedagogy – Are the learning activities well-chosen for the stated objectives, subject, and educational level? Does the learning resource use appropriate vocabulary and sequencing, clear explanations, and recognized best practices in instructional design? Are assessments carefully aligned to objectives and learning activities?
Reviewers rate a resource from 1-3 in each of these three areas, applying a rubric developed by Curriki and extended over time by Curriki reviewers and interested members of the Curriki community. The resource is then assigned an overall score equal to the average of its subscores. Content creators whose resources fall short in one area may work to improve their resource in the deficient area in order to meet the next threshold.

Reviewers

In each of the four core subject areas, Curriki has hired a master teacher or instructional designer to serve as Lead Reviewer. They conduct reviews as well as organize the work of volunteer reviewers, who are being recruited from pools of experienced teachers, curriculum developers, and teacher educators. Curriki is always seeking volunteer reviewers. If you would like to join the effort, email review@curriki.org and tell us about your relevant interest and your area of expertise.

Process

All members of Curriki can nominate resources for review—sometimes because they are suspect or worthy of recognition, other times because they have changed significantly since their last review. Reviewers in each subject work through a queue of nominated resources, usually in the order in which they were nominated. When a resource has been nominated, but not yet reviewed, the 'Nominate' link are replaced with the words 'Review Pending.' When the queue is empty, Curriki reviewers browse the site for resources to review. For each resource, reviewers submit both a numerical rating and textual feedback.


 

More information about the criteria used by the reviewers and other details are available in the Curriki Review System Collection. You may also want to watch the video tutorial "What is a Reviewed Resource?"

Introduction to CRS Criteria

CRS criteria exist at three levels of specificity:

  • Overall criteria - Very short statements of what an overall rating of 1, 2, 3, or Not Ratable means
  • Subscore criteria - Short statements of what 1, 2, 3, and Not Rateable mean in each of the subscore areas (technical completeness, content accuracy, and appropriate pedagogy)
  • Detailed criteria - Lengthy lists, given where necessary, that extrapolate from the subscore criteria to each Instructional Component Type (ICT), subject area, or level. For example, detailed criteria tell what is expected from a particular type of resource, such as a lesson plan, to earn a 3 for technical completeness
Each set of criteria can be found in this collection, below.  Reviewers should update this document as they encounter cases that are not already explained. Other Curriki members are encouraged to leave comments suggesting additions or changes.

Principles of Interpretation for CRS Criteria

1) The spirit of the general criteria statements should guide the writing of more detailed criteria. The point of the more detailed criteria is to ensure consistency across reviewers, not to create unnecessary hoops for content creators to jump through. If they are unsuccessful, the detailed criteria may be revised or extended by the community of reviewers and the Lead Reviewers in each subject area, at the final discretion of Curriki’s Chief Academic Officer.

2) One challenge for CRS is to define pedagogy criteria with enough specificity to identify exemplary resources, and yet with enough generality to allow for a range of pedagogical approaches, both existing and innovative. We attempt to negotiate this in several ways:

  • When possible, reviewers should evaluate resources on their own terms. Is the pedagogy well chosen to meet the stated objectives? For the stated age group?
  • Pedagogy criteria have been written to require certain elements of good pedagogy that should be relatively uncontroversial (vocabulary, sequencing, alignment w/ assessments), and on less universal matters, to allow content creators to choose from a list of best practices. The list of best practices may be extended as necessary
  • CRS reviews are posted anonymously and should represent a consistent application of the review criteria, with a little leeway for editorializing in the textual reviews, but not much. However, reviewers who would like to leave more personal, idiosyncratic, or opinionated thoughts are encouraged to post them under their own name, as comments.

Overall Criteria

This document presents the overall criteria to be used when rating resources in the Curriki Review System. It includes a very simple statement of the what each of the rating values means.  The Subscore and Detailed criteria, presented below in this collection, are all extrapolations from these simple statements.

3 Exemplary

A complete and highly polished resource of publishable quality.

2 Good

A resource that will be immediately useful to teachers but has gaps or lacks polish.

1 Basic

A resource that requires a significant amount of cleanup or expansion to be used in a classroom.

- Not Rateable

A resource of small scope that is part of a larger resource, or is otherwise too small to be rated by itself

Criteria for Curriculum

Criteria for CURRICULUM 

Full Course, Unit, Lesson Plan, Scope and Sequence, Standards, Syllabus 

Technical Completeness:

The technical completeness score is determined by how easy it is for the community to locate this resource on Curriki.  The appropriate metadata needs to be set which includes:  educational level, instructional component type (ICT), keywords, subject area, and an accurate, error-free description of the resource. 

Exemplary: 3

The resource has been given complete information/metadata.  The description accurately defines the content submitted. Minor adjustments may have been made by the CRT. 

Good: 2

The resource has most of the information/metadata correct; however, the metadata does not accurately reflect the ICT, education level, subject area and language. Keywords do not reflect content. The description has many errors and/or does not adequately describe the resource. Significant adjustments need to be made by the user. 

Basic: 1

The resource is missing significant metadata or metadata is inaccurate or does not reflect the content. Major adjustments need to be made before used in a classroom. 


 

Content Accuracy:

The Content Accuracy score is determined by the correctness and thoroughness of the resource. Alignment between the content and the targeted grade level is appropriate. The resource clearly identifies its bias toward a particular perspective, if one exists. If work sheets and problem sets are included, answer keys are essential. 

Exemplary: 3

Content is accurate, thorough, and may include sources or links to other relevant material. Formatting is complete, spelling is correct and content is general enough to be useful in a variety of classrooms. Content is appropriate for the targeted grade level, is consistent with expert consensus in the field, and if there is bias, it is addressed. A spot check of questions and answer keys reveals no errors. 

Good: 2

Content is mostly accurate and thorough. There may be minor formatting errors, misspellings, or specific references to a particular teacher's classroom; however, it can be used with minor modifications. Content presented is consistent with expert consensus, but some explanations are ambiguous or incomplete. Content does not adequately address a bias. A spot check of questions and answer keys reveals an occasional, easily correctable error in worksheets and problem sets. 

Basic: 1

The content is inaccurate or incomplete. There may be major formatting and spelling errors or specific references to a particular teacher’s classroom. Content does not represent expert consensus or presents a strong bias. Content may not be appropriate for the targeted grade level. A spot check of questions and answer keys reveals many errors, or there is no answer key. 


 

Pedagogy:

The pedagogy score is determined by the variety of strategies, techniques and approaches used to teach all type of learners. Resources need: clear objectives and directions; a list of required materials; and graphs, charts or worksheets to enhance the content, if appropriate. Scaffolding of material is encouraged to support higher order thinking. Activities allow students to actively engage in the material. Ideas for further extensions or modifications may be included. An evaluation tool or assessment is provided. 
 
Exemplary: 3

Multiple techniques or strategies are used to enhance the content. Directions and objectives are clear. Many suggestions for modifications and differentiation are included. An evaluation tool or assessment is provided which aligns itself to the content. 

Good: 2

Some different techniques or strategies are explicit, but may not be developed. Directions and objectives may be vague or not easily adaptable to any classroom. Some suggestions for modifications and differentiation are included. An evaluation tool is included, but it is not aligned, or is not sufficiently organized. 

Basic: 1

Only one strategy/technique is used. Directions and objectives are confusing or not included. Resource only requires basic recall of information. No modifications or extensions of content are present. No evaluation tool is included and mistakes, formatting, and organization make it difficult to use the resource.

Criteria for Activity

Criteria for ACTIVITY

Exercise, Graphic Organizer/Worksheet, Experiment/Lab, Problem Set

Technical Completeness:

The technical completeness score is determined by how easy it is for the community to locate this resource on Curriki. The appropriate metadata needs to be set which includes: educational level, instructional component type (ICT), keywords, subject area, and an accurate, error-free description of the resource.

Exemplary: 3

The resource has been given complete information/metadata. The description accurately defines the content submitted. Minor adjustments may have been made by the CRT.

Good: 2

The resource has most of the information/metadata correct; however, metadata does not accurately reflect the ICT, education level, subject area and language. Keywords do not reflect content. The description has many errors and/or does not adequately describe the resource. Significant adjustments need to be made by the user.

Basic: 1

The resource is missing significant metadata or metadata is inaccurate or does not reflect the content. Major adjustments need to be made before used in a classroom.


 

Content Accuracy:

The content accuracy score for an activity is determined by how easily it can be utilized in the classroom. The activity is appropriate for the targeted grade level. If directions are needed to use this resource, then they must be included. The activity is used to help students further understand a more complex idea.

Excellent: 3

This activity is easily used in a classroom. Directions are clear and needed in order to use the activity. Students will gain a better understanding of a more complex idea by participating in this activity.

Good: 2

The activity can be used in the classroom, but teachers would need to make some minor adjustments. The activity could be used at the targeted grade, but may need some modifications. Directions exist but may not be clear. There are some benefits of using this activity in order to better understand a more complex idea.

Basic 1:

The activity is not useful as it stands.


 

Pedagogy:

An ICT type labeled activity does not qualify for a pedagogy score. It will be rated N/A.

Criteria for Asset

Criteria for ASSET

Index List, Presentation/Slide Show, Reference Collection, Script/Transcript,

Pictures, Videos, Animation/Simulation,

Table, Graph, Chart, Book, Weblinks, Test/Assessment

Technical Completeness:

The technical completeness score is determined by how easy it is for the community to locate this resource on Curriki. The appropriate metadata needs to be set which includes: educational level, instructional component type (ICT), keywords, subject area, and an accurate, error-free description of the resource.

Exemplary: 3

The resource has been given complete information/metadata. The description accurately defines the content submitted. Minor adjustments may have been made by the CRT.

Good:2

The resource has most of the information/metadata correct; however, metadata does not accurately reflect the ICT, education level, subject area and language. Keywords do not reflect content. The description has many errors and/or does not adequately describe the resource. Significant adjustments need to be made by the user.

Basic: 1

The resource is missing significant metadata or metadata is inaccurate or does not reflect the content. Major adjustments need to be made before used in a classroom.


 

Content Accuracy:

The content accuracy score is determined by how easily the resource could be integrated into curriculum on a relevant topic. Bias must be identified, if it exists. The resource needs to be appropriate for the targeted grade level. The resource should help s a student to further understand the relevant topic. If necessary, outside sources must be referenced.

Exemplary: 3

The resource could be easily integrated into relevant curriculum. Any bias is identified. The resource is appropriate for the targeted grade level. The resource helps a student further understand the topic. Outside sources are referenced.

Good: 2

The resource is usable, however, teacher would need to put in time figuring out how to integrate it into the curriculum. Bias is apparent, but is not clearly identified. Resource could be used at grade level but requires some modification. There are some benefits to using the resource to help students understand the topic. References are noted but may be incomplete.

Basic: 1

The resource is not usable as it stands.


 

Pedagogy:

An ICT type labeled Asset does not qualify for a pedagogy score. It will be rated N/A.

Understanding CRS * Ratings

If you find a resource that shows a CRS rating where the icon includes an asterix (*) or the text reads "Part of a [Rating Term] Resource," this resource has not been fully reviewed and rated by a CRS reviewer.

However, this resource is part of a folder or collection that did receive a full review and rating by a CRS reviewer. In other words, resources with CRS * ratings form the contents of, or are sub-resources within, fully reviewed resources.

Rating
 Description
 crs3star.gif Part of an Exemplary Resource
 crs2star.gif Part of a Good Resource
 crs1star.gif Part of a Basic Resource
If you find a resource with a * rating, scroll down to review the comment about the resource, submitted by a Curriki editorial staffer. This comment will provide a hyperlink to or URL for the fully reviewed "parent" collection or folder for the resource that received the * rating.