What Makes an Effective Curriculum?

Curriculum has the potential to have  a dramatic effect on student achievement and teacher quality. When teachers thoroughly learn to use a highly effective curriculum, they learn about how to teach a particular subject and have ready-made lessons and materials organized in a developmental sequence that promotes student learning.

How can you tell what makes a good curriculum? It’s not simple. People looking for simple answers will base decisions on the most superficial things: cover design, greater number of pages, special features, or that they like the sales person. But people looking to implement a quality curriculum have work to do.

First of all if it is going to improve student achievement, a quality curriculum will require changes, changes that faculty and administrators may find uncomfortable. But unless changes are made, how can anyone expect any changes in student achievement?

So, what kinds of changes?

1. Content. Content in a quality curriculum is more thorough and deeper. Instead of mentioning topics, concepts in history, math, or science are developed and explored.

2. Concept and Skill Development. Concepts and Skills are developed along student learning trajectories. One concept builds on the previous one without abrupt changes that leave students perplexed.

3. Lesson Plans. The introduction, development, guided practice, practice, and assessment of each concept and skill is rich, practical, and engages the majority of students at their level of development. Lessons are not overly complicated or too simple. They build on previous lessons and lay the foundation for subsequent lessons. They don’t introduce new skills or concepts out of the blue or drop them without sufficient development and practice.

4. Research Based Teaching Methods. The teaching methods in an effective curriculum are based in educational research or a history of effective practice. Educational research is not comprehensive, but it does provide guidance for curriculum developers. There is research about how to teach reading, beginning with phonemic awareness and developing the alphabetic principle and phonics. There is research that identifies the most effective ways to teach math facts. There is research that identifies the most effective ways to teach spelling. There is no reason teachers have to discover these methods through trial and error and implement them on their own. A quality curriculum would incorporate research based practices into the daily lessons.

5. Student Population. An effective curriculum meets the needs of a given student population. If you have a population of struggling learners, the curriculum concentrates on developing foundational skills and concepts. If you have a high achieving population, the curriculum challenges students with advanced concepts and extended learning opportunities. No one curriculum will be appropriate for all learners at all levels.

6. Standards. An effective curriculum meets the state or national standards in a comprehensive way. A standard, such as the grade 4 Common Core reading standard (“Determine the main idea of a text and explain how it is supported by key details; summarize the text.” ) is not just addressed in one lesson. It is comprehensively developed throughout a grade level.

Determining whether a program meets standards qualitatively, is difficult. Most new educational materials will reflect the new Common Core standards in English Language Arts and Mathematics. But most curriculum developers will simply relabel existing curriculum with the Common Core standards and fill in any gaps. Yes, they will look as if and claim to “meet” the Common Core standards. But the curriculum will not be developed based on the Common Core.

To do this would require that curriculum developers start with each standard at each grade and lay out a series of lessons so that students build understanding throughout the year. These lessons would be interwoven so that the different strands of the subject area build together.  The materials would clearly reflect each standard and how it is developed and assessed, not isolated mentions of the standard.

This would most likely require that a developer would completely revamp an existing curriculum that was not based on the Common Core, which requires a significant investment…  or would start anew, which requires an enormous investment. Educational publishers will not make that investment unless they absolutely have to because customers are demanding it. And customers won’t demand it if they don’t know what they’re looking for or think curriculum doesn’t really matter. Without a real change in curriculum, we can’t expect the Common Core or any reforms to have any substantive effect on educational practices, teaching quality, or student achievement.

Quantifying Education

So much of school reform has come about from the desire to quantify everything and then use data to improve decision-making. “Simply” measure student performance and then “simply” assess school and teacher performance on this hard data. It looks so good on paper. On a very superficial level it sounds as if the quantifiers are helping schools run like a business, and that’s supposed to be a good thing. A business can be judged successful or unsuccessful based on its bottom line. There is no question that this can be helpful information for schools, but it’s just one piece of the puzzle.

Even in business there are different definitions of success and failure and these can be manipulated by skilled accountants and different perspectives on the numbers. Successful long-term companies are failures if they are not meeting the profit expectations of their stockholders, or in education terms, rising test scores. Short-term fixes that are not sustainable, even unethical fixes, can be temporarily rewarded because they provide short-term profits, like changing students’ standardized test scores. Drastically cutting costs with staff layoffs, decreased production, offshoring labor, or using cheaper materials can improve a balance sheet but have the potential to destroy a business in the long run. When the stakes are high, some are willing to go to extraordinary lengths to make the numbers look good in business or in education.

The problem in education is that so much of the good work that schools do  is not quantifiable.  Love of learning, personal student achievement, pursuing interests, and actual learning isn’t easily measurable. You can go to any high school graduation and see these unquantifiable measures. There you will see the total joy expressed by parents, teachers, and students for having “made it,” even if some just barely made it. That joy has nothing to do with standardized test scores.

Teachers judge their own teaching performance on how much students are engaged and how much of an effect they have on their students. Teachers are motivated by overhearing students talk about a lesson at recess or in between classes, or coming in the next day with new insights because they’ve been thinking about it. There is no greater reward that having students come back in subsequent years and express how much a teacher meant to them. A student–who becomes a teacher because of a teacher;  is inspired because of a teacher’s support, encouragement, or belief in the student’s abilities; has developed a love or reading, or history,or science–this is what provides the greatest rewards for teachers. None of these is quantifiable.

Daniel Pink in his book, Drive, offers these three motivations for everyone, including students and teachers: 1. the need to achieve autonomy, to be able to something on your own; 2. the desire to be good at something that matters to you, and 3. purpose to make a contribution to something greater and more enduring than yourself. These motivate you to work hard, engage in tedious practice, go beyond your comfort zone, and be willing to endure even painful and difficult experiences to achieve. All three are much more powerful than pay, winning, or even being rewarded, which are all transitory. Learning to play the piano well, or memorizing Poe’s “Annabel Lee” is an enduring achievement that can’t be taken away. Motivation is not quantifiable.

In addition to quantifying student test scores, the focus of educational reform efforts should also be related to the true motivating factors for students and teachers, providing increased opportunity to develop autonomy, mastery, and purpose. These efforts would necessarily focus on improving curriculum, methods, and teacher competence, and meeting the needs of individual students.

Text Complexity

The Common Core State Standards for English Language Arts & Literacy place a major emphasis on developing students’ ability to read increasingly complex text as they progress through school. The Standards cite research that shows that although the reading demands of college, careers, and citizenship have held steady or increased, the demands of reading in K-12 educational materials have trended downwards with “precipitous declines (relative to the period from 1946 to 1962) in average sentence length and vocabulary level in reading textbooks for a variety of grades–a general, steady decline–over time, across grades in the difficulty and likely also the sophistication of content of the texts students have been asked to read in school since 1962.” (CCSS Appendix A)

This is really simple to explain–the free market! For-profit educational materials do not lead the market, they respond to customer concerns and try to meet consumer demands. This is the reason for the behemoth textbooks and programs that students lug around today. The simple fact is that the best selling educational materials were those that had the most pages in the 1980s and 1990s. This started a textbook poundage war. If a competitor had a 750 page book, the next edition must be 800 pages. Because curriculum quality is otherwise difficult to determine, customers adopted an impression that more pages and more components must make for a better program. At the very least, a bigger program would have more options from which to pick and choose. So many educators think that published curriculum is all the same anyway, so bigger is better. The textbooks of today are typically twice as big as the ones in the 1950s, and have thousands more components.

The same mechanism was at work in the changes in text complexity. Teachers  complained that their students couldn’t read text materials, not only in reading programs, but in all content areas–science and social studies. Even paragraphs of explanation in math were too difficult for students. In response, publishers increased type sizes, added pictures with captions that delivered much of the content, and reduced the complexity of the text. Teachers themselves didn’t want explanations of strategies or detailed instruction in teachers editions–just bulleted lists and and answer keys. Programs with less text complexity were the most popular and were the best sellers, but many students still couldn’t or wouldn’t read them. Publishers moved toward including “verbo-visual” teaching elements and with the increasing use of technology have complete audio recordings of the texts and animated math and science explanations and video links in science and social studies.

Instead of teaching students to read more complex text, teachers bought textbooks that were the easiest to read. Educational publishers responded with increasingly less complex text. There are some materials today that are basically all captioned photographs to explain science and social studies concepts. These are, in fact, more difficult to read because they do not have a narrative sequence or the development of an argument or explanation. Students just flit from one image to another without understanding the point.

This is not to say that multimedia with pictures, audio, and video is not a good thing. But it should support reading not substitute for it. Students do need to read increasingly complex materials. Multimedia can add dimensional depth, breadth, and understanding, taking advantage of the resources that are available with technology. But that does not eliminate the need to be able to read and comprehend increasingly complex text, now more than ever.

It is also not to say that the for-profit system of educational materials should be substituted for a national curriculum. The free market should work to improve the quality of materials–but only if consumers make their purchases based on quality–not quantity or ease of use.

If consumers of educational materials take the Common Core Standards seriously and evaluate and purchase curricular materials on the basis of the quality of the instruction and teach students to develop the reading skills they need to comprehend increasingly complex text, student achievement will advance. If, on the other hand, educational materials are simply relabeled with the Common Core Standards and there is no consumer demand to improve them, we will continue the status quo.

Educational Research

There should not be educational reforms without research. Too often educators embrace fads (like the Kahn Academy) that have absolutely no foundation in any research. They spend hundreds of thousands of dollars and inordinate amounts of time and energy in implementation only to find out that it has no effect on student achievement. There are many different types of educational research to consider when developing curriculum.

1. Teaching Method Research. These are the types of studies you find in American Educational Research Association publications. They are most often empirical studies that use scientific methods to identify methods that are more effective than other methods. These methods identified as most effective when incorporated into a complete curriculum are extremely valuable and on a small scale promote changes in practices and student achievement.

2. Subject Area Studies. These studies, such as Adams’s Beginning to Read and more recently Clements and Sarama’s Early Childhood Mathematics Education, can have a profound effect on curriculum development. The serious evaluation of these studies changes what content is included in a curriculum, the way the concepts are sequenced to maximize  student learning, and how concepts are taught. Excellent work is constantly being done that starts with an analysis of established research and then takes it to the next level. Using this research, we would always be innovating, moving toward a more perfect curriculum,  as we understand more about learning.

3. Program Empirical Research. These studies based on scientific methods test one curriculum against another. They can help determine which type of curriculum is more effective with different groups of students.

4. Field Test Research. These studies are often conducted during the development of a curriculum. Lessons are created based on educational theories and then tested in the classroom, refined, and retested. Some of the most effective curriculum has been developed in this way, including Sharon Griffin’s Number Worlds,  but it takes years and years.

5. Market Research. Market research is conducted during the development of a curriculum to identify marketable features and to hone a marketing message. Because market research including focus groups and surveys, is based on quick impressions rather than use of educational materials, it rarely has any dramatic effect on any significant reforms in education.

Why on earth would we not learn from research and design innovative curriculum based on a foundation of knowledge so we don’t keep repeating the same mistakes over and over?

Publishers and curriculum developers have the resources to create innovative, effective research-based materials but they don’t make that a priority because educators DON’T BUY THEM. Educators don’t need to know all the research themselves but they need to know enough or designate people like curriculum specialists to build expertise in a discipline and then review and recommend the most effective materials–none will be perfect, and if they are really good, they will be changing in response to new information coming from researchers. It is totally unrealistic that all teachers will know all things about educational research.

Teaching is an emotionally, intellectually, and physically exhausting profession in and of itself. Learning about effective teaching methods, organization of concepts, and how students learn is also a full-time job. Once an effective curriculum has been adopted, educators will develop their expertise by implementing the new curriculum WITH FIDELITY. This means CHANGING practices and discussing with other teachers and refining the strategies in the new curriculum to meet needs. Doing this will provide the on-the-job, relevant professional development teachers need to advance student achievement. If educators incentivize curriculum developers to produce the MOST EFFECTIVE materials, rather than the ones with the most pages, best covers, most components, flashiest technology or features, educational materials in whatever form they take (textbooks, online, kits…) will improve. If the criteria for purchase is proven effectiveness, publishers will rise to the challenge. Educators need to demand the following of curriculum developers.

1. Show how the curriculum is based on research in the most effective teaching methods in a discipline.

2. Prove that the organization of concepts and skills is based on student learning trajectories.

3.Provide usage studies that describe the experience of other educators in schools with similar demographics who used the curriculum either in field test studies or sales implementations.

4. Demonstrate that the curriculum’s content coverage will build a deep understanding, not superficial coverage of all required standards.

Currently educators don’t ask publishers any of these questions and accept very faulty, superficial explanations of what “research-based” means. An effective research-based curriculum should make educators change and upgrade their practices and move incrementally to more effective education and improved student achievement.

The Arts and Dimensions of Teaching

It is amazing that so many people well into adulthood can remember the names of many of their teachers.  When you think about your favorite teacher, it’s usually easy to recall a particular person who made a difference in your life.  The reasons that one teacher is a favorite are varied, but usually it is because of a personal connection. The favorite teacher may have demonstrated confidence or faith in you, inspired you to do something, or challenged you to do your best. Interestingly favorite teachers are rarely those who let you do whatever you wanted, had no discipline or classroom management skills, or didn’t challenge you to learn anything–even though, if you ask most students, being able to do whatever they want is what they say they want. It makes you realize that Steve Jobs was right to forget about market research since “it’s not the consumers’ job to know what they want.”

There’s no one type who is consistently the favorite teacher. Some people love teachers who were strict and orderly. Others love teachers who were warm and funny. Some love teachers who strictly followed the textbook. Others love teachers who incorporated innovative methods into their teaching.

Determining what effective teaching looks like is a difficult task and often in the eye of the beholder. Is the effective teacher the one who has children seated in rows and on task? Is the effective teacher the one who has children moving from whole class, to individual, to small group work with seamless transition? Is the effective teacher the one who knows the most about a subject or the one who understands how children learn and can present concepts incrementally along students’ individual learning trajectories? Is the effective teacher the one whose students have the highest scores on a single test that is given in the spring?

The truth is that human beings are a diverse group and both students and teachers reflect that diversity. One of the beauties of public education is that in a formalized way, in addition to learning to read, write, and do mathematics and to understand foundational science and social studies, children also learn how to interact with different personalities. This diversity accounts for the reality that one person’s favorite teacher may be another person’s least favorite.

There are five major dimensions or arts of teaching that affect teacher performance and student achievement. It’s easy to see these arts reflected in teachers in every school.

1. Classroom Management–In a classroom taught by an expert in classroom management, students know what to do and how to do it. It is a productive, relaxed atmosphere. There’s no time wasted transitioning from one activity to another or unnecessary disruptions or derailments. Students feel as though they are treated fairly and respect the teacher.

2. Lesson Planning–An expert lesson planner pulls together appropriate resources, plans out activities, and comes prepared to present an appropriate concept or skill, provide guided practice or exploration, check for understanding, engage students in appropriate independent or group work, and administer relevant assessment–all within the allotted time period.

3. Content Knowledge–An expert in content knowledge not only knows the subject matter inside and out, but also knows how to teach the content. Being able to multiply fractions is vastly different from teaching someone else to multiply fractions. Being able to read is vastly different from being able to teach someone to read.

4. Teaching Methods–An expert in teaching methods knows when to employ discussion, group work, practice, exploration, direct instruction, problem solving and other types of methods to teach or reinforce particular concepts and skills. The expert can transition from one method to another if something is not working and does not waste time explaining concepts that students already know or use group work if students aren’t ready to engage.

5. Children’s Learning Trajectories–An expert in children’s learning trajectories understands how children develop reading and math skills, and knowledge. The expert can identify where an individual child is on a  learning trajectory and provide appropriate practice and instruction to move to the next level. There are no leaps from one concept to another that lose most of the class because the expert’s scope and sequence of concepts is built along the learning trajectory.

Rarely is one teacher an expert in each of the arts. It’s easy to remember teachers who had great classroom management but didn’t really teach much. It’s easy to identify teachers with in-depth content knowledge who cannot relate to children or who don’t employ appropriate methods to help build student understanding.

Each of these arts of teaching can be learned and developed, but they will be reflected in different ways by different teachers, just as an artist develops his or her own style using the same tools, elements, and principles of art. The goal is not to copy the masters but to learn the skills, concepts, and methods, and then employ them in a reflection of ones own personal style. The nature of expertise is not complete mastery of any area. Instead the difference between an expert and a nonexpert is that the expert is interested and always learning, always problem solving, always trying use knowledge and experience to make things better. A nonexpert, even a proficient nonexpert, is one who has learned a particular task and repeats it over and over. The nonexpert is rattled when a new problem is presented. The expert is motivated to find a solution. The artist is the expert. The nonexpert attempts to copy masterpieces but has no style of his or her own.

Few teachers are experts in all five arts, but every school has experts in each area who can share their knowledge with others. Over time every teacher can develop expertise in each of the arts. In this way the arts of teaching will be elevated and we can begin to rise above the plateau of student achievement.

Implementing the Common Core Standards

There is so much hope for advancing student achievement riding on the Common Core Standards released for PreK-12 reading/language arts and mathematics. After all, having low standards is supposed to be a major reason for stagnant test scores. Students just aren’t being challenged. I predict, however, after all the work that has been done in writing the new standards and getting them approved, and all the work that will be done to implement them, there will be no significant change in student achievement.

Blasphemy! Don’t we have all kinds of technology for online training for teachers?  Don’t we have all kinds of new cool gadgets, IPads and cell phones, that will make the standards accessible to students? Well, yes, we do, but neither standards nor technology will make a significant difference. If having new standards and new technology affected student achievement, we would have seen it by now after 30 years of one standards movement after another and a massive influx of technology applications in our daily lives.

The one thing that would make a significant difference is improved curriculum that has been created  based on the Common Core standards and that is taught with fidelity by competent teachers. Yet in all the hype about the standards and implementation, very little attention has been paid to how to evaluate curriculum for effectiveness and adherence to standards. Nor has there been any attention to educating teachers about the importance of using a solid curriculum and teaching them how to use it with fidelity.

Instead the professional development squads are out training teachers about the standards and how to develop and pick and choose individual lessons. Maine West, for example has done a wonderful job of collecting a series of websites with information about the standards https://sites.google.com/a/maine207.org/mw-math-department/home/common-core. Ohio.gov has descriptions of model curriculum. But the chances that the vast majority of teachers will be able to piece together a coherent curriculum from all these resources are nil. Left to their own devices, teachers will pick and choose lessons as they always have, based on things they think their students will like or at least “get” that are labeled with the new standards. Teachers learn to plan lessons, but they do not learn how to develop curriculum. The result will be continued curricular chaos.

In the meantime the major educational publishers are busy re-labeling their existing materials with the new standards, and why shouldn’t they? It is a major effort to create a new curriculum when the one that is currently being sold can be revised. With all the standards, both state and national, that have been developed over the past thirty years, publishers have gotten really good at identifying where topics are covered and where there are holes and new content must be added and edits must be made. The fact that the Common Core have been adopted by the vast majority of states actually makes the publishers’ job easier. Expect new Common Core editions of existing educational materials to be forthcoming with minor changes and major labeling.

If we really wanted to “move the needle,” educational publishers would start afresh with experienced content authors and writers who have studied the Common Core and understand the content and the intention of the standards. They would review the research in the most effective teaching methods for addressing each concept and study research about student learning trajectories, or how students learn concepts and develop skills. Then with a particular group of students in mind (below average, average, and above average) they should plot out a careful scope and sequence of lessons with developmental progressions that follow student learning trajectories so there are no major gaps between concepts. Next they should develop a lesson plan template that will introduce concepts and provide guided practice, checking for understanding, practice, assessment, and review using appropriate  teaching methods. Then they should write the curriculum and field test it in actual classrooms to see if it works and make adjustments as necessary. No teacher has the time, knowledge, or energy to make this effort. The idea that each of the 6.2 million teachers in the U.S. would take this on is absurd. But curriculum developers surely do have the resources, knowledge, and distribution networks to make a difference.

If the major and minor publishers developed new curriculum, teachers would have a variety of standards-based materials to choose from to meet their specific needs. As part of their professional development, they would need to learn what the standards are so that they could recognize effective and ineffective coverage in the educational materials they are reviewing. They would need to work with curriculum supervisors to be able to identify the most effective teaching methods and evaluate how they are incorporated in new curriculum. The job is not to have 6.2 million teachers each write a curriculum based on standards but to evaluate curriculum and demand through their careful selection that the materials are the most effective for their population of students.

Once a curriculum has been selected, the teachers’ job would be to implement the curriculum with fidelity. If teachers pick and choose, skip lessons and chapters on a whim, the careful development of concepts and skills will be of no advantage to students in the class or in subsequent years. Of course teachers should be ever critical of lessons that work or do not work, but if they choose to stray off the path, they should do it intentionally and communicate with other faculty members so it can be a coordinated effort.

I see no sign of this major change happening anywhere. Curriculum developers have the resources and knowhow to create standards- based materials but they get by without doing it because educators don’t know how to review curriculum and don’t value what a curriculum can do. They don’t value it because all of the effort in colleges of education and professional development is targeted on individual lesson planning and implementing one standard at a time.

A quality curriculum can help teachers in so many ways in what they do every day, day in and day out. It can play an incredibly important role in improving teacher lesson planning, content development, addressing student developmental learning progressions, and using the most effective teaching methods, all of which is needed to promote student achievement.

The tragedy is teachers don’t learn to do what they really need to do: evaluate curriculum based on solid criteria and as a result, schools will continue to purchase the same types of materials they always have, only with the Common Core labels. In order for student achievement levels to change, the curriculum needs to change and teachers need to change their practices to implement a truly standards-based new curriculum.

What Does It Take to Design and Write Curriculum?

It’s easy to think that any teacher is a curriculum expert and a competent instructional designer. After all teachers use curriculum and work with students to deliver it. But it takes a lot more than using a curriculum to create one.

Anyone who has raised or been around children will know that it takes more than knowing how to read to be able to teach a child to read. It takes more than knowing how to add or multiply fractions or decimals yourself to be able to teach someone else how to do that. There are skills and concepts that are part of a child’s learning trajectory that must be understood and then introduced  and developed in a coherent and comprehensive sequence. It’s easy to believe that all teachers by virtue of being teachers know who to teach anyone anything. But that is not the case.

I was a middle school English/literature teacher for several years in the days when spelling was actually taught. We gave a 20-word pretest on Monday, the kids worked through the workbook pages, studied the words throughout the week, and then took a spelling test on Friday. What could be simpler than that? When I left teaching and went into publishing, I was astonished to learn what it takes to put even a seemingly simple curriculum together.

When I joined Merrill Publishing (subsequently acquired by McGraw-Hill) my first assignment as assistant editor trainee was as the level editor of the grades 7 and 8 levels of the revision of the 1986 revision of the spelling program. Merrill had been publishing spelling since 1891. I was amazed that the project editor had spent six months studying research-based word lists to find high frequency word, sorting them  by pattern, and then organizing them into grade levels and outlining lessons. To ensure that students had a thorough development and exposure to the spelling words, each lesson had several parts: pattern instruction, spelling words in context, meaning exercises, word building, proofreading, and writing the words in a short composition. As a teacher, I couldn’t even remember which spelling program we used, let alone identify grade level appropriate, high-frequency words grouped to teach and reinforce specific spelling patterns. I just used the spelling program we had.

I later became the project editor  and did substantial research into the history of spelling instruction and research and how children learn to spell. In the 1980s there was a revolution in understanding how children learn to spell centered at the University of Virginia. Spelling was not memorized word by word, but was learned pattern by pattern and generalized to new words. Spelling patterns were phonetic (long e=ee, ea, e-consonant-e), structural (drop e and and ing) and meaning based (sign/signal/significant). It turned out very few words needed to be memorized.  I was able to recruit as authors two of the researchers, Charlie Temple and Jean Wallace Gillet, who had been part of this research. They helped us outline the lessons for the K-8 program and then wrote drafts. A team of eight editors took their drafts and created lessons. A team of four production editors proofread and copyedited the work. A team of designers laid out pages with appropriate artwork and room for students to write. It took almost two years from beginning to published books. It was a heady experience for me because I realized a subject that as a teacher I thought I understood, had such a rich tradition and had such important connections to reading and writing.

The history of spelling instruction since the 1980s is a perfect example of educators’ lack of understanding of curriculum development. During the whole language movement of the 1980s spelling was abandoned as a separate subject. It wasn’t necessary, many educators claimed. With the Reading First Initiative in 2004, phonics, phonemic awareness, vocabulary, comprehension, and fluency became the priorities. Spelling and phonics go hand in hand, but spelling was an adjunct to reading. Totally misunderstanding the importance of spelling pattern instruction many teachers and reading programs simply selected words from student reading or identified words students could not spelling and had students memorize them.

There is no way teachers have time to devote to researching a subject, identifying how children learn the subject, and develop rich lessons to develop the concepts in a coherent and comprehensive way throughout a year and from year to year. That is a set of skills and knowledge that teachers do not get in pre-service or in-service training and they should not be expected to know how to do that. It is more than a full-time job. Teachers don’t even get an adequate education in evaluating curriculum. Many of the curriculum decisions that schools make based on teacher recommendations are based on superficial design impressions or superfluous features, rather than the quality of the instruction or the coherent development of the concepts.

Curriculum writing, including planning out a scope and sequence of concepts and skills based on children’s developmental progressions involves a completely different skill and knowledge set than delivering the curriculum. Instructional designers should be steeped in educational research, content knowledge for students, and content knowledge for teachers so that they can incorporate not only an appropriate developmental sequence but also employ the most effective strategies and methods for teachers. Teachers are way too busy teaching to keep up with all the research. Teaching, working with students, and implementing curriculum is more than a full-time job. It helps to have teaching experience as an instructional designer but you also need a depth of knowledge and a mindset that most teachers do not have time to develop.

Well designed curriculum should be a significant contributor to our educational process, but when teachers feel they must design their own curriculum or choose and implement curriculum haphazardly, the result is curriculum chaos. Under our current system, teachers rarely allow the curriculum to support their efforts.  Good teachers don’t need published curriculum, many are led to believe. So instead of helping them with concept development, lesson planning, content coherence, and teaching methods, which a quality curriculum has conscientiously and thoroughly developed, teachers are left on their own. As a result, student achievement is totally dependent upon the teacher at each grade level or in each subject. The lack of quality curriculum implementation as a result of educators not knowing that they don’t know what they don’t know, is very likely a significant factor in the stagnation of student achievement.

Educational Materials Review: The Kahn Academy

It’s hard to avoid hearing about the Kahn Academy  http://www.khanacademy.org/. It’s like the Chickenman theme, “He’s everywhere. He’s everywhere.” I’ve seen him on Charlie Rose, PBS The News Hour, The Colbert Report, and TedTalks. With backing from among others, the Gates Foundation, Sal Kahn has produced over 2400 videos covering everything from simple arithmetic to physics to finance and history.  This is certainly a tremendous achievement, Yet I have been troubled by the wholesale acceptance of these efforts and the adoption of these materials into use in schools across the country. Kahn has even been promoting a “flip” model of instruction in which students watch the videos at home, relieving teachers of the burden of presenting the material, and then use class time to help students work through the material.

In conversations with several self-motivated people looking to recall how a mathematical formula works or the basic facts about a topic, I have been rebuffed when I question the information, presentation, or credibility of the Kahn Academy offerings. It’s a free service and is associated with low stakes learning. For this purpose, it is probably not that harmful because the people using it have a sense of the subject and have some critical judgment, although there are so many other resources that are so much richer with better production values.

But there are so many troubling things about the Kahn Academy.

1. Credibility and Accuracy.

By his own admission, Sal Kahn produces four or five videos a day. For those topics he doesn’t think he already knows, he looks up information on Wikipedia, and then translates the topic into an video drawing and audio presentation that he produces himself. There appears to be no planning, script writing, or even rehearsing, just have at it. Kahn may be a smart guy, but is he really an expert on virtually everything?

The answer is no, he is not. Most of his presentations are the quality of a first year teacher who has very little background in what students know and misconceptions they may have. In fact he often perpetuates naive conceptions. In the evolution video, for example, he draws a tail on an ape.

He’s no expert, but that would be ok if he had people who were knowledgeable about the content reviewing his material, but that is not the case. He simply produces the videos and posts them. This would be ok, too, if the Kahn Academy was like Wikipedia in which knowledgeable people had the opportunity to correct and amend the lessons, but they do not.

Sometimes videos include guest lecture experts, for example, the Precocious Puberty completed at Stanford University, but even then, one can have very little confidence in the accuracy or credibility of the information.

In contrast, curriculum developers can spend years working with authors to identify those critical concepts that children at different grade levels need to learn according to standards, organizing those concepts along learning trajectories, and then incorporating effective teaching methods to introduce concepts, relate them to what students already know, check for understanding, provide appropriate practice, and evaluate understanding. After drafts have been written, curriculum developers typically have a cadre of content and education expert reviewers who can identify errors and make suggestions for more effective methods. They also question bias, inappropriate content, poor language, and misconceptions. Curriculum is then edited, designed, proofread, and published. After publication, curricular materials are evaluated by teachers, parents, students, and schools. Often newly published materials are reprinted to address the errors and omissions that have been identified.

With all this effort, often mistakes get through, so materials need to be continually updated.

In the Kahn Academy, there is none of this rigor. We have to hope that Sal Kahn knows what he’s talking about and gets it right.

2. Production Quality

The Kahn Academy has been praised for its simple, straight-forward presentation of concepts. Another way to say this is a simplistic, simpleminded, and totally inadequate use of the power of technology. If one teacher were to spend day after day in a classroom poorly drawing out his interpretation of mathematics, history, biology, finance and a multitude of other subjects we would think he was insane. Yet this is what Sal Kahn is doing and with the power of technology, making it available to the world. On virtually any topic he has produced, there are far better, even free lessons available. Just look at the PBS offerings on evolution, for example. At PBS there is a whole variety of rich multimedia resources and activities available.

The Kahn Academy Electoral College video is a good example of a video on a topic with which most people should be familiar. If you understand how the electoral college works, you can follow this pretty easily and it confirms what you already know. Now put yourself in the mind of a high school freshman who has no idea what the electoral college is. The video in this case becomes rambling with no clear introduction or conclusion. It assumes that the student understands what the political parties and how the Federal Government is organized. The video is a Sal Kahn stream of consciousness about this topic that includes not only factual errors and misconceptions, but grammatical errors, as well. He makes all kinds of seemingly innocuous errors, like saying California is a large state so it has more representatives. Yet someone who doesn’t understand may wonder why Alaska, which is much larger than California, doesn’t get more representatives.  Kahn spends a lot of time discussing a scenario that has not happened in modern history, that a presidential candidate not receive 270 votes, which makes this way out of perspective. When he finally gets to an example from the 2010 election, it is almost incomprehensible. He also uses data and graphs that have no references, so you don’t know where he got the information. This lesson represents incredibly shoddy presentation and research that any professor would be embarrassed to present. Again, if you already know this, it might be interesting or helpful as a little refresher. If you don’t already understand the topic, you would be lost, or worse, accept what Kahn is saying without question.

3. Teaching Methods

The Kahn Academy is strictly a lecture format with Sal Kahn narrating his scratch drawings of every topic.  If a teacher were to present material in a lecture format day after day, topic after topic, students would be bored to death. Struggling students, who need a lot of interaction, including models, encouragement, feedback, reteaching, checking for understanding, and evaluating, would be particularly lost. A good teacher using a good curriculum  assesses whether concepts should be introduced with questions, hands on activities, small groups, or other experiences. There is no mechanism in the Kahn Academy videos for any of this.

In mathematics all of the Kahn Academy lessons are simply procedural explanations of how to solve a particular problem or use a particular formula. This is great for people who need a quick refresher of how to factor or solve a quadratic equation, but it is useless for someone who does not understand the underlying concepts. This type of teaching is exactly what math educators have been working to overcome to help our nation become mathematically literate.

4. Audience

The first thing any teacher of writing will teach is that the writer must tailor his or her message and form to the intended audience. After watching many Kahn Academy videos, it is clear that the audience is not PreK-8 students and not average or struggling high school students who need an introduction to topics, skills, and concepts. It appears the audience is above average high school students, college students, and adults who want a quick refresher for something they already know. It would be extremely difficult to even bribe a struggling high school student to watch more than one minute of any of these videos before he or she would be itching to turn it off. It’s horrifying to think that this is the future of education for them.

RATINGS

The excitement of technology too often obscures mediocracy. Because the Kahn Academy videos are available online and available to all and because the Gates Foundation is funding the effort and promoting it as the “future of education,” it is easy to think that it must be good. But apart from the fact that these videos are free and widespread, it is difficult to find much credible value in them for most educational purposes.

On a scale of 0-5 below is an evaluation of the Kahn Academy.

Content Accuracy     Rating 2 of 5

The material has not been reviewed or vetted by content experts, so there can be little confidence that the content is accurate. Virtually every video has some minor or major error or misconception.

Content Depth Rating 2 of 5

Although the Kahn Academy has produced 2400 videos, the group is guilty of a serious lack of depth. Only the most basic surface understanding of each topic is presented.

Content Scope Rating 4 of 5

The Kahn Academy is attempting to cover all topics covered in school and many that aren’t addressed in standards.

Design 0 of 5

The design is consistently simple, difficult for anyone who doesn’t already understand the concept to follow, and far below any current design standards.

Ease of Use Rating 3 of 5

The program is easy to access and easy for individual students to use to review concepts. It is inadequate for classroom use.

Lesson Plan Model  Rating 1 of 5

A typical lesson plan includes assessing prior knowledge, concept presentation, checking for understanding, appropriate practice, feedback, and assessment. The Kahn Academy includes only the concept presentation.

Program Philosophy Rating 1 of 5

The program philosophy appears to be that Sal Kahn is able to translate and explain even the most complicated topics so that people who already are familiar with the concepts can understand them.

Standards Coverage  Rating 1

Although topics that are included in the Common Core and state standards are covered in the Kahn Academy, the lessons make no effort to address the intent of the standards. A third grade Common Core math standard in Operations and Algebraic Thinking is: Solve two-step word problems using the four operations. Represent these problems using equations with a letter standing for the unknown quantity. Assess the reasonableness of answers using mental computation and estimation strategies including rounding. The Kahn Academy video Rounding to Estimate Differences should address this standard. Yet the video is simply a procedural explanation of rounding the number of microbes growing in a culture and subtracting to see what the difference is in growth. First this would be inappropriate to use with third graders because one would have to explain what microbes are. Secondly it assumes the student already knows what rounding and estimation are. There is no support for the purpose of rounding, any real world examples of estimation, or context.

Student Learning Trajectories Rating 1 of 5

The Kahn Academy videos are produced without consideration of how students learn. Each video is a simple explanation of a topic from Sal Kahn’s personal point of view.

Teaching Methods Rating 1 of 5

The only teaching method employed is lecture with very limited visual support. This method has limited value to students who are unfamiliar with concepts, are unmotivated, or are struggling learners.

Conclusion

It’s frightening that schools would use the Kahn Academy to replace the teacher presentation of material. The material is only well suited to above average high school students, college students, and adults who just need a quick review or refresher on specific topics. And not surprisingly it is these people who have been promoting the Kahn Academy and giving it such recognition. If I can’t remember what imaginary numbers are or how to calculate slope, it is helpful to have a resource to refresh my memory. I would argue that there are much better resources that have actually been planned, written, and produced with much more accurate information and effective production values, but there is no question that the Kahn Academy is useful for this purpose.

For students, like the vast majority who are not self-motivated to learn in school, or students who need a solid introduction and development of critical concepts, the use of the Kahn Academy represents a cruel joke for unknowing students and educators. The uncritical acceptance of this kind of fad is a distraction from the type of instruction needed to increase student achievement. I hope schools have the good sense not to waste their precious time on this type of material.

Educational Standards and Student Achievement

For almost thirty years, the United States has been on a standards writing binge. In 1983, A Nation at Risk: The Imperative For Educational Reform, written by President Ronald Reagan’s National Commission on Excellence in Education, was published. This report argued against grade inflation and made a strong case for higher educational standards. Following A Nation at Risk many states and organizations began to review and rewrite existing standards and create new standards.

Individual states have the responsibility and authority to create their own standards, which led to some unusual, custom, and chaotic standards from state to state. A student, for example, who moved from one state to another, may find a very different science curriculum in one state as opposed to another. One state may require that astronomy is taught in the third grade, while another may not require it until the fourth grade. One state might require teaching cursive handwriting in second grade, while another may not require it at all. Some of these discrepancies are insignificant. Others could have repercussions if a state’s standards are more or less rigorous than another state’s.  In 1989, the National Council of Teachers of Mathematics (NCTM) published a set of math standards that many states used in developing their state standards. This set of standards really helped to standardize mathematics content requirements across the states. In 2000 NCTM revised its standards, which inspired states to further refine their own individual standards as they consulted the NCTM Standards, and created their own.

The Goals 2000 initiative started under President Bush and continued under President Clinton inspired further standards development. Virtually every national organization including the International Reading Association (IRA), National Council of Teachers of English (NCTM), National Science Teachers Association (NSTA), National Council for the Social Studies (NCSS), and even the National Art Education Association (NAEA), among others, created sets of national standards that individual states could use as a basis for their own standards in each subject area.

The No Child Left Behind legislation under President George W. Bush in 2001 tied funding to states for setting high educational standards and establishing and testing measurable goals. This inspired states to rewrite their state standards and align the standards with their new state proficiency tests. Most recently the Common Core Standards for K-12 English and Mathematics have been approved by over 44 states and have largely replaced each state’s reading and math standards. The Common Core, while not perfect, have been shown to be an improvement over most of the individual state standards.

With all the effort toward standards writing and implementation in schools and assessments, one would expect that students would be achieving in their educational pursuits at higher and higher levels, but this has not happened. Test scores on long-range tests over the last thirty years, such as the National Assessment of Educational Progress (NAEP) are stagnant.

Many intelligent, well-meaning people in all levels of education have been involved in writing standards to improve education. Some believe scores are stagnant because even the Common Core Standards are not the right ones. Others blame teachers and schools for not implementing them properly. Still others blame parents for not supporting their children in their educational efforts.

I suggest that although it is important to set high standards, setting the standards in and of itself will do little to affect educational outcomes either for teachers or students. It is not the standards, but the curriculum that matters and the curriculum, even when it incorporates all kinds of new technologies, is unchanged and fundamentally unaffected by standards.

This does seem impossible, but it is very likely. The major educational publishing companies (Pearson, Houghton Mifflin Harcourt, and McGraw-Hill) publish almost 75% of the educational materials that provide the curriculum foundation used in the majority of American classrooms. Granted many teachers pick and choose lessons from a variety of sources and many educational materials are not used in the manner they were intended, but this situation is the same as it has been for at least fifty years.

The critical  issue is that publishers and teachers have a core set of organized lessons they use for each subject area. For publishers, when new standards are implemented, they typically take their existing programs with established organization and lessons, look for places where the new standards coverage is weak or missing, and then add lessons, activities, or content to fill the holes. They may redesign the program and put a new copyright date and label the standards, but they rarely, if ever, re-conceive an entire curriculum based on standards. Teachers behave the same way. Why should an experienced teacher throw out everything she had developed over the years and start from scratch? In these ways the basic curriculum may be slightly modified but never overhauled when new standards are implemented.

There’s no villain here, just a systemic problem. Standards are written by good people. Publishers incorporate the new standards in good faith into the curriculum. Teachers make every effort to teach to the standards. Yet, if we want significant improvement in student achievement, we need significant changes in the curriculum that establishes what students and teachers do in the classroom every day, including the order in which concepts are developed, and how concepts and skills are introduced, developed, practiced, and assessed. Higher standards really have very little impact on curriculum. Instead of panicking about the Common Core, most publishers, who have been frustrated with fifty different sets of standards for each subject area, are relieved that the Common Core will only require one set of standards and very little change.

If we want real change, we need to change the curriculum that is used in schools. This means that publishers and schools need to do things differently. The Common Core Standards are an incredible opportunity for curriculum developers to stop trying to come up with accommodations for all fifty states. Instead it is an opportunity to invest their time and energy into developing materials that incorporate proven teaching methods and use effective educational research into how children learn and reorganize the curriculum based on developmental progressions, rather than tradition. Instead of trying to fit the new standards into their existing lesson plans, individual teachers can invest their energies into reviewing and selecting the most effective curriculum that their schools can implement system wide. This would provide coherence and the best chance of significant change. Of course any new curriculum will be difficult to teach because it is different and will require much re-education on the part of teachers. But if we don’t change what we’re doing day after day and begin to incorporate system-wide, curriculum that is developed and re-imagined to truly address high standards, Einstein’s definition of insanity will be realized:  doing the same thing over and over again and expecting different results.

Rubrics for Evaluating Curriculum

There are two key ingredients in providing a quality education: teachers and the content they teach, which is the curriculum. Teachers always get a lot of credit and blame for student achievement and a lot of media attention. We focus on teacher motivation, merit pay, professional development, getting rid of bad teachers, and teacher evaluations. The curriculum, whether it is teacher or school created or selected, or whether it is produced by educational publishers, rarely gets any attention. I suspect this is because most people don’t understand what it is or how to evaluate it.

If most people think about curriculum, they think textbooks and other educational materials are like references, such as dictionaries or encyclopedias, that are handed down from on high. Some people who think about it become alarmed and give more credence to educational materials than is due. Textbooks have been criticized for promoting a secular, liberal point of view by people concerned about religious values. They have been criticized for having too much commercialism or not enough diversity by people concerned about promoting good models of behavior. It seems implausible that a paragraph or picture in a textbook that most students won’t even see or understand could be so controversial, let alone compete with all of the other influences in a child’s life. Rarely are they criticized for the organization of the skills and concepts, the quality of the content, or the effectiveness of the instruction.

Very few people, including teachers, administrators, and academics know how to evaluate curriculum. Teachers tend to assess whether materials will be easy to teach and whether their students will be able to read and access the content. They are often impressed by design, special features, or components that have little to do with the presentation of the content. Academics tend to evaluate whether the content is accurate. Although both are valid, neither approach will help select the most effective materials to use in a particular school or with a particular group of students.

Having been in teaching and educational publishing for over 35 years, I have experienced using different curricula and developing different curricula. The most effective curricula, if used with fidelity, should disrupt teaching methods and practices. Otherwise, no change will occur and student achievement will not be affected.

Of course an effective curriculum is accurate and presents a depth of content and activities that meet educational standards.  But an effective curriculum does a lot more than that. Developers of effective curricula spend an inordinate amount of time developing and testing a scope and sequence of skills and concepts so that one lesson builds on previous lessons and leads students through a logical, developmental progression. In an effective curriculum, there are not major leaps from one concept to another that leave students and even teachers behind.

Developers of effective curricula do intensive research in teaching methods that are most appropriate for concept development. Grouping suggestions, fact practice, explicit explanation, open-ended questions, use of games, exploration activities, inclusion of video experiences, or hands-on activities are all examples of teaching methods that can be used to present concepts in effective ways.

Effective curricula provide comprehensive lesson plans, so educators have support for introducing concepts, checking for understanding, student practice, evaluation, and remediation.

Developing a quality curriculum is not easy. After teaching middle school language arts, I was amazed at the amount of time that publishers spend developing a simple spelling program. This involved researching the most effective practices in spelling instruction. It involved selecting a high frequency word list and organizing by and verifying the sound and letter patterns and developing a lesson template to make sure that pattern instruction, as well as use of words in reading, writing, meaning, and word building activities was included. Then it involved a team of writers to write the lessons, a team of editors to review and proofread the lessons, a team of designers to lay out the lessons in an approachable design, a team of artists to illustrate the lessons, and then production companies to develop the pages and covers and publish them. No teacher or school staff would be able to do this nearly as effectively. Nor should they.

The full-time job of the school staff is to deliver curriculum, not to develop it. These are two very different careers. To deliver effective curriculum teachers must be able to identify the qualities that make one curriculum more effective than another and then select the superior materials. Otherwise, their jobs of advancing student understanding will be more difficult and their students will suffer. A curriculum that is easy to teach might not be very effective.

Following are rubrics for evaluating curriculum that any one can use to help select and implement the most effective curriculum. In future posts, I will apply the rubrics to existing curriculum.

	5	4	3	2	1
Content Accuracy	Content is thorough and accurate with credible authorship and reviewers.	Content appears accurate.	Some inaccuracies are found.	Many inaccuracies are found.	There is no reason to be confident about the accuracy of the content.
Content Depth	Content coverage is rich. Opportunities to explore depth of content are numerous.	Content is covered but there are few opportunities to explore content in depth.	Content coverage is superficial.	Content coverage is weak.	Significant amounts of important content are not covered.
Content Scope	Thoroughly covers foundational concepts	Covers key concepts.	Covers some key concepts.	Mentions but does not cover foundational concepts.	Does not address the majority of foundational concepts.
Design	Design facilitates use with appealing features and navigation ease.	Design helps in organization of content but is not appealing.	Design does not help or distract from use.	Design distracts from ease of use.	Design hinders use.
Ease of Use	After training, program is well laid out and intuitive. Distinctive materials are worth the time to implement because they are effective.	Program requires little or no training because it is like other programs we have used.	Some materials in the program will not be used because they are unnecessarily confusing and ineffective.	Most materials are not effective and not worth the effort it will take to learn how to use them.	Even after training, program is incomprehensible.
Lesson Plan Model	Lesson plan design includes effective concept introduction, practice, summarizing, and assessment of key concepts.	Lesson plan design organizes lesson into stages of introduction, development, and assessment.	Lesson plan design omits important features critical to concept understanding.	Lesson plan design distracts from the development of concept development.	Lesson plan design is nonexistent or  impedes concept development.
Program Philosophy	Program has a sound philosophy grounded in credible evidence, research, and/or experience. The philosophy is evidenced throughout the program.	Program philosophy is sound and based on credible information, but the philosophy is only evidenced in specific locations.	Program philosophy is not strong and is not clearly evident.	Program philosophy is not apparent.	Program philosophy reflects ineffective practices.
Standards Coverage	Thoroughly covers all grade level standards and meets the intention of the standards.	Thoroughly covers some of the standards and meets the intention of the standards.	Addresses standards but does not meet the intention of the standards.	Does not thoroughly address the standards or meet the intention of the standards.	Does not address the standards.
Students Learning Trajectories	Carefully develops incremental concepts along children’s learning trajectories.	Follows children’s learning trajectories within sections or subjects.	Organizes content mostly by subject rather than children’s learning trajectories.	Does not use children’s learning trajectories effectively to organize content.	Concept development runs counter to student learning trajectories.
Teaching Methods	Employs effective, innovative, and engaging teaching methods that are founded in research.	Employs effective traditional teaching methods.	Employs some ineffective teaching methods.	Employs mostly ineffective teaching methods.	Employs ineffective teaching methods.