Students and families may not be getting as much help as they think from commercial admission test preparation, according to a report (http://www.nacacnet.org/PublicationsResources/Research/Documents/TestPrepDiscussionPaper.pdf) commissioned by the National Association for College Admission Counseling (NACAC). Existing academic research suggests average gains as a result of commercial test preparation are in the neighborhood of 30 points on the SAT and less than one point on the ACT, substantially lower than gains marketed by test preparation companies. However, the research report also indicates that some colleges and universities may make inappropriate distinctions among applications based on small differences in admission test scores, making even minimal test score gains potentially important in those decisions. The report suggests more comprehensive research is needed to further understand the impact of specific types of test preparation, as distinct from other factors that may improve test scores.
"We believe it is important for educators, students and families to be familiar with independent evaluations of the effects of test preparation," stated Joyce Smith, NACAC CEO. "Test scores play a prominent role in admission decisions at many colleges. As NACAC's Commission on the Use of Standardized Tests in Undergraduate Admission suggested, understanding the effects of test preparation is an important factor in balancing institutional admission decision-making with student considerations."
A survey of NACAC-member colleges included in the study revealed that in a small number of cases, colleges report either that they use a cut-off test score in the admission process or that a small increase in test score could have a significant impact on an applicant's chances of being admitted. Such practices run counter to guidance from NACAC and testing agencies as to the appropriate use of admission test scores. These realities are likely to complicate the decisions of students and families trying to determine how best to allocate resources (both time and money) related to test preparation.
"It is important for colleges and universities to understand and practice appropriate test use in the admission process," noted Smith. "On the heels of this research, NACAC will initiate a new round of communications and training with our member colleges and universities to ensure an understanding of and adherence to appropriate test practice, as well as the standards for admission practice contained in NACAC's Statement of Principles of Good Practice." (http://www.nacacnet.org/AboutNACAC/Policies/Documents/SPGP_8_18_08.pdf).
The paper, "Preparation for College Admission Exams" (http://www.nacacnet.org/PublicationsResources/Research/Documents/TestPrepDiscussionPaper.pdf) was written for NACAC by Derek Briggs, chair of the Research and Evaluation Methodology Program and associate professor of quantitative methods and policy analysis at the University of Colorado at Boulder. Briggs' research points to the need for continued research on the effects of test preparation, particularly as it becomes more widely accessible through a variety of formats and delivery systems. Although the existing academic research base suggests a consensus on the magnitude of test preparation effects, some important practical questions remain unanswered:
- Is the newest version of the SAT more or less "coachable" than previous versions, which have been the subject of academic studies? Since research on ACT test prep is limited, what is the effect of the full range of test preparation for the ACT?
- Are there certain characteristics of particular test prep programs (quality, setting, duration) that may result in higher than average test score increases?
- Is the magnitude of test preparation effects influenced by any student characteristics that have yet to be identified?
- Are commercial forms of test preparation any more effective than student-driven test preparation?
As recommended by the Commission on the Use of Standardized Tests in Undergraduate Admission (http://www.nacacnet.org/PublicationsResources/Marketplace/research/Pages/TestingCommissionReport.aspx), NACAC will continue to play a role in developing the research base in order to provide the best information to students and families about how to allocate test preparation resources and to provide guidance and training to admission offices about appropriate use of test scores in admission decisions.
Direct link to report: http://www.nacacnet.org/PublicationsResources/Research/Documents/TestPrepDiscussionPaper.pdf
Using Assessments to Improve Learning and Student Progress
This issue paper is designed to help policymakers, educators, and administrators evaluate how assessment can support teaching and learning. Through the paper, Pearson examines the role of assessment in the teaching and learning process, and explores how to help teachers and students utilize both "assessments for learning" and "assessments of learning" to gauge and improve student progress.
Pearson's issue paper discusses how assessments provide objective information to support evidence-based or data-driven decision making. While there are many contributing factors that support successful teaching and learning, assessment continues to be an important piece in the learning puzzle. The paper explores the growing body of evidence that shows when teachers use well-constructed, professionally developed assessments students can see larger gains in their performance.
"President Barack Obama and Education Secretary Arne Duncan have made it clear that high standards, assessments that are aligned with college and career expectations, and international benchmarks are essential to reversing the 'race to the bottom,'" said Will Ethridge, CEO of Pearson's North American education businesses. "Pearson's paper on assessments offers policymakers, educators, and administrators additional support for the work they do to improve the quality of education in this new environment of reform."
According to research cited in Using Assessments to Improve Learning and Student Progress, curriculum-embedded instructional assessments and assessments of accountability can improve student achievement by targeting instruction to the skills and needs of the individual learner. Additionally, the paper discusses the importance of a balanced approach to assessment that uses both summative (used after instruction to collect information regarding mastery of content) and formative (process to determine how students are learning) assessments. This balanced approach ultimately benefits not only the administrators, teachers and parents, but, most importantly, the students.
Despite the evidence supporting assessments, the general understanding of how they work and are applied to enhance learning is lagging. Pearson's paper suggests that states and districts need to embrace a balanced assessment system and recommends ways in which leaders can foster teacher and student success. A sampling of these recommendations includes:
• Development and growth of formal teacher education on assessment for learning
•
• Development of tools and reports that enable teachers to manage and use "just in time" results to improve individual student instruction
•
• Implementation of student-friendly learning systems, reports, and tools that provide clear information to students for managing their own learning
•
• Establishment of longitudinal data-management systems so student progress can be measured over time, and so educators and parents can project whether a student is on a path to proficiency and to reach important benchmarks, such as college readiness.
•
Pearson's issue paper encourages assessment literacy training for all teachers, explaining the value of using reliable, professionally developed assessments for learning. This strategy, combined with targeted professional development, will empower teachers to bring new relevance to the teach-and-learn cycle and result in improved learning.
In addition, students can benefit from playing an active role in their own assessment. The paper points out a variety of ways, including:
• Training and implementation of student self-evaluation
• More student peer review and collaboration
• Involvement of students in their own parent/teacher meetings, allowing them a key role in presenting their own goals and progress to their parents
The paper closes by addressing what's next for assessment, providing concrete solutions for improving assessment, such as increased assessment literacy for teachers, students, and parents; funding and support for securing well-constructed, reliable, professional formative assessments; ongoing professional development for educators and districts; and funding and support for building longitudinal data systems.
For a downloadable copy of the Using Assessments to Improve Learning and Student Progress issue paper including an Assessment Glossary of Terms, go to:
http://www.pearsoned.com/pr_2009/051309.pdf
Report: To Close Achievement Gaps, Close Gaps in Life Experiences and Conditions
Five years after a landmark Educational Testing Service (ETS) study identified racial/ethnic and income gaps in 14 life conditions and experiences that are associated with academic success, a new analysis acknowledges little progress in closing the gaps.
Parsing the Achievement Gap II updates ETS’s 2003 Policy Information Center study, Parsing the Achievement Gap: Baselines for Tracking Progress. The updated report identifies 16 factors ranging from birth weight and hunger to lead poisoning, parental involvement, and teacher quality that are related to academic performance. The report then looks at whether these important factors were distributed evenly across different racial/ethnic and income groups.
The new report concludes that while a few of the gaps in the achievement correlates have narrowed and a few have widened, overall, the gaps identified in the previous study remain unaltered and disturbing.
“The undeniable fact is that disparities in life experiences and conditions directly affect, for better or worse, cognitive development and academic achievement,” Paul E. Barton, co-author of the report says. “These 16 correlates span from birth to adolescence and include life experiences in the home before school, during school, after school and in the summer.”
Barton and co-author Richard J. Coley, however, make it clear that these 16 factors include school experiences as well as before-school and out-of-school experiences, and emphasize that the results should in no way diminish the profound importance of schools and their quality as keys to raising achievement and closing gaps.
Coley, Director of the ETS Policy Information Center explains, “Analyzing these correlates will help to further our understanding of what causes achievement gaps and how we can successfully address them. The correlates are best viewed as three clusters of factors—school factors, factors related to the home and school connection, and factors that are present both before and beyond school.”
A few of the report findings include:
School Factors
• Teacher preparation – Minority and low-income students are less likely to be taught by certified teachers and more likely to have math teachers with neither a major nor a minor in mathematics. The gap in students having teachers prepared in the subjects they teach widened between White and Hispanic students and remained about the same for the other populations.
•
• Fear and safety at school – Minority students are more likely to report issues of fear and safety at school. The gaps widened for students reporting the presence of street gangs and fights in school, and remained unchanged for students reporting feeling fearful in school.
•
Home and School Connection
• Parent participation – White students’ parents are more likely to attend a school event or to volunteer at school. The gap in parents volunteering in schools remained unchanged; the gap in parents attending school events narrowed.
•
Before and Beyond School
• Hunger and nutrition – Minority and low-income children were more likely to be food insecure. The White-Black gap was unchanged; the White-Hispanic gap narrowed.
•
• Excessive television watching – Minority and lower socioeconomic status children watch more television. The gap was unchanged between White and Black students; the gap widened among students whose parents have different education levels.
•
Barton and Coley conclude, “…the finding that these critical gaps in the life and school experiences of minority and low-income children still mirror gaps in school achievement as they did five years ago is very troubling and shows how much work there is to do and how early we need to start. And we have to start with a clear recognition of the depth of the causes of achievement gaps, and expand the scope of our efforts to eliminate them both inside and outside the schools.”
In addition to closing gaps in “beyond school” conditions, Coley emphasizes that we still need to improve the quality of children’s teachers and schools. Only by improving conditions both inside and outside of schools will we successfully close achievement gaps.
Download Parsing the Achievement Gap II for free
http://www.ets.org/Media/Research/pdf/PICPARSINGII.pdf
Tying education to future goals may boost grades more than helping with homework
Helping middle school students with their homework may not be the best way to get them on the honor roll. But telling them how important academic performance is to their future job prospects and providing specific strategies to study and learn might clinch the grades, according to a research review.
"Instilling the value of education and linking school work to future goals is what this age group needs to excel in school, more than parents' helping with homework or showing up at school," said lead researcher Nancy E. Hill, PhD, of Harvard University. She examined 50 studies with more than 50,000 students over a 26-year period looking at what kinds of parent involvement helped children's academic achievement.
These findings are reported in the May issue of Developmental Psychology, published by the American Psychological Association.
"Middle school is the time when grades and interest in school decline," said Hill. "Entering puberty, hanging out with friends, wanting distance from parents and longing to make one's own decisions win over listening to parents and studying."
But adolescence is also a time when analytic thinking, problem-solving, planning and decision-making skills start to increase, Hill said. At this age, "teens are starting to internalize goals, beliefs and motivations and use these to make decisions. Although they may want to make their own decisions, they need guidance from parents to help provide the link between school and their aspirations for future work."
This type of parental involvement works for middle school students because it is not dependent on teacher relationships, like in elementary school. Middle school students have different teachers for each subject so it is much more difficult for parents to develop relationships with teachers and to influence their teenagers through their teachers, Hill said.
Parents' involvement in school events still had a positive effect on adolescents' achievement, Hill said, but not as much as parents' conveying the importance of academic performance, relating educational goals to occupational aspirations and discussing learning strategies.
Helping with homework had mixed results. Some students felt that parents were interfering with their independence or putting too much pressure on them. Some found that their parents' help was confusing because they didn't use the same strategies as their teachers. Still others felt that parents helped them complete or understand their homework, said Hill and co-author Diana F. Tyson, PhD, of Duke University.
Another possible explanation for the negative return on homework, said Hill, "was that those students who needed help with their homework were already doing poorly in school and this showed up as being associated with lower levels of achievement."
The review did not rule out ethnic and socioeconomic influences. Findings showed no difference between whites and blacks in which types of parental involvements influenced achievement but the same interventions did not necessarily produce the same results for Hispanics and Asian-Americans. Some of the studies showed that parental involvement had different meanings across different ethnic groups, which could be the result of differences in economic resources.
"Lack of guidance is the chief reason that academically able students do not go to college," said Hill. "So communicating the value of education and offering curriculum advice about what to focus on helps these students plan their long-term goals."
Full text of the article is available at http://www.apa.org/journals/releases/dev453740.pdf
MERIT PAY HAS PITFALLS FOR SCHOOLS
New study reveals significant downsides of reward-punishment systems based on quantitative outcomes, whether in public or private sector
Some school policymakers are promoting a new idea for improving the schools: merit pay plans that would tie teachers’ pay to the scores their students earn on standardized math and reading tests. Advocates of this approach base their support on two assumptions: first, that merit pay is long-established and widespread in the private sector, and second, that students’ test scores are a reliable way to gauge how well teachers are doing their jobs. Both assumptions, according to a new research report issued today by the Economic Policy Institute, are faulty.
In Teachers, Performance Pay, and Accountability: What Education Should Learn from Other Sectors Scott J. Adams, John S. Heywood and Richard Rothstein examine the evidence that underlies these assumptions, concluding that the use of merit pay systems based on quantitative measures is fraught with perverse consequences that often thwart the larger goal of improving the quality of services and outcomes and that such systems are not widespread among private sector professionals.
As Daniel Koretz writes in his preface to this volume, “In large part because available numerical measures are necessarily incomplete, holding workers accountable for them – without countervailing measures of other kinds – often leads to serious distortions.” He notes that the limitations of numerical measures of success are especially acute among professionals with complex roles, since “the available objective measures are seriously incomplete indicators of value to firms, and therefore, other measures, including subjective evaluations, have to be added to the mix.”
In Part One of the study, entitled “Performance Pay in the U.S. Private Sector,” Adams and Heywood offer a detailed description of performance pay systems utilized by businesses and track the trends in their use. They find that, contrary to the claims of advocates of teacher merit pay, “relatively few private sector workers have pay that varies in a direct formulaic way with their productivity, and that the share of such workers is probably declining.” Haywood added, "Formulaic reward structures often reward only a few dimensions of productivity and run the risk of causing workers to abandon effort in the dimensions not rewarded."
Their research shows that even though many workplaces pay “bonuses,” these are generally not regular performance-related pay of the kind that is being promoted for teachers. And even though the use of bonus pay has grown, that expansion has not been widespread but rather has been focused in certain occupations and industries. The authors describe this growth as “largely a non-union, male phenomenon concentrated among managers and professionals and in finance, insurance, and real estate.” Performance pay now covers only about one in seven workers and represents only a small portion of their compensation.
EPI economist Joydeep Roy, co-editor with NYU’s Sean Corcoran of this series, noted that “Policymakers should probably think twice before they transfer to education the pay system that has helped generate the global financial crisis.” In addition, Corcoran said, “Rewarding workers for good performance is the mark of any successful organization. But, as this book shows, private and public sector organizations have long been aware of the perils of narrowly focused incentive schemes."
Adams and Heywood observe that private-sector performance pay tends to be concentrated in areas whose activities and goals bear little, if any, resemblance to those of schools or other public sector institutions – in sales, for example, where a clear individual-output measure correlates with the firm’s goal of maximizing profits. Adams noted, "Use of performance pay in the private sector, which aligns workers’ interests in greater wages and secure long-term employment with employers’ interests in maximizing profits, need not alone justify the suitability of such schemes for the public sector, where employment is more secure and there are typically no profits to maximize.”
They note, further, that many government functions involve substantial team production and multi-dimensional measures of success, ranging, for example, from disease research teams to police and firefighters. Attempts to establish measures of individual success in these circumstances often lead to unintended outcomes – such as “quotas” for arrests that can produce an overemphasis on less serious crimes, and many other distortions that are detailed in the report.
In Part Two, Richard Rothstein explores “The Perils of Quantitative Performance Accountability” in the field of education, as well as a broad range of other areas extensively studied and documented by social scientists and management theorists. Rothstein’s work shows how even the best-intentioned attempts to create systems for measuring performance often subvert the goals and values of the firm or organization being measured.
Rothstein paints a vivid picture of the perverse consequences created when numbers-based accountability measures encounter the human talent for gaming the system. He draws upon familiar examples such as body counts employed by the military during the Vietnam War, ticket quotas and crime clearance rates used by law enforcement agencies, TV sweeps week, best-seller lists, and college rankings, as well as examining the impact of health care report cards on health care delivery.
He also cites less familiar cases, such as a system devised in Santiago, Chile, to prevent buses from “clumping” together by paying drivers per-passenger. The system was meant to encourage drivers to increase their trailing distance behind the bus in front in order to allow more passengers to gather at each bus stop; instead, it inspired drivers to speed up in order to overtake and pass the lead bus to reach the passengers further along the route first. As a consequence, drivers paid under this system are involved in 67% more accidents per mile than fixed-wage drivers.
One of the chief shortcomings of test-based accountability in education, Rothstein notes, is that it doesn’t take into account the wide variations in student characteristics. He writes, “A school with large numbers of low-income children, high residential mobility, great family stress, little literacy support at home, and serious health problems may be a better school even if its test scores are lower than another whose pupils do not have such challenges; similarly for teachers.” Rothstein does not conclude schools and teachers cannot or should not be held accountable; rather, he urges that any accountability system must be built on the extensive experience and research inside and outside of education and on an informed assessment of the gains and losses inherent in any system. As he writes in his conclusion: “In education, most policy makers who now promote performance incentives and accountability, and scholars who analyze them, seem mostly oblivious to the extensive literature in economics and management theory documenting the inevitable corruption of quantitative indicators and the perverse consequences of performance incentives that rely on such indicators. Of course, ignorant of this literature, many proponents of performance incentives are unable to engage in careful deliberation about whether, in particular cases, the benefits are worth the price.”
You have read this article with the title May ERR #7. You can bookmark this page URL http://universosportinguista.blogspot.com/2009/05/may-err-7.html. Thanks!
No comment for "May ERR #7"
Post a Comment