As expected and in typical bureaucratic doublespeak, the Florida Department of Education used the report by the sole bidders for the allegedly independent review of the 2015 FSA test to magically deem that Florida's test scores could be used in the aggregate for school grades and teacher evaluations. Commissioner Pam Stewart who has presided over much chaos and bears much responsibility for the many problems and lack of accuracy of her own statements in the Common Core standard and testing fiasco, called this statement by Alpine "welcome news." However, even the authors of the study, who have multiple incestuous relationships to the development of Common Core standards and the testing industry, admit numerous problems with their work blaming it all on a fast timeline or other factors beyond their control. Here are some of the most important issues:
Admitted lack of rigor and adequate standardization The report's executive summary said, ""The evaluation team can reasonably state that the spring 2015 administration of the [Florida Standards Assessments] did not meet the normal rigor and standardization expected with a high-stakes assessment program like the FSA."
Basing the FSA on the Utah Test which has no Validity As FSCCC has previously reported, Florida is renting test questions from Utah for three years at a cost to the taxpayers of $16.2 million in addition to the reported $220 million cost of the 6 year testing contract. The Utah test still has no validity information, despite being used for the last two years. However, this should be no surprise as the Department has admitted doubts about the validity and usefulness of the FCAT 2.0 that had been in use since 2011 (see page 137).
The computer adaptive national SBAC test also developed by AIR also has zero validity and reliability data Here are the comments of testing specialist Dr. Doug McCrae presented to the California State Board of Education on 9/3/15:
"My Read more
Dr. Gene V. Glass, a seminal figure in the field of educational psychometrics and data mining has had enough. According to his blog post republished on the Washington Post's Answer Sheet blog, this pioneer in the field of education statistics is "no longer comfortable being associated with the discipline of educational measurement."
This is a man who actually developed the term and process "meta-analysis," which is the statistical procedure of combining the results of multiple smaller studies into one larger analysis to try to get more information and reliability from greater numbers.
Glass described his history and the history of psychometrics in general: My mentors both those I spoke with daily and those whose works I read had served in WWII. Many did research on human factors -- measuring aptitudes and talents and matching them to jobs. Assessments showed who were the best candidates to be pilots or navigators or marksmen. We were told that psychometrics had won the war; and of course, we believed it
The next wars that psychometrics promised it could win were the wars on poverty and ignorance. The man who led the Army Air Corps effort in psychometrics started a private research center. (It exists today, and is a beneficiary of the millions of dollars spent on Common Core testing.) My dissertation won the 1966 prize in Psychometrics awarded by that man's organization. And I was hired to fill the slot recently vacated by the world's leading psychometrician at the University of Illinois. Psychometrics was flying high, and so was I.
In the emphasized language above, Glass appears quite likely to be describing the American Institutes for Research (AIR), which has definitely received not just millions of dollars, but hundreds of millions of dollars from states and the federal government for the Common Core tests (an advertised $220 million from Florida alone). Glass is also probably describing their studies on fighter pilots Read more
The American Institutes for Research (AIR) is already extremely problematic both as the creator of the Smarter Balance (SBAC) computer adaptive testing platform and as statewide test provider for numerous other states that include Florida, Arizona, Montana, West Virginia, and now Ohio. We have chronicled these problems that include: Psychosocial testing and data collection
Manipulative computer adaptive testing
Controversial social emotional intelligence and experimental education programs
Admits that Common Core is a national curriculum
Its founder was involved in racial eugenics experiments
Collects data on individual students and teachers through their national longitudinal database
As evidence of their student psychological and attitudinal profiling and data mining mounts, AIR continues to prove that they should not be the ones responsible for testing millions of American students via federally mandated, funded, and or supervised tests for either the SBAC federal consortium or for individual states. Here are the latest revelations:
1) AIR is involved with the US Departments of Education and Justice to implement an Orwellian data gathering, monitoring, and psychological screening program in the Miami Dade Schools and across the country called The Campus Shield Initiative. This program involves an alarming amount of data collection combined with subjective and ineffective mental health screening of every child in a school, ostensibly to allow law enforcement and school officials to prevent violence. The program plans are revealed on the website:
"One major recent development in preventative policing efforts is the use of social media by police to circumvent threats. The use of social media in school crime prevention is particularly relevant, considering the rise of Internet threats as precursors to school violence. With little effort, police are able to access information posted on Facebook, Twitter, Read more
It is getting very hard to keep up with the American Institute for Research's educational malfeasance in the testing realm. Parent Advocate Deb Herbage and Dr. Karen Effrem of FSCCC have reviewed the 2245 page contract between AIR and the Florida Department of Education. Here are just a few of those revelations as Alpine spends another $600,000 of Florida taxpayer funds to do a validity study that should have already been done and that Commissioner Stewart promised was done. The public has been told this test is going to cost the public $220 million over six years. It is very important to verify what has been paid for has been done.
The following information was taken directly from the executed contract Contract #14-652 for the development of the state assessment (FSA) and Algebra I, Algebra II, and Geometry End of Course (EOC's) exams executed on 6/3/14. The contract was signed by Pam Stewart (FLDOE) and Vickie Brooks (AIR). [The full 2245 page contract is available at the Florida CFO website with all page numbers referencing this document].
1) Potentially Missing Linking, Validity and or Field Studies Have Florida Taxpayers Paid for Work Not Completed? The FLDOE required in its ITN (Invitation to Negotiate) that:
7.6.0. Scaling, Equating, Scoring and Special Psychometric Studies (Pg. 77)
Excerpt from the ITN" Other than the annual regular psychometric operations, such as sampling, test construction, and SES for the assessment system, the contractor will conduct a set of special psychometric studies for these assessments, described in Section 220.127.116.11. (Pg. 77)
The contractor must show evidence that the Department's preferences are psychometrically defensible and operationally feasible. The respondent may include in the reply a different proposal for scaling, equating, and scoring of these assessments to obtain assessment results that are valid, reliable, and accurate. (Pg. 165)
AIR said in numerous places in Read more