Alice in Wonderland Validity Study: FSA is Valid Except When It Is Not

September 11, 2015

As expected and in typical bureaucratic doublespeak, the Florida Department of Education used the report by the sole bidders for the allegedly independent review of the 2015 FSA test to magically deem that Florida's test scores could be used in the aggregate for school grades and teacher evaluations. Commissioner Pam Stewart who has presided over much chaos and bears much responsibility for the many problems and lack of accuracy of her own statements in the Common Core standard and testing fiasco, called this statement by Alpine "welcome news." However, even the authors of the study, who have multiple incestuous relationships to the development of Common Core standards and the testing industry, admit numerous problems with their work blaming it all on a fast timeline or other factors beyond their control. Here are some of the most important issues:
Admitted lack of rigor and adequate standardization The report's executive summary said, ""The evaluation team can reasonably state that the spring 2015 administration of the [Florida Standards Assessments] did not meet the normal rigor and standardization expected with a high-stakes assessment program like the FSA."

Basing the FSA on the Utah Test which has no Validity As FSCCC has previously reported, Florida is renting test questions from Utah for three years at a cost to the taxpayers of $16.2 million in addition to the reported $220 million cost of the 6 year testing contract. Read more

Posted in Testing. Tagged as AIR, FSA, Validity Study.

Renowned Researcher who Likely Studied Under AIR Founder “No Longer Comfortable” in Education Data Mining

August 25, 2015

Dr. Gene V. Glass, a seminal figure in the field of educational psychometrics and data mining has had enough. According to his blog post republished on the Washington Post's Answer Sheet blog, this pioneer in the field of education statistics is "no longer comfortable being associated with the discipline of educational measurement."

This is a man who actually developed the term and process "meta-analysis," which is the statistical procedure of combining the results of multiple smaller studies into one larger analysis to try to get more information and reliability from greater numbers.

Glass described his history and the history of psychometrics in general: My mentors both those I spoke with daily and those whose works I read had served in WWII. Many did research on human factors -- measuring aptitudes and talents and matching them to jobs. Assessments showed who were the best candidates to be pilots or navigators or marksmen. We were told that psychometrics had won the war; and of course, we believed it
The next wars that psychometrics promised it could win were the wars on poverty and ignorance. The man who led the Army Air Corps effort in psychometrics started a private research center. (It exists today, and is a beneficiary of the millions of dollars spent on Common Core testing.) My dissertation won the 1966 prize in Psychometrics awarded by that man's organization. And I was hired to fill the slot recently vacated by the world's leading psychometrician at the University of Illinois. Psychometrics was flying high, and so was I.

In the emphasized language above, Glass appears quite likely to be describing the Read more

Posted in Testing. Tagged as AIR, Dr. Gary Thompson, Gene V. Glass, John Flanagan, Pearson.

Revelations of AIR Involvement in Miami Dade & Pasco Mental Screening & Attitudinal Profiling

July 30, 2015

The American Institutes for Research (AIR) is already extremely problematic both as the creator of the Smarter Balance (SBAC) computer adaptive testing platform and as statewide test provider for numerous other states that include Florida, Arizona, Montana, West Virginia, and now Ohio. We have chronicled these problems that include: Psychosocial testing and data collection
Manipulative computer adaptive testing
Controversial social emotional intelligence and experimental education programs
Admits that Common Core is a national curriculum
Its founder was involved in racial eugenics experiments
Collects data on individual students and teachers through their national longitudinal database

As evidence of their student psychological and attitudinal profiling and data mining mounts, AIR continues to prove that they should not be the ones responsible for testing millions of American students via federally mandated, funded, and or supervised tests for either the SBAC federal consortium or for individual states. Here are the latest revelations:

1) AIR is involved with the US Departments of Education and Justice to implement an Orwellian data gathering, Read more

Posted in Psychological Manipulation. Tagged as AIR, American Institutes for Research, Campus Shield, Gallup Student Survey, mental screening, perceived sexual orientation.

Startling Validity Failures from Florida's AIR FSA Contract

July 30, 2015

It is getting very hard to keep up with the American Institute for Research's educational malfeasance in the testing realm. Parent Advocate Deb Herbage and Dr. Karen Effrem of FSCCC have reviewed the 2245 page contract between AIR and the Florida Department of Education. Here are just a few of those revelations as Alpine spends another $600,000 of Florida taxpayer funds to do a validity study that should have already been done and that Commissioner Stewart promised was done. The public has been told this test is going to cost the public $220 million over six years. It is very important to verify what has been paid for has been done.

The following information was taken directly from the executed contract Contract #14-652 for the development of the state assessment (FSA) and Algebra I, Algebra II, and Geometry End of Course (EOC's) exams executed on 6/3/14. The contract was signed by Pam Stewart (FLDOE) and Vickie Brooks (AIR). [The full 2245 page contract is available at the Florida CFO website with all page numbers referencing this document].

1) Potentially Missing Linking, Validity and or Field Studies Have Florida Taxpayers Paid for Work Not Completed? The FLDOE required in its ITN (Invitation to Negotiate) that:
7.6.0. Scaling, Equating, Scoring and Special Psychometric Studies (Pg. 77)
Excerpt from the ITN" Other than the annual regular psychometric operations, such as sampling, test construction, and SES for the assessment system, the contractor will conduct a set of special psychometric studies for these assessments, described in Section 7.6.3.6. (Pg. 77)
The contractor must show evidence that the Department's preferences are psychometrically defensible and operationally feasible. The respondent may include in the reply a different proposal for scaling, equating, and scoring of these Read more

Posted in Testing. Tagged as AIR, Alpine, American Institutes for Research, Deb Herbage, Dr. Karen Effrem, FSA, Pam Stewart, testing, Utah, validity.

Keep in Touch

Subscribe