News
California Department of Education Admits Total Lack of Validity & Reliability Data for AIR’s SBAC
September, 2015In astonishing public testimony before the California State Board of Education on 9/3/15, retired testing specialist and psychometrician Dr. Doug McRae gave evidence that the SBAC (Smarter Balanced Assessment Consortium) statewide test given in the spring of 2015 had absolutely no validity, reliability, or fairness data available in the entire department. SBAC is one of two federally funded, federally supervised national tests developed to assess the Common Core standards in English and math, PARCC (Partnership for the Assessment of Readiness for College and Careers), the test that Florida was about to adopt until Governor Scott removed the state from the PARCC consortium and the test by AIR (American Institutes for Research) was adopted instead. AIR is also responsible for the computer adaptive testing platform for SBAC as documented in their Florida testing contract:
We are pleased to be able to offer a mature solution. AIR has been delivering adaptive and fixed-form online tests for 7 years. In addition to millions of operational tests in Oregon, Delaware, Hawaii, Minnesota, and Ohio, our online testing system recently delivered the Smarter Balanced Assessment Consortium's pilot test to over half a million students across 21 states. This spring, we will serve approximately 5 million Smarter Balanced students Our system is versatile, works well within schools, and is easy to use--all characteristics that enable us to extend service to so many states, schools, and students in such short order.
We have documented many, many problems with AIR and its tests, including a focus on controversial social issues instead of academics, basing and field testing Florida's test on AIR's Utah Common Core test, which is also completely devoid of validity data, and the necessity for a validity study that was supposed to be completed before the administration of the 2015 AIR test in Florida necessitating the completion of a $600,000 study after the test was given. That validity study was also fraught with major problems which are documented HERE.
Here is Dr. McRae's amazing and very important statement:
"My name is Doug McRae, a retired testing specialist from Monterey.
The big question for Smarter Balanced test results is not the delay in release of the scores, or the relationships to old STAR data on the CDE website, but rather the quality of the Smarter Balanced scores now being provided to local districts and schools. These scores should be valid reliable and fair, as required by California statute as well as professional standards for large scale K-12 assessments.
When I made a Public Records Request to the CDE last winter for documentation of validity reliability and fairness information for Smarter Balanced tests, either in CDE files or obtainable from the Smarter Balanced consortium, the reply letter in January said CDE had no such information in their files. I provided a copy of this interchange to the State Board at your January meeting. There has been no documentation for the validity, reliability, or fairness for Smarter Balanced tests released by Smarter Balanced, UCLA, or CDE since January, as far as I know.
Statewide test results should not be released in the absence of documented validity reliability and fairness of scores. Individual student reports should not be shared with parents or students before the technical quality of the scores is documented. But, the real longer lasting damage will be done if substandard information is placed in student cumulative academic records to follow students for their remaining years in school, to do damage for placement and instructional decisions and opportunities to learn, for years to come. To allow this to happen would be immoral, unethical, unprofessional, and to say the least, totally irresponsible. I would urge the State Board to take action today to prevent or (at the very least) to discourage local districts from placing 2015 Smarter Balanced scores in student permanent records until validity reliability and fairness characteristics are documented and made available to the public." [Emphasis added]
This, combined with the total lack of validity of AIR's Utah test that is based on the SBAC test, is even more evidence that the contract with AIR should be cancelled and that Florida should return to tried and true pre-Common Core nationally norm referenced tests administered on paper. It is also major evidence that the federal mandate for annual testing should be removed.
