Data Misrepresented Is Destructive!

Data is the best thing since Apple pie - or so techies selling software would have us believe.  With the right data and enough of it, teachers could all be effective, students could all be National Merit Scholars and John White could make every charter in the RSD an "A" school.  But data can be a dangerous tool to mislead an unknowing public.

Here is an analysis of the recent AP (Advanced Placement) test results for Louisiana assembled by math teacher/blogger Gary Rubenstein preceded by Louisiana State Supt. John White's rose-colored glasses version.  You be the judge as to which has more "clarity" combined with accuracy!

Release Header 
                                                                                            Follow Us:Like us on Facebook Follow us on Twitter View our videos on YouTube
Date: 02/11/14
Contact: LDOE Public Affairs, (225) 342-3600, Fax: (225) 342-0193
Growth Among all Louisiana High School Students is Tops in the Nation

BATON ROUGE, La. - The Department of Education today announced Louisiana high school seniors are outpacing the nation in the increase of graduates scoring a 3 or higher on AP exams and qualifying for college credit, according to the College Board's 10th Annual AP Report to the Nation. The report shows the number of seniors scoring a qualifying score of 3 or higher increased from 1,531 in 2012 to 1,911 in 2013, representing a 25 percent increase, ranking Louisiana first in the nation. Nationwide, the increase from 2012 to 2013 was six percent. Additionally, Louisiana ranks third in the nation for the increase in the percentage of graduates taking an AP exam during high school, increasing 4.1 percentage points from 11.1 to 15.2 percent, representing an increase of more than 1,500 high school seniors taking an AP exam from 3,931 in 2012 to 5,516 in 2013. Likewise, the number of AP exams taken by high school seniors increased by 42 percent from 2012 to 2013, ranking Louisiana first in the nation.

For all Louisiana high school students in grades 9-12, the number of college credits earned has increased by more than 1,000 from 2012 to 2013 - the greatest individual increase in state history. The number of students earning a qualifying score of 3 or higher rose from 4,112 in 2012 to 5,144 in 2013, representing a 25 percent increase. Likewise, the number of students taking an AP exam increased from 6,645 in 2012 to 10,553 in 2013, an increase of nearly 4,000 students or 59 percent. The number of AP exams those students took also increased, from 9,644 in 2012 to 15,070 in 2013, for an increase of 56 percent. Enrollment in AP courses has also grown. In, 2013, Louisiana students took approximately 6,000 more AP courses. There were 23,435 students enrolled in AP courses in 2013, compared to 17,496 in 2012, representing a 33 percent increase. With more students than ever before participating in AP, Louisiana is on track to being a leader in expanding college access to all students.

"As more seniors in Louisiana leave high school with college credits already under their belt it is further proof that the hard work being done in our classrooms is paying off for our students," said State Superintendent John White. "And, it's not just the most recent cohort of high school graduates. We are seeing proven success and enrollment increases across all grade levels in Advanced Placement."

Advanced Placement courses allow high school students to earn college credit while working towards their high school diplomas. Students enrolled in the college-level courses receive high school credit for the subject in which they take the AP class, and college credit upon earning a qualifying score on a standardized AP exam.

AP tests are scored on a 1 to 5 scale as follows:

5 - Extremely well qualified
4 - Well qualified
3 - Qualified
2 - Possibly qualified
1 - No recommendation

Increasing AP participation is a critical component of Louisiana Believes, the state's comprehensive plan to put every student on track to a college degree or a professional career. With AP, students can get a feel for the rigors of college-level studies while they still have the support of a high school environment. When students take AP courses, they demonstrate to college admission officers they have sought out an educational experience that will prepare them for success in college and beyond. Research shows a strong link between taking AP courses and success in college, finding students taking AP courses are more likely to graduate in four years and have higher college GPAs.

"Louisiana is poised to continue improving upon this success," said Supt. White. "We are dedicated to providing not only increased opportunities for our students but the preparation and support educators need to offer these opportunities to students."

Louisiana is utilizing multiple strategies as part of its plan to increase AP participation including the Advanced Placement Exam Fee Program and the AP Summer Institute Fee Reimbursement Program. Through the AP Exam Fee Program, the Department will subsidize the $55 for each AP exam taken by students enrolled in both public and BESE-approved nonpublic school who meet the low-income eligibility criteria. In addition, the Department will reimburse the full $89 for each AP Exam taken by students enrolled in a public school which is implementing or expanding an AP program. The fee reimbursements are for students who were enrolled in a rigorous course preparing them for the AP exam taken during the May 2014 testing cycle, and were between the ages of 5 and 19 on exam day. 

Also, over the last two years, the Department has trained approximately 1,000 educators to teach rigorous college-preparatory courses. Through the 2014 AP Summer Institute Fee Reimbursement Program, the Department will offer the professional development opportunity to an additional 600 educators seeking to ensure a smooth transition to the redesigned courses that feature 21st century skills, inquiry-based approaches, and strong alignment to best practices in higher education. The deadline to apply for both programs is March 7. Applications can be found on the Department's website by clickinghere.

The expanding of access to AP and student participation is leading to cost savings in college for students and families. In May 2013, Louisiana public and private high school students took a total of 8,617 AP exams that resulted in scores of 3 or higher. Based on students' opportunity to earn at least 3 college credits for each AP Exam score of 3 or higher, this represents an estimated 25,851 college credits. At an average rate of $218.20 per credit hour, the total potential cost savings for Louisiana students and families was $5,640,688.

To read the College Board's 10th Annual AP Report to the Nation, please click here.

To view 2012-2013 AP Results by District and School, please click here.

Louisiana still ranks second to last in AP results

One argument that common core supporters sometimes use is that without common exams across the country, it is impossible to measure how different states are doing.  But of course there are plenty of existing tests already which do this, like the SAT and the APs.
Though I’ve got plenty of issues with The College Board, I will admit that the AP tests are decent tests.  And The College Board is pretty good about publishing its annual data, which is something that I find helpful when looking for somewhat objective numbers.
Last year I read about the miracle in Louisiana where they had a huge increase in the number of test takers.  This year, the College Board released the 2014 data, and Louisiana did it again.  They are truly closing the gap between the percent of Louisiana students taking the AP compared to the national average.
Screen shot 2014-10-07 at 10.52.54 PMWith 13.6% participation in 2014, Louisiana has catapulted to 14th from the bottom in their participation.
This caused TFA alum John White to celebrate:Screen shot 2014-10-07 at 11.10.08 PM

Screen shot 2014-10-07 at 11.21.02 PM
Screen shot 2014-10-07 at 11.08.57 PM
But participation rate is of course something that can be rigged.  All it takes is some money to give to The College Board and as many students as you want to pay for can sit and take the AP test.  What usually matters to ‘reformers’ are results.
But as the Times-Picayune noted, the percent of students passing the test declined in 2014 from about 34% to about 30%.  Of course John White had a response to that:
But detailed data showed that overall pass rate declined by 4 percentage points. Education Superintendent John White said that drop was expected, given the higher number of students taking the test.
“Because we have had such a large increase in test takers, it is possible for the number of overall success stories to go up and the rate of success to remain flat or go down,” he said.
And it is true that it is possible to increase test takers by a lot and even if both test takers and test passers increase, it is possible for the percent of passers to go down.  So if percent participation isn’t really a good measure of success, as I suggest, and if percent passing isn’t a good measure of success, as John White suggests, what metric could be used to measure AP success which cannot possibly be gamed?
Fortunately, there is one.  The College Board keeps a statistic of the percent of the graduating class who pass an AP exam.  This is a number that, for 2014 ranges from the lowest, Mississippi at 3.2%, to the highest, Maryland at 22%.
How did Louisiana rank on this metric in 2014?
aprn_2014_ap_national_6So Louisiana ranked second to last with just 4.1%.
The College Board produced graphs like this for 2012 and 2013, which can be used to see Louisiana’s progress over the past three years.
Screen shot 2014-10-07 at 10.46.55 PMIn 2013 they were also second to last.  The 5.3% was higher than the 4.1% in 2014, but the numbers can’t be truly compared because the 2014 numbers are for juniors and seniors while the 2013 numbers are just for seniors.
Screen shot 2014-10-07 at 10.36.37 PMAnd, yes, in 2012 they were also second to last.  The 2012 number is 6.3% which means that they did drop in this metric from 2012 to 2013 and they were calculated the same way.
So with all the cheering coming from Louisiana about the AP results, I’d say that what they have is an embarrassment, only better than Mississippi for the past three years.
I also found this chart ranking the ‘progress’ of the different states in AP results over the past ten years.  In it we see that Louisiana isn’t just low on achievement, but had the fourth lowest amount of ‘growth’ down there with the other miracle state, Tennessee.
Screen shot 2014-10-07 at 10.48.37 PM
Looking for results from individual schools, I found the New Orleans newspaper made an easy to use database with the AP results for any school.  In the RSD, things are looking really bad.  The first school I checked out was the famous miracle school Sci Academy.
Screen shot 2014-10-07 at 11.19.59 PMThough they had 155 students take an AP test, only 14 students even got a 3, which was 9%.
In general, the RSD did not have many passing scores.
Screen shot 2014-10-07 at 11.17.02 PMYou’ve got to love the school that had over 200 test takers with less than 10 passers.  The leader was a KIPP school which had 16.1% of their test takers pass.
Forcing kids to take an AP course or an AP test when they are not ready for it is not ‘raising the bar’ and ‘increasing rigor.’  If I were teaching a course and 9% of my students passed the final exam, there would need to be a serious discussion about what the problem was.  In this case the problem is surely the Louisiana Department of Education forcing students to take tests to pump up their participation numbers.  But the numbers, at least the right ones, don’t lie.  Louisiana, even after all their years of being allowed to perform their experiments on the kids down there, is still second to last in AP results in the one metric they are not able to rig.

No comments:

Post a Comment

Your Comments Welcome