Integrity of Data from #SLPSgovernance Online Survey

Today, I made phone call to Canada regarding St. Louis Public Schools.

The #SLPSgovernance online survey (more info here), in addition to default accepting multiple submissions from the same person or device, lacks a mechanism for verifying a survey respondent’s authenticity through any form of email, Google, or Facebook account-linking.

Upon contacting Tiffany, Support Agent for MetroQuest, the Vancouver-based company handling the SLPS Governance online survey, I learned some troubling technical details about the data collection and analysis methods.

Instead of implementing backend safeguards to ensure unique data is tied to a specific respondent, the final output is a raw data set exported to an Excel file which is to be manually sorted by Vector Communications‘ data management team, comprised of two individuals.

While IP addresses and timestamps for each entry are included in this Excel output file, the data management team possess unilateral authority for pruning all captured data. These individuals may sort entries by IP address, timestamp, email address, or any other data field, including survey answers, when determining how many duplicate survey submissions to strike or retain for the final data set presented to the Task Force (who will make a recommendation to the SAB on the future of SLPS governance structure).

Vector Communications’ official reason for not requiring email addresses is that the survey yields more participation because, in their experience, requiring an email address causes more people to “give up” at this final step and users do not complete/submit the survey.

“I believe the integrity of this process is further compromised because not only does it allow for manipulation of data, it does not account for our most marginalized citizens. Households who share computers and mobile devices, or people who use public library or school computers, may not necessarily have their opinions counted–meanwhile special interest organizations, particularly those with access to large networks of various IP addresses, may flood the results with numerous submissions. In short, there is excessive room for human error in both the survey response process and data handling.”

Natalie Vowell
Former Computer Lab Manager, University of Arkansas
Founder, Executive Director, Project Raise The Roof
Member, St. Louis City Board of Education

________________

“The poll in question accepts data without regard to the source. Meaning the data is not checked for duplicates, or anything that indicates multiple submissions from one user.

What’s more troubling, in my opinion, is any translation of the data after this point is left to an individual or group to determine its veracity. This means the data obtained will be presented in a way determined by that individual or group, which may not necessarily represent the raw data collected; regardless if the data is properly collected or not.”

Walter Holley
Applications Developer
SLPS Graduate, Gateway Institute of Technology, Class of 2001
Walnut Park West Native

________________

“There are steps that can be taken to prevent manipulation of an internet poll by a dedicated or interested actor/party. None of those steps seem to have been taken here. No internet poll is completely free from manipulation, but it is possible to make it non-trivial to inflate results, by requiring authentication (even third-party authentication, like log in with Google or Facebook) or having email validation with followup email confirmation.

Again, none of that has been done here. It is therefore both trivially easy to manipulate this poll either manually or automatically (with a simple script) and difficult to track any such manipulation (unless there is cookie information saved, or server-side IP logging, or some other means of at least rudimentarily de-duplicating multiple submissions).

Owing to these flaws, the information obtained from this survey should not be used for any official recommendations.”

Josh Smith
Solutions Engineer, Merkle, Inc.
SLPS Graduate, Gateway Institute of Technology, Class of 2000
Resident, Parent, St. Louis City

 

 

The best way to make it more difficult to skew the outcome of the SAB’s governance recommendation process is to complete the online survey.  The survey takes less than five minutes, but allows for long answer comments if you would like to include further opinions.  Survey closes at midnight on November 28th, 2017.

Leave a Reply