If there are any lawyers here, I'd love thoughts on whether these tools (used in employment/interview settings) violate 29 CFR § 801.4 - Prohibitions on lie detector use. (I know this article is about schools and in Canada). I've been thinking these tools do violate, but never filed a case alleging it (yet). Specifically, these anti-cheat "algorithms" seem to be "a polygraph, deceptograph, voice stress analyzer, psychological stress evaluator, or any other similar device (whether mechanical or electrical) that is used, or the results of which are used, for the purpose of rendering a diagnostic opinion regarding the honesty or dishonesty of an individual." I mean, the entire purpose is to determine if the prospective employee is honest. Any thoughts?
Exactly -- and who knows how invasive these programs are. We have no idea how they work because the companies keep the code private. They could be using the microphone, video camera, keystrokes, mouse movements etc, to feed some dubious pseudoscientific code to predict "cheating" and we're entirely reliant on their claims of robustness of it all--that troubles me.
Detecting behaviors to assess what though? Isn't that exactly what a lie detector does? It claims to detect electrical signals of behaviors (stress) and then that goes to determine if the person is honest.
A lie detector does (claim to) not detect facts. It (claims to) detect psychological states via its physiological measurements, which is obviously what the law is talking about.
No one would claim that fact-checking statements is a "lie detector" under this law.
We can go down the rabbit hole all we want, but a "lie detector" is a specific type of device. Not everything that can detect lies is a lie detector.
(And not everything called a "lie detector" can detect lies, either. A polygraph test doesn't detect lies at all... and yet it is exactly the type of test banned. In fact, that's one of the main reasons why it is banned: because it doesn't work for its stated purpose.)
Right, but the "specific type of device" is defined more broadly than that -- so if I code a "lie detector" that tracks mouse movements during an online interview (my hypothesis would be that liars and cheats move their mouse more sporadically), could I use that code as a part of an interview process?
> Right, but the "specific type of device" is defined more broadly than that...
More broadly than what? I can't recall giving a specific definition for lie detector.
It sounds like what you are describing is basically a way to use mouse movements to detect psychological stress, which is more or less the same way a polygraph is designed to work. So it would be banned, because it's the same type of thing that a polygraph is.
Uhmm yeah this is just so wrong. Proctorio CEO Mike Olsen accused Linkletter of ...
" potentially showing students ways to circumvent the software, thus risking the safety and security of the millions of students who use our platform.”
Say what? In what way(s?) did his actions impact on the students' safety or security? The videos have been pulled but note that the CEO is talking about how showing people how to 'possibly' (his words not mine) circumvert the proctoring in any way shape or form affects the students' safety or security?
[Edit for the pedantic, CEO said 'potentially' rather than 'possibly'.]
Proctorio's support is also fucking absurd! It reflects their CEO's attitude in that article.
I had an issue where their plugin, which only worked on chrome at the time IDK if that has changed, stopped working. And I had about 1 hour before my final exam.
They went through the normal expected steps, then started recommending crazy shit! For example, they recommended that I delete all my bookmarks from my browser and "see if that works". And if it didn't, to re-install the operating system. Their other recommendation was to go buy another laptop -- in an hour!
Then, they basically just said ¯\_(ツ)_/¯
I understand that there is only so much a phone tech can do in that situation, but these exams are very, very important. I am not sure I would have been able to get a waiver to take it at another time and could have ruined my GPA and been out thousands of dollars. Luckily I got it working on another computer that I had, so everything worked out in the end.
Why anyone would pay these folks anything is beyond me. While cheating is certainly a thing, and easier to do online, violating people's personal space and trusting in some average behavior BS to flag things as suspicious does not seem worth the cost. Add in ridiculous behavior by the company, if I were an educational institution I'd stay far away from these people. I remember reading about something similar being used to take screening tests for programmers which watched via the camera to ensure the test taker was not cheating; I'd tell any employer who wanted such a thing to get lost.
I took the GRE this summer. Because of the pandemic, I used ETS's new (and temporary) at-home testing option. Someone from ProctorU watched me through my webcam the whole time, and it felt totally invasive and creepy.
But, what else was the ETS supposed to do? For a high-stakes exam like the GRE, I don't think "trust people to not cheat" is a viable option. That isn't to say these remote proctoring solutions will necessarily prevent cheating either (and having now used one of these services, I think cheating would have been relatively easy), but it at least should make test takers think twice.
I think we can all agree that an in-person exam would have been better, but the state of the world made that untenable.
>That isn't to say these remote proctoring solutions will necessarily prevent cheating either (and having now used one of these services, I think cheating would have been relatively easy), but it at least should make test takers think twice.
Sounds like security theater.
>I think we can all agree that an in-person exam would have been better, but the state of the world made that untenable.
Why not? Just rent out a large open space (stadium, convention center, school gym), and space the students 20ft apart. The risk should be negligible. Let's be honest here, the real reason is that [cost of exam surveillance software] < [cost of conducting a physically distanced exam]
I think a bit of security theater is exactly what's called for in this scenario.
This isn't like hacking into a computer network, or bringing an explosive onto an airplane, where one dedicated bad actor can defeat the system for everyone. It's much closer to something like shoplifting, where your goal is to make instances of bad behavior as low as possible, given other constraints. A security camera does not need to be 100% effective to be worth installing—and doesn't even necessarily need to be turned on.
A remotely-administered multiple-choice test is like a retail store without any staff, which asks customers to kindly leave money in a basket. You're much better off with even a single cashier, and even one who spends most of their day watching Netflix.
People may not die if I cheat on my exam, but casting doubt on the accuracy of the results by eroding trust in the examination process certainly affects all who took the exam when they end up not getting a job.
Security theater doesn't apply evenly, however. I know a couple students who could pretty blatantly cheat without getting caught simply because they owned a second computer they could position in the right spot, and had a hardware disconnect on their microphone. People that aren't well-off or tech savvy are going to be at a significant "disadvantage" here.
... has been discussed at length by people with direct knowledge. E.g. Dr. David Joyner. In-person proctoring works for a niche set of situations but leaves a lot of other students unable to participate.
Why is a webcam more invasive and creepy than holding your entire body hostage at a testing location?
Only because you haven't taken the time to make sure your personal testing location is prelared for public viewing, since usually you don't have to worry about that.
It's a fair point. I don't actually have an issue with the webcam so much as the monitoring software they installed—it's a giant black box with admin permissions that I don't understand at all. (And, using a virtual machine was explicitly not allowed, for understandable reasons.)
I assume it hasn't left behind a giant security hole or explicit keylogger—and I did make a time machine backup before the test which I restored afterwards...
> Why anyone would pay these folks anything is beyond me.
Because they are worried about reputational damage from cheating. Even with tools like Proctorio, ProctorTrack, HonorLock, etc, the cheating rate is pretty high in programs like OMSCS.
That solution is existentially terrifying to them both in the "job depends upon not understanding it" sense and their very ideas of how they feel the world works/should work so are institutional non-starters until they are replaced one way or another (generationally or competition rendering them irrelevant).
Make a small abstract painting. Hang it in background. Now if proctorio records video of you they are violating the copyright of your painting. Sue for ONE BILLION DOLLARS!
This (for example) should be wildly illegal, if it isn't already:
"[Proctorio CEO] Olsen posted a copy of the student’s chat logs"
But also - colleges and universities should simply not be using this software. Students are not, in general, able to give active consent for such software to be used against their interest.
I wonder if the real issue is the format of the exams itself.
Caltech had take-home exams for a long time and students still adhere to the honor code.
We have pretty good tools for plagiarism detection. It's time to shift from rote learning exams to another format (essay, lab assignments for computer science). Else it's simply an arm race for cheating (soon some kid will just get an e-ink display in a picture frame and as soon as it goes out of the proctor's webcam field of view will turn into a cheat sheet).
But at the same time, if you have to put draconian measures to discourage cheating as the last resort, maybe it's because the school is turning into a diploma mill and should review it's admission standards.
Isn't the issue more that administrators are panicking over the loss of control from the pandemic causing a norms break and are pushing dubious anti-cheating software on everyone out of do-somethingism more than any amount of cheating sufficent for dillution of graduate quality?
Granted the "value evaluation" gets a bit silly to begin with given that either they trust the name enough to use it as a filter regardless of its merit or lack of it or they ignore it entirely and go based off of their own assessment measures with their own sets of issues.
Don't need to be out of the field of view. You just sync the camera with your picture frame and you see the cheat sheet while the webcam sees a Picasso.
Just needs a common trigger signal and some tweaking and you are good to go.