A Melbourne startup, which has developed facial recognition technology for schools, is continuing its trials in classrooms — despite serious concerns being raised about data collection and a lack of regulation.
In October 2018, The Age revealed a number of schools were trialling a facial recognition technology in classrooms that removed the need for teachers to manually mark rolls each class.
The technology was designed by a Melbourne-based startup called LoopLearn, which has received a number of federal and local government grants in order to make its product commercially viable.
Despite Australia’s own government dropping its plans to implement a controversial biometric database, this startup’s technology has gone ahead with little oversight and regulation.
In response to The Age‘s report, the Victorian government’s Minister for Education James Merlino banned public schools in the state from using the technology unless they received approval from parents, students and the education department.
“I remain concerned about any rollout of this initiative. The fact that Scott Morrison is supporting this Big Brother-style system in our classroom does not change that,” Merlino told The Age, at the time. “Teachers are best placed to record student attendance, not robots.”
With the software company currently regulating itself, there are concerns about how comprehensive its data protection efforts are and whether the convenience of automating roll call procedures outweighs the sensitivity of collecting biometric information.
Australian Human Rights Commissioner Edward Santow has been an advocate of digital rights and is leading a project to advance protections against emerging AI technologies. He told Gizmodo Australia there needed to be sufficient justification to use such sensitive data.
“This use of technology inevitably has an impact on individuals’ privacy,” Santow told Gizmodo Australia in an email. “This is only acceptable if there is an open, compelling justification. A small increase in convenience or efficiency would rarely justify such a privacy impact.”
LoopLearn defended its use of the technology pointing to the fact that accurate roll call reporting is a legal requirement and the technology makes it more accurate than ever.
“Under the law, schools are required to accurately record student attendance at regular intervals throughout the school day. In some instances, this can be up to eight times a day and, unfortunately, is not always completed accurately,” a LoopLearn spokesperson said to Gizmodo Australia over email.
“LoopLearn automates the roll call process, improving accuracy, saving time, and ultimately helping schools ensure students are safe.”
Waverley College is currently undertaking a trial of Looplearn
Though initial criticism led to a number of schools backing out of trials, Gizmodo Australia has learned of at least one school still conducting trials.
Waverley College’s Waterford campus in Sydney’s Eastern Suburbs is currently undertaking a trial of the software in five classrooms, which started at the end of 2019 and is expected to run until the end of the first term in 2020.
“[We are running the] trial of LoopLearn as part of our efforts to reduce the time spent taking the roll in school,” a spokesperson from Waverley College told Gizmodo Australia.
“Students attend school to learn, but when roll call is taken both at the start of every day, as well as at every single class period throughout the day, that is a lot of time spent on admin instead of learning and it can be a distraction before a lesson has a chance to start.
“The only commitment is to assess the technology, look at the benefits, ask for feedback, and then if we move forward there will be further consultation with our school community.”
The college explained to Gizmodo Australia that parents and students were informed of the trial with no pushback.
“We communicated with parents and students that we wanted to run a very limited trial, with no feedback or concerns raised,” the spokesperson said. Gizmodo Australia has contacted the college’s Parents’ Association to attempt to independently verify this information but it declined to comment.
The school confirmed the trial, which runs to April, is costing them $2,700. In a 2019 post on the school’s site, it said it was considering extending the trial to music classes, the library, the health centre and would introduce mobile kiosks to register more students into the system.
Despite repeated attempts by Gizmodo Australia, the school’s governing body, Edmund Rice Education Australia (EREA), did not provide an explanation regarding its oversight of the trial and whether it’s aware of other schools conducting trials within the network.
[referenced url=”https://gizmodo.com.au/2019/10/australias-facial-recognition-bill-has-been-postponed-for-now/” thumb=”https://gizmodo.com.au/wp-content/uploads/2019/10/frecog-410×231.jpg” title=”Australia’s Facial Recognition Bill Has Been Postponed For Now” excerpt=”Proposed laws paving the way for a facial recognition database in Australia have been abandoned after a parliamentary report found they required stronger protections for the privacy of citizens.”]
How does LoopLearn work?
The concern with facial recognition technology being introduced in schools mostly revolves around how children’s personal information may be stored and used. LoopLearn argues this is not an issue with its technology as it’s not storing the data on its own servers – instead the data is temporarily stored on the school’s own server – and the company claims no data is shared with a third party.
“Instead of a teacher calling out student names to manually mark attendance, a single image is taken by a secure LoopLearn sensor situated on the wall of the classroom,” a LoopLearn spokesperson told Gizmodo Australia. “This sensor is connected to the school’s closed network, and facial recognition technology is used to accurately and instantly match the students presence against de-identified numerical values. Once roll call is complete, the image is permanently deleted.”
It’s demonstrated in a promotional video by the company, which explains the image corresponds to a series of digits. A fact sheet also points out the image in the database only has a student number attached to it — no name, date of birth or anything else.
“Data is de-identified and fully encrypted, and stored on secure servers based in Australia. Data is privately owned by each school, and is not shared with or accessible by any third party, including government bodies,” LoopLearn said in statement provided to Gizmodo Australia.
“Any image that may be captured is for the specific purpose of enabling schools to record attendance, and is deleted once the roll call is complete.
“Recognition data is de-identified, which means no identifying information on students is stored with this data — not their name, date of birth, address or gender, or any identifying information.”
LoopLearn explained it was providing the childrens’ guardians with the information needed to make an informed decision. Three schools Gizmodo Australia spoke to, however, did not specify whether a permission slip was required and whether students could opt out if they had concerns about the technology.
“Parents and carers at schools where LoopLearn is or has been trialled are provided extensive information about the technology, and all feedback to date has been positive. Schools also complete a Privacy Impact Assessment prior to the technology being trialled,” LoopLearn said in a statement to Gizmodo Australia.
Concerns about the use of children’s data to test new technology
Electronic Frontiers Australia (EFA), a digital rights advocacy group, is not so convinced about the software’s collection of data — especially data as sensitive as biometric.
“[LoopLearn] will need to have stored some data so that it has something to compare the student’s face to in order to see if it’s a match. That’s how these things work,” an EFA spokesperson said to Gizmodo Australia.
“Biometric data is much more personal than other kinds of data because it is intrinsic to your physical person. It’s much harder to change your face than to change your password or your credit card number when it leaks all over the internet, even if you are Nicholas Cage.”
Santow said the problem is that technologies are being introduced before being properly considered and tested. It’s the most vulnerable of us, like school children, that face the brunt of it.
“Too often we see new technology ‘beta tested’ on vulnerable people, including children. We need careful testing of any new technology before it is used to make sure that it doesn’t cause harm, and that any limitation on human rights can be justified. This should take place before the technology is used in a ‘real world’ scenario,” Santow said to Gizmodo Australia.
Despite vocal critics arguing the concept of facial recognition technology in schools is concerning on both a practical and ethical level, the company has been awarded a number of grants from both local and federal governments.
“The City of Melbourne’s Small Business Grants program aims to help passionate, innovative small businesses based within the City of Melbourne to realise their dreams and reach new markets” a City of Melbourne spokesperson said to Gizmodo Australia in late 2019.
The City of Melbourne told Gizmodo Australia it had awarded LoopLearn a $15,000 small business grant in August 2018.
“Grants are assessed by an independent panel, with a shortlist of recommended recipients then considered by Council. Grants assessors noted Looplearn’s proposal was innovative, and that Looplearn founders had extensive experience in education and firsthand understanding of the challenges teachers face,” the spokesperson said.
Just a few months later, in January 2019, it was provided with federal government support too.
“LoopLearn received a grant to help commercialise its product, which automates the roll marking process,” a spokesperson from the Federal Department of Industry, Innovation and Science told Gizmodo Australia.
That grant — awarded under Accelerating Commercialisation — was confirmed to come in at a grand total of $517,207.90 including GST.
[referenced url=”https://gizmodo.com.au/2020/01/facial-recognition-australian-federal-police-afp-clearview-ai/” thumb=”https://gizmodo.com.au/wp-content/uploads/2019/12/bodycam-410×231.jpg” title=”AFP Says It Isn’t Using That Creepy Facial Recognition App” excerpt=”Last week the New York Times revealed that an Australian startup had developed an alarming facial recognition app that’s being used by hundreds of law enforcement agencies in the U.S. and Canada. While the app isn’t public, that doesn’t rule out the potential for other agencies to be using it. As it turns out, the Australia Federal Police (AFP) apparently isn’t one of them.”]
Waverley College is just one school among many
Clarendon College in Victoria was one of the first schools to be publicly named in The Age’s report back in 2018. A spokesperson for the school confirmed to Gizmodo Australia in November 2019 that while it did undertake the trial, it did not go ahead with the software.
“We trialled two student attendance devices in two classrooms over several weeks in Term 4 in 2018. Our assessment of the effectiveness of the devices was inconclusive and they have not been operational in 2019,” the spokesperson said.
“Our association with the technology was brief. The hardware has not been operational in 2019 and has been removed.”
It’s understood that rationale was provided for parents but specific permission slips were not necessarily required.
Another Victorian school set to take on the trial was Geelong’s Sacred Heart College, but the school denied ever using the software to Gizmodo Australia.
To participate in trials, Victorian government schools are meant to ensure the trial is compliant with Victorian Protective Data Security Standards and to undertake a privacy impact assessment, to understand how the data is collected, used and stored. Gizmodo Australia confirmed with the Victorian Education Department in November 2019 that no government school had yet sought permission to undertake trials of the software.
In NSW, the state’s education department confirmed to Gizmodo Australia it had no plans to use facial recognition technology and was not aware of any schools using it. Additionally, the federal Department of Education, Skills and Employment said it was not engaged with Looplearn and had nothing to do with its operations or the data it collects.
It’s understood there are, however, a number of other schools across Australia undertaking the trial but LoopLearn has not released the names of schools who have completed — or are completing — the trial. Gizmodo Australia was also unable to find any schools that had gone ahead with a full implementation of the software.
It’s not stopping the company from expanding its product line. According to the company’s site, it also offers LoopKiosks, which allows visitors to sign in with the facial database through a free-standing device located in school entry points, like a reception. The products promotional video suggests contractors and non-school personnel will be able to pre-approved on the system before arrival and students will be able to sign in if they have schedules outside of regular class time.
[referenced url=”https://gizmodo.com.au/2019/10/australias-facial-recognition-database-is-causing-concern/” thumb=”https://gizmodo.com.au/wp-content/uploads/2019/10/facial-410×231.jpg” title=”Australia’s Facial Recognition Database Is Causing More Concerns” excerpt=”Ahead of the 2019 Australian federal election, two bills that would pave the way for a nation-wide facial database were ditched. Since then they have been re-introduced and a parliamentary inquiry is looking into their viability. A public hearing into the bills that was scheduled for next week has been unexpectedly cancelled and it’s not yet clear why. Here’s what we know so far.”]
Legislation isn’t keeping up with the technology
While it would be easy to chalk up the schools’ adoption of the technology as something nefarious, the reality is there is very little regulation or legislation meaning the advancements are a bit of a free-for-all.
“The legislation has not kept pace,” Associate Professor Bruce Baer Arnold from University of Canberra’s Law School told Gizmodo Australia.
“The default policy position in private [and] public schools is that use of the technology is justified by the need to protect minors and that it is cost effective.”
Professor Arnold pointed to the Privacy Act 1988 and The Workplace Privacy Act 2011 (ACT) as some examples of laws that protect aspects of a person’s biometric information but ultimately, there’s “no inherent restriction on facial recognition technologies.”
In the Privacy Act 1988, sensitive information can’t be collected by an organisation without consent unless it is “reasonably necessary for one or more of the entity’s functions or activities.” Sensitive information includes health, genetic and biometric information as well as biometric templates.
The problem is the decision makers, who decide whether to implement new software and technologies, aren’t very likely to be keeping up with the latest debates around the ethics and regulation to do with biometric data. This is something that concerns Associate Professor Erica Southgate deeply.
“We can’t expect school principals and teachers and parents and students to be really [up to speed] with that on purpose because we’re all playing catch up at the moment around the globe,” Southgate said.
“My key concerns would be that people working at schools or people who work at educational policy, for instance, just don’t have the technical background to ask the right questions of this technology and of the type of data it’s collecting. What it’ll be used for, how it’s stored, what happens if there are data breaches and what you’re giving away when somebody when a camera takes a photo of your face.”
It’s something Australia’s Human Rights Commission is looking at too. The commission is working on a discussion paper right now, which is open for submissions until March 23, 2020. It’s aware of how quickly technology is evolving and how governments are failing to keep up.
“Governments and companies are increasingly relying on biometric technology to perform a range of tasks. We need to ensure our laws are properly enforced to protect our human rights, especially our privacy,” Santow said.
“We are particularly concerned that some technology is moving faster than our laws. For example, the Commission proposes a moratorium on potentially harmful uses of facial recognition technology until effective human rights safeguards are in place.
“No matter who is using these technologies, whether the government or the private sector, our human rights and other laws must be rigorously enforced.”
If you’ve got any information on facial recognition technologies being used in Australian classrooms, send an email to sarah.basford@pedestriangroup.com.au.
[referenced url=”https://gizmodo.com.au/2019/10/australian-government-wants-to-scan-your-face-before-you-watch-porn/” thumb=”https://gizmodo.com.au/wp-content/uploads/2019/10/peterdutton-410×231.jpg” title=”Australian Government Wants To Scan Your Face Before You Watch Porn” excerpt=”An Australian government department has shown interest in forcing pornography sites to verify a user’s age, and it’s willing to offer its facial recognition services to get it done.”]