As the ACLU highlights the problem of Algorithms that discriminate we need to remember that Algorithms are only as good as the assumptions that they’re based on.
In reality we’d be better off if we replaced the idea of Algorithms as “clean unbiased maths” with “prose instructions written by fallible people.”
But alongside the potential for bringing about social progress, the Internet also holds the possibility of contributing to unlawful discrimination. An example of this potential negative impact is a patent recently acquired by Facebook that could conceivably permit loan servicers to gain access to the credit ratings of a loan applicant’s social network and then use that information to determine whether the applicant qualifies for a loan. The patent combines the possibility of serious invasions of privacy with the realistic prospect of illegal lending discrimination.
More here. The Guardian also had a piece on some of these problems two years ago.
Image of Justice via Wikimedia