The Lindahl Letter
The Lindahl Letter
Ethics, fairness, bias, and privacy (ML syllabus edition 7/8)
1
0:00
-7:11

Ethics, fairness, bias, and privacy (ML syllabus edition 7/8)

Part 7 of 8 in the ML syllabus series
1

This set of topics was either going to be the foundation to start this series or it was going to be collected as a set of thoughts at the end. You can tell that obviously I demurred from starting with ethics, fairness, bias, and privacy in machine learning until the full foundation was set for the topics under consideration. These topics are not assembled as an afterthought and are very important to any journey within the machine learning space. This technology in terms of machine learning and artificial intelligence has the potential to be near omnipresent in day to day life and certainly within anything where decision making or anything digital persists. Each of these topics is going to receive a solid overview followed by a series of scholarly articles like the previous lectures. You are now well aware from seeing dozens of other scholarly articles that these topics do not appear in each and every work and while they are conceptually foundational as intellectual guardrails they are not consistently presented that way in literature reviews or considerations for the practical work occurring within the machine learning space. I would clearly argue and have for years that just because you can do a thing does not mean that you should. You have to consider the consequences and realities of bringing that thing forward in a world where models and methods are so readily shared on GitHub and other platforms. 

Overlap certainly occurs between the topics of ethics, fairness, bias, and privacy within the machine learning academic space. I have tried to sort the articles to help enhance readability within the different categories, but you will see some overlap. 

Ethics - This topic got covered back in week 65. I’m going to rework part of that content here so if it feels familiar that is consistent with it appearing before about 20 weeks ago. Anybody preparing machine learning content should be comfortable with presenting ethics as a topic of consideration. I firmly believe and hope you would support that effort after coming along for this journey so far into this independent study syllabus. Ethics should be covered as a part of every machine learning course. Perhaps the best way to sum it up as an imperative would be to say, “Just because you can do a thing does not mean you should.” Machine learning opens the door to some incredibly advanced possibilities for drug discovery, medical image screening, or just spam detection to protect your inbox. The choices people make with machine learning use cases is where the technology and ethics have to be aligned.

No one really solid essay or set of essays on AI/ML ethics jumped out and caught my attention this week during my search. Part of my search involved digging into results from Google Scholar that yielded a ton of different options to read about “ethics in machine learning” [1]. A lot of those articles cover how to introduce ethics to machine learning courses and about the need to consider ethics when building machine learning implementations. Given that those two calls to action are the first things that come up and they are certainly adjacent to the primary machine learning content being shared it might make you take a moment to pause and consider how much the field of machine learning should deeply consider the idea that just because it can do something does not mean you should. Some use cases are pretty basic and the ethics of what is happening is fairly settled. Other use cases walk right up to the edge of what is reasonable in terms of fairness and equity.

Lo Piano, S. (2020). Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward. Humanities and Social Sciences Communications, 7(1), 1-7. https://www.nature.com/articles/s41599-020-0501-9.pdf 

Greene, D., Hoffmann, A. L., & Stark, L. (2019). Better, nicer, clearer, fairer: A critical assessment of the movement for ethical artificial intelligence and machine learning. https://scholarspace.manoa.hawaii.edu/server/api/core/bitstreams/849782a6-06bf-4ce8-9144-a93de4455d1c/content 

Fairness and Bias - Implementing machine learning algorithms generally involves working with imperfect datasets that have different biases that have to be accounted for and ultimately corrected. 

Corbett-Davies, S., & Goel, S. (2018). The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023. https://arxiv.org/pdf/1808.00023.pdf 

Chouldechova, A., & Roth, A. (2018). The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810. https://arxiv.org/pdf/1810.08810.pdf 

Barocas, S., Hardt, M., & Narayanan, A. (2017). Fairness in machine learning. Nips tutorial, 1, 2. https://fairmlbook.org/pdf/fairmlbook.pdf 

Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6), 1-35. https://arxiv.org/pdf/1908.09635.pdf 

Yapo, A., & Weiss, J. (2018). Ethical implications of bias in machine learning. https://scholarspace.manoa.hawaii.edu/server/api/core/bitstreams/d062bd2a-df54-48d4-b27e-76d903b9caaa/content 

Privacy - No conversation about machine learning would be complete without a consideration of privacy. A part of the ethical considerations surrounding the use of machine learning algorithms is inherently privacy of data and privacy of the outputs. 

Ji, Z., Lipton, Z. C., & Elkan, C. (2014). Differential privacy and machine learning: a survey and review. arXiv preprint arXiv:1412.7584. https://arxiv.org/pdf/1412.7584.pdf 

Rigaki, M., & Garcia, S. (2020). A survey of privacy attacks in machine learning. arXiv preprint arXiv:2007.07646. https://arxiv.org/pdf/2007.07646.pdf 

Conclusion - I wanted to refocus my efforts on the macro considerations related to ethics in machine learning at this point. I remembered that Rob May shared a weekend commentary as a part of the Inside AI newsletter recently about the dark side of reducing friction in taking action with advanced technology [2]. Rob even went as far as sharing an article from one of my favorite technology related sources “The Verge” about just how easy and low friction it was to use machine learning to suggest new chemical weapon builds [3]. That is a very real example of where reducing friction to doing a thing opens the door to very problematic actions that illustrate the need for a foundational set of ethics.

Links and thoughts:

“How to design algorithms with fairness in mind”

“Why our Screwdriver took 3 YEARS”

Top 5 Tweets of the week:

Footnotes:

[1] https://scholar.google.com/scholar?q=ethics+in+machine+learning&hl=en&as_sdt=0&as_vis=1&oi=scholart 

[2] https://inside.com/campaigns/inside-ai-31781/sections/inside-ai-commentary-by-robmay-275419 

[3] https://www.theverge.com/2022/3/17/22983197/ai-new-possible-chemical-weapons-generative-models-vx 

Research Note:

You can find the files from the syllabus being built on GitHub. The latest version of the draft is being shared by exports when changes are being made. https://github.com/nelslindahlx/Introduction-to-machine-learning-syllabus-2022

What’s next for The Lindahl Letter?

  • Week 87: MLOps (ML syllabus edition 8/8)

  • Week 88: The future of publishing

  • Week 89: your ML model is not an AGI

  • Week 90: What is probabilistic machine learning?

  • Week 91: What are ensemble ML models?

  • Week 92: National AI strategies revisited

  • Week 93: Papers critical of ML

  • Week 94: AI hardware (RISC-V AI Chips)

  • Week 95: Quantum machine learning

I’ll try to keep the what’s next list forward looking with at least five weeks of posts in planning or review. If you enjoyed this content, then please take a moment and share it with a friend. If you are new to The Lindahl Letter, then please consider subscribing. New editions arrive every Friday. Thank you and enjoy the week ahead.

Discussion about this podcast

The Lindahl Letter
The Lindahl Letter
Thoughts about technology (AI/ML) in newsletter form every Friday