1 Introduction
This book started as a collection of notes from several classes I gave in the Department of Biological Sciences at the Universit�� de Montréal, as well as a few workshops I ran for the Québec Centre for Biodiversity Sciences. In teaching data synthesis, data science, and machine learning to biology students, I realized that the field was missing a stepping stone to proficiency. There are excellent manuals covering the mathematics of data science and machine learning; there are many good papers giving overviews of some applications of data science to biological problems; and there are, of course, thousands of tutorials about how to write code (some of them are good!).
But one thing that students commonly called for was an attempt to tie concepts together, and to explain when and how human decisions were required in ML approaches (Sulmont, Patitsas, and Cooperstock 2019). This is this attempt.
There are, broadly speaking, two situations in which reading this book is useful. The first is when you are done reading some general books about machine learning, and want to see how it can be applied to problems that are more specific to biodiversity research; the second is when you have a working understanding of biodiversity research, and want a stepping stone into the machine learning literature. Note that there is no scenario where you stop after reading this book – this is by design. The purpose of this book is to give a practical overview of “how data science for biodiversity happens”, and this needs to be done in parallel to even more fundamental readings.
These are examples of books I like. I found them comprehensive and engaging. They may not work for you.
A wonderful introduction to the mathematics behind machine learning can be found in Deisenroth, Faisal, and Ong (2020), which provides stunning visualization of mathematical concepts. Yau (2015) is a particularly useful book about the ways to visualize data in a meaningful way. (Watt, Borhani, and Katsaggelos 2020) TK
When reading this book, I encourage you to read the chapters in order. They have been designed to be read in order, because each chapter introduces the least possible quantity of new concepts, but often requires to build on the previous chapters. This is particularly true of the second half of this book.
note on the meaning of colors
1.1 Core concepts in data science
1.1.1 EDA
1.1.2 Clustering and regression
1.1.3 Supervised and unsupervised
1.1.4 Training, testing, and validation
1.1.5 Transformations and feature engineering
1.2 An overview of the content
In Chapter 2, we introduce some fundamental questions in data science, by working on the clustering of pixels in Landsat data. The point of this chapter is to question the way we think about data, and to start a discussion about an “optimal” model, hyper-parameters, and what a “good” model is.
In Chapter 3, we revisit well-trodden statistical ground, by fitting a linear model to linear data, but uisng gradient descent. This provides us with an opportunity to think about what a “fitted” model is, whether it is possible to learn too much from data, and why being able to think about predictions in the unit of our problem helps.
In Chapter 4, we start introducing one of the most important bit element of data science practice, in the form of cross-validation. We apply this technique to the prediction of plant phenology over a millenia, and think about the central question of “what kind of decision-making can we justify with a model”.
In Section 6.2, we discuss data leakage, where it comes from, and how to prevent it. This leads us to introducing the concept of data transformations as a model, which will establish some best practices we will keep on using throughout this book.
In Chapter 5, we introduce the task of classification, and spend a lot of time thinking about biases in predictions, which are acceptable, and which are not. We start building a model for the distribution of the Reindeer, which we will improve over a few chapters.
In ?sec-variable-selection, we explore ways to perform variable selection, think of this task as being part of the training process, and introduce ideas related to dimensionality reduction. We further improve our distribution model.
In Chapter 7, we conclude story arcs that had been initiated in a few previous chapters, and explore training curves, the tuning of hyper-parameters, and moving-threshold classification. We provide the final refinements to out model of the Reindeer distribution.
In Chapter 8, we will shift our attention from prediction to understanding, and explore techniques to quantify the importance of variables, as well as ways to visualize their contribution to the predictions. In doing so, we will introduce concepts of model interpretation and explainability.
In ?sec-bagging, …
1.3 Some rules about this book
When I started aggregating these notes, I decided on a series of four rules. No code, no simulated data, no long list of model, and above all, no iris dataset. In this section, I will go through why I decided to adopt these rules, and how it should change the way you interact with the book.
1.3.1 No code
This is, maybe, the most surprising rule, because data science is programming (in a sense). But sometimes there is so much focus on programming that we lose track of the other, important aspects of the practice of data science: abstractions, relationship with data, and domain knowledge.
This book did involve a lot of code. Specifically, this book was written using Julia (Bezanson et al. 2017), and every figure is generated by a notebook, and they are part of the material I use when teaching from this content in the classroom. But code is not a universal language, and unless you are really familiar with the language, code can obfuscate. I had no intention to write a Julia book (or an R book, or a Python book). The point is to think about data science applied to ecological research, and I felt like it would be more inclusive to do this in a language agnostic way.
And finally, code rots. Code with more dependencies rots faster. It take a single change in the API of a package to break the examples, and then you are left with a very expensive monitor stand. With a few exceptions, the examples in this book do not use complicated packages either.
1.3.2 No simulated data
I have nothing against simulated data. I have, in fact, generated simulated data in many different contexts, for training or for research. But the limit of simulated is that we almost inevitably fail to include what makes real data challenging: noise, incomplete or uneven sampling, data representation artifacts. And so when it is time to work on real data, everything seems suddenly more difficult.
Simulated data have immense training value; but it is also important to engage with the imperfect actual data, as we will overwhelmingly apply the concepts from this book to them. For this reason, there are no simulated data in this book. Everything that is presented correspond to an actual use case that proceeds from a question we could reasonably ask in the context, paired with a dataset that could be used to answer this question.
1.3.3 No model zoo
My favorite machine learning package is MLJ (Blaom et al. 2020). When given a table of labels and a table of features, it will give back a series of models that match with these data. It speeds up the discovery of models considerably, and is generally a lot more informative than trying to read from a list of possible techniques. If I have questions about an algorithm from this list, I can start reading more documentation about how it works.
Reading a long enumeration of things is boring; unless it’s sung by Yakko Warner, I’m not interested, and I refuse to inflict it on people. But more importantly, these enumerations of models often distract from thinking about the problem we want to solve in more abstract terms. I rarely wake up in the morning and think “oh boy I can’t wait to train a SVM today”; chances are, my thought process will be closer to “I need to tell the mushroom people where I think the next good foraging locations will be”. The rest, is implementation details.
In fact, 90% of this book uses only two models: linear regression, and the Naïve Bayes Classifier. Some other models are involved in a few chapters, but these two models are breathtakingly simple, work surprisingly well, run fast, and can be tweaked to allow us to build deep intuitions about how machines learn. They are perfect for the classroom, and give us the freedom to spent most of our time thinking about how we interact with models, and why, and how we make methodological decisions.
1.3.4 No iris dataset
From a teaching point of view, the iris dataset is like hearing Smash Mouth in a movie trailer, in that it tells you two things with absolute certainty. First, that you are indeed watching a movie trailer. Second, that you could be watching Shrek instead. There are datasets out there that are infinitely more exciting to use than iris.
But there is a far more important reason not to use iris: eugenics.
Listen, we made it several hundred words in a text about quantitative techniques in life sciences without encountering a sad little man with racist ideas that academia decided to ignore because “he just contributed so much to the field, and these were different times, maybe we shouldn’t be so quick to judge?”. Ronald Aylmer Fisher, statistics’ most racist nerd, was such a man; and there are, of course, those who want to consider the possibility that you can be outrageously racist as long as you are an outstanding scientist (Bodmer et al. 2021).
The iris dataset was first published by Fisher (1936) in the Annals of Eugenics (so, there’s a bit of a red flag there already), and draws from several publications by Edgar Anderson, starting with Anderson (1928); Unwin and Kleinman (2021) have an interesting historiographic deep-dive into the correspondence between the two. Judging by the dates, you may think that Fisher was a product of his time. But this could not be further from the truth. Fisher was dissatisfied with his time, to the point where his contributions to statistics were done in service of his views, in order to provide the appearance of scientific rigor to his bigotry.
Fisher advocated for forced sterilization for the “defectives” (which he estimated at, oh, roughly 10% of the population), argued that not all races had equal capacity for intellectual and emotional development, and held a host of related opinions. There is no amount of contribution to science that pardon these views. Coming up with the idea of the null hypothesis does not even out lending “scientific” credibility to ideas whose logical (and historical) conclusion is genocide. That Ronald Fisher is still described as a polymath and a genius is infuriating, and we should use every alternative to his work that we have.
Thankfully, there are alternatives!
The most broadly known alternative to the iris dataset is penguins, which was collected by ecologists (Gorman, Williams, and Fraser 2014), and published as a standard dataset (Horst, Hill, and Gorman 2020) so that we can train students without engaging with the “legacy” of eugenicists. The penguins dataset is also genuinely good! The classes are not so obviously separable, there are some missing data that reflect the reality of field work, and the data about sex and spatial location have been preserved, which increases the diversity of questions we can ask. We won’t use penguins either. It’s a fine dataset, but at this point there is little that we can write around it that would be new, or exciting. But if you want to apply some of the techniques in this book? Go penguins.
References
Anderson, Edgar. 1928. “The Problem of Species in the Northern Blue Flags, Iris Versicolor l. And Iris Virginica l.” Annals of the Missouri Botanical Garden 15 (3): 241. https://doi.org/10.2307/2394087.
Bezanson, Jeff, Alan Edelman, Stefan Karpinski, and Viral B. Shah. 2017. “Julia: A Fresh Approach to Numerical Computing.” SIAM Review 59 (1): 65–98. https://doi.org/10.1137/141000671.
Blaom, Anthony, Franz Kiraly, Thibaut Lienart, Yiannis Simillides, Diego Arenas, and Sebastian Vollmer. 2020. “MLJ: A Julia Package for Composable Machine Learning.” Journal of Open Source Software 5 (55): 2704. https://doi.org/10.21105/joss.02704.
Bodmer, Walter, R. A. Bailey, Brian Charlesworth, Adam Eyre-Walker, Vernon Farewell, Andrew Mead, and Stephen Senn. 2021. “The Outstanding Scientist, R.A. Fisher: His Views on Eugenics and Race.” Heredity 126 (4): 565–76. https://doi.org/10.1038/s41437-020-00394-6.
Deisenroth, Marc Peter, A. Aldo Faisal, and Cheng Soon Ong. 2020. “Mathematics for Machine Learning,” February. https://doi.org/10.1017/9781108679930.
Fisher, R. A. 1936. “The Use Of Multiple Measurements In Taxonomic Problems.” Annals of Eugenics 7 (2): 179–88. https://doi.org/10.1111/j.1469-1809.1936.tb02137.x.
Gorman, Kristen B., Tony D. Williams, and William R. Fraser. 2014. “Ecological Sexual Dimorphism and Environmental Variability Within a Community of Antarctic Penguins (Genus Pygoscelis).” Edited by André Chiaradia. PLoS ONE 9 (3): e90081. https://doi.org/10.1371/journal.pone.0090081.
Horst, Allison M, Alison Presmanes Hill, and Kristen B Gorman. 2020. Allisonhorst/Palmerpenguins: V0.1.0. Zenodo. https://doi.org/10.5281/ZENODO.3960218.
Sulmont, Elisabeth, Elizabeth Patitsas, and Jeremy R. Cooperstock. 2019. “Can You Teach Me to Machine Learn?” Proceedings of the 50th ACM Technical Symposium on Computer Science Education, February. https://doi.org/10.1145/3287324.3287392.
Unwin, Antony, and Kim Kleinman. 2021. “The Iris Data Set: In Search of the Source of Virginica.” Significance 18 (6): 26–29. https://doi.org/10.1111/1740-9713.01589.
Watt, Jeremy, Reza Borhani, and Aggelos Katsaggelos. 2020. “Machine Learning Refined,” January. https://doi.org/10.1017/9781108690935.
Yau, Nathan. 2015. “Visualize This,” October. https://doi.org/10.1002/9781118722213.