Table of Contents

 

Preface

 

xi

 

 

 

 

 

Chapter 1

Introduction

1

 

 

1.1†† Machine learning vs. statistics

4

 

 

1.2†† This book and the ancillary material

5

 

 

1.3†† Types of machine learning models

6

 

 

1.4†† Validation and testing

8

 

 

1.5†† Data cleaning

16

 

 

1.6†† Bayesí theorem

18

 

 

Summary

20

 

 

Short concept questions

21

 

 

Exercises

22

 

 

 

 

 

Chapter 2

Unsupervised Learning

23

 

 

2.1†† Feature scaling

24

 

 

2.2†† The k-means algorithm

25

 

 

2.3†† Choosing k

30

 

 

2.4†† The curse of dimensionality

33

 

 

2.5†† Country risk

34

 

 

2.6†† Alternative clustering approaches

39

 

 

2.7†† Principal components analysis

41

 

 

Summary

45

 

 

Short concept questions

46

 

 

Exercises

47

 

 

 

 

 

Chapter 3

Supervised Learning: Linear and Logistic Regression†††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††††

 

49

 

 

3.1†† Linear regression: one feature

50

 

 

3.2†† Linear regression: multiple features

51

 

 

3.3†† Categorical features

54

 

 

3.4†† Regularization

56

 

 

3.5†† Ridge regression

56

 

 

3.6†† Lasso regression

60

 

 

3.7†† Elastic Net regression

63

 

 

3.8†† Application to Predicting House Prices

64

 

 

3.9†† Logistic regression

69

 

 

3.10 Decision criteria

71

 

 

3.11 Application to credit decisions

72

 

 

3.12 The k-nearest neighbor algorithm

79

 

 

Summary

80

 

 

Short concept questions

81

 

 

Exercises

82

 

 

 

 

 

Chapter 4

Supervised Learning: Decision Trees

85

 

 

4.1†† Nature of decision trees

86

 

 

4.2†† Information gain measures

87

 

 

4.3†† Application to LendingClub data

89

 

 

4.4†† The naÔve Bayes classifier

94

 

 

4.5†† Continuous target variables

98

 

 

4.6†† Ensemble learning

102

 

 

Summary

103

 

 

Short concept questions

104

 

 

Exercises

105

 

 

 

 

 

Chapter 5

Supervised Learning: SVMs

107

 

 

5.1†† Linear SVM classification

107

 

 

5.2†† Modification for soft margin

114

 

 

5.3†† Non-linear separation

117

 

 

5.4†† Predicting a targetís value

119

 

 

Summary

122

 

 

Short concept questions

123

 

 

Exercises

124

 

 

 

 

 

Chapter 6

Supervised Learning: Neural Networks

125

 

 

6.1†† ANNs with a single hidden layer

125

 

 

6.2†† Multi-layer ANNs

130

 

 

6.3†† Other activation functions

132

 

 

6.4†† Gradient descent algorithm

134

 

 

6.5†† Variations on the basic method

140

 

 

6.6†† The stopping rule

141

 

 

6.7†† The Black−Scholes−Merton formula

143

 

 

6.8†† Extensions

146

 

 

6.9†† Understanding volatility movements

147

 

 

Summary

152

 

 

Short concept questions

152

 

 

Exercises

 

 

153

 

 

Chapter 7

Further Applications of Neural Networks

155

 

 

7.1†† Autoencoders and PCA

155

 

 

7.2General autoencoder design

158

 

 

7.3†† Variational autoencoders

160

 

 

7.4†† Generative adversarial networks

161

 

 

7.5†† Recurrent neural networks

163

 

 

7.6 ††Convolutional neural networks

165

 

 

Summary

169

 

 

Short concept questions

169

 

 

Exercises

170

 

 

 

 

 

 

 

 

 

Chapter 8

Reinforcement Learning

171

 

 

8.1†† The multi-armed bandit problem

172

 

 

8.2†† Changing environment††

177

 

 

8.3†† The game of Nim

179

 

 

8.4†† Temporal difference learning

182

 

 

8.5 ††When the opponent learns

185

 

 

8.6 ††Deep Q-learning

186

 

 

8.7†† Playing chess

186

 

 

8.8†† Applications

187

 

 

8.9†† Optimal trade execution

189

 

 

8.10 Data issues

192

 

 

Summary

192

 

 

Short concept questions

193

 

 

Exercises

194

 

 

 

 

 

Chapter 9

Natural Language Processing

195

 

 

9.1Sources of data

198

 

 

9.2†† Pre-processing

199

 

 

9.3†† Bag-of-words model

200

 

 

9.4†† Application of naÔve Bayes classifier

202

 

 

9.5†† Application of other algorithms

206

 

 

9.6†† Information retrieval

207

 

 

9.7†† Other NLP applications

208

 

 

Summary

210

 

 

Short concept questions

211

 

 

Exercises

211

 

 

Chapter 10†††

 

Model Interpretability

 

213

 

 

10.1†† Linear regression

215

 

 

10.2†† Logistic regression

219

 

 

10.3†† Black-box models

222

 

 

10.4†† Shapley values

223

 

 

10.5†† LIME

226

 

 

Summary

226

 

 

Short concept questions

227

 

 

Exercises

228

 

 

 

 

 

Chapter 11

Issues for Society

229

 

 

11.1Data privacy

230

 

 

11.2Biases

231

 

 

11.3Ethics

232

 

 

11.4Transparency

233

 

 

11.5Adversarial machine learning

233

 

 

11.6Legal issues

234

 

 

11.7Man vs. machine

235

 

 

 

 

 

Answers to End of Chapter Questions

 

237

Glossary of Terms

 

255

Index

 

265