machine-learning

Artificial Intelligence (AI) aims to build systems that can learn, adapt, and make decisions.
One powerful tool is the neural network, inspired by the brain.

From Statistics to AI

• Regression predicts Y from X
• Logistic regression predicts probability (0–1)
• Neural networks generalize this idea: many inputs, many layers, nonlinear patterns

The Structure of a Neural Network

1. Input layer — variables (X₁, X₂, …)
2. Hidden layers — units that transform the input
3. Output layer — prediction or classification

Each connection has a weight (like a slope in regression).

Formula for a Neuron

A single unit in the network:

$$z = \sum w_i X_i + b$$

$$y = f(z)$$

Where:

• $$w_i$$ = weights
• $$X_i$$ = inputs
• $$b$$ = bias (like an intercept)
• $$f(z)$$ = activation function (e.g., logistic, ReLU)

Learning in a Network

The network predicts outputs and compares them with the true answers.
The error is sent backward through the network to adjust weights.
This is called backpropagation.

Example

Predicting if a student will pass or fail based on:

• Study hours
• Attendance
• Practice problems completed

Inputs → combined with weights → logistic activation → output: probability of passing.

Visuals

Figure 18.1 — Simple Neural Network (Inputs → Hidden → Output)

Figure 18.2 — Activation Functions

Why This Matters

• Neural networks extend regression and logistic regression.
• They allow learning from large, complex datasets (images, speech, language).
• Modern AI (translation, recognition, chatbots) is powered by these models.

Practice self-test quiz

In the space below, please find practice problems and self-test quizzes. For full access, please signup free.

Machine learning is where statistics meets computers.
Instead of only writing formulas, we teach a computer to learn patterns from data.

What is Machine Learning?

Machine learning uses algorithms to improve automatically with experience.

• Supervised learning: the computer is given examples with correct answers.
• Unsupervised learning: the computer finds patterns without answers.

Supervised Learning

Goal: predict Y from X.

Examples:

• Predict exam scores from study hours
• Predict house price from size, location, and age

Steps:

1. Split data into training set and test set
2. Train the model on training data
3. Test accuracy on new (unseen) data

Formula (simple linear regression as machine learning):
$$\hat{Y} = a + bX$$

Here, the computer “learns” $$a$$ and $$b$$ from the data.

Unsupervised Learning

Goal: find hidden structure in the data.

Examples:

• Group students by study habits
• Cluster shoppers by buying patterns

Algorithms:

• k-means clustering
• Hierarchical clustering

No “correct answer” is given — the computer organizes the data.

Overfitting vs. Generalization

• Overfitting: the model memorizes the training data but fails on new data.
• Generalization: the model captures the underlying pattern and works on new data.

Example:
If a student memorizes past exam answers (overfit), they may fail a new test.
If they learn the concepts (generalize), they succeed.

Key Concepts

• Training set: data used to build the model
• Test set: data used to evaluate performance
• Accuracy: how well the model predicts new data

Visuals

Figure 16.1 — Supervised learning example: regression line predicting Y from X.

Figure 16.2 — Unsupervised learning example: scatterplot with clusters (k-means).

Figure 16.3 — Overfitting vs. generalization: wiggly curve vs. smooth line.

Why This Matters

Machine learning grows directly out of statistics:

• Regression → prediction
• ANOVA → group classification
• Clustering → organizing data

By learning the basics of ML, students see how statistics powers AI.

Practice self-test quiz

In the space below, please find practice problems and self-test quizzes. For full access, please signup free.

In the past, statistics dealt with small datasets: 20 students in a class, 50 patients in a trial.
Today, we live in the age of big data: millions of tweets, billions of web pages, streams of data from phones, sensors, and satellites.

Big data changes the scale of statistics.

What is Big Data?

Big data is often described by the 3 Vs:

1. Volume — enormous amounts of data (terabytes, petabytes)
2. Velocity — data generated quickly (social media streams, stock markets)
3. Variety — many forms (numbers, text, images, audio, video)

Sometimes a fourth V is added: Veracity (how reliable are the data?).

Why Big Data Matters

• Traditional statistics assumes small, clean datasets.
• With big data, we need algorithms and computers to process information.
• Sampling becomes less important when entire populations are measured (e.g., all tweets in a week).
• Visualization and summaries are critical to make sense of huge datasets.

Example

• A teacher records grades for 30 students → small dataset.
• YouTube collects billions of video views per day → big data.

Statistical tools remain the same (mean, median, regression), but the scale requires computational methods.

Tools for Big Data

• Databases (SQL, NoSQL) to store data
• Distributed computing (Hadoop, Spark) to process data
• Statistical programming (R, Python) for analysis

Visuals

Figure 14.1 — Big Data and the 3 Vs. Diagram showing Volume, Velocity, Variety (and Veracity) in overlapping circles.

Why This Matters

Big data connects statistics to the modern world:

• Online behavior, medical records, GPS signals, shopping patterns
• Algorithms detect patterns too large for humans to see
• Big data powers modern AI and machine learning

Practice self-test quiz

In the space below, please find practice problems and self-test quizzes. For full access, please signup free.