Multivariate analysis:
what can it do?
Multivariate analysis sounds rather technical. What is actually happening to the data? Let's move a few diagrams and take a look.
First, let's lay out the café survey responses.
Six responses from a fictional café survey of 90 people
These are separate analyses using the same survey, not steps in one procedure.
1. Starting with a café survey
The food was good, but the seating wasn't comfortable. Let's turn impressions like these into six scores.
Six ratings, side by side.
Taste, service, quietness, seating, décor and value for money / 0–100 each
Six of the 90 fictional responses. The numbers let you compare cells without relying on color.
Figure 1: Part of a fictional dataset of 90 people. Each rating is on a 0–100 scale. Overall satisfaction is a separate question.
Two people may give the same taste score but different scores for service or seating. Looking at several variables together lets us explore these differences. This page uses “multivariate analysis” broadly, including multiple regression with a single outcome.
2. Multiple regression: predicting satisfaction
Let's predict overall satisfaction from ratings such as taste, service and seating. Start by changing just the taste score and watch the prediction.
Change the taste and seating scores.
Six ratings → predicted overall satisfaction / model fitted to fictional data
How the bars relate to the prediction
Each bar is bⱼ × (rating xⱼ − that rating's mean). Add all six contributions to the prediction for an average response. The four ratings held at their means contribute zero, so only the taste and seating bars are shown. This is the same calculation as ŷ = b₀ + b₁x₁ + … + b₆x₆.
Figure 2: A regression model using six ratings. Only the two adjustable ratings are shown; the other four stay at their means. Bars show changes from the prediction for an average response.
The prediction moves as you change taste. Now leave taste alone and change seating. The two scores change the prediction by different amounts.
Holding the other ratings fixed, how much does the prediction change when one rating increases by one point? That change is its partial regression coefficient.
A prediction is not an intervention effect
The diagram shows a model prediction. The survey alone does not establish that improving the food would raise satisfaction by the same amount.
3. Principal component analysis: two scores, one axis
Next, let's combine taste and service scores on one number line. How much of the original difference remains when we use fewer dimensions? Follow A and B in the diagram.
Rotate the axis and compare A with B.
The same 90 people and two ratings. Only the direction changes.
Taste and service are standardized. A and B keep their colors and shapes throughout.
Why this direction is a principal component
The value on the line is a weighted sum of the two original ratings. With the weight vector's length held fixed, the direction giving the largest variance is the first principal component. “Variation retained” is the projected variance divided by the sum of the two original variances. A and B help us follow individual points; the direction is chosen using all 90 people.
Figure 3: Standardized taste and service scores. Each point keeps its color and shape. The number line's scale stays fixed as the direction changes.
In the “Collapse” direction, A and B end up in almost the same place. Choose “Spread out” and they appear at different positions on the line.
The points haven't changed, but the direction makes quite a difference.
This diagram finds the direction that retains the most variation across all 90 people after projection onto one line. That is the first principal component direction.
Six ratings are harder to draw, but the idea is similar: combine the original values into fewer coordinates.
What is lost when we use one axis?
Differences perpendicular to that axis. The displayed percentage tells us how much of the two ratings' total variance remains on the line. Preserving variation and predicting satisfaction are different goals.
4. Factor analysis: suppose there is something in common
What if quietness, seating and décor ratings tend to rise and fall together? They might have something in common. For this diagram, let's assume a shared factor called “comfort” and see what happens.
Suppose there is a “comfort” factor.
Illustration of an assumed common-factor model / unique components held fixed
Each of the three ratings changes according to its own weight
The model used in this diagram
Each rating = 60 + 15 × weight × common factor + unique component. The weights are 0.85 for quietness, 0.75 for seating and 0.65 for décor. The unique components are fixed at −3, +4 and −1 points. Arrows show an assumed model, not evidence of causation. In an actual factor analysis, the model is estimated from correlations among observed variables.
Figure 4: An assumed common-factor model, not a factor estimated from the survey responses.
Press “Watch the shared movement” and the three bars grow and shrink together. They move by different amounts because each rating has a different weight linking it to the factor.
In an actual factor analysis, correlations among responses are used to fit a model like this. Variation specific to each item and measurement error also matter.
This can look similar to principal component analysis. The starting point is different, though: factor analysis assumes common factors to model relationships among items, while PCA combines the observed values into new coordinates.
Does the calculation discover “comfort”?
The name is an interpretation based on the items. The calculation alone cannot establish a psychological entity called “comfort” or a causal relationship.
5. Cluster analysis: grouping similar responses
Finally, let's group people with similar taste and seating scores. Press “Watch the grouping” to see memberships and centers change while the observed points stay in place.
The points stay put. Their groups change.
Grouping by taste and seating / both on a 0–100 scale
What counts as “similar” here?
The diagram uses Euclidean distance between taste and seating scores. K-means reduces the sum of squared distances from points to their group centers. Different starting centers or variables can produce different results. The other four ratings are not used in this grouping.
Figure 5: K-means using taste and seating scores. Colors and shapes show groups; crosses show centers.
Assign each point to the nearest center, then move each center to the average position of its group. Repeat these steps to form the groups.
Three groups do not establish that customers fall into three natural types. Change the number of groups and the same data will be divided differently.
6. Different questions, the same data
| What you want to do | Method | What is calculated |
|---|---|---|
| Predict one outcome | Multiple regression | Coefficients and predicted outcome values |
| View the data with fewer coordinates | Principal component analysis | New axes and each person's coordinates |
| Model shared variation | Factor analysis | Factors and their links to observed items |
| Group people with similar responses | Cluster analysis | Group memberships |
All four examples use the same survey, but the tasks differ: predict an outcome, combine measurements, model shared variation or find similar responses.
“Multivariate analysis” may sound like a lot at once. These diagrams offer a place to start. If one method catches your interest, try its controls and compare what you see with a textbook explanation.