The medieval modes are as close as we get to scales in pre-common-practice music (that is, music prior to 1600 or so). They aren’t really scales, though.
To start, let’s look at the diatonic scale. It’s what modern scales are built on, and what the medieval modes are built on. The diatonic scale is this pattern:
H = Half step
W = Whole step.
If you think of a piano keyboard, a whole step is the distance between two keys that have only one key in the middle. C – D is a whole step. E to F♯ is a whole step. A half step is the distance between two keys that have no keys have no keys in between. E to F is a half step. C to C♯ is a half step.
If you look at the entire piano keyboard, the white keys form the diatonic scale. The pattern mentioned above is repeated over and over again.
The C major scale is when you start on C and then repeat the diatonic scale pattern over and over making C your most important note. The pattern looks like this:
The brackets contain one period of the C major pattern.
The A (natural) minor scale is when you start on A and then repeat the diatonic scale pattern over and over making A the most important note. The pattern will look like this
The brackets contain one period of the A minor pattern. In bold is the C major pattern, just to emphasize that this is still the diatonic scale, we’ve only shifted the start and end point of the period.
This idea of shifting the starting point of the diatonic scale period is the basis of modes. In fact, you could call C major and A minor modes in this sense. Ionian mode is another name for the major scale, and Aeolian mode is another name for natural minor. These terms were both coined in the 16th century by music theorist Heinrich Glarean. For whatever reasons, those modes weren’t part of the musical theory before then.
The modes that were in use were the ones starting on D, E, F, and G.
The mode starting on D was called Dorian. Using a keyboard, if you play and octave from D to D using only white keys you get the Dorian mode.
The pattern in steps looks like this:
Again I’ve bolded the C major pattern. It’s still there!
So if you’re in D it’s easy to play in the Dorian mode. Just play D a lot and don’t play any sharps or flats. But it’s not too hard to remember if you shift to starting on a different note. The modern key of D minor has a B♭, and B♭ is the sixth of the D minor scale. So if you sharp the 6th of a minor scale (and leave the 7th flat) you get Dorian. Or, you can think of it as playing an ascending harmonic minor scale, but leaving the 7th flat.
For example, A Dorian would look like this:
A B C D E F♯ G A
G Dorian would look like this:
G A B♭ C D E F G
G Dorian happens a lot in renaissance music. The way to identify it is if you have a piece with one flat that ends on G.
Another way to identify dorian pieces is if the piece would be in minor, but the last flat is missing from the key signature. The modern key of G minor has both B♭ and E♭ in the key signature. G Dorian only has B♭.
In Lydian, you start on F and play only white keys.
There’s a weird thing about Lydian, though. Because of (I believe) hexachordal reasons, the B is flat most of the time. So you effectively get a major scale. This may be why Medieval music theorists hadn’t considered Ionian mode, since they got what it provides with Lydian.
So, if you play in F major, you’re probably playing Lydian.
Start on G and play only white keys.
Or, play a G scale without the F♯. Or, play any major scale without the leading tone.
Here’s C mixolydian:
C D E F G A B♭ C
Start on E and play only white keys.
You’ll hear people talk about Phrygian with adjectives like “weird” or “eerie”. In the middle ages it was associated with the choleric humor, i.e. angry. In fact all of the modes had an associated humor.
Another way to think of Phrygian is to play E minor without the F♯. Or, play a minor scale and flat the second.
A Phrygian would look like this:
A B♭ C D E F G A
What I’ve described up to now is how modern musicians think and use modes. (With the exception of Lydian — modern musicians would play an F scale without the B♭. Otherwise you’re just playing a major scale.)
Medieval modes have a few more wrinkles.
Authentic vs. Plagal
The modes were used as a classification system for chants. Chants are melodies. A way you can classify a melody is by its range (how low it goes to how high it goes) and where the “tonic” (as we would think of it) or “final” (as the medieval music theorist would think of it) is in relation to the range. Is it at the bottom like, say “Twinkle, Twinkle Little Star?”
The range for Twinkle is a sixth, in this case middle C to A above middle C. Its final is the bottom note, in this case C.
Or, is it in the middle of the melody, like “O Come All Ye Faithful?”
The range is an octave (from D to D an octave higher), and its final is in the middle of the range. (In this case G)
In the middle ages, Twinkle Twinkle would be classified as “Authentic”: the final is at the bottom of the range. “O Come All Ye Faithful” would be classified as “Plagal”: the final is in the middle of the range. Plagal modes have the prefix “Hypo”.
And with that, here’s a chart of the Medieval modes:
There’s a bit more to the medieval modes than this, like when does one flat the Bs? And what if the range of a melody is really large? And how does hexachord theory fit into all of this? But I don’t understand that well enough to answer accurately.
How do the modes fit into playing and composing music in a medieval or Renaissance style?
I think that the modern understanding of modes is the most relevant part. That is, knowing how to modify a major or minor scale to make a mode. Want to make Greensleeves in A minor sound modal? Don’t sharp the G! Do sharp the F.
When you’re trying to play along with a modal piece, the easiest and most convincing accompaniment is a drone on the final. Or maybe the final and a fifth above it. Greensleeves in Dorian with a drone on the final will sound good. Carelessly applying chords will sound weird and probably bad.
The plagal vs. authentic part I wouldn’t worry overmuch about. Does it really matter if your piece’s final is in the middle of the range vs. the bottom? No. Just have a final.
In performing polyphony, it can be useful to know what mode you’re in if you’re trying to sight sing using movable-do solfege. For instance, if you’re learning a piece in G Dorian, you might be tempted, via the key signature of one flat, to sing the piece as if it was in F major or D minor – that is, assigning the syllable ‘do’ to F, ‘re’ to G, and ‘la’ to D. But, your ear might be better served by assigning the ‘la’ to G and singing ‘fi’ (a sharped ‘fa’) for the E♮ – that is, singing as if it was in G minor with a sharped sixth. (Neither approach is relevant to how it would have been sight-sung at the time – again, see hexachord theory for more on that…)
In writing polyphony, modes have basically nothing to say about how the parts interact. You have to look to treatises on counterpoint for that.
And that’s an introduction to modes. Hope it helps!