Helter Skelter in a summer swelter

"Helter Skelter" is a Beatles song which appears on the "white" album. Charles Manson, claiming to have been "inspired" by the song (through which he thought God and/or the devil were taking to him) led his followers in the Tate-LaBianca murders.

Is "summer swelter" a reference to the "Summer of Love" or perhaps to the "long hot summer" of Watts?

The birds flew off with the fallout shelter
Eight miles high and falling fast

The Byrd's "Eight Miles High" was on their late 1966 release "Fifth Dimension". It was one of the first records to be widely banned because of supposedly drug-oriented lyrics.

It landed foul on the grass

One of the Byrds was busted for possesion of marijuana.

The players tried for a forward pass

Obviously a football metaphor, but about what? It could be the Rolling Stones, i.e. they were waiting for an opening which really didn't happen until the Beatles broke up. With the jester on the sidelines in a cast

On July 29, 1966, Dylan crashed his Triumph 55 motorcycle while riding near his home in Woodstock, New York. He spent nine months in seclusion while recuperating from the accident.

Now the halftime air was sweet perfume

Drugs, man.

Well, now, wait a minute; that's probably too obvious. It's possible that this line and the next few refer to the 1968 Democratic National Convention. The "sweet perfume" is probably tear gas.

While sergeants played a marching tune

Following from the thought above, the sergeants would be the Chicago Police and the Illinois National Guard, who marched the protestors out of the park and into jail.

Alternatively, this could refer to the Beatles' "Sgt. Pepper's Lonely Hearts Club Band". Or, perhaps McLean refers to the Beatles' music in general as "marching" because it's not music for dancing. Or, finally, the "marching tune" could be the draft.

We all got up to dance
Oh, but we never got the chance

The Beatles' 1966 Candlestick Park concert only lasted 35 minutes. Or, following on from the previous comment, perhaps he meant that there wasn't any music to dance to.