The roots go back to the late 1950s, when a handful of musicians started feeling cramped by bebop's own conventions. Bebop had been radical once, but by the mid-fifties it had its own orthodoxies — its chord changes, its tempos, its soloists-over-rhythm-section hierarchy. Ornette Coleman was the first to make that discomfort impossible to ignore. When he arrived in New York and started playing his plastic alto saxophone with a quartet that included Don Cherry, the jazz world split hard. Some heard chaos; others heard a completely coherent new language that just happened to have thrown out the harmonic rulebook. Free Jazz (1961), his double-quartet recording, put the argument on wax in the most confrontational way possible — two full groups improvising simultaneously across two stereo channels for nearly forty minutes.

At roughly the same time, John Coltrane was moving in a related but distinct direction. His 'sheets of sound' approach in the late fifties gave way to modal experimentation and then to something that had more in common with Coleman's freedom than with his own earlier hard bop. By the mid-sixties, Coltrane's quartet with McCoy Tyner, Jimmy Garrison, and Elvin Jones was playing music of almost unbearable intensity, and then — after Tyner and Jones departed — his final groups went further still. Ascension (1965), a large-ensemble free improvisation recorded for Impulse!, showed that collective freedom at scale was possible without collapsing into noise, though whether the noise would have been a problem is exactly the kind of question avant-garde jazz asks you to sit with.

Cecil Taylor belongs in a category almost by himself. A pianist who approached the instrument with a percussive physicality and a harmonic vocabulary that owed as much to European modernism as to jazz tradition, Taylor built a body of work that remains genuinely difficult — not difficult in a gatekeeping way, but difficult in the way that demands you meet it halfway. The AACM, the Association for the Advancement of Creative Musicians, founded in Chicago in 1965, gave the avant-garde an institutional home and a philosophical framework. Musicians like Muhal Richard Abrams, Anthony Braxton, and the Art Ensemble of Chicago came out of that community and brought a collective, politically conscious approach to experimental music that separated them from the more individualist New York scene.

What makes avant-garde jazz endure isn't the shock value — that wears off fast. It's the fact that the best of it is made by musicians who have deeply internalized the tradition they're pushing against. Coleman knew blues. Coltrane came up through rhythm and blues and hard bop. Taylor studied at the New England Conservatory. The freedom in free jazz isn't ignorance of the rules; it's a conscious decision about which ones still serve the music. That's why the genre keeps producing interesting work decades on — there's always more to find when you've made exploration the whole point.