Update: right after I wrote this I went back and it let me continue playing. Turn 57 I emptied the Chapel pile (it's slow when Bandits have trashed everything bigger than copper) and the game ended. Yeah! Turns were very, very slow though.
Maybe you need to empty four piles with six players?
Maybe you need to empty four piles with six players?