Before she was a pop superstar, Katy Perry performed under the name “Katy Hudson” and released an album of Christian music. But along with her self-titled solo album, Perry also collaborated with Christian rock band P.O.D.: Her vocals are featured on the song “Goodbye for Now.”

Back in 2006, she was even featured in the band’s performance on The Tonight Show.

You can take a trip down Christian music memory lane in the video above.