Here is the text of the Treaty of Tripoli dated 4 November 1797. See Article 11, where the US government explicitly states that the country was NOT founded on the Christian religion. This is as explicit as it gets and as contemporary to the founding as possible.