The club was founded in 1876, with 30 members, which made it bigger than Greenock Morton, founded in the same year. It took its name from the public park where it played its earliest matches. The first recorded match for the club was against Union in January 1877.
