Swarthmore is located in east-central Delaware County at (39.901788, −75.347083). It is bordered to the north, east, and southwest by Springfield Township, to the southeast by Ridley Township, and to the west by Nether Providence Township. Crum Creek, a southward-flowing tributary of the Delaware River, forms the western boundary of the borough.
