I think it probably does refer to rowing here, given that "pull" can mean row, that Jack Cardigan (to whom it refers) is a fitness fanatic (and this passage is all about his physical accomplishments) and that Caversham is on the river. Moreover, "pull" is used in this sense elsewhere in the same text; for example:
"Fleur did not land at once, but pulled in among the reeds, into the sunshine"
"Fleur thrust her letters back, got out her sculls, and pulled up to the landing-stage."
In another context, however, this could refer to driving. "Pull" can be used to refer to moving in a vehicle:
"pull
13.
also intr; foll by
away, out, over, etc to move (a vehicle) or (of a vehicle) be moved in a specified manner - "
he pulled his car away from the roadside"."
http://www.collinsdictionary.com/dictionary/english/pullIt's true that nowadays "pull" is not normally followed by "down" when used in this sense, but I would not be very surprised to find "pull down to" being used to mean "drive down to". However, there is some doubt about that, and in any case I don't think that's the meaning here.