Raymond was established when in 1886 a branch railroad line was built from Berenda in the valley into the foothills. Travel to Yosemite Park shifted to the route from the community through Ahwahnee, Fish Camp and Wawona.

The town was called "Wildcat Station" by some after, so the story goes, the arrival of an early train so overwhelmed the hotel that when all the meat was gone, the chefs substituted wildcat meat.

The Raymond-Knowles area is famed for its granite. Materials from quarries there helped rebuild San Francisco after the earthquake. Raymond granite also is found in buildings in Los Angeles, Salt Lake City and other cities as far away as Texas.