James died suddently on July, 17, 1907, at his summerhouse in Percé, Québec, where he used to go every summer for over 20 years, enjoying a tremendous panorama on the Percé Rock from the top of the hill and welcoming fellow artists occasionnally. His dead was mourned by many, according the newspapers of the time, for he was highly appreciated by the locals. Today his house is known as Villa Frederick-James and preserved as Quebec's Built Heritage.