See the physics library guide for some resources including library guides relating to history of thermodynamics and statistical mechanics, history of electromagnetism, as well as astronomy.

Of possible interest:

Stephen G. Brush, Allen L. King, Editors. History in the Teaching of Physics: Proceedings of the International Working Seminar on The Role of the History of Physics in Physics Education.