Baldwin served through the Civil War in the 19th Michigan Infantry, fighting in all his regiment's battles from 1862 to 1865. Upon the postbellum reorganization of the Regular Army, he was commissioned into the 19th United States Regular Infantry as a Second Lieutenant. He eventually was assigned to the 5th U.S. Infantry, with whom he fought in the various frontier conflicts with the Indians. His actions in an attack on an Indian village on the Red River in Montana on December 18, 1876, earned him a brevet of Captain, U.S. Regular Army (awarded on February 27, 1890). He served with distinction under General Nelson A. Miles as chief of scouts during campaigns against Sitting Bull and Crazy Horse.