His family removed to Sweden, New York in 1814. At the age of fourteen, he was apprenticed to a printer in Batavia, New York. He removed to Ogdensburg in 1831, and edited there the Northern Light, a weekly newspaper. He later became co-owner of the Times and Advertiser, the Whig paper of St. Lawrence County.
He studied law, was admitted to the bar in 1838, and commenced practice in Ogdensburg.
He was a justice of the New York Supreme Court (4th District) from 1854 to 1876, and was ex officio a judge of the New York Court of Appeals in 1861 and 1869. He was a member of the peace convention of 1861 held in Washington, D.C., in an effort to devise means to prevent the impending war.
James was elected as a Republican to the 45th and 46th United States Congresses, and served from March 4, 1877, to March 3, 1881. While serving his second term in Congress, he was stricken with "paralysis", from which he partially recovered.
