He was born on February 5, 1818, in Amsterdam, Montgomery County, New York, the son of William Kline. He moved to the Village of Amsterdam and became a carpenter. In 1843, he began the construction of machines for weaving figures into carpets and the next year became a partner of Congressman John Sanford who ran a large carpet factory. In 1844, Kline married Bata A. Simons. In 1847, he moved across the Mohawk River to Port Jackson. He was Supervisor of the town of Florida in 1849.
In 1857, he opened a knit goods factory in Amsterdam. He was Supervisor of the Town of Amsterdam, and Chairman of the Board of Supervisors of Montgomery County, in 1858 and 1859. He was Treasurer of Montgomery County from 1861 to 1863. In 1866, he established a flour mill. He also traded wool and was vice president of the First National Bank of Amsterdam.
