In 1832, Eston married a free woman of color, Julia Ann Isaacs (1814–1889). She was the daughter of the successful Jewish merchant David Isaacs, from Germany, and Ann (Nancy) West, a free woman of mixed race. Nancy West was the daughter of Priscilla, a former slave, and Thomas West, her white master. Thomas West left property to his children Nancy and James West in his will. Prohibited by law from marrying, Isaacs and West maintained separate households and businesses for years (she was a successful baker.) They had seven children together, and later in their lives shared a household.