In his poetry, Kiss broke with the classical rules of verse, creating a distinctive modern rhythm combined with emphatic meter. The main theme of his poetry is the bitterness of neglect. His diverse poems combined elements that were Hungarian and Jewish, folk and modern, rural and urban, epic and lyrical, and realistic and fairy-tale.
Born to István Klein, a poor Jewish village shopkeeper, and the daughter of a Lithuanian Jewish cantor teacher who had fled to Hungary to escape the pogroms, Kiss moved with his family in 1850 to Gömör-Kishont County. His father worked as a tenant farmer, and Kiss was introduced to literature by Reform pastor Sámuel Balogh Almási. His parents, intending that he become a rabbi, sent him to Miskolc to begin his studies.
He fled to Vienna at the age of 13. After returning home, he attended Rimaszombat High School and then the Reformed College in Debrecen. For five years (1862–1867) he was a traveling teacher. In 1867, when the emancipation of the Jews was announced, he went to Pest to publish his poems. He worked as a proofreader at the Deutsch publishing house and, between 1870 and 1873, was editor of the Képes Világ, a magazine published by the printing house, which he took over from Arnold Vértesi.
In 1874, he became seriously ill but continued to write his novel Budapesti rejtelmek in bed. His first success came in 1875 with his ballad “Simon Judit,” which was presented to Ferenc Toldy at a meeting of the Kisfaludy Society. (In 1915, Adolf Mérei made a film based on the ballad). The poem, and with it the name of the poet, soon became known throughout the country.
From 1876, he was the notary of the Jewish community of Timisoara for six years and then moved again to Budapest in 1882 to work as an official of the Hungarian-French insurance company. His reputation as a poet grew, although the first volume of his poems was met with controversy and some severe criticism from Jenő Rákosi and the Budapest Review. His poems were also translated into German.
The Jewish community in Pest commissioned him to compose Hungarian hymns, but when Kiss reworked old psalms in a more modern setting, he was asked to make some changes. Refusing to do so, he then published the hymns himself under the title Ünnepnapnapok (“Holidays”). Kiss became the founding editor of the journal A Hét (“The Week”) after the insurance company he worked at went out of business in 1889.
He was elected a member of the Petőfi Society in 1877 and the Kisfaludy Society in 1914. While his ballads at the beginning of his career were known for depicting the village life of Jews, he later abandoned this genre to focus on city life and the problems of modern man. His most famous poem was written in the aftermath of the 1905 Russian Revolution, and a collection of his poems was set to music by Károly Szelle.
His grave can be found in the Jewish cemetery on Salgótarjáni Street in Budapest.