Kittelsen had his debut as a poet in 1970. He has since than published several poetry cycles and collections, as well as fables, dramatic works and translations of poetry.

He is known for his dialogues, partly with colleagues and partly in the work with the translation of poetry from distant languages in relation to the Nordic language area like Arabian, Persian, Korean, Latvian and Sumerian. He also has poetical dialogue with the most ancient poetical traditions in the Nordic Countries – the Old Norse Poetic Edda where at first he translates the old text and than presents a contemporary literary answer. Erling is known as a writer who renews language, a poet and storyteller. He is a writer who moves in untraditional ways, both with the language in his books and dramatic works and his literary activity through events and happenings. His last play has been translated and performed several times in the Middle East.

He has received several literary prices, amongst them The Aschehoug Prize (awarded on a binding recommendation by the Norwegian Critics Organization) and The Dobloug Prize (awarded by the Swedish Academy).