When a letter is poorly written (i.e. written without due care and attention, written without concern for the reader) or if a poet fails to rehearse for a reading there's a case for describing the author as negligent or even impolite.
If I read an application form that's messily filled in, I wouldn't be impressed; my time's been wasted. If I think the applicant is trying to bluff me, I grow suspicious; I feel I'm being taken advantage of. When I see difficult poetry, or a rough draft presented at a workshop, I sometimes wonder whether the poet's thinking more about themselves than the hapless reader.
Some standard guidelines for communication concern relevance - don't include material (e.g. line-breaks) that has no purpose, and certainly don't add features that have a negative effect on communication. Equally, don't delete too much - removing a few words to make a poem "denser" means that the reader will take longer to read it (i.e. the poem becomes in some sense longer - less dense - rather than shorter).
So when does "inconsiderate" become "rude"? When the behaviour's intentional? When it's continued despite it being pointed out? Of course, poets aren't mere communicators, and they can't be all things to all people, but if they make their work gratuitously difficult (e.g. by not providing notes, not explaining foreign words, adding skew-whiff line-breaks), if they don't bother spending just a little more time trying to make their work a lot easier without compromising artistic integrity, isn't rudeness sometimes a valid description?
And yet, I've never seen the term used in this context ("elitist" or "socially inept" yes, but not "rude"). Should poets think about their readers? Perhaps difficult poets do, but they don't want to insult the readers' intelligence. Considerate poets of various types exist. On the back cover of Billy Collins' "Ballastic" it says "No poet writing today insists on such open, direct and courteous engagement with the reader". Andrew McMillan in "Eyewear" wrote that "Constant consideration of the reader, of an audience, is the mark of a great poet. In [Emily] Berry, that is exactly what we have". I'm not convinced by the first sentence, and great though Emily Berry might be, her poems don't seem especially reader-centred, but at least the reviewer's addressing the issue. I think poets are well advised to anticipate the reader's reaction when rewriting a poem in order to weigh up whether any loss of reader-friendliness is sufficiently compensated for. There are poets (especially after receiving workshop feedback) who consider line by line how the poem will be received, how the reader's state of mind might change with each phase.
Some readers don't look for the author behind the text. Some poets don't actively consider the reader, concentrating instead perhaps on authenticity, on expressing what they really feel inside. The poet (though much more often the novelist) may wish to be invisible, discouraging a poet-reader relationship. Nevertheless, the poet might still show through. More often with poetry than with prose, there might be an assumed one-to-one connection between author and reader.
Readers may become irritated if they think the poet's Sexist, Racist, Anti-semite, Anti-gay, etc. Readers might become more than just irritated if they belong to the aggrieved set of people. Elitism or aloofness doesn't tend to provoke similar reactions - the poet's behaviour is less personal, less targeted, and could be described as style rather than attitude, and style isn't, as far as I can tell, considered a legitimate justification for becoming angry about a poet/artist.
Perhaps rudeness isn't an applicable emotion in this context; readers should be engaging with the text, not the poet. Or perhaps the presumption is that readers voluntarily enter into this unequal relationship with the writer, and should be prepared to walk away feeling disappointed, humiliated or inadequate. Perhaps it's felt that the editor or publisher rather than the poet is really the culprit. If more poets were criticised as being rude, perhaps they'd write more clearly. Describing them as elitist only encourages them.