When Homeric epic became famous, it was in the context of very public performances in a major festival, the Great Panathenaia in Athens. Individual rhapsodes would declaim or perhaps chant parts of the epics, keeping time by beating a staff. They would presumably have had a sizeable audience, though presumably not as colossal as the audiences in the theatre of Dionysos, at Dionysiac theatre festivals.

This much is fairly well documented in ancient sources. We can infer that there are likely to have been other cultic and festival contexts in which Homer was performed: the Hymn to Apollo indicates epic performance at the Delia, a festival of Apollo on Delos, with the narrator adopting the persona of Homer; and the Hymn to Demeter shows strong links to the Mysteria at Eleusis. We can also be pretty confident that the Panathenaic performances in Athens also originally included poems other than the Iliad and Odyssey: in the early 500s BCE, the main poem associated with the name Homer was actually a lost poem called the Thebaid, and we have one Attic vase showing a rhapsode beginning a performance with a line about the city of Tiryns, which doesn’t appear in the surviving poems. The Panathenaia also included other kinds of contests, including citharoidic (musical) contests, but to all appearances these were separate from performances of Homer.
Poetic battles,
or ‘agonistic’ performance: essentially ancient Greek rap battles. (Only epic, not rap, obviously.) Two poets could engage in an aggressive face-to-face contest, displaying their virtuosity and trying to one-up each other in various ways — answering riddles, giving the best moral aphorisms, completing seemingly incompletable lines. Some of the evidence for this practice is late, but not all: Aristophanes’ Frogs portrays a comic version of a battle, and his Clouds has two characters engage in a similar (but non-poetic contest) in a cage-fight, literally in a cage. We have material going back to the 4th century BCE writer Alkidamas, who in turn drew on older material, depicting a battle between Homer and Hesiod; it’s likely that the Hesiodic Works and days contains an allusion to this story. There are also stories of a battle between Lesches and Arktinos; and two accounts of what appears to be someone choking in a battle (Kalchas chokes in a contest with Mopsos, in the Hesiodic Melampodia; Thamyris chokes in a battle with the Muses, related in Iliad 2.599-600). The individual stories are presumably mostly or completely fictional, but they must be premised on a practice that was known from real situations.
Court poets.
The Homeric poems themselves depict the poets Phemios, Demodokos, and Apollo and the Muses as honoured guests in the halls of great lords (Odysseus, Alkinoos, and Zeus respectively). We shouldn’t take this picture too literally, because to some extent they’re self-insertions: they’re suggestions for how the real poet ought to be treated. Demodokos, in particular, is a clear self-insertion because his blindness evokes the Homer persona.
Sympotic performance,
or performance at aristocratic parties. This, I think, is the closest we get to an ‘original’ performance scenario for Homer. The reason I think that is because the court poets, above, sound like an aggrandised and/or archaified version of sympotic performance. That is, a sympotic performance in a small-ish gathering is all very well and good; from the performer’s point of view, the ideal scenario would be similar to that only bigger, better, and wealthier — in other words, the court of a great lord. The court poet seems like a natural imaginative extension of real symposia, rather than a truly separate kind of performance.

And sympotic performance is very strongly indicated in other hexameter poems, outside Homer. The opening of the Catalogue of women evokes themes that also appear in sympotic poetry, talking explicitly about feasts and who gets next to whome. The Hymn to Hermes, though much later, is also very clear: it depicts Hermes giving the first-ever performance of a hexameter poem —
The god sang beautifully with his accompaniment,
experimentally, impromptu, as when young men
at feasts compete with each other using double meanings …
The Hymn makes it clear that Hermes’ poem includes standard formal elements used in hexameter poems: a hymnic prelude, a praise poem, and a genealogy. Later, Apollo compares the invention of the lyre to the idea of passing things to the right at feasts; and Hermes gives him the lyre and suggests that he treat it as a hetaira, a professional escort at a symposium.

This may still be some way removed from early 7th century performance practices. There’s a complication in that sympotic performance generally comes with musical accompaniment, while actual performance of Homer doesn’t: the idea of ‘singing’ a poem is essentially a poetic conceit — like when poets in the modern era refer to poems as ‘odes’ and ‘songs’, though their poems are only ever spoken or read silently. The proem of the Hesiodic Theogony shows that the same conceit was present in antiquity: the narrator talks of ‘singing’ songs, but with the aid of a rhapsode’s staff (i.e. not sung). So there is a dissonance between the pictures we get.

But for the record, my suspicion of what an ‘original’ performance looked like is: a performer declaiming, probably with a beat given by a staff or perhaps even by the audience, at an aristocratic social event. The performance would begin with a hymn to one god or another, then move into narrative. A poem in that situation can’t have been anything like as long as the extant Homeric poems, so there’s still a lot that we don’t know, and probably never can know. But it does seem to be where the evidence is pointing.

This account is a distilled version of what I wrote in my 2015 book Early Greek hexameter poetry (pp. 83-87). For a second opinion, I’ll turn to Oliver Taplin’s Homeric soundings (1992, ch. 1). Taplin argues that the Iliad:
was created to be performed in extracts of (say) one or two hours’ length. The prime argument in favour of this is the practice of the bards Phemius and Demodocus in the Odyssey. It is also not uncommon for oral performances in various places in the twentieth century to be of this sort of length …
He identifies certain types of narrative cues in the story as pointing to this ‘extract’ model. He gets into the setting for performances at page 39:

It is possible that the Iliad was created to be performed through three successive nights in the banqueting-hall of a lord. We have seen Odysseus himself supplying some sort of precedent (see pp. 30-1 above); and the feast or symposium are recognized more and more as an important occasion for poetry. …
But he is inclined against this.

I find it hard to believe that it is coincidence that big panegyric festivals were becoming firmly established during the very period that produced the Homeric epics. At this stage these were local, though some were to become Panhellenic before long, most famously the festivals of Zeus at Olympia and Apollo at Delphi. … The time would typically be spent in sacrifice and ritual, in feasting, and in witnessing athletics and other activities in honour of the god. These activities might include poetry; and in later classical times competitions between rhapsodes were not uncommon. This seems to me to be the most plausible opportunity for the three successive all-night sessions for the Iliad and two for the Odyssey which I have posited.
Taplin is broadly in agreement with Nagy about the centrality of cultic performance, and that the panhellenisation of festivals and of the epics occurred concurrently. I’m not going to say they’re definitely wrong: just that there are reasons for some reservations. The poems themselves, aside from the Hymns to Demeter and Apollo, are very insistent on sympotic or proto-sympotic scenarios. Perhaps some combination is the truer answer; but we can’t be very sure of how cultic and sympotic contexts played against each other, or in conjunction with each other.