What Clarke is saying in short is that our Evolution and development is guided and protected by an outside force, that there is an existence beyond out physical realm which we'll evolve to if we please our overlords. However this is so far off Clarke's philosophical opinions in general we must consider it a 'muse' or a 'fancy' in part and also as an allegory. He could well be saying that the first neurological brainwave that started humanity on it's quest for glory was a dream of something powerful and magnificent on the moon - certainly the huge black obsidian phallus as hard and sharply defined as anything imaginable is a lovely metaphor for man's early pre-linguistic mental image of shear awesome wonder - was it that dark night when one monkey's eyes raised to the sky did some change overcome him when for the first time eyes looked upon our moon and wanted to see close, to touch whatever great power was on that magnificent mysterious ball of light in the sky? That early fantasy, soon to turn into gods and kings and all sorts of erroneous dreams, now in a void of understand - just a powerful image adrift in a simian brain...
The actual meat of story is of mankinds desire to reach into space, the absolute top secrecy and urgency with which we jealously hide our inner most desires, how somewhere deep inside all of us is that huge black monolith calling us from the ether beckoning us to rise up and touch the moon. Yet after all these thousands of years of fighting and fussing finally our species has reached up and grown into the masters of our domain, those early days of moon settlement will be amazing pioneering days and yet still there is that thing left to grasp, that mighty black obelisk of the soul.
As we reach out and touch the dreams we've held for so long we develop a new dream, a new desire - reaching out across space mankind heads for that huge dark box, that massive mental void that now looms before us. Interesting that the music should be Strauss -Also Sprach Zarathustre, a nod at 'God is dead, weap children for now ye have no father' Nietzsche of course is dealing with the issue of humanities lack of binding shared dream as humanity realizes that there is no god and so i suspect on another level is Clarke but lets leave that for the moment, he's saying from the perspective we were at before that 'the moon is dead, weap children for now ye have no dreams' -inside us now we have this great void, a mighty blackness. That mysterious and powerful thing that the monolith pointed us towards draws us closer, as we stand before it and gaze upon it we're grabbed and pulled inside... it's full of stars it's....it's [massive rolling visions of a thousand wonders, billions of blurred and fractured images of wondrous glory of space] it's full of stars
Our dream rises from that primordial innocence and explodes now into the next level of wonder it's full of stars humanity now again a babe, now again at the start of a new quest -as vast in scale as reaching the moon was to a monkey... we look down on earth as a baby, we've grown to fill her how we must grow to fill the galaxy.
it's the classic type one, type two civilization level up tale we all know so well (kaku has a few bits about it ;) ) - the reason it as a film speaks to everyone so universally is because we all experience this transformation on a day to day basis.
oh and the whole hal thing, thats an aside about wrestling with pure logic rather than human desire - it represents the importance of remaining human, the mission IS our humanity, no mission is more important than that. well thats what i took from it.