The structure is different depending on which Leninist state your'e talking about and what time period your'e talking about. All of them are republics that held elections regularily, but they're one-party states so the party always wins. Generally, the party has regular party congresses where elected representatives from the local party branches (only party members can vote in these) get together, pass resolutions, elect the central committee and such. The party congress is officially the highest organ in the party, but between congresses the central committee runs things. The central committee elects several committees to help the party control the state (and do other things), including the political committee or politburo. The politburo basically runs the government. The general secretary is the leader of the party.
Although real power is in the party, most leninist states have some sort of parliament or pseudo-parliament and various government officials which are elected by the general public. Officially they play a role similar to what they do in the western "democracies" - passing laws and whatnot. However, the Communist party is the only party allowed to win and party members are subject to party discipline so real power lies with the party congress, central committee & politburo. Government officials like Presidents, etc. generally do the work necessary to run the government. Sometimes the head of state is also the head of the party, some times they are not.
Some Leninist states don't have Presidents or Prime Ministers. The early Bolshevik state in Russia was run by a "Council of People's Commissars" or SOVNARKOM. The commissars in this council were like the ministers in an ordinary parliamentary state, except this council of commissars had the power to pass laws. Executive and Legislative were combined. There was also a Soviet Congress, but it didn't meet very often (once every three months at the beginining, less later). The chairman of the Sovnarkom (Lenin) was the head of state, the closest equivalent of a Prime Minister but not really. This structure was later changed, especially after the new constitution in 1936.