Preheat the oven to 350 degrees F. Lightly oil an 11-by-16-inch pan (or similar size). Place a sheet of parchment paper over this, and lightly grease this too.
To make the cake, beat the butter and sugar until light and fluffy. Add egg yolks, one at a time, incorporating fully between additions.
In a medium bowl, sift together the flour, baking powder, and vanilla sugar. Add the flour mixture to the egg mixture a little at a time, alternating with the milk. Spread the batter into the pan.
Next, make the meringue: In a separate bowl, beat the egg whites with a pinch of salt until stiff. Gradually add the sugar and continue to beat until stiff peaks form.
Using a spatula, carefully spread the egg whites on top of the cake batter. Sprinkle with chopped almonds and bake for 25 to 30 minutes, until the meringue is dry and golden. Cool in the pan.
When the cake and meringue have cooled, cut in half vertically. Place one half on a platter. Spread pastry cream over this, using enough to add a luscious layer without it spilling out the sides (you won't need the full portion). Top with the remaining half of the cake.