Preheat the oven to 350°F (175°C). Grease and flour a 9×9 inch baking pan.
In a medium bowl, whisk together the flour, baking powder, baking soda, and salt. Set aside.
In a large bowl, beat the softened butter and granulated sugar together until light and fluffy, about 3 minutes.
Add the eggs one at a time, beating well after each addition. Stir in 1 teaspoon vanilla extract.
Mix in the mashed bananas.
Alternately add the flour mixture and buttermilk to the banana mixture, beginning and ending with the flour mixture. Mix until just combined.
Pour the batter into the prepared baking pan and spread evenly.
Bake for 30 to 35 minutes or until a toothpick inserted into the center comes out clean.
Remove from oven and let the cake cool completely in the pan on a wire rack.
While the cake cools, prepare the vanilla pudding according to package instructions using 2 cups whole milk. Refrigerate until set, about 5 minutes.
In a large mixing bowl, beat the heavy whipping cream, powdered sugar, and 1 teaspoon vanilla extract until stiff peaks form. Refrigerate until ready to use.
Once the cake is completely cooled, cut it into 1-inch cubes.
In a large trifle bowl or individual serving glasses, layer one-third of the banana cake cubes.
Top with one-third of the vanilla pudding.
Add a layer of sliced bananas (about one banana per layer).
Add one-third of the whipped cream over the bananas.
Repeat the layers two more times, ending with whipped cream on top.
Refrigerate the assembled trifle for at least 2 hours before serving to allow flavors to meld.