In a medium bowl, combine the softened cream cheese and unsalted butter.
Using an electric mixer, beat the cream cheese and butter together on medium speed until smooth and creamy, about 2 minutes.
Add the vanilla extract and mix until incorporated.
Gradually add the sifted powdered sugar, about 1/2 cup at a time, beating on low speed after each addition to avoid a sugar cloud.
Once all the powdered sugar is incorporated, add the whole milk and beat on medium speed until the icing is smooth and spreadable, about 1-2 minutes.
If the icing is too thick, add a little more milk, 1 teaspoon at a time, until desired consistency is reached.
Spread the icing evenly over cooled pumpkin bread before serving.