Line an 8×8 inch baking pan with parchment paper, leaving some overhang on the sides for easy removal later.
In a dry skillet over medium heat, toast the chopped pecans for a few minutes until fragrant. Set aside.
In a medium saucepan over medium heat, melt the butter.
Add the brown sugar and milk to the melted butter. Stir continuously until the mixture comes to a boil. Let it boil for 2 minutes while stirring constantly.
Remove the saucepan from the heat and let it cool slightly.
Gradually stir in the powdered sugar and vanilla extract until the mixture is smooth and well combined.
Fold in the toasted pecans until evenly distributed.
Pour the fudge mixture into the prepared baking pan, spreading it out evenly with a spatula.
Let the fudge cool at room temperature for about 2 hours, or until fully set.
Once set, lift the fudge out of the pan using the parchment paper overhang. Cut into squares and serve.