Preheat your oven to 350°F. Grease a 9 by 13 inch baking dish with butter or nonstick spray.
Spread the 8 cups brioche cubes evenly in the prepared dish. Scatter the ½ cup raisins over the bread.
In a large bowl, whisk the 2 cups whole milk, 1 cup heavy cream, ¾ cup granulated sugar, ½ cup brown sugar, 5 large eggs, 2 teaspoons vanilla extract, 1 teaspoon cinnamon, ½ teaspoon nutmeg, and ¼ teaspoon salt until smooth and the sugar has dissolved.
Pour the custard evenly over the bread. Scatter the ½ cup chopped pecans on top. Let it soak for 15 minutes, pressing the bread down gently every few minutes so the top cubes absorb the custard.
Bake for 45 to 55 minutes, until the top is golden and the center is set with a slight wobble. An instant read thermometer in the center should read about 160°F.
Let it rest for 15 minutes before cutting. It continues to set as it cools.
While it rests, melt the ¼ cup butter in a small saucepan over medium heat. Stir in the ½ cup brown sugar until dissolved, then slowly whisk in the ¼ cup heavy cream. Cook for 2 to 3 minutes until slightly thickened. Remove from the heat and stir in the 1 teaspoon vanilla and pinch of salt.
Cut into squares and drizzle with the warm sauce.