Cut the brioche or challah into approximately 1-inch cubes.
Lightly butter a 9 × 13-inch baking dish and arrange the bread cubes loosely inside.
In a large bowl, whisk together the eggs, whole milk, heavy cream, brown sugar, vanilla, cinnamon, nutmeg, and salt.
Pour the custard mixture evenly over the bread.
Gently press some of the bread pieces into the custard without completely flattening them.
Cover the baking dish and refrigerate for at least 8 hours or overnight.
The next morning, remove the casserole from the refrigerator while the oven preheats to 350°F (175°C).
Combine the flour, brown sugar, cinnamon, and salt for the topping.
Cut in the cold butter until coarse crumbs form.
Sprinkle the crumble evenly over the casserole.
Bake uncovered for 45–55 minutes, until the center is set and the top is golden brown.
Remove from the oven and rest for 10 minutes.
Serve warm with maple syrup, fresh berries, or powdered sugar.