Preheat oven to 400°F (200°C) and line a large baking sheet with parchment paper.
Combine pumpkin puree, brown sugar, cornstarch, cinnamon, ginger, nutmeg, salt, and vanilla in a bowl.
Stir until smooth and thick.
Roll cold pie dough on a lightly floured work surface to approximately ⅛-inch thickness.
Cut approximately sixteen 3 × 4½-inch rectangles, rerolling scraps gently when necessary.
Arrange eight rectangles on the prepared baking sheet.
Spoon approximately 1½ tablespoons pumpkin filling into the center of each, leaving about a ½-inch border.
Brush the exposed borders lightly with egg wash.
Place the remaining pastry rectangles over the filling.
Press the edges together gently.
Crimp all four edges with a fork.
Cut 3–4 small steam vents into each pastry.
Refrigerate the assembled pastries for 10–15 minutes.
Brush the tops lightly with egg wash.
Bake for approximately 18–22 minutes, until puffed and golden.
Transfer to a wire rack and cool for at least 15–20 minutes.
Whisk powdered sugar, cinnamon, vanilla, and 1½ tablespoons milk together.
Add optional maple syrup and additional milk in very small amounts if needed.
Spoon the thick cinnamon glaze over the cooled Pop Tarts.
Let the glaze set for 10–15 minutes before serving.