In a medium saucepan, combine the cocoa powder, sugar, water, and salt over medium heat. Heat this chocolate base mixture, stirring constantly, until smooth.
Once the cocoa is dissolved, reduce the heat a little and pour in the milk. Continue stirring as it heats up, and you see steam surface. Do not let it boil; you don't want to burn the milk.
Turn off the heat and stir in the vanilla. You add the vanilla last to keep the flavor; otherwise, it evaporates.
Pour the hot cocoa into mugs and top with marshmallows.
Notes
You can swap sugar for the same amount of honey, coconut sugar, or turbinado. It will slightly change the flavor, but it's just as good.