Put the peeled chestnuts in a small saucepan with 3/4 cup of the milk. Bring to a bare simmer over medium-low heat and cook for 3 to 4 minutes, until the chestnuts are soft enough to mash easily.
Blend the chestnuts and milk together — a stick blender or regular blender both work — adding the remaining milk a little at a time until you get a smooth, thick paste roughly the texture of hummus. Stir in the sugar and vanilla while it's still warm so the sugar dissolves, then let it cool completely in the fridge, about 30 minutes.
Beat the cream cheese on its own until smooth and free of lumps, about a minute.
In a separate bowl, whip the heavy cream to soft peaks.
Fold the cooled chestnut paste into the cream cheese first, mixing until even. Then fold in the whipped cream in two additions, keeping the mixture as light as you can — this is what makes the final texture airy rather than dense.
Dip each ladyfinger or biscuit briefly in the cooled coffee — a quick one-second dunk per side, not a soak, or they'll turn to mush. Arrange a layer in the bottom of a serving dish or individual cups.
Spread half the chestnut cream over the biscuit layer. Repeat with a second layer of dipped biscuits and the remaining cream.
Cover and refrigerate for at least 2 hours, ideally overnight, so the biscuits soften properly and the layers set.
Dust with cocoa powder and scatter a few pieces of roasted chestnut on top just before serving.