Blend orange juice concentrate and coconut milk in a bowl with a whisk or an immersion blender. (Yes, leave the oj frozen and scoop it out of can with a spoon. It will melt into the coconut milk and blend up fine.)
Add vanilla extract and mix in.
Pour liquid into popsicle mold and add sticks.
Freeze for at least 12 hours or until completely solid.