Not recommended — #29 of 31 Ramie ranks #29 of 31 fabrics for warmth without weight. It is a poor fabric for this. Ranked on warmth relative to weight, with a packability bonus. The top pick is Cashmere.
Ramie measures 88/100 for breathability, 10% moisture regain, 15/100 warmth, at 170 gsm. Warmth-to-weight is the metric: 15/100 warmth at 170 gsm is unremarkable for packable insulation.