Bring 4 cups of water to a boil. Add tea bags and remove from heat.
Let tea bag steep for about 15-20 minutes. Stir in sugar and add fresh lemon juice. Let the tea cool to room temperature.
Pour sweetened tea into the popsicle molds. Leave about ¼ inch at the top for the liquid to expand. Seal with popsicle lid.
Freeze for at least 6 hours, preferably overnight.
To remove popsicle, run some warm water over the mold to loosen.