Microresonator based dual frequency combs offer an integrated photonic solution to a wide range of technological challenges, including spectroscopy, range finding, material characterization and hyperspectral imaging. These applications require a high level of mutual coherence between combs, but achieving such stability can be demanding. Here we experimentally demonstrate that comb generation exploiting the rich structure of the nonlinear electro-optic tensor in lithium niobate can generate ultra-stable dual combs, with the two combs naturally having orthogonal polarizations. We theoretically explore the application of our technique to dual frequency comb generation from the far infrared (IR) to the ultraviolet (UV). Our combs have relative linewidths as low as ~400 μHz, and require no stabilization or post-processing methods.