In the standard model there are charges with abelian anomaly only (e.g. right-handed electron number) which are effectively conserved in the early universe until some time shortly before the electroweak scale. A state at finite chemical potential of such a charge, possibly arising due to asymmetries produced at the GUT scale, is unstable to the generation of hypercharge magnetic field. Quite large magnetic fields (of order 1022 gauss at T=100 GeV with typical inhomogeneity scale 106/T) can be generated. These fields may be of cosmological interest, potentially acting as seeds for amplification to larger scale magnetic fields through non-linear mechanisms. Previously derived bounds on exotic B-L violating operators may also be evaded.