https://cea.hal.science/cea-01336932Messio, LauraLauraMessioIPHT - Institut de Physique Théorique - UMR CNRS 3681 - CEA - Commissariat à l'énergie atomique et aux énergies alternatives - Université Paris-Saclay - CNRS - Centre National de la Recherche ScientifiqueCorboz, PhilippePhilippeCorbozETH-Hoenggerbeg - Theoretische Physik - ETH Zürich - Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich]Mila, FrédéricFrédéricMilaUNIL - Université de Lausanne = University of LausanneCompetition between three-sublattice order and superfluidity in the quantum 3-state Potts model of ultracold bosons and fermions on a square optical latticeHAL CCSD2013[PHYS] Physics [physics]De Laborderie, Emmanuelle2016-06-24 11:14:492023-08-31 16:14:032016-06-24 16:20:42enJournal articleshttps://cea.hal.science/cea-01336932/document10.1103/PhysRevB.88.155106text/html; charset=utf-81We study a quantum version of the three-state Potts model that includes as special cases the effective models of bosons and fermions on the square lattice in the Mott insulating limit. It can be viewed as a model of quantum permutations with amplitudes J_parallel and J_perp for identical and different colors, respectively. For J_parallel=J_perp>0, it is equivalent to the SU(3) Heisenberg model, which describes the Mott insulating phase of 3-color fermions, while the parameter range J_perp<min(0,-J_parallel) can be realized in the Mott insulating phase of 3-color bosonic atoms. Using linear flavor wave theory, infinite projected entangled-pair states (iPEPS), and continuous-time quantum Monte-Carlo simulations, we construct the full T=0 phase diagram, and we explore the T>0 properties for J_perp<0. For dominant antiferromagnetic J_parallel interactions, a three-sublattice long-range ordered stripe state is selected out of the ground state manifold of the antiferromagnetic Potts model by quantum fluctuations. Upon increasing |J_perp|, this state is replaced by a uniform superfluid for J_perp<0, and by an exotic three-sublattice superfluid followed by a two-sublattice superfluid for J_perp>0. The transition out of the uniform superfluid (that can be realized with bosons) is shown to be a peculiar type of Kosterlitz-Thouless transition with three types of elementary vortices