Phase diagrams capture the essential features of a system in many areas of physics. Distinguishing one phase from another is often done by hand-crafted selection rules and an automated approach could accelerate this process. Here, we use a machine learning technique called contrastive learning to classify 18,000 magnetic ground state configurations into 12 distinct clusters. This is done by using a hybrid approach of increasing the number of clusters given by the model to 40 and then merging these clusters into the 12 phases by hand. The ground states of two-dimensional magnetic atomic lattices on metallic substrate are generated by fitting a tight-binding model to a classical Heisenberg model and subsequent classical Monte Carlo calculations. The symmetries of the system are utilized as transformations to cluster identical phases together. Furthermore, we investigate the representation space created by the model as a quick overview for understanding large amounts of physical data. Because of the lack of labeled phases, we judge the quality of the phase diagrams by taking random samples of the resulting clusters. The approach contributes to a better understanding of the connection between magnetism and topological electronic matter. Our results are generalizable to the automated identification of phases in condensed matter physics and beyond.