Ferenc Xavér Haller was born on December 22, 1716, in Varasd. He entered the Society of Jesus in 1733; according to sources, either on October 27 in Trenčín or on November 3 in Vienna. He completed his novitiate in Vienna. In 1736, he studied at a teacher training college in Zagreb; from 1737, he studied logic, metaphysics, and physics in Graz, and subsequently studied and taught in Zagreb, Varasd, Graz, and Vienna. Between 1744 and 1747, he studied theology in Nagyszombat; in 1748, he completed his tertianship in Besztercebánya; and from 1749 to 1750, he taught mathematics in Ljubljana.

Later sources often confused him with the Jesuit József Haller, and they also misinterpreted the designation “Quito.” Ferenc Xavér Haller was included in the preparations for the Spanish–Portuguese border expedition not merely as a missionary in Quito, but as a Jesuit scholar, due to his knowledge of mathematics and astronomy. He arrived in Cumaná in 1754 and subsequently worked in the Orinoco–Guayana region. In May or on June 24, 1755, somewhere between Trinidad and Guayana, he died of a tropical fever.