Kassel and Reutenauer computed the zeta function of the Hilbert scheme of n points on a two-dimensional torus and showed it satisfies several number-theoretical properties via modular forms. Classifying the singularities of this rational function into zeros and poles, we define a word which contains a lot of number-theoretical information about n (the above-mentioned number of points). This nontrivial connection between natural numbers and words can be used to define many classical subsets of natural numbers in terms of rational and context-free languages (e.g. the set of semi-perimeters of Pythagorean triangles, the set of numbers such that any partition into consecutive parts has an odd number of parts). Also, some arithmetical functions can be described in way (e.g. the Erdös-Nicolas function, the number of middle divisors). Finally, this approach provides a new technique to prove number-theoretical results just using relationships among context-free languages.