Context. The black hole mass measurement in active galaxies is a challenge, particularly in sources where the reverberation method cannot be applied. Aims. We aim to determine the black hole mass in a very special object, RE J1034+396, one of the two active galactic nuclei (AGN) with quasi-periodic oscillations detected in X-rays, and a single bright AGN with optical band totally dominated by starlight. Methods. We fit the stellar content using the code STARLIGHT, and the broad band disk contribution to optical/UV/X-ray emission is modeled with OPTXAGNF. Based on sSTARLIGHT, we develop our own code OPTGAL for simultaneous fitting of the stellar, Fe II, and BC content in the optical/UV/X-ray data. We also determine the black hole mass using several other independent methods. Results. Various methods give contradictory results. Most measurements of the black hole mass are in the range 106 107 M-circle dot, and the measurements based on dynamics give higher values than measurements based on H beta and Mg II emission lines.