When you user a Daydream viewer with a controller, camera is on 0,0,0 perhaps is related with this three.js issue > https://github.com/mrdoob/three.js/issues/11619 