Resting state functional MRI (rs-fMRI) is a popular and widely used technique to explore the brain's functional organization and to examine if it is altered in neurological or mental disorders. The most common approach for its analysis targets the measurement of the synchronized fluctuations between brain regions, characterized as functional connectivity (FC), typically relying on pairwise correlations in activity across different brain regions. While hugely successful in exploring state- and disease-dependent network alterations, these statistical graph theory tools suffer from two key limitations. First, they discard useful information about the rich frequency content of the fMRI signal. The rich spectral information now achievable from advances in fast multiband acquisitions is consequently being under-utilized. Second, the analyzed FCs are phenomenological without a direct neurobiological underpinning in the underlying structures and processes in the brain. There does not currently exist a complete generative model framework for whole brain resting fMRI that is informed by its underlying biological basis in the structural connectome. Here we propose that a different approach can solve both challenges at once: the use of an appropriately realistic yet parsimonious biophysical signal generation model followed by graph spectral (i.e. eigen) decomposition. We call this model a Spectral Graph Model (SGM) for fMRI, using which we can not only quantify the structure-function relationship in individual subjects, but also condense the variable and individual-specific repertoire of fMRI signal's spectral and spatial features into a small number of biophysically-interpretable parameters. We expect this model-based inference of rs-fMRI that seamlessly integrates with structure can be used to examine state and trait characteristics of structure-function relations in a variety of brain disorders.