Identifying protein functions is important for many biological applications. Since experimental functional characterization of proteins is time-consuming and costly, accurate and efficient computational methods for predicting protein functions are in great demand for generating testable hypotheses guiding large-scale experiments. Here we proposed Graph2GO, a multi-modal graph-based representation learning model that can integrate heterogeneous information including multiple types of interaction networks (sequence similarity network and protein-protein interaction network) and protein features (amino acid sequence, subcellular location and protein domains) to predict protein functions on Gene Ontology. Comparing Graph2GO to BLAST, a baseline model and two popular protein function prediction methods: Mashup and deepNF, we demonstrated that our model can achieve state-of-the-art performance. We show the robustness of our model by testing on multiple species. We also provide a web server supporting function query and downstream analysis on-the-fly. Graph2GO is the first model that utilized attributed network representation learning methods to model both interaction networks and protein features for predicting protein functions, and achieved promising performance. Our model can be easily extended to include more protein features to further improve the performance. Besides, Graph2GO is also applicable to other application scenarios involving biological networks and the learned latent representations can be used as feature inputs for machine learning tasks in various downstream analysis.