Knowledge graphs are increasingly built using complex multifaceted machine learning based systems relying on a wide of different data sources. To be effective these must constantly evolve and thus be maintained. I present work on combining knowledge graph construction (e.g. information extraction) and refinement (e.g. link prediction) in end to end systems. I then discuss the challenges of ongoing system maintenance, knowledge graph quality and traceability.