The QT interval is an electrocardiographic measure representing the sum of ventricular depolarization and repolarization, estimated by QRS duration and JT interval, respectively. QT interval abnormalities are associated with potentially fatal ventricular arrhythmia. Using genome-wide multi-ancestry analyses (>250,000 individuals) we identify 177, 156 and 121 independent loci for QT, JT and QRS, respectively, including a male-specific X-chromosome locus. Using gene-based rare-variant methods, we identify associations with Mendelian disease genes. Enrichments are observed in established pathways for QT and JT, and previously unreported genes indicated in insulin-receptor signalling and cardiac energy metabolism. In contrast for QRS, connective tissue components and processes for cell growth and extracellular matrix interactions are significantly enriched. We demonstrate polygenic risk score associations with atrial fibrillation, conduction disease and sudden cardiac death. Prioritization of druggable genes highlight potential therapeutic targets for arrhythmia. Together, these results substantially advance our understanding of the genetic architecture of ventricular depolarization and repolarization.