The significant degradation of reconstructed image resolution and the presence of apparent artifacts can result from sub-millimetric misalignment of scanner configuration in highspatial resolution positron emission tomography (PET) systems. Therefore, ensuring accurate scanner configuration alignment is crucial for obtaining high-quality reconstructed images in such systems. Moreover, incorporating time-of-flight (TOF) technology has demonstrated its efficacy in enhancing image quality within PET. Achieving substantial reductions in statistical noise within reconstructed PET images via TOF information necessitates excellent timing resolution across the scanner's entire field of view (FOV). This work aims to explore the impact of misalignment within the Prism-PET brain scanner system and to introduce an inverse rendering-based technique for geometric calibration. Additionally, with regard to time-of-flight (TOF), we update our iterative TOF offset calibration method and deploy it on GPU to expedite computation. The results of the proposed inverse rendering-based calibration method illustrate its capacity to rectify misalignment configurations of detectors on the scanner rings, aligning them with the ideal geometry configuration to effectively eliminate image artifacts. The GPU-accelerated TOF offset calibration maintains a consistent coincidence time resolution (CTR) of 320 ps full width half maximum (FWHM) compared to the previous method. However, it achieves a remarkable acceleration, now completing the process approximately 240 times faster than before, requiring only about 12 minutes.