ThetaGPU will undergo hardware replacements and a major software upgrade in the latter half of June. The hardware upgrades will be conducted in 3 phases where 8 nodes are taken down for service each time. The software upgrade work across all the nodes will be combined with the hardware work in phase 1. The tentative schedule is June 14-16 for phase 1, June 21 for phase 2, and June 28 for phase 3. It is important to note that there will be a new default CUDA library (11.3) after the software upgrade on June 16. Software built on the current CUDA 11.0 should still work, but we recommend rebuilding any dependent software.