Deep-hole trepanning is challenged by enclosed cutting conditions and limited chip evacuation, leading to increased cutting loads, chip blockage, and reduced tool reliability. These challenges are particularly severe in trepanning because the retained core material confines the annular cutting zone and narrows the chip evacuation path. To optimize the geometric parameters of deep-hole trepanning tools, a constrained multi-objective optimization model is developed to simultaneously minimize axial force, cutting torque, and chip blockage risk, with rake angle, clearance angle, point angle, edge radius, chip-breaker width, and chip-groove guide angle as design variables. To solve this model, a Sobol-Opposition Initialization, Reference-Vector Dual-Archive Management, and Reinforcement Learning-Regulated Multi-Objective Grey Wolf Optimizer (SORD-RL-MOGWO) is developed. Sobol-opposition initialization enhances initial population diversity and feasible-space coverage, reference-vector dual-archive management balances convergence and distribution uniformity, and reinforcement learning adaptively regulates exploration and exploitation during the search. Under validation conditions of a cutting speed of 20 m/min and a feed rate of 0.060 mm/r, the mechanical performance model achieved coefficients of determination of 0.98 and 0.99 for axial force and cutting torque, with average relative errors of 1.38% and 1.04%, respectively. SORD-RL-MOGWO achieved a hypervolume of 0.78 ± 0.01 and a feasible solution ratio of 95.93 ± 1.72%, outperforming MOGWO, C-TAEA2, CMOEA-TSRA, and DRLOS-EMCMO. Across feed rates of 0.040–0.100 mm/r at 20 m/min, the optimized tool consistently reduced axial force, cutting torque, and chip blockage. At 0.100 mm/r, axial force and cutting torque were reduced by 14.23% and 14.00%, respectively, while the chip blockage rate decreased from 55.0% to 20.0%. These results demonstrate that the proposed method provides a practical approach to reducing cutting loads and chip-evacuation failure while maintaining structural feasibility, thereby improving tool reliability and process stability in the industrial deep-hole trepanning of difficult-to-machine alloys such as GH4169.
