Field experience shows that the cutting transportation and hole-cleaning phenomena are essential during the drilling phase. Particularly in directional drilling, when the accumulation of cutting has caused some drilling problems such as drill string sticking, formation failure, slow rate of penetration, drill bit abrasion, and the like. Through the study, a novel method for efficient hole cleaning, considering different parameters such as flow rate, the drill bit nozzles’ flow area, the consistency and flow behavior indices in the same time using PSO and ACO algorithms were implemented. Moreover, Power Law has been considered for the fluid rheology model. Based on this, the research parameter shows that the PSO algorithm is much more accurate than the ACO algorithm, improving objective function by 50% and 4%, respectively. The performance of each algorithm was evaluated, and the results show that hole cleaning has been significantly improved. The flow rate and the bit nozzle size, which play key roles, were selected as optimization variables. Effective parameters on hole cleaning were evaluated, and the results before and after optimization showed a significant improvement in the model. The PSO and ACO algorithms have been coded in MATLAB software, and the results are compared to the results of the ant colony. The amount of PV and YP has an inverse effect on the increment of minimum velocity required for cutting transport. Various model analyses reveal that the PSO algorithm is more accurate and robust than the Ant colony algorithm.