In this paper an image outlier technique, which is a hybrid model called SVM regression based DWT optimization have been introduced. Outlier filtering of RGB image is using the DWT model such as Optimal-HAAR wavelet changeover (OHC), which optimized by the Least Square Support Vector Machine (LS-SVM) . The LS-SVM regression predicts hyper coefficients obtained by using QPSO model. The mathematical models are discussed in brief in this paper: (i) OHC which results in better performance and reduces the complexity resulting in (Optimized FHT). (ii) QPSO by replacing the least good particle with the new best obtained particle resulting in “Optimized Least Significant Particle based QPSO” (OLSP-QPSO). On comparing the proposed cross model of optimizing DWT by LS-SVM to perform oulier filtering with linear and nonlinear noise removal standards.