Lời giải chi tiết:

Ta có \(5f\left( x \right) - 7f\left( {1 - x} \right) = 3\left( {{x^2} - 2x} \right)\).

Thay \(x\) bởi \(1 - x\) ta có:

\(\begin{array}{l}\,\,\,\,5f\left( {1 - x} \right) - 7f\left( x \right) = 3\left( {{x^2} - 2x + 1 - 2 + 2x} \right)\\ \Leftrightarrow 5f\left( {1 - x} \right) - 7f\left( x \right) = 3\left( {{x^2} - 1} \right)\end{array}\)

Do đó ta có hệ phương trình: \(\left\{ \begin{array}{l}5f\left( x \right) - 7f\left( {1 - x} \right) = 3\left( {{x^2} - 2x} \right)\\5f\left( {1 - x} \right) - 7f\left( x \right) = 3\left( {{x^2} - 1} \right)\end{array} \right.\) .

Suy ra

\(\begin{array}{l}25f\left( x \right) - 49f\left( x \right) = 15\left( {} \right){x^2} - 2x + 21\left( {{x^2} - 1} \right)\\ \Leftrightarrow  - 24f\left( x \right) = 36{x^2} - 30x - 21\\ \Leftrightarrow f\left( x \right) =  - \dfrac{1}{8}\left( {12{x^2} - 10x - 7} \right)\\ \Rightarrow f'\left( x \right) =  - \dfrac{1}{4}\left( {12x - 5} \right)\end{array}\)

Do đó \( - \dfrac{a}{b} = \int\limits_0^1 {x.f'\left( x \right)dx}  =  - \dfrac{1}{4}\int\limits_0^1 {x\left( {12x - 5} \right)dx}  =  - \dfrac{3}{8} \Rightarrow \left\{ \begin{array}{l}a = 3\\b = 8\end{array} \right.\).

Vậy \(8a - 3b = 8.3 - 3.8 = 0\).

Chọn B.