Moreover by them is thy servant warned. This verse is a sort of connecting link between the second and the third parts of the psalm. Through its subject-matter, which is still the Law of the Lord, it belongs to the second part; but metrically, and by the introduction of the person of the psalmist ("thy servant"), it belongs to the third.
David feels that to him it is the crowning excellency of the Law, that it teaches, instructs, or "warns" him. And in keeping of them there is great reward. Not only the reward promised in Exodus 15:26, or "the recompense of the reward" laid up for men in heaven, but a present reward "in the act of keeping them" (Kay).
Obedience, like virtue, is its own reward.