Though he fall, he shall not be utterly cast down. "The difference," as Hengstenberg observes, "is that between misfortune or loss, and absolute ruin." The good man may be afflicted; he may even fall into some fault (Galatians 6:1) or grievous sin (2 Samuel 11:4); but so long as "the root of the matter is in him" (Job 19:28), God will not suffer him to be prostrated.
For the Lord upholdeth him with his hand; literally, the Lord supports his hand. If he falls, God (as Luther says) "catches him by the hand, and raises him up again." So David had himself experienced (2 Samuel 12:13).