The only time God may take His hands off you is when you are not worthy of your calling. Saul was not worthy of God's calling because he ignored God's commande adn changed the rules.
2 Thess 1:11 reads:...we constantly pray for you, that our God may count you worthy of His calling, and that by His power He may fulfill every good purpose of yours and every act prompted by yoru faith. NIV
IF we disobey God's directions, He reserves the right to remove us as the one to fulfill His plans.
When God looks at your life are you close to being removed because you have been disobedient? Or are you like David who is a person after God's own heart.