Robotic harvesting research has seen significant achievements in the past decade, with breakthroughs being made in machine vision, robot manipulation, autonomous navigation and mapping. However, the missing capability of obstacle handling during the grasping process has severely reduced the harvest success rate and limited the overall performance of robotic harvesting. This work focuses on leaf interference caused by slip detection and handling, where solutions to robotic grasping in an unstructured environment are proposed. Through analysis of the motion and force of fruit grasping under leaf interference, the connection between object slip caused by leaf interference and inadequate harvest performance is identified for the first time in the literature. A learning-based perception and manipulation method is proposed to detect slip that causes problematic grasps of objects, allowing the robot to implement timely reactions. Our results indicate that the proposed algorithm detects grasp slips with an accuracy of 94%. The proposed sensing-based manipulation demonstrated great potential in robotic fruit harvesting and could be extended to other pick-place applications.