you could set the sessions on Jenkins to be shorter than those on your IdP. Number of seconds since user was authenticated in IdP while his authentication is considering as active.
Default is 24h * 60 min * 60 sec = 86400