WCF call has 15 seconds delay after idle for couple of minutes

Customer has a WCF application which uses ASP.Net as a client. The service is hosted on another IIS as well. For secure purpose, customer enabled transport layer security for the WCF service. Customer found the ASP.Net web page took more than 15+ seconds sometimes while normally it takes less than 1 second. With some investigation, we found the slowness happens only when the web application idle