Executives expect lender numbers to drop from more than 1,500 to as few as 50