Actually, there is no specific way to check FESCO bill without a reference number online. You need some information to use to fetch your bill online.
The alternative way is to visit the FESCO revenue office with your account details, reference number, or customer ID, or you must have your bill to get the updates.