Do they have a back-up system in place to protect your data in case there is a power outage?
Do they require you to make payments in advance?
Do they require a long-term contract?
Do they charge set up fees?
How is their customer support? Test them.
Do they offer shopping cart software to process your orders?
Do they provide you with an online Web Site Manager?
Do they provide you with an online support manual?
Then you should check if they have:
reliable tech support
Your own domain name (www.yourname.com)
At least good of monthly transfer (traffic)
A minimum of 20MB of server space
POP email accounts - name@yourdomain.com
Unlimited email aliases
Your own unrestricted CGI-Bin
MySQL Database
Perl
htaccess password protection
Server Side Includes (SSI) support
Easy access to your log files
Statistics on visits to your site