I suspect there's a file in your Startup directory that's invoking Internet Explorer. Try this:
In Windows XP:
1. In Windows Explorer, navigate to the directory:
C:\Documents and Setting\your username\Start Menu\Programs\Startup
2. Look for any files of the type Internet Shortcut.
3. If you find one, double-click it to try opening it. If it's your offending page, delete the file.
In Windows Vista, possibly Windows 7. (This would be the same as the above, except that the Start Menu directory is hard to get to.):
1. In Windows Explorer, enter this in the address window at to the top of the window:
C:\Users\your username\Start Menu\Programs\Startup
2. Look for any files of the type Internet Shortcut.
3. If you find one, double-click it to try opening it. If it's your offending page, delete the file.
Hope that helps.