The simple answer is, trust me, you don't want to.
The more complicated answer is...
I am unfortunately not familiar with Wamp or Joomla. However, I run an Apache server, and I actually did create an Apache folder that will run from a removable device.
Before I get started, I should hit some obvious (or not so obvious) points. The server running on C must be stopped if you want to run a server on E. The server on E will be http://localhost/, just like the one on C, and you can only run one of these servers at a time. (The exception is if you configure one of them to use a port other than port 80, but I will not go into that...) Your hostname does not change; the server is still running on your physical machine.
First, you MUST realise that E:\ is great, but not really. Take your flash drive somewhere else, plug it into another computer, and it might be F or G or even H, I, J, K, or something else... not good! Windows will give it the first available drive letter. If the computer has a couple of hard disk partitions, a few CD drives, perhaps a zip drive, a USB 9-in-1 multi-flash card reader, and a few network drives, your drive might end up being N or O. Applications generally will not run if they think they are on C and they really are on E. Same goes for if the application thinks it has E and it got F.
If you installed to your hard drive then copied the application folder to your flash disk, the app still thinks it is running from the C drive. This poses major problems. First of all, the app is actually still installed on your C drive. This means it might borrow any or all of the settings and data from your C drive without you even realising it. You might think you finally got it to work only to discover that your www files are coming from the www folder on your C drive. If you want to try to make this work from your flash drive, the first thing I would recommend is renaming the program folder on the C drive. If it is "C:\Program Files\Wamp\", rename it to "Wamp2" or "Wamp Backup" or "teapot7" or "3.14159" or anything you want. Now, at least your E installation will not be able to "borrow" files from the C installation... it will probably crash instead. This is good, though; at least you know there is a problem. Rename the folder back when you get the E installation working on its own... or more likely when you give up.
Hopefully, Wamp does not use the registry to store your settings. If it does, kiss it goodbye and switch to Apache.
If the settings are stored in a config file somewhere, you have hope. Go through the entire config file (or files, if there are more than one config file) and change all of the paths from absolute (perhaps "C:\Program Files\Wamp\www\") to relative (maybe "..\www\"). You will probably have to base the relative paths off the folder the binary is stored in... in other words, if the binary is called "C:\Program Files\Wamp\bin\wamp.exe", your "C:\Program Files\Wamp\www\" folder would probably be "..\www\".
Hypothetically speaking, with every path changed to a relative path, it SHOULD run from your flash drive (even if you put it in another computer and it gets a different drive letter). Good luck with that.
As soon as you think you have it working perfectly, you need to put it to the real test. Take the flash disk to a different computer, plug it in, give it a different drive letter (use the computer management to change the drive letter if it gave it E ), and see if it runs. You might very well discover that some DLLs are missing; they are probably in the Windows, system, or system32 folder. If you ARE missing DLLs, find them and copy them into the same folder on the flash disk as the server executable... it SHOULD be able to find them now.
As I said, I did this with Apache. If you want, I would be willing to zip and send you a stand-alone, portable Apache server that ought to unzip and run as-is, Apache + PHP5. I am not sure how similar it would be to your current server. If you want it, let me know.