The bitwise operators (& | ^ ~) operate on integer-type values and treat them as bit arrays, with each bit interpreted as (0=false, 1=true). The logical versions (&& || ~) operate on boolean values and return a boolean result. (There isn't an operator specifically "logical XOR", but x != y is equivalent to x XOR y for boolean x and y.)
Bit manipulations (including the bitwise operations, plus shifts) are most often used for efficiently packing multiple binary fields into an int or long-sized package. One way to number bits ("little-endian") is to count from bit 0 as the least-significant bit. If you want to extract bits 3-5 from an int value m into another int k, you can use:
k = (m >> 3)&7
The reverse operation of storing the low 3 bits of k into bits 3-5 of m, without changing any other bits of m is:
m = (m & ~(7<<3)) | ((k^7) << 3);
Normally, you use symbolic constants instead of raw numbers. Note that 7 represents an int with 111 in the low 3 bits and 0 bits everywhere else.
Other uses arise in more serious data compression for both data and streaming media, encryption/decryption of binary data, and interpreting/converting binary data that doesn't fit one of the Java data types.
Plus there are a number of rare curiosities. For example:
static boolean isPowerOfTwo(int n) { return (n ^ -n) == n; }
...returns (true) if n is a power of 2 (1, 2, 4, 8, 16, ...) or (false) if not.