This is typically a 'symmetric' type of cryptosystem. You use the same key for encryption and decryption; which is why the key needs to be kept 'secret'. The most commonly used encryption algorithm is the DES (Data Encryption Standard).
DES is a block cipher. It operates on 64-bit blocks. It has a 56-bit key. The actual encryption algorithm is very complex. I won't go into the details (that's because I don't remember most of it! ;-)). However it uses several permutations; generates 16 keys from the first main key and then does a whole lot of stuff before coming up with the cipher text.
The way to crack DES is to try out all the 2^56 keys! That entails *massive* computing power; however it can be done. May take days and months to crack the key for one message but it can be done. So it's not entirely secure.
So what they do is they use Triple-DES. It uses 3 keys - that gives an effective key size of 168 bits. Now that's almost impossible to crack with the current computing power that we have. Security guru Bruce Schneiner (not sure of the spelling) said - There isn't enough silicon in the galaxy or enough time before the sun burns out to brute-force Triple-DES.
Here's a lot of info about DES.
BTW, the unix password encryption uses DES... 25 times in a row.