There are 3 classes which can be caught with a try ... catch : throw , error and exit .

• throw is generated using throw/1 and is intended to be used for non-local returns and does not generate an error unless it is not caught (when you get a nocatch error).
• error is generated when the system detects an error. You can explicitly generate an error using error/1 . The system also includes a stacktrace in the generated error value, for example {badarg,[...]} .
• exit is generated using exit/1 and is intended to signal that this process is to die.

The difference between error/1 and exit/1 is not that great, it more about intention which the stacktrace generated by errors enhances.
The difference between them is actually more noticeable when doing catch ... : when throw/1 is used then the catch just returns the thrown value, as is expected from a non-local return; when an error/1 is used then the catch returns {'EXIT',Reason} where Reason contains the stacktrace; while from exit/1catch also returns {'EXIT',Reason} but Reason only contains the actual exit reason. try ... catch looks like it equates them, but they are/were very different.

[UPDATED]

• I glossed over the important difference between throw and error, pointed out by Robert Virding. This edit is just for the record!*

throw error is to be used where one would use throw in other languages. An error in a running process has been detected by your code, which signals an exception with error/1 . The same process catches it (possibly higher up in the stack), and the error is to be handled within the same process. error always brings with it a stacktrace.
throw is to be used not to signal an error, but just to return a value from a deeply nested function. Since it unwinds the stack, calling throw returns the thrown value to the place it was caught. As in the case of error , we're catching stuff that was thrown, only what was thrown wasn't an error but rather just a value passed up the stack. This is why throw does not bring with it a stacktrace.
As a contrived example, if we wanted to implement an exists function for lists, (similar to what list:any does) and as an exercise without doing the recursing ourselves, and using just list:foreach , then throw could be used here:

exists(P, List) ->
  F = fun(X) -> 
    case P(X) of 
      true -> throw(true); 
      Whatever -> Whatever 
    end
  end,
  try lists:foreach(F, List) of
    ok -> false
  catch
   true -> true
  end.

A value thrown but not caught is treated as an error : a nocatch exception will be generated.
EXIT is to be signaled by a process when it 'gives up'. The parent process handles the EXIT, while the child process just dies. This is the Erlang let-it-crash philosophy.
So exit/1 's EXIT is not to be caught within the same process, but left to the parent. error/1 's errors are local to the process - i.e., a matter of what happens and how it is handled by the process itself; throw/1 is used for control flow across the stack.

[UPDATE]

1. This tutorial explains it well: http://learnyousomeerlang.com/errors-and-exceptions
2. Note there is also a exit/2 - called with a Pid of a process to send the EXIT to. exit/1 implies the parent process.

I'm new to Erlang, but here's how I think about what these things are, their differences, what they're used for, etc.:
throw : a condition that should be handled locally (i.e. within the current process). E.g. caller is looking for an element in a collection, but does not know if the collection actually contains such an element; then, the callee could throw if such an element is not present, and the caller detect absence by using try[/of]/catch . If caller neglects to do this, then this gets turned into an nocatcherror (explained below).
exit : The current process is done. E.g. it has simply finished (in that case, you'd pass normal , which is treated the the same as the original function returning), or its operation was cancelled (E.g. it normally loops indefinitely but has just received a shut_down message).
error : the process has done something and/or reached a state that the programmer did not take into account (E.g. 1/0), believes is impossible (E.g. case ... of encounters a value that does not match any case), or some precondition is not met (E.g. input is nonempty). In this case, local recovery doesn't make sense. Therefore, neither throw nor exit is appropriate. Since this is unexpected, a stack trace is part of the Reason.
As you can see, the above list is in escalating order:
throw is for sane conditions that the caller is expected to handle. I.e. handling occurs within the current process.
exit is also sane, but should end the current process simply because the process is done.
error is insane. Something happened that can't reasonably be recovered from (usually a bug?), and local recovery would not be appropriate.
vs. other languages:
throw is analogous to the way checked exceptions are used in Java. Whereas, error is used in a manner more analogous to unchecked exceptions. Checked exceptions are exceptions you want the caller to handle. Java requires you to either wrap calls in try/catch or declare that your method throws such exceptions. Whereas, unchecked exceptions generally propagate to the outermost caller.
exit does not have a good analog in more "conventional" languages like Java, C++, Python, JavaScript, Ruby, etc. exit vaguely like an uber- return : instead of returning at the end, you can return from the middle of a function, except you don't just return from the current function, you return from them ALL.
exit Example

serve_good_times() ->
  receive
    {top_of_the_mornin, Sender} ->
      Sender ! and_the_rest_of_the_day_to_yourself;
    {you_suck, Sender} ->
      Sender ! take_a_chill_pill;
    % More cases...

    shut_down ->
      exit(normal)
  end,
  serve_good_times()
end

Since serve_good_times calls itself after almost all messages, the programmer has decided that we don't want to repeat that call in every receive case. Therefore, she has put that call after the receive. But then, what if serve_good_times decides to stop calling itself? This is where exit comes to the rescue. Passing normal to exit causes the process to terminate just as though the last function call has returned.
As such, it's generally inappropriate to call exit in a general purpose library, like lists . It's none of the library's business whether the process should end; that should be decided by application code.
What About Abnormal exit ?
This matters if another process (the "remote" process) is linked to the "local" process that calls exit (and process_flag(trap_exit, true) was not called): Just like the last function returning, exit(normal) does not cause remote process to exit. But if the local process makes a exit(herp_derp) call, then the remote process also exits with Reason=herp_derp . Of course, if the remote process is linked to yet more processes, they also get exit signal with Reason=herp_derp . Therefore, non- normal exits result in a chain reaction.
Let's take a look at this in action:

1> self().   
<0.32.0>
2> spawn_link(fun() -> exit(normal) end).
<0.35.0>
3> self().
<0.32.0>
4> 
4> 
4> spawn_link(fun() -> exit(abnormal) end).
** exception exit: abnormal
5> self().
<0.39.0>
6>

The first process that we spawned did not cause the shell to exit (we can tell, because self returned the same pid before and after spawn_link ). BUT the second process did cause the shell to exit (and the system replaced the shell process with a new one).
Of course, if the remote process uses process_flag(trap_exit, true) then it just gets a message, regardless of whether the local process passes normal or something else to exit . Setting this flag stops the chain reaction.

6> process_flag(trap_exit, true).
false
7> spawn_link(fun() -> exit(normal) end).  
<0.43.0>
8> self().
<0.39.0>
9> flush().
Shell got {'EXIT',<0.43.0>,normal}
ok
10>                                         
10> 
10> spawn_link(fun() -> exit(abnormal) end).
<0.47.0>
11> self(). 
<0.39.0>
12> flush().
Shell got {'EXIT',<0.47.0>,abnormal}

Recall that I said that exit(normal) is treated like the original function returning:

13> spawn_link(fun() -> ok end).
<0.51.0>
14> flush().
Shell got {'EXIT',<0.51.0>,normal}
ok
15> self().
<0.39.0>

What do you know: the same thing happened as when exit(normal) was called. Wonderful!