I have a tiny program that reads a csv file (100M). The problem is that my program makes the Erlang interpreter crash:
Crash dump was written to: erl_crash.dump
eheap_alloc: Cannot reallocate 3563526520 bytes of memory (of type "heap").
Aborted
Here is the program:
readlines(FileName) ->
{ok, Device} = file:open(FileName, [read]),
try get_all_lines(Device)
after file:close(Device)
end.
get_all_lines(Device) ->
case io:get_line(Device, "") of
eof -> [];
Line -> [Line | get_all_lines(Device)]
end.
And I do:
Path="...csv".
Lines=tut6:readlines(Path).
And this procudes a crash.
Can someone please tell me what the problem is? Maybe something is wrong with my program? How can I avoid the crashes?
Thanks in advance
2条答案
按热度按时间yks3o0rb1#
Did you realize that 3563526520 is 3.3 GB? How much memory does your system have? The gigantic memory consumption stems from the fact that you have chosen the least optimal algorithm for reading the lines:
So, to fix the code:
Learn You Some Erlang has an excellent discussion about tail recursive functions if you want to know how to properly implement such functions.
If the function was written in a tail-recursive manner the whole algorithm could look like this:
huwehgph2#
尝试
然后重新思考你真正在做什么。