I am trying to use sort to sort a 812 MB file:

mmap2(NULL, 4118106112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
--- SIGSEGV (Segmentation fault) @ 0 (0) ---

I guess sort ignores the failure of the mmap system call. Here's a stack
trace:

#0  0xb7e0c42d in strcoll_l () from /lib/i686/cmov/libc.so.6
#1  0xb7e08a31 in strcoll () from /lib/i686/cmov/libc.so.6
#2  0x08053249 in memcoll (
    s1=0x548b2008 "\033[1;24r\033[0;10m\033[4l\033[?7h\033[39;49m\033[?25l\033[?1c\033[39;49m\033[37m\033[40m\033[H\033[J\033[24d\033[0;10;1m\033[33m\033[44m\033[J\033[H\033[K\033[2d\033[0;10m\033[34m\033[47m\033[K\033[3d\033[K\033[4d\033[K\033[5d\033[K\033[6d\033[K\033[7d\033[K\033[8d\033[K\033[9d\033[K\033[10d\033[K\033[11d\033[K\033[12d\033[K\033[13"..., s1len=769181578,
    s2=0x8263e806 "\b\033[0;10;1m\033[33m\033[44m \033[36m\033[44mG\033[33m\033[44mames", ' ' <repeats 12 times>, "\033[23;28H\033[0;10m\033[39;49m\033[37m\033[40m\033[9;32H\033[0;10;1m\033[33m\033[40m \033[36m\033[40mG\033[33m\033[40mames", ' ' <repeats 12 times>, "\033[23;28H\033[0;10m\033[39;49m\033[37m\033[40m\033[9;33H\b\033[0;"..., s2len=54438834) at memcoll.c:56
#3  0x08050264 in xmemcoll (
    s1=0x548b2008 "\033[1;24r\033[0;10m\033[4l\033[?7h\033[39;49m\033[?25l\033[?1c\033[39;49m\033[37m\033[40m\033[H\033[J\033[24d\033[0;10;1m\033[33m\033[44m\033[J\033[H\033[K\033[2d\033[0;10m\033[34m\033[47m\033[K\033[3d\033[K\033[4d\033[K\033[5d\033[K\033[6d\033[K\033[7d\033[K\033[8d\033[K\033[9d\033[K\033[10d\033[K\033[11d\033[K\033[12d\033[K\033[13"..., s1len=769181577,
    s2=0x8263e806 "\b\033[0;10;1m\033[33m\033[44m \033[36m\033[44mG\033[33m\033[44mames", ' ' <repeats 12 times>, "\033[23;28H\033[0;10m\033[39;49m\033[37m\033[40m\033[9;32H\033[0;10;1m\033[33m\033[40m \033[36m\033[40mG\033[33m\033[40mames", ' ' <repeats 12 times>, "\033[23;28H\033[0;10m\033[39;49m\033[37m\033[40m\033[9;33H\b\033[0;"..., s2len=54438833) at xmemcoll.c:43
#4  0x0804c6eb in compare (a=0x8d8a2d78, b=0x8d8b2638) at sort.c:1910
#5  0x0804d488 in mergelines (t=0x8d8b2658, lo=0x8d8a2d88, nlo=1,
    hi=0x8d8b2648, nhi=2) at sort.c:2211
#6  0x0804d468 in sortlines (lines=0x8d8b2658, nlines=3, temp=0x8d8a2d88)
    at sort.c:2274
#7  0x0804d612 in sortlines_temp (lines=0x8d8b2658, nlines=7, temp=0x8d8a2d88)
    at sort.c:2303
#8  0x0804d41d in sortlines (lines=0x8d8b2658, nlines=15, temp=0x8d8a2d88)
    at sort.c:2270
#9  0x0804d612 in sortlines_temp (lines=0x8d8b2658, nlines=31, temp=0x8d8a2d88)
    at sort.c:2303
#10 0x0804d41d in sortlines (lines=0x8d8b2658, nlines=62, temp=0x8d8a2d88)
    at sort.c:2270
#11 0x0804d612 in sortlines_temp (lines=0x8d8b2658, nlines=124,
    temp=0x8d8a2d88) at sort.c:2303
#12 0x0804d41d in sortlines (lines=0x8d8b2658, nlines=248, temp=0x8d8a2d88)
    at sort.c:2270
#13 0x0804d612 in sortlines_temp (lines=0x8d8b2658, nlines=497,
    temp=0x8d8a2d88) at sort.c:2303
#14 0x0804d41d in sortlines (lines=0x8d8b2658, nlines=995, temp=0x8d8a2d88)
    at sort.c:2270
#15 0x0804d612 in sortlines_temp (lines=0x8d8b2658, nlines=1990,
    temp=0x8d8a2d88) at sort.c:2303
#16 0x0804d41d in sortlines (lines=0x8d8b2658, nlines=3981, temp=0x8d8a2d88)
    at sort.c:2270
#17 0x0804db84 in sort (files=0x805d85c, nfiles=0, output_file=0x0)
    at sort.c:2490
#18 0x0804f16f in main (argc=2, argv=0xbfb9e9f4) at sort.c:3171

Oddly, s1 and s2 are both non-NULL, so maybe this is a glibc bug?

It was suggested that I work out the size of the largest line:

        $ wc -L xsession-errors
        wc: xsession-errors:8083: Invalid or incomplete multibyte or wide character
        662677599 xsession-errors

That's quite a long line!

Oh, BTW, my system has 2 GB of RAM and 1 GB of swap.

What happens if you use the -S option to sort? (I actually don't think
it will matter, but give it a shot.)

Mike Stone

Indeed, it doesn't make a difference. I don't think so much that it is
the length of the file that's the issue--but the length of the longest
line (~600 MB or something ridiculous).
-- 
Sam Morris <sam@robots.org.uk>

This actually seems to be a libc problem--there are some mmap calls
within the strcoll function, and the segfault seems to happen after
that.

Mike Stone

Salaam,



Almana Financial Broker, is the home of discerning investors. We offer
independent financial advice and assist our clients to make sound investment
decisions from the array of investment opportunities available. We are
structured to render personalized services to our clients thereby ensuring
safety of capital, adequate returns on investments.



Our investors are ready to provide funding for your business expansion as
Debt/Equity finance. If you require funding, we would be able to partner
with you.



We look forward to your response, thanks and stay safe,



Abdulkader Abdi, CFA

Almana Financial Broker

Dubai, United Arab Emirates