Text of github.com › curl › curl › issues › 4919 as Styx crawled it on Nov 19, 2026. The page may have changed since. Only its words are kept, without pictures or layout.

Open the live page

problems sending large data on macOS over unix domain sockets · Issue #4919 · curl/curl · GitHub

Skip to content You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert problems sending large data on macOS over unix domain sockets #4919 New issue Copy link New issue Copy link Closed Closed problems sending large data on macOS over unix domain sockets #4919 Copy link Assignees Description bagder opened on Feb 12, 2020 Issue body actions (I received this bug report by a user who shall remain anonymous) curl 7.68.0 seems to have problems sending large data on macOS over unix domain sockets.

You can reproduce by using netcat as a server like this: rm -rf /tmp/foofoo.sock && nc -U -l /tmp/foofoo.sock And then invoke curl: curl -v --unix-sock /tmp/foofoo.sock -H "$(python -c 'print "x"*90000'): OK" http://localhost:8888/ (This has curl send a very long single header that ends with OK) If you look at the output from curl, you’ll notice that it never gets around to sending the OK part of the string. This seems to be specific to recent curls and to macOS Taking a look at dtrace it seems like curl stops asking select whether the socket is writable $ sudo dtrace -n 'syscall::sendto:entry / execname == "curl" && arg0 == 5 / { printf("%s(%d): %d, %s, %d", execname, pid, arg0, copyinstr(arg1), arg2); } syscall::sendto:return / execname == "curl" / { printf("return %d %d", arg1, errno); } syscall::select:entry / execname == "curl" / { printf("select(%d, %p, %p, %p)\n", arg0, arg1, arg2, arg3); printf("READ:\n"); tracemem(copyin(arg1, 128), 16); printf("WRITE:\n"); tracemem(copyin(arg2, 128), 128); } syscall::select:return / execname=="curl" / { printf("%s return: %d %d", probefunc, arg1, errno); }' Password: dtrace: description 'syscall::sendto:entry ' matched 4 probes CPU ID FUNCTION:NAME 2 347 select:entry select(4, 7ffeebbdf230, 7ffeebbdf2b0, 7ffeebbdf330) READ: 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 0: 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

WRITE: 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 2 348 select:return select return: 0 0 2 347 select:entry select(6, 7ffeebbdf170, 7ffeebbdf0f0, 7ffeebbdf1f0) READ: 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ WRITE: 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 0: 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ...............

10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

2 348 select:return select return: 1 0 2 427 sendto:entry curl(4116): 5, GET / HTTP/1.1 Host: localhost:8888 User-Agent: curl/7.68.0 Accept: */* xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx, 90084 2 428 sendto:return return 8192 0 6 347 select:entry select(6, 7ffeebbdf230, 7ffeebbdf1b0, 7ffeebbdf2b0) READ: 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 0: 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ...............

WRITE: 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 6 348 select:return select return: 1 0 (This is a printout of the send calls and then the fdsets passed to select, the relevant FD is 5 so 0x20) Activity bagder added help wanted KNOWN_BUGS material on Mar 31, 2020 bagder commented on Apr 5, 2020 bagder on Apr 5, 2020 Member Author More actions What puzzles me the most about this bug is that very little internals even know what kind of socket it is working with and most of the code is just agnostic and work with "a socket" independent of what kind of connection it is.

It might imply that unix domain sockets somehow behaves differently than SOCK_STREAM ones do - on macos. Research is required. The bug triggers if the custom header is larger than 8100 bytes something. Then only the first piece of the request is sent and nothing more is ever sent. The socket just doesn't seem to signal itself as writable anymore... bagder commented on Apr 7, 2020 bagder on Apr 7, 2020 Member Author More actions The bug triggers on macOS if the custom header is larger than 8100 bytes something.

Then only the first piece of the request is sent and nothing more is ever sent. Clearly the logic for handling a "split" request send is broken, and I believe it is broken on all platforms. This bug also reproduces on Linux but then at around ~220K so the custom header needs to be made much larger there. bagder commented on Apr 7, 2020 bagder on Apr 7, 2020 Member Author More actions I arrived at this little patch that lets me write a simple test case for this problem using the regular test suite, which makes it easier to work on a fix: diff --git a/lib/http.c b/lib/http.c index 324dd7252..814d69fd2 100644 --- a/lib/http.c +++ b/lib/http.c @@ -1227,12 +1227,25 @@ CURLcode Curl_add_buffer_send(Curl_send_buffer **inp, return result; } memcpy(data->state.ulbuf, ptr, sendsize); ptr = data->state.ulbuf; } - else + else { + #ifdef CURLDEBUG + /* Allow debug builds override this logic to force short initial sends */ + char *p = getenv("CURL_SMALLREQSEND"); + if(p) { + size_t altsize = (size_t)strtoul(p, NULL, 10); + if(altsize) + sendsize = CURLMIN(size, altsize); + else + sendsize = size; + } + else + #endif sendsize = size; + } result = Curl_write(conn, sockfd, ptr, sendsize, &amount); if(!result) { /* bagder removed KNOWN_BUGS material help wanted on Apr 7, 2020 bagder self-assigned this on Apr 7, 2020 bagder added a commit that references this issue on Apr 7, 2020 http: don't consider upload done if the request isn't completely sent… This issue will close once commit 02d03c2 is merged into the 'master' branch.

02d03c2 bagder linked a pull request that will close this issue Fix split http GET request #5197 on Apr 7, 2020 bagder closed this as completed in 0ef54ab on Apr 8, 2020 Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment Metadata Metadata Assignees bagder Labels No labels No labels Type No type Projects No projects Milestone No milestone Relationships None yet Development No branches or pull requests Issue actions You can’t perform that action at this time.