File 1151-Handle-multi-giga-byte-writes-to-files.patch of Package erlang

Overview Repositories Revisions Requests Users Attributes Meta

File 1151-Handle-multi-giga-byte-writes-to-files.patch of Package erlang (Revision a19498298f166adf5d0de3a98d4a48a2)

Currently displaying revision a19498298f166adf5d0de3a98d4a48a2 , Show latest

From 5ae46e823a8a52ed4e5b960ff62975894b1a8302 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Bj=C3=B6rn=20Gustavsson?= <bjorn@erlang.org>
Date: Tue, 29 Mar 2016 09:10:13 +0200
Subject: [PATCH] Handle multi-giga byte writes to files

Test cases that write 4Gb to a file at once would fail on
OS X and FreeBSD.

By running a simple test program on OS X (El Capitan 10.11.4/Darwin
15.4.0), I found that writev() can handle more than 4Gb of data, while
write() only can handle less than 2Gb. (Note that efile_drv.c will use
write() if there is only one element in the io vector, and writev() if
there is more than one.)

It is tempting to attempt to piggy-back on the existing mechanism
for segmenting write operations in efile_drv.c, but because of the
complex code I find it too dangerous, both from a correctness and
performance perspective.

Instead do the change in unix_efile.c, which is considerably
simpler.
---
 erts/emulator/drivers/unix/unix_efile.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/erts/emulator/drivers/unix/unix_efile.c b/erts/emulator/drivers/unix/unix_efile.c
index ac9b681..c097bc1 100644
--- a/erts/emulator/drivers/unix/unix_efile.c
+++ b/erts/emulator/drivers/unix/unix_efile.c
@@ -638,12 +638,21 @@ efile_writev(Efile_error* errInfo,   /* Where to return error codes */
 		do {
 		    w = writev(fd, &iov[cnt], b);
 		} while (w < 0 && errno == EINTR);
+		if (w < 0 && errno == EINVAL) {
+		    goto single_write;
+		}
 	    } else
+	    single_write:
 		/* Degenerated io vector - use regular write */
 #endif
 		{
 		    do {
-			w = write(fd, iov[cnt].iov_base, iov[cnt].iov_len);
+			size_t iov_len = iov[cnt].iov_len;
+			size_t limit = 1024*1024*1024; /* 1GB */
+			if (iov_len > limit) {
+			    iov_len = limit;
+			}
+			w = write(fd, iov[cnt].iov_base, iov_len);
 		    } while (w < 0 && errno == EINTR);
 		    ASSERT(w <= iov[cnt].iov_len ||
 			   (w == -1 && errno != EINTR));
-- 
2.1.4

Places

File 1151-Handle-multi-giga-byte-writes-to-files.patch of Package erlang (Revision a19498298f166adf5d0de3a98d4a48a2)

Places