auto_open.email

Envelope-to: itz@ahiker.mooo.com
Delivery-date: Fri, 26 Sep 2014 06:39:08 -0700
Received-SPF: pass (google.com: domain of w.rouesnel@gmail.com designates 2607:f8b0:400e:c03::230 as permitted sender) client-ip=2607:f8b0:400e:c03::230;
From: Will Rouesnel <w.rouesnel@gmail.com>
To: bup-list@googlegroups.com
Cc: Will Rouesnel <w.rouesnel@gmail.com>
Subject: [RFC] Parallalize running test script components.
Date: Fri, 26 Sep 2014 22:54:00 +1000
Message-Id: <1411736040-6885-1-git-send-email-w.rouesnel@gmail.com>
MIME-Version: 1.0
Precedence: list
Mailing-list: list bup-list@googlegroups.com; contact bup-list+owners@googlegroups.com
Sender: bup-list@googlegroups.com
Content-Type: multipart/mixed; boundary="------------1.9.1"

This is a multi-part message in MIME format.
--------------1.9.1
Content-Type: text/plain; charset=UTF-8; format=fixed
Content-Transfer-Encoding: 8bit


This patch does two things: firstly it turns the regular shell script
test executor into a wildcard match, to make adding new shell script tests
easier. It matches on t/t*.sh, same as the Python test runner.

Secondly: it turns the script runner into a set of regular make dependencies,
singular to each test file. This enables running make -jN to parallize the test
runner.

This seems to yield impressive improvements to test times: 166 sec -> 44 sec
with make -j16 on an i7.

Since our tests are king, this is a big win. The new behavior can be trivially
switched off by simply running regular make. There may be some race conditions
in test scripts.
---
 Makefile | 39 +++++++++++++++------------------------
 1 file changed, 15 insertions(+), 24 deletions(-)


--------------1.9.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Disposition: inline

--
You received this message because you are subscribed to the Google Groups "bup-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bup-list+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--------------1.9.1
Content-Type: text/x-patch; name="0001-Parallalize-running-test-script-components.patch"
Content-Transfer-Encoding: 8bit
Content-Disposition: inline; filename="0001-Parallalize-running-test-script-components.patch"

diff --git a/Makefile b/Makefile
index 97f9338..4fbb891 100644
--- a/Makefile
+++ b/Makefile
@@ -12,6 +12,9 @@ else
   test_tmp := $(CURDIR)/t/tmp
 endif

+TEST_SH := $(wildcard t/t*.sh)
+TEST_PY := $(wildcard t/t*.py) $(wildcard lib/*/t/t*.py)
+
 default: all

 all: bup Documentation/all
@@ -82,33 +85,21 @@ lib/bup/_version.py:

 runtests: all runtests-python runtests-cmdline

-runtests-python: all
+.PHONY: ${TEST_PY}
+${TEST_PY}: all
 	test -e t/tmp || mkdir t/tmp
-	TMPDIR="$(test_tmp)" $(PYTHON) wvtest.py t/t*.py lib/*/t/t*.py
+	TMPDIR="$(test_tmp)" $(PYTHON) wvtest.py $@
+
+.PHONY: runtests-python
+runtests-python: ${TEST_PY}

-runtests-cmdline: all
+.PHONY: ${TEST_SH}
+${TEST_SH}: all
 	test -e t/tmp || mkdir t/tmp
-	TMPDIR="$(test_tmp)" t/test-fuse.sh
-	TMPDIR="$(test_tmp)" t/test-drecurse.sh
-	TMPDIR="$(test_tmp)" t/test-cat-file.sh
-	TMPDIR="$(test_tmp)" t/test-compression.sh
-	TMPDIR="$(test_tmp)" t/test-fsck.sh
-	TMPDIR="$(test_tmp)" t/test-index-clear.sh
-	TMPDIR="$(test_tmp)" t/test-index-check-device.sh
-	TMPDIR="$(test_tmp)" t/test-ls.sh
-	TMPDIR="$(test_tmp)" t/test-meta.sh
-	TMPDIR="$(test_tmp)" t/test-on.sh
-	TMPDIR="$(test_tmp)" t/test-restore-map-owner.sh
-	TMPDIR="$(test_tmp)" t/test-restore-single-file.sh
-	TMPDIR="$(test_tmp)" t/test-rm-between-index-and-save.sh
-	TMPDIR="$(test_tmp)" t/test-command-without-init-fails.sh
-	TMPDIR="$(test_tmp)" t/test-redundant-saves.sh
-	TMPDIR="$(test_tmp)" t/test-save-creates-no-unrefs.sh
-	TMPDIR="$(test_tmp)" t/test-save-restore-excludes.sh
-	TMPDIR="$(test_tmp)" t/test-save-strip-graft.sh
-	TMPDIR="$(test_tmp)" t/test-import-rdiff-backup.sh
-	TMPDIR="$(test_tmp)" t/test-xdev.sh
-	TMPDIR="$(test_tmp)" t/test.sh
+	TMPDIR="$(test_tmp)" $@
+
+.PHONY: runtests-cmdline
+runtests-cmdline: ${TEST_SH}

 stupid:
 	PATH=/bin:/usr/bin $(MAKE) test

--------------1.9.1--