cspool117:schemas stain$ ./run_test.sh --junit-xml=result.xml RUNNER=cwltool
--- Running conformance test v1.0 on /Users/stain/cwl/bin/cwltool ---
/Users/stain/cwl/bin/cwltool 1.0.20170224141733
Test [18/86] Test failed: /Users/stain/cwl/bin/cwltool --tmp-outdir-prefix=/Users/stain/cwltool/cwltool/schemas/v1.0C6ENY9 --tmpdir-prefix=/Users/stain/cwltool/cwltool/schemas/v1.0C6ENY9 --outdir=/Users/stain/cwltool/cwltool/schemas/v1.0C6ENY9 --quiet v1.0/wc-tool.cwl v1.0/wc-job.json
Test command execution in with stdin and stdout redirection
Compare failure expected: {
    "output": {
        "checksum": "sha1$631bfbac524e2d04cdcc5ec33ade827fc10b06ae",
        "class": "File",
        "location": "output",
        "size": 15
    }
}
got: {
    "output": {
        "basename": "output",
        "checksum": "sha1$ae852d273ed56a04c9efd6aaa6f1376d6c03f312",
        "class": "File",
        "location": "file:///Users/stain/cwltool/cwltool/schemas/v1.0C6ENY9/output",
        "path": "/Users/stain/cwltool/cwltool/schemas/v1.0C6ENY9/output",
        "size": 25
    }
}
caused by: failed comparison for key 'output': expected: {
    "checksum": "sha1$631bfbac524e2d04cdcc5ec33ade827fc10b06ae",
    "class": "File",
    "location": "output",
    "size": 15
}
got: {
    "basename": "output",
    "checksum": "sha1$ae852d273ed56a04c9efd6aaa6f1376d6c03f312",
    "class": "File",
    "location": "file:///Users/stain/cwltool/cwltool/schemas/v1.0C6ENY9/output",
    "path": "/Users/stain/cwltool/cwltool/schemas/v1.0C6ENY9/output",
    "size": 25
}
caused by: field 'checksum' failed comparison: expected: "sha1$631bfbac524e2d04cdcc5ec33ade827fc10b06ae"
got: "sha1$ae852d273ed56a04c9efd6aaa6f1376d6c03f312"
Test [79/86] Test failed: /Users/stain/cwl/bin/cwltool --tmp-outdir-prefix=/Users/stain/cwltool/cwltool/schemas/v1.0FUMThK --tmpdir-prefix=/Users/stain/cwltool/cwltool/schemas/v1.0FUMThK --outdir=/Users/stain/cwltool/cwltool/schemas/v1.0FUMThK --quiet v1.0/envvar2.cwl v1.0/empty.json
Test $HOME and $TMPDIR are set correctly in Docker
Returned non-zero
HOME=/private/var/spool/cwl TMPDIR=/tmp
[job envvar2.cwl] completed permanentFail
Final process status is permanentFail

Test [86/86] 84 tests passed, 2 failures, 0 unsupported features

1 tool tests failed