API tools faq
paste
Guest User

Untitled

a guest
Dec 12th, 2018
117
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 17.24 KB | None | 0 0
raw download clone embed print report
  1. --- results.md 2018-12-09 19:12:39.636689970 -0500
  2. +++ hardened_results.md 2018-12-09 19:12:39.173350525 -0500
  3. @@ -26,6 +26,7 @@
  4. * CtsAppComponentFactoryTestCases - pass
  5. * CtsAppSecurityHostTestCases - pass (flaky)
  6. - very flaky: android.appsecurity.cts.EphemeralTest#testNormalStartEphemeral
  7. + - flaky: android.appsecurity.cts.ExternalStorageHostTest#testExternalStorageNone
  8. - flaky: android.appsecurity.cts.InstantCookieHostTest#testCookieUpdateAndRetrieval
  9. * CtsAppTestCases - pass
  10. - flaky: android.app.cts.ExpandableListActivityTest#testSelect
  11. @@ -42,16 +43,27 @@
  12. * CtsBackgroundRestrictionsTestCases - pass
  13. * CtsBackupHostTestCases - pass
  14. * CtsBackupTestCases - pass
  15. - - flaky: android.backup.cts.FullBackupQuotaTest#testQuotaExceeded
  16. * CtsBatterySavingTestCases - pass
  17. * CtsBionicTestCases - pass
  18. + - 2 intentional failures from returning EINVAL on invalid non power of 2 alignments:
  19. + - malloc#memalign_multiple
  20. + - malloc#memalign_non_power2
  21. + - 1 intentional failure from using safer standards compliant realloc(ptr, 0)
  22. + - malloc#realloc_0
  23. * CtsBluetoothTestCases - pass
  24. * CtsBootStatsTestCases - pass
  25. * CtsCalendarcommon2TestCases - pass
  26. * CtsCameraApi25TestCases - pass
  27. -* CtsCameraTestCases - pass
  28. - - overall quite flaky due to not having a great setup for lighting, etc.
  29. - - flaky: android.hardware.camera2.cts.StillCaptureTest#testAfRegions
  30. +* CtsCameraTestCases - seemingly random crashes in random tests [HARDENED_MALLOC]
  31. + - android.hardware.camera2.cts.NativeImageReaderTest#testImageReaderCloseAcquiredImages
  32. + - crash, likely use-after-free (null pointer dereference)
  33. + - android.hardware.camera2.cts.StillCaptureTest#testAfRegions
  34. + - didn't seem to fail without hardened malloc, need to verify that, no
  35. + apparent crash though just a time out on auto focus
  36. + - android.hardware.camera2.cts.CameraDeviceTest#testCameraDeviceRepeatingBurst
  37. + - android.hardware.camera2.cts.PerformanceTest#testMultipleCapture
  38. + - android.hardware.camera2.cts.ReprocessCaptureTest#testMixedBurstReprocessing
  39. + - etc. (not consistent)
  40. * CtsCarTestCases - pass
  41. * CtsCarrierApiTestCases - pass
  42. - 6 passed, 14 require SIM card with carrier privilege rule to run
  43. @@ -63,9 +75,93 @@
  44. * CtsCurrentApiSignatureTestCases - pass
  45. * CtsDatabaseTestCases - pass
  46. * CtsDebugTestCases - pass
  47. -* CtsDeqpTestCases - pass
  48. +* CtsDeqpTestCases - fail
  49. + - memory corruption bug triggered repeatedly on armv8a only, SEGV_ACCERR
  50. + 0x7200dfc074 (always ends in 0x074)
  51. +
  52. +12-08 22:42:37 I/ModuleListener: [29/3673] dEQP-EGL.functional.query_context.get_current_context#rgb565_window fail:
  53. +12-08 22:42:50 I/ModuleListener: [32/3673] dEQP-EGL.functional.query_context.get_current_context#rgb888_window fail:
  54. +12-08 22:43:03 I/ModuleListener: [35/3673] dEQP-EGL.functional.query_context.get_current_context#rgba4444_window fail:
  55. +12-08 22:43:16 I/ModuleListener: [38/3673] dEQP-EGL.functional.query_context.get_current_context#rgba5551_window fail:
  56. +12-08 22:43:29 I/ModuleListener: [41/3673] dEQP-EGL.functional.query_context.get_current_context#rgba8888_window fail:
  57. +12-08 22:43:43 I/ModuleListener: [45/3673] dEQP-EGL.functional.query_context.get_current_surface#rgb565_window fail:
  58. +12-08 22:43:56 I/ModuleListener: [48/3673] dEQP-EGL.functional.query_context.get_current_surface#rgb888_window fail:
  59. +12-08 22:44:09 I/ModuleListener: [51/3673] dEQP-EGL.functional.query_context.get_current_surface#rgba4444_window fail:
  60. +12-08 22:44:22 I/ModuleListener: [54/3673] dEQP-EGL.functional.query_context.get_current_surface#rgba5551_window fail:
  61. +12-08 22:44:42 I/ModuleListener: [57/3673] dEQP-EGL.functional.query_context.get_current_surface#rgba8888_window fail:
  62. +12-08 22:44:55 I/ModuleListener: [61/3673] dEQP-EGL.functional.query_context.get_current_display#rgb565_window fail:
  63. +12-08 22:45:08 I/ModuleListener: [64/3673] dEQP-EGL.functional.query_context.get_current_display#rgb888_window fail:
  64. +12-08 22:45:27 I/ModuleListener: [67/3673] dEQP-EGL.functional.query_context.get_current_display#rgba4444_window fail:
  65. +12-08 22:45:41 I/ModuleListener: [70/3673] dEQP-EGL.functional.query_context.get_current_display#rgba5551_window fail:
  66. +12-08 22:45:54 I/ModuleListener: [73/3673] dEQP-EGL.functional.query_context.get_current_display#rgba8888_window fail:
  67. +12-08 22:46:07 I/ModuleListener: [77/3673] dEQP-EGL.functional.query_context.query_context#rgb565_window fail:
  68. +12-08 22:46:20 I/ModuleListener: [80/3673] dEQP-EGL.functional.query_context.query_context#rgb888_window fail:
  69. +12-08 22:46:40 I/ModuleListener: [83/3673] dEQP-EGL.functional.query_context.query_context#rgba4444_window fail:
  70. +12-08 22:46:53 I/ModuleListener: [86/3673] dEQP-EGL.functional.query_context.query_context#rgba5551_window fail:
  71. +12-08 22:47:12 I/ModuleListener: [89/3673] dEQP-EGL.functional.query_context.query_context#rgba8888_window fail:
  72. +
  73. +12-08 22:48:54 I/ModuleListener: [995/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_texture_rgb8_texture fail:
  74. +12-08 22:49:07 I/ModuleListener: [997/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_texture_rgb565_texture fail:
  75. +12-08 22:49:20 I/ModuleListener: [999/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_texture_rgba8_texture fail:
  76. +12-08 22:49:40 I/ModuleListener: [1001/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_texture_rgb5_a1_texture fail:
  77. +12-08 22:49:53 I/ModuleListener: [1003/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_texture_rgba4_texture fail:
  78. +12-08 22:50:07 I/ModuleListener: [1005/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_positive_x_rgba8_texture fail:
  79. +12-08 22:50:20 I/ModuleListener: [1007/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_positive_y_rgba8_texture fail:
  80. +12-08 22:50:33 I/ModuleListener: [1009/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_positive_z_rgba8_texture fail:
  81. +12-08 22:50:46 I/ModuleListener: [1011/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_negative_x_rgba8_texture fail:
  82. +12-08 22:51:00 I/ModuleListener: [1013/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_negative_y_rgba8_texture fail:
  83. +12-08 22:51:13 I/ModuleListener: [1015/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_negative_z_rgba8_texture fail:
  84. +12-08 22:51:26 I/ModuleListener: [1017/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_positive_x_rgb8_texture fail:
  85. +12-08 22:51:39 I/ModuleListener: [1019/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_positive_y_rgb8_texture fail:
  86. +12-08 22:51:58 I/ModuleListener: [1021/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_positive_z_rgb8_texture fail:
  87. +12-08 22:52:12 I/ModuleListener: [1023/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_negative_x_rgb8_texture fail:
  88. +12-08 22:52:25 I/ModuleListener: [1025/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_negative_y_rgb8_texture fail:
  89. +12-08 22:52:38 I/ModuleListener: [1027/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_cubemap_negative_z_rgb8_texture fail:
  90. +12-08 22:52:58 I/ModuleListener: [1029/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_renderbuffer_rgba4_texture fail:
  91. +12-08 22:53:17 I/ModuleListener: [1031/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_renderbuffer_rgb5_a1_texture fail:
  92. +12-08 22:53:30 I/ModuleListener: [1033/3673] dEQP-EGL.functional.image.render_multiple_contexts#gles2_renderbuffer_rgb565_texture fail:
  93. +
  94. +12-08 22:45:14.559 27696 27696 F DEBUG : #00 pc 0000000000236358 /vendor/lib64/egl/libGLESv2_adreno.so (EsxGfxMem::FreeAllocationOnTimestampCommon(EsxContext*, EsxMemAllocDescInstance)+552)
  95. +12-08 22:45:14.559 27696 27696 F DEBUG : #01 pc 0000000000236534 /vendor/lib64/egl/libGLESv2_adreno.so (EsxGfxMem::Destroy(EsxContext*)+64)
  96. +12-08 22:45:14.559 27696 27696 F DEBUG : #02 pc 0000000000335394 /vendor/lib64/egl/libGLESv2_adreno.so (A5xResource::HwFreeSubResource(EsxContext*, EsxSubResource*) const+56)
  97. +12-08 22:45:14.559 27696 27696 F DEBUG : #03 pc 00000000001ae0dc /vendor/lib64/egl/libGLESv2_adreno.so (EsxResource::FreeSubResource(EsxContext*, EsxSubResource*)+52)
  98. +12-08 22:45:14.559 27696 27696 F DEBUG : #04 pc 00000000001adf70 /vendor/lib64/egl/libGLESv2_adreno.so (EsxResource::Destroy(EsxContext*)+100)
  99. +12-08 22:45:14.560 27696 27696 F DEBUG : #05 pc 0000000000202cf4 /vendor/lib64/egl/libGLESv2_adreno.so (EglSurface::Destroy()+72)
  100. +12-08 22:45:14.560 27696 27696 F DEBUG : #06 pc 00000000001eb580 /vendor/lib64/egl/libGLESv2_adreno.so (EglApi::DestroySurface(void*, void*)+176)
  101. +12-08 22:45:14.560 27696 27696 F DEBUG : #07 pc 0000000000015f24 /system/lib64/libEGL.so (eglDestroySurface+100)
  102. +12-08 22:45:14.560 27696 27696 F DEBUG : #08 pc 0000000001178608 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (eglu::UniqueSurface::~UniqueSurface()+32)
  103. +12-08 22:45:14.560 27696 27696 F DEBUG : #09 pc 000000000138a340 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (deqp::egl::RenderCase::executeForConfig(void*, void*)+1004)
  104. +12-08 22:45:14.560 27696 27696 F DEBUG : #10 pc 00000000013af5c4 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (deqp::egl::SimpleConfigCase::iterate()+44)
  105. +12-08 22:45:14.560 27696 27696 F DEBUG : #11 pc 0000000001124ef8 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (tcu::TestSessionExecutor::iterateTestCase(tcu::TestCase*)+52)
  106. +12-08 22:45:14.560 27696 27696 F DEBUG : #12 pc 0000000001124744 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (tcu::TestSessionExecutor::iterate()+332)
  107. +12-08 22:45:14.560 27696 27696 F DEBUG : #13 pc 00000000010e6bd4 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (tcu::App::iterate()+60)
  108. +12-08 22:45:14.560 27696 27696 F DEBUG : #14 pc 00000000012a20f4 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (tcu::Android::TestThread::render()+36)
  109. +12-08 22:45:14.560 27696 27696 F DEBUG : #15 pc 00000000012a195c /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (tcu::Android::RenderThread::run()+256)
  110. +12-08 22:45:14.560 27696 27696 F DEBUG : #16 pc 0000000001fe99bc /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (startThread+12)
  111. +12-08 22:45:14.560 27696 27696 F DEBUG : #17 pc 00000000000842c4 /system/lib64/libc.so (__pthread_start(void*)+36)
  112. +12-08 22:45:14.560 27696 27696 F DEBUG : #18 pc 000000000002473c /system/lib64/libc.so (__start_thread+68)
  113. +
  114. +12-08 22:53:24.145 2922 2922 F DEBUG : #00 pc 0000000000236358 /vendor/lib64/egl/libGLESv2_adreno.so (EsxGfxMem::FreeAllocationOnTimestampCommon(EsxContext*, EsxMemAllocDescInstance)+552)
  115. +12-08 22:53:24.145 2922 2922 F DEBUG : #01 pc 0000000000236534 /vendor/lib64/egl/libGLESv2_adreno.so (EsxGfxMem::Destroy(EsxContext*)+64)
  116. +12-08 22:53:24.145 2922 2922 F DEBUG : #02 pc 0000000000335394 /vendor/lib64/egl/libGLESv2_adreno.so (A5xResource::HwFreeSubResource(EsxContext*, EsxSubResource*) const+56)
  117. +12-08 22:53:24.145 2922 2922 F DEBUG : #03 pc 00000000001ae0dc /vendor/lib64/egl/libGLESv2_adreno.so (EsxResource::FreeSubResource(EsxContext*, EsxSubResource*)+52)
  118. +12-08 22:53:24.145 2922 2922 F DEBUG : #04 pc 00000000001adf70 /vendor/lib64/egl/libGLESv2_adreno.so (EsxResource::Destroy(EsxContext*)+100)
  119. +12-08 22:53:24.145 2922 2922 F DEBUG : #05 pc 0000000000202cf4 /vendor/lib64/egl/libGLESv2_adreno.so (EglSurface::Destroy()+72)
  120. +12-08 22:53:24.145 2922 2922 F DEBUG : #06 pc 00000000001eb580 /vendor/lib64/egl/libGLESv2_adreno.so (EglApi::DestroySurface(void*, void*)+176)
  121. +12-08 22:53:24.145 2922 2922 F DEBUG : #07 pc 0000000000015f24 /system/lib64/libEGL.so (eglDestroySurface+100)
  122. +12-08 22:53:24.145 2922 2922 F DEBUG : #08 pc 000000000134d358 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (deqp::egl::Image::ImageFormatCase::deinit()+144)
  123. +12-08 22:53:24.145 2922 2922 F DEBUG : #09 pc 0000000001124a14 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (tcu::TestSessionExecutor::leaveTestCase(tcu::TestCase*)+60)
  124. +12-08 22:53:24.145 2922 2922 F DEBUG : #10 pc 0000000001124690 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (tcu::TestSessionExecutor::iterate()+152)
  125. +12-08 22:53:24.145 2922 2922 F DEBUG : #11 pc 00000000010e6bd4 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (tcu::App::iterate()+60)
  126. +12-08 22:53:24.145 2922 2922 F DEBUG : #12 pc 00000000012a20f4 /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (tcu::Android::TestThread::render()+36)
  127. +12-08 22:53:24.145 2922 2922 F DEBUG : #13 pc 00000000012a195c /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (tcu::Android::RenderThread::run()+256)
  128. +12-08 22:53:24.145 2922 2922 F DEBUG : #14 pc 0000000001fe99bc /data/app/com.drawelements.deqp-LuaHQDu6JmFONVIcUNsJww==/lib/arm64/libdeqp.so (startThread+12)
  129. +12-08 22:53:24.145 2922 2922 F DEBUG : #15 pc 00000000000842c4 /system/lib64/libc.so (__pthread_start(void*)+36)
  130. +12-08 22:53:24.145 2922 2922 F DEBUG : #16 pc 000000000002473c /system/lib64/libc.so (__start_thread+68)
  131. +
  132. * CtsDeviceIdleHostTestCases - pass
  133. -* CtsDevicePolicyManagerTestCases - pass
  134. +* CtsDevicePolicyManagerTestCases - fail
  135. + - random failures with 'Failed to parse result: cmd: Can't find service: package'
  136. * CtsDexMetadataHostTestCases - pass
  137. * CtsDisplayTestCases - pass
  138. * CtsDpiTestCases - pass
  139. @@ -100,8 +196,11 @@
  140. * CtsHostsideWebViewTests - 1 failure
  141. - failure: com.android.cts.webkit.WebViewHostSideStartupTest#testStrictMode
  142. * CtsIcuTestCases - pass
  143. -* CtsIncidentHostTestCases - pass
  144. - - flaky: com.android.server.cts.IncidentdTest#testIncidentReportDumpAuto
  145. +* CtsIncidentHostTestCases - 4 failures
  146. + - failure: com.android.server.cts.ErrorsTest#testNativeCrashA
  147. + - failure: com.android.server.cts.IncidentdTest#testIncidentReportDumpAuto
  148. + - failure: com.android.server.cts.IncidentdTest#testIncidentReportDumpExplicit
  149. + - failure: com.android.server.cts.IncidentdTest#testIncidentReportDumpLocal
  150. * CtsInlineMockingTestCases - pass
  151. * CtsInputMethodServiceHostTestCases - pass
  152. * CtsInputMethodTestCases - pass
  153. @@ -217,18 +316,22 @@
  154. * CtsLocation2TestCases - pass
  155. * CtsLocationTestCases - pass
  156. - flaky indoors
  157. + - need to look into this more with hardened malloc due to existing
  158. + flakiness making it hard to identify new issues
  159. * CtsLogdTestCases - pass
  160. * CtsMediaBitstreamsTestCases - pass
  161. * CtsMediaHostTestCases - pass
  162. * CtsMediaStressTestCases - pass
  163. -* CtsMediaTestCases - 1 failure + flaky tests
  164. +* CtsMediaTestCases - 2 failures
  165. + - 1 failure (crash): android.media.cts.DecoderConformanceTest#testVP9Other
  166. - the flaky network tests seem to be flaky themselves in general, so others may also occasionally fail
  167. - flaky: android.media.cts.MediaPlayerFlakyNetworkTest#test_S3P00001
  168. - flaky: android.media.cts.MediaPlayerFlakyNetworkTest#test_S4P00001
  169. - flaky: android.media.cts.MediaPlayerFlakyNetworkTest#test_S5P00001
  170. - flaky: android.media.cts.VideoDecoderPerfTest#testHevcGoog0Perf0640x0360 (maybe others too)
  171. - flaky: android.media.cts.StreamingMediaPlayerTest#testPlayHlsStream (maybe others too)
  172. - - failure: android.media.cts.MediaSessionManagerTest#testRemoteUserInfo
  173. + - framerate too high? android.media.cts.VideoDecoderPerfTest#testAvcGoog0Perf0720x0480
  174. + - failure (timeout): android.media.cts.VideoEncoderTest#testGoogVP9SurfArbitraryH
  175. * CtsMidiTestCases - pass
  176. * CtsMockingDebuggableTestCases - pass
  177. * CtsMockingTestCases - pass
  178. @@ -274,7 +377,9 @@
  179. * CtsPrintTestCases - pass
  180. - flaky
  181. * CtsProtoTestCases - pass
  182. -* CtsProviderTestCases - pass
  183. +* CtsProviderTestCases - 2 failures
  184. + - android.provider.cts.contacts.ContactsContract_DumpFileProviderTest#testQuery_throwsErrorWithIllegalFileName
  185. + - android.provider.cts.contacts.ContactsContract_DumpFileProviderTest#testOpenFileDescriptor_throwsErrorWithIllegalFileName
  186. * CtsRenderscriptLegacyTestCases - pass
  187. * CtsRenderscriptTestCases - pass
  188. * CtsRsBlasTestCases - pass
  189. @@ -300,7 +405,8 @@
  190. * CtsSkQPTestCases - pass
  191. * CtsSliceTestCases - pass
  192. * CtsSpeechTestCases - pass
  193. -* CtsStatsdHostTestCases - pass
  194. +* CtsStatsdHostTestCases - 1 failure
  195. + - failure: android.cts.statsd.atom.UidAtomTests#testCpuTimePerUid
  196. * CtsSustainedPerformanceHostTestCases - pass
  197. * CtsSyncAccountAccessOtherCertTestCases - pass
  198. * CtsSyncContentHostTestCases - pass
  199. @@ -348,10 +454,10 @@
  200. * CtsVoiceSettingsTestCases - pass
  201. * CtsVrTestCases - pass
  202. * CtsWebkitTestCases - pass
  203. - - flaky: android.webkit.cts.WebChromeClientTest#testOnJsBeforeUnloadIsCalled
  204. * CtsWidgetTestCases - pass
  205. * CtsWindowManagerDeviceTestCases - pass
  206. - flaky: android.server.wm.DialogFrameTests#testExplicitSizeBottomRightGravity
  207. + - flaky: android.server.wm.DialogFrameTests#testExplicitSizeDefaultGravity
  208. * CtsWrapNoWrapTestCases - pass
  209. * CtsWrapWrapDebugMallocDebugTestCases - pass
  210. * CtsWrapWrapDebugTestCases - pass
Add Comment
Please, Sign In to add comment
Public Pastes
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!