API tools faq
paste
Advertisement
wellthatsucks

Untitled

wellthatsucks
Sep 24th, 2018
106
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 12.98 KB | None | 0 0
raw download clone embed print report
  1. Matthew Waters
  2. 4:48 PM
  3.  
  4. what even does crossfade between three inputs even mean?
  5. meh profile image
  6. Mathieu Duponchelle
  7. 4:48 PM
  8.  
  9. ?
  10. meh profile image
  11. Mathieu Duponchelle
  12. 4:49 PM
  13.  
  14. You crossfade the result of the crossfade
  15. slomo profile image
  16. Sebastian Dröge
  17. 4:49 PM
  18.  
  19. i'm just replying something to that
  20. bilboed profile image
  21. Edward Hervey
  22. 4:50 PM
  23. Commented on Mathieu Duponchelle's message: You crossfade the result of the crossfade
  24.  
  25. damnit, there's no :xzibit_yo_dawg emoticon
  26. (edited)
  27. slomo profile image
  28. Sebastian Dröge
  29. 4:51 PM
  30. Commented on Matthew Waters' message: what even does crossfade between three inputs even mean?
  31.  
  32. something for which i would use two times the same element
  33. meh profile image
  34. Mathieu Duponchelle
  35. 4:51 PM
  36.  
  37. eg: https://cave.centricular.com/paste/ptne84x7x
  38. meh profile image
  39. Mathieu Duponchelle
  40. 4:52 PM
  41. Commented on Matthew Waters' message: what even does crossfade between three inputs even mean?
  42.  
  43. note that this part is not mandatory for pitivi, but it would eventually allow us to use a single mixer for the whole timeline
  44. ystreet00 profile image
  45. Matthew Waters
  46. 4:53 PM
  47.  
  48. crossfade and single mixer for the whole timeline are not workable with the painters algorithm
  49. slomo profile image
  50. Sebastian Dröge
  51. 4:53 PM
  52.  
  53. a single mixer for the whole pipeline is also not so good performance-wise
  54. meh profile image
  55. Mathieu Duponchelle
  56. 4:53 PM
  57.  
  58. making things easier for GES, and also reducing the likelihood of race conditions sneaking in
  59. slomo profile image
  60. Sebastian Dröge
  61. 4:54 PM
  62.  
  63. it does not parallelize
  64. meh profile image
  65. Mathieu Duponchelle
  66. 4:55 PM
  67.  
  68. that's a valid concern, though we could always parallelize if we found that to be a problem
  69. slomo profile image
  70. Sebastian Dröge
  71. 4:55 PM
  72.  
  73. or parallelize the code inside compositor, yes
  74. meh profile image
  75. Mathieu Duponchelle
  76. 4:56 PM
  77.  
  78. Anyway, that's not a mandatory feature for us
  79. 4:56 PM
  80.  
  81. What's mandatory is the positioning and mixing of differently-sized input streams
  82. 4:56 PM
  83.  
  84. things that compositor does for us
  85. slomo profile image
  86. Sebastian Dröge
  87. 4:56 PM
  88.  
  89. that's what compositor did before crossfading was added
  90. meh profile image
  91. Mathieu Duponchelle
  92. 4:57 PM
  93.  
  94. yes of course, but we need both features in combination
  95. tim profile image
  96. Tim Müller
  97. 4:57 PM
  98.  
  99. should prolly move this to #gstreamer at some point (my bad for starting it)
  100. meh profile image
  101. Mathieu Duponchelle
  102. 4:57 PM
  103.  
  104. an element that would crossfade two perfectly overlapping input streams is useless for us
  105. slomo profile image
  106. Sebastian Dröge
  107. 4:59 PM
  108.  
  109. i see, and you don't want a cascade of compositors (which you can use for crossfading just fine without the crossfade property) because then you have to create some kind of "schedule" from the timeline and then build the pipeline topology based on that
  110. 4:59 PM
  111.  
  112. while now you just put everything to a single compositor and done
  113. meh profile image
  114. Mathieu Duponchelle
  115. 4:59 PM
  116.  
  117. no, we currently cascade
  118. 4:59 PM
  119.  
  120. one compositor per transition
  121. 4:59 PM
  122.  
  123. and one coompositor to mix the different layers
  124. 5:00 PM
  125.  
  126. that's why the "more than 2-pad crossfade" feature is not useful for us right now, but we were quite happy that we'd be able to use it if we wanted in the future
  127. slomo profile image
  128. Sebastian Dröge
  129. 5:00 PM
  130.  
  131. i see
  132. 5:01 PM
  133.  
  134. but at that point it probably degenerates sooner or later into a pitivi-compositor, not a generic compositor
  135. meh profile image
  136. Mathieu Duponchelle
  137. 5:03 PM
  138.  
  139. there's nothing pitivi-specific to that feature
  140. slomo profile image
  141. Sebastian Dröge
  142. 5:03 PM
  143.  
  144. the way how crossfade is currently implemented in compositor
  145. 5:04 PM
  146.  
  147. and later you'd probably also want to support other kinds of transitions in pitivi and then we're back at this point
  148. 5:04 PM
  149.  
  150. i think the only way how to do this generically is a pitivi-compositor that does just that. or a lot of smaller elements that you can combine to get that effect
  151. meh profile image
  152. Mathieu Duponchelle
  153. 5:05 PM
  154.  
  155. fwiw I would be completely happy with a compositor subclass, it's just the copy paste part I'm not enthusiastic about
  156. slomo profile image
  157. Sebastian Dröge
  158. 5:05 PM
  159.  
  160. me neither but i prefer it over stabilizing the current API
  161. meh profile image
  162. Mathieu Duponchelle
  163. 5:11 PM
  164.  
  165. I suppose the real question is, would new blend modes in compositor allow us to get the expected behaviour?
  166. 5:12 PM
  167.  
  168. If so, then I don't care and I don't think thibault does either about the exact API that gets used
  169. New Messages
  170. slomo profile image
  171. Sebastian Dröge
  172. 5:12 PM
  173.  
  174. not with a single compositor unless you move away from the painter's algorithm
  175. meh profile image
  176. Mathieu Duponchelle
  177. 5:13 PM
  178.  
  179. hrm, even in our current usage scenario where we use one compositor per transition, with two and only two pads ?
  180. slomo profile image
  181. Sebastian Dröge
  182. 5:14 PM
  183.  
  184. no because you want to mix the background after the two inputs
  185. 5:15 PM
  186.  
  187. unrelated, i also just looked at the code in compositor, it breaks zorder if there are multiple pads with crossfading and in between pads without
  188. 5:16 PM
  189.  
  190. ah no, nevermind
  191. meh profile image
  192. Mathieu Duponchelle
  193. 5:16 PM
  194.  
  195. couldn't we solve this by instead using the "reverse painter's algorithm"?
  196. nirbheek profile image
  197. Nirbheek Chauhan
  198. 5:17 PM
  199.  
  200. That's already how composition works, right?
  201. 5:17 PM
  202.  
  203. Except when you have alpha
  204. slomo profile image
  205. Sebastian Dröge
  206. 5:17 PM
  207.  
  208. no, it's bottom to top
  209. meh profile image
  210. Mathieu Duponchelle
  211. 5:18 PM
  212.  
  213. I'd need to see the code I suppose
  214. 5:18 PM
  215.  
  216. also wouldn't the reverse algorithm make the whole "is_that_stream_covered" code unnecessary ?
  217. slomo profile image
  218. Sebastian Dröge
  219. 5:19 PM
  220.  
  221. painters algorithm is
  222. ((A op B) op C) op D) etc
  223.  
  224. what you need here is (for crossfading B and C
  225.  
  226. (A op (B op C)) op D
  227. 5:19 PM
  228.  
  229. A being the background
  230. nirbheek profile image
  231. Nirbheek Chauhan
  232. 5:20 PM
  233.  
  234. The reverse algo also requires you to implement an occlusion culling algorithm, doesn't it?
  235. slomo profile image
  236. Sebastian Dröge
  237. 5:20 PM
  238.  
  239. yes
  240. nirbheek profile image
  241. Nirbheek Chauhan
  242. 5:20 PM
  243.  
  244. So it increases complexity, unless you're using a GPU which does that for you
  245. slomo profile image
  246. Sebastian Dröge
  247. 5:20 PM
  248.  
  249. and i'm not sure how that would work with SIMD for the blending
  250. meh profile image
  251. Mathieu Duponchelle
  252. 5:21 PM
  253.  
  254. hrm, correct me if I'm wrong, but wouldn't occlusion culling in our case basically be "is this pixel opaque" ?
  255. nirbheek profile image
  256. Nirbheek Chauhan
  257. 5:21 PM
  258.  
  259. I have never implemented a culling algo
  260. 5:22 PM
  261.  
  262. Except around the edges I guess, where you want to over-render
  263. slomo profile image
  264. Sebastian Dröge
  265. 5:22 PM
  266.  
  267. it also wouldn't solve the problem, the result would be the same or your algorithm has a bug
  268. 5:26 PM
  269.  
  270. i wonder if one could get your behaviour by adding some kind of "first with following pad"-boolean flag to each pad. which would then first do whatever operation there is with the following pad, and only then do the previous operation with the background
  271.  
  272. but at that point we maybe just want to implement a way to define arbitrary parenthesis
  273. meh profile image
  274. Mathieu Duponchelle
  275. 5:26 PM
  276.  
  277. back to the point then
  278. 5:26 PM
  279.  
  280. yes I was about to ask that
  281. 5:26 PM
  282.  
  283. instead of saying "crossfade"
  284. 5:27 PM
  285.  
  286. we would be able to specify a relational operation with the next pad
  287. slomo profile image
  288. Sebastian Dröge
  289. 5:28 PM
  290.  
  291. that's then again taking the pitivi use-case as the goal. why only the following pad?
  292.  
  293. maybe you want to crossfade pad D and E, and then crossfade the result on pad C?
  294. meh profile image
  295. Mathieu Duponchelle
  296. 5:28 PM
  297.  
  298. we need to quantify that operation however
  299. slomo profile image
  300. Sebastian Dröge
  301. 5:28 PM
  302.  
  303. how that operation would look like i wrote in the bug this morning
  304. meh profile image
  305. Mathieu Duponchelle
  306. 5:29 PM
  307.  
  308. we had that idea in our initial discussions with thibault fwiw (arbitrary relations)
  309. 5:29 PM
  310.  
  311. but iirc we couldn't find a solution to do that without introducing a new object type
  312. ystreet00 profile image
  313. Matthew Waters
  314. 5:29 PM
  315.  
  316. order is signified by zorder already
  317. slomo profile image
  318. Sebastian Dröge
  319. 5:29 PM
  320.  
  321. yes the API is the problem, not the processing itself
  322. 5:30 PM
  323.  
  324. @Matthew Waters i meant
  325.  
  326. (((A op B) op (C op (D op E))) op F) etc
  327. 5:31 PM
  328.  
  329. or you mean that in that case things could be moved according to the zorder?
  330. ystreet00 profile image
  331. Matthew Waters
  332. 5:31 PM
  333.  
  334. right, doing that in one element is tricky
  335. meh profile image
  336. Mathieu Duponchelle
  337. 5:32 PM
  338.  
  339. I'm sure we're smart enough to figure out an implementation
  340. ystreet00 profile image
  341. Matthew Waters
  342. 5:32 PM
  343.  
  344. and I think requires an intermediate buffer anyway
  345. meh profile image
  346. Mathieu Duponchelle
  347. 5:32 PM
  348.  
  349. It's just what the API should look like
  350. slomo profile image
  351. Sebastian Dröge
  352. 5:32 PM
  353.  
  354. it alraedy uses intermediate buffers in the current crossfading code
  355. 5:32 PM
  356.  
  357. because it can't work otherwise
  358. ystreet00 profile image
  359. Matthew Waters
  360. 5:33 PM
  361.  
  362. right, so then multiple mixers and be done with it
  363. slomo profile image
  364. Sebastian Dröge
  365. 5:33 PM
  366.  
  367. which is also why i say that using multiple compositors wouldn't be bad, you do the same processing after all
  368. 5:33 PM
  369.  
  370. but @Mathieu Duponchelle and thibault disagree
  371. (edited)
  372. meh profile image
  373. Mathieu Duponchelle
  374. 5:33 PM
  375.  
  376. I'm afraid thibault is not in this channel
  377. nirbheek profile image
  378. Nirbheek Chauhan
  379. 5:33 PM
  380.  
  381. Just so I understand correctly, the problem is new crossfade ops that are not commutative?
  382. 5:34 PM
  383.  
  384. Err, I meant the other thing, not commutative
  385. slomo profile image
  386. Sebastian Dröge
  387. 5:34 PM
  388.  
  389. associative
  390. nirbheek profile image
  391. Nirbheek Chauhan
  392. 5:34 PM
  393.  
  394. Right that one
  395. slomo profile image
  396. Sebastian Dröge
  397. 5:35 PM
  398.  
  399. the ones we have right now are not associative already
  400. nirbheek profile image
  401. Nirbheek Chauhan
  402. 5:35 PM
  403.  
  404. Oh
  405. meh profile image
  406. Mathieu Duponchelle
  407. 5:35 PM
  408.  
  409. @Sebastian Dröge , note that I don't disagree with anything, just looking for a solution that does not involve expropriating pitivi
  410. slomo profile image
  411. Sebastian Dröge
  412. 5:36 PM
  413.  
  414. using multiple compositors would always solve the pitivi problem
  415. meh profile image
  416. Mathieu Duponchelle
  417. 5:36 PM
  418.  
  419. didn't you say blending the background would be a problem ?
  420. ystreet00 profile image
  421. Matthew Waters
  422. 5:37 PM
  423.  
  424. you can specially craft the background (e.g. black) so that it doesn't matter
  425. 5:37 PM
  426.  
  427. which is how we can do 'crossfading' with glvideomixer currently
  428. meh profile image
  429. Mathieu Duponchelle
  430. 5:38 PM
  431.  
  432. Then I don't understand
  433. slomo profile image
  434. Sebastian Dröge
  435. 5:39 PM
  436.  
  437. @Mathieu Duponchelle you would just take another background you have two backgrounds with two compositors. you use the first compositor to crossfade, and the second to blend over the background
  438. 5:39 PM
  439.  
  440. the first would have a fully transparent background i guess
  441. 5:40 PM
  442.  
  443. which is basically what you implement as a special case inside compositor right now. there's a second "background" being created for an intermediate surface for the crossfading
  444. meh profile image
  445. Mathieu Duponchelle
  446. 5:41 PM
  447.  
  448. OK, then IMO the plan forward should be:
  449. 5:41 PM
  450.  
  451. Implement blend modes in compositor
  452.  
  453. 5:42 PM
  454.  
  455. let us try and use that in pitivi
  456.  
  457. 5:42 PM
  458.  
  459. it works as expected -> remove crossfade
  460.  
  461. 5:42 PM
  462.  
  463. don't copy anything
  464.  
  465. slomo profile image
  466. Sebastian Dröge
  467. 5:42 PM
  468.  
  469. and e.g. in my previous case of (((A op B) op (C op (D op E))) op F)
  470.  
  471. you would have one compositor doing (D op E), connected to one compositor doing (C op (D op E)), to one compositor doing the remainder
  472. 5:43 PM
  473.  
  474. that's all more obvious if you create a tree out of those expressions
  475. meh profile image
  476. Mathieu Duponchelle
  477. 5:43 PM
  478.  
  479. it's obvious enough don't worry
  480. slomo profile image
  481. Sebastian Dröge
  482. 5:43 PM
  483.  
  484. ok
  485. 5:44 PM
  486.  
  487. so then i put that on my list, but not for today. and show that it works
  488. meh profile image
  489. Mathieu Duponchelle
  490. 5:44 PM
  491.  
  492. but yeah, bottom line for me is "let's not copy anything"
  493. 5:44 PM
  494.  
  495. cool
  496. 5:44 PM
  497.  
  498. anyone cares if I paste the convo to thiibault ?
  499. slomo profile image
  500. Sebastian Dröge
  501. 5:44 PM
  502.  
  503. and i don't even need to implement new code, just fix the broken additive blending (or maybe i misread the orc code and it's actually correct)
  504. meh profile image
  505. Mathieu Duponchelle
  506. 5:45 PM
  507.  
  508. you should be able to test with ges-launch btw
  509. 5:45 PM
  510.  
  511. the current behaviour
  512. slomo profile image
  513. Sebastian Dröge
  514. 5:45 PM
  515.  
  516. should be fine to paste, just double-check. should've been in #gstreamer anyway
  517. meh profile image
  518. Mathieu Duponchelle
  519. 5:46 PM
  520.  
  521. just need to add two clips with overlapping times on the same layer
  522. slomo profile image
  523. Sebastian Dröge
  524. 5:46 PM
  525.  
  526. there's a crossfade compositor example app in git. i don't want to learn GES first
  527. meh profile image
  528. Mathieu Duponchelle
  529. 5:46 PM
  530.  
  531. and with non-overlapping geometries
  532. slomo profile image
  533. Sebastian Dröge
  534. 5:47 PM
  535.  
  536. ok need to shower now, have to leave in 45 minutes and only got half of what i wanted to do done today because of the compositor stuff
  537. meh profile image
  538. Mathieu Duponchelle
  539. 5:48 PM
  540.  
  541. compositor is a pretty important thing to get right I think
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!