API tools faq
paste
Advertisement
Guest User

Hard vs Soft bounces, In band out of band postfix

a guest
Feb 6th, 2016
83
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 11.05 KB | None | 0 0
raw download clone embed print report
  1. [00:44:57] <gansbrest_> hi. Is there a way to distinguish between bounce generated by my own mail server and bounce generated by recipient's mail server (soft bounce)
  2. [00:45:37] <Patrickdk> the error message?
  3. [00:47:10] *** smica has quit IRC
  4. [00:47:47] <gansbrest_> I use transport to direct bounces to my script and I pass it $recipient variable for later processing in the script
  5. [00:47:55] <gansbrest_> argv=/var/www/mirtrik/bounces.php -u ${recipient}
  6. [00:48:37] <gansbrest_> Ideally I would like to avoid reading of full bounce message
  7. [00:49:16] <gansbrest_> all I need to know if it's hard bounce (bounced right away) or soft bounce (received from recipient mail server after message was accepted)
  8. [00:49:51] *** cafuego has joined #postfix
  9. [00:51:05] <gansbrest_> is there a way to know that without parsing full bounce message
  10. [00:53:55] <seekwill> That's not generally what's called a soft bounce
  11. [00:54:46] <seekwill> Your Postfix logs will tell you if it couldn't deliver. How you handle out of band messages is a different story
  12. [00:55:54] <gansbrest_> so basically the most reliable way is to parse the message
  13. [00:56:29] <seekwill> No
  14. [00:56:46] <seekwill> Re-read my line above...
  15. [00:57:13] <seekwill> Out of band bounces are those that initially get accepted and then rejected.
  16. [00:58:46] <gansbrest_> I'm not sure what band bounce is, sorry
  17. [00:59:05] <seekwill> (received from recipient mail server after message was accepted) <-- out of band bounce
  18. [00:59:19] <seekwill> The "band" being the SMTP conversation your server has with the recipient's
  19. [00:59:19] <gansbrest_> ah ok, thanks
  20. [01:00:09] <gansbrest_> ok, so when my server tries to deliver and fails then it writes that to log and generate bounce message
  21. [01:00:33] <gansbrest_> so theoretically I can read log to figure that out
  22. [01:00:37] <seekwill> Yes
  23. [01:01:00] <gansbrest_> but my bounce handling script is triggered when I receive bounce message
  24. [01:01:21] <seekwill> Is your bounce handling script located on the host sending out the mail?
  25. [01:01:30] <gansbrest_> currently yes
  26. [01:02:40] <gansbrest_> my script will receive ${recipient} + whole message in stdin
  27. [01:02:54] <gansbrest_> theoretically I can look up a log entry
  28. [01:03:00] <gansbrest_> based on recipient
  29. [01:03:23] <gansbrest_> but what if there are multiple entries for that
  30. [01:05:34] *** orogor has quit IRC
  31. [01:07:39] <seekwill> You want to use VERP
  32. [01:07:49] <seekwill> (If I understand your question correctly)
  33. [01:07:56] <gansbrest_> yes, I do use it
  34. [01:08:06] <gansbrest_> i receive user-123 at mydomain dot com
  35. [01:08:07] <seekwill> Why would there be multiple VERP entries?
  36. [01:08:27] <seekwill> You may need to be more specific about what you're doing
  37. [01:09:18] <gansbrest_> basically I want to remove not existent users from the list right away and allow some number of band bounces
  38. [01:10:08] <seekwill> The OOB bounces are less of a worry for the time being. Most "good" receivers will tell you about non existant users in band
  39. [01:10:08] <gansbrest_> so I guess I can read the log for that specific user (user-123 in my case)
  40. [01:10:10] <seekwill> Focus on that
  41. [01:10:22] <seekwill> Read your Postfix log, and parse for it
  42. [01:10:32] *** magyar has quit IRC
  43. [01:11:32] *** loddafnir has quit IRC
  44. [01:18:31] <gansbrest_> what about this approach, if I have to worry only about not existent users I can close my port 25 and avoid all bounces sent after message was accepted.
  45. [01:19:09] <seekwill> Some stupid people don't do recipient validation, and would legitimately send an OOB bounce
  46. [01:19:10] <gansbrest_> that way the only bounce I will receive is the one generated by my postfix
  47. [01:22:31] <gansbrest_> How you handle out of band messages is a different story
  48. [01:22:59] <gansbrest_> can you please explain your approach on how to handle band messages?
  49. [01:24:14] <seekwill> In our sending environment, we have a dedicated inbound MTA that collects the bounces. Once we process the VERP as being a legimate message, we scrape the body to see if we can figure out why it bounced.
  50. [01:24:41] <rob0> Invasion of the Body Scrapers
  51. [01:25:01] <seekwill> Depends who's body... I'd go for KB1JWQ's!
  52. [01:25:32] <gansbrest_> do you read logs at all or you just scrape the body and if there is 550 you remove the user
  53. [01:25:45] <seekwill> Always read the logs
  54. [01:26:32] <seekwill> Throwing out a guess, maybe less than 5% of bounces are OOB
  55. [01:26:47] <gansbrest_> oob?
  56. [01:26:48] <seekwill> (sorry, of our OOB bounces)
  57. [01:26:51] <seekwill> Out of band
  58. [01:27:32] *** shasta has quit IRC
  59. [01:27:58] <gansbrest_> could you please take a look at my band bounce log http://pastebin.com/Rcgdh3GY
  60. [01:28:11] <seekwill> "In band"
  61. [01:28:32] <seekwill> Yeah, Line 4 is the one you want to parse
  62. [01:29:05] <rob0> oob syas tsohg
  63. [01:29:23] <seekwill> ho
  64. [01:32:26] <gansbrest_> so I mostly care about status=bounced and to=<>
  65. [01:32:55] <tharkun> seekwill: does the ipad support DtD?
  66. [01:33:08] <seekwill> No flash :(
  67. [01:33:19] <seekwill> I think...
  68. [01:33:28] <seekwill> gansbrest_: And the response
  69. [01:34:33] <seekwill> tharkun: Nope :(
  70. [01:34:44] <gansbrest_> but don't you get all of that in the bounce body generated by postfix later on?
  71. [01:34:47] <seekwill> Might be able to jailbrak
  72. [01:34:52] <tharkun> seekwill: back to square one
  73. [01:35:02] *** googlah has joined #postfix
  74. [01:36:06] <seekwill> gansbrest_: Sure, whatever you want to do
  75. [01:36:49] <seekwill> It seems easier to me to parse a log line vs a bounce message
  76. [01:36:54] <gansbrest_> I'm just thinking if you still need body scraper, you might as well parse for 550 messages
  77. [01:36:57] *** Zblakany has quit IRC
  78. [01:40:46] <seekwill> Sure
  79. [01:43:05] <pj> well, if you have to parse bounces from remote servers anyways, then it would probably be easier just to parse the body in all of them.
  80. [01:43:49] <seekwill> I prefer to not generate a bounce message
  81. [01:45:25] <pj> and he seems to be confused, thinking that OOB vs REJECT is the same as soft vs hard or something.
  82. [01:45:29] <pj> they are completely different
  83. [01:45:52] <seekwill> I think from his point of view, he's understand right
  84. [01:46:08] <seekwill> He was definitely confused on soft vs hard
  85. [01:46:34] <pj> I don't know, it looks to me like you corrected his terminology to OOB vs In band, but he still thinks it's the same as soft vs hard.
  86. [01:46:48] <seekwill> oh
  87. [01:46:58] <pj> he wants to be parsing for soft vs hard, from his POV he shouldn't care if it's OOB or not.
  88. [01:47:24] <seekwill> I didn't think he was really talking about soft bounces (I prefer calling it transient failures)
  89. [01:48:17] <pj> yeah, it looks that way to me. His question kinda looked like, "how do I differentiate between soft vs hard" and you led him down talking about OOB, heh
  90. [01:48:41] <seekwill> oops
  91. [01:48:43] <seekwill> :)
  92. [01:48:49] <seekwill> Someone hand me another beer!
  93. [01:48:53] <gansbrest_> wait wait :)
  94. [01:48:57] <pj> here is what he was asking that led down that path...
  95. [01:48:59] <pj> [15:48] <gansbrest_> all I need to know if it's hard bounce (bounced right away) or soft bounce (received from recipient mail server after message was accepted)
  96. [01:49:04] <pj> but later he says this ...
  97. [01:49:36] <pj> [16:08] <gansbrest_> basically I want to remove not existent users from the list right away and allow some number of band bounces
  98. [01:49:51] <pj> that latter statement seems to suggest that he really wants to know about soft vs hard.
  99. [01:49:55] <seekwill> true
  100. [01:50:07] <pj> and it makes more sense that way too.
  101. [01:50:12] <seekwill> I say we hang gansbrest_ outside
  102. [01:50:14] <gansbrest_> could you guys explain more about soft and hard
  103. [01:50:51] <lunaphyte_> well, i'm glad you asked.
  104. [01:51:02] <pj> gansbrest_: a soft bounce is a condition where the server will try again (also known as "deferred"). It is simply a way of informing you that the message was not delivered yet but don't do anything because it will retry again for up to five days
  105. [01:51:17] <pj> a hard bounce is where the server gives up right away and won't try again.
  106. [01:51:23] <seekwill> Quitter
  107. [01:51:25] <pj> it has nothing to do with OOB vs In band.
  108. [01:51:49] <gansbrest_> will try again to deliver?
  109. [01:52:05] <seekwill> 4xx code vs 5xx code
  110. [01:52:08] <pj> right
  111. [01:52:21] <pj> that's how you differentiate, witht he code#
  112. [01:52:48] <pj> when maintaining a mailing list, you usually want to discard soft bounces and only consider hard bounces in terms of removing someone from the list.
  113. [01:53:11] <gansbrest_> ok, ignore 4xx, remove 5xx
  114. [01:53:17] <pj> yes, exactly
  115. [01:53:25] <Aprogas> Mailman has a good heuristic for handling bounces, borrow inspiration from their implementation.
  116. [01:53:53] <gansbrest_> yes, maybe I will take a class from there
  117. [01:53:57] <seekwill> "Borrow" :)
  118. [01:54:03] <gansbrest_> borrow :)
  119. [01:54:04] <pj> yes, it can get more complex than that, too, for example different types of hard bounces might be treated differently as well, but that's the gist of it.
  120. [01:54:14] <Aprogas> Basically they keep a score, a single bounce is not enough for removal.
  121. [01:54:35] <gansbrest_> even for 550?
  122. [01:54:57] <seekwill> Even for some 4xx :)
  123. [01:54:58] <seekwill> Damn Yahoo
  124. [01:55:12] <pj> right, what I said was a generalization
  125. [01:55:15] <seekwill> 451 Hey, yeah, I don't want this email. We're never going to accept it.
  126. [01:55:25] <pj> just to give you an idea.
  127. [01:55:32] <gansbrest_> nice
  128. [01:56:12] <pj> there are libraries for various programming languages that you can use to parse bounces, btw.
  129. [01:56:37] <Aprogas> I think "mailbox full / quota exceeded"-errors are 5xx even though usually they are quickly fixed by the user.
  130. [01:56:44] <gansbrest_> so let me clarify a little: IN BAND can produce hard and soft bounce, is that correct?
  131. [01:57:02] *** famicom has quit IRC
  132. [01:58:02] <pj> gansbrest_: hard and soft have nothing to do with In band vs OOB.
  133. [01:58:09] <pj> either one can produce both.
  134. [01:58:14] <gansbrest_> ok
  135. [01:58:18] *** lusted_gay has quit IRC
  136. [01:58:26] <seekwill> OOB transient failures...
  137. [01:58:28] <seekwill> ?
  138. [01:59:02] <Aprogas> NOOB failures
  139. [02:00:14] <gansbrest_> so most good providers produce hard bounce in band, while it's also possible that some mail server will accept my message and will send me a bounce with 550 later on
  140. [02:01:46] <Aprogas> Good providers produce the relevant response code in-band, whether that is a 2xx 4xx or 5xx response. Good providers don't come back on their earlier claim afterwards.
  141. [02:02:03] <Aprogas> From the perspective of your mailinglist this is not very relevant though.
  142. [02:02:19] <gansbrest_> yes, I just want to understand it better
  143. [02:02:24] <Aprogas> Either the mailinglist gets a bounce from your Postfix, or it gets a bounce from a remote mailserver (delivered through your Postfix)
  144. [02:02:26] <gansbrest_> thank you guys for a lot of useful info
  145. [02:02:41] <gansbrest_> so mostly I care about error codes
  146. [02:02:48] <gansbrest_> in the body
  147. [02:03:21] <gansbrest_> scrape - get the code, remove from the list or ignore
  148. [02:04:43] <gansbrest_> is there a standard for those codes?
  149. [02:05:28] <Aprogas> There is a non-exclusive list in RFC 5321.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!