unix log processing

I have a requirement to read from a continuously running logs which refreshes in milliseconds, because 1000's of people will be accessing in each and every minute.
My requirement is:

Suppose there is an xml request like this:
<dataReq>
  <id>1</id>
  <name>test</name>
  <reqId>ddaabyy234</reqid>
   ....
   ...
   ....
</dataReq>

And the response for the above request is:

<dataRes>
   <id>1</id>
   <dept>railway</dept>
   <reqId>ddaabyy234</reqid>
   .......
   .......
   <errcode>
       <errNum>404</errNum>
   </errcode>
 </dataRes>


 **As you seen above, if the <errNum> is anything apart from 200, then I need to capture that <dataReq>..</dataReq> and <dataRes>..</dataRes>**

The problem here is, there can be multiple <dataReq>...</dataReq> tags. Because, inorder to genererate a response it will take some time for the backend systems and in the meanwhile many users will be sending their requests from webUI (or) mobile (or) apps to the same application.

The log will also be getting archived into archived files every 5 minutes as the log file  will be growing its size to huge withing every 5 minutes.

Please suggest me on this.