daily pastebin goal
91%
SHARE
TWEET

Carson file uploads

AntonShauchenia May 23rd, 2018 126 in 89 days
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. *Residents file upload logic*
  2.  
  3. How to upload:
  4. - Send the web-form (specify the Content-Type as 'multipart/form-data') to specific endpoint
  5.  
  6.  
  7. Currently in staging:
  8.  
  9.  
  10. 1. CP file upload for building.
  11.  
  12. Flow:
  13.  
  14. - Get file from request. NOTE: file field should have name 'file'
  15.  
  16. - Read data from file one row at a time:
  17.  
  18.  * Rename row fields
  19.  
  20.  * Search for existing unit:
  21.    If not exists: create unit, update the report
  22.  
  23.  * Search for person:
  24.    If not exists: skip row - this is a temp. measure since we don't have person name in file.
  25.    The actual flow is to create new person and update the report
  26.  
  27.  * Search for person role in unit:
  28.    If not exists: create person_role, update the report
  29.  
  30.  * Save person payment data - payment_provider_id in person_role
  31.  
  32.  * Save unit payment data - payment_external_id in unit
  33.  
  34. - Return the report:
  35.   [
  36.     {'unit_name': 'unit2'}, - if unit was created
  37.     {'unit_name': 'unit2', 'email': 'smth@gmail.com', 'first_name': 'Bobby', 'last_name': 'Fisher'}, - if person was created or if existing person was assigned new role
  38.   ]
  39.  
  40.  
  41.  
  42. Currently not in staging:
  43.  
  44.  
  45. 1. Yardi file sync.
  46.  
  47. We assume that PMs will setup a sceduled action in Yardi system that will send email with up-to-date residents list to the email we provider them with.
  48. We determine building using the 'to' email address. We take file from the mail attachments
  49.  
  50. Flow:
  51.  
  52. a) For empty building - initial upload:
  53.     - For each row in file:
  54.       * Rename row fields
  55.       * Check row for errors - currently only that row has one of 'email' and 'phone_number'
  56.         If error: store row to report
  57.  
  58.       * Search for existing unit
  59.         If not exists: create unit, update counter in report
  60.  
  61.       * Search for existing person
  62.         If not exists: create person, update counter in report
  63.  
  64.       * Search for existing person role in unit
  65.         If not exists: create person_role, update counter in report
  66.  
  67.       * Save person payments data
  68.       * Update total counter
  69.  
  70.     - Return success flag and report:
  71.       If there any row with error - success flag is false
  72.       report:
  73.       {
  74.         'records': 10, - total number of successfully processed rows
  75.         'units': 2, - number of created units
  76.         'persons': 3, - number of created persons
  77.         'associations': 2, - number of newly assigned roles
  78.         'skip_rows': [
  79.             {...row from the file}
  80.         ] - rows that contain errors
  81.       }
  82.  
  83.     - If not success flag: send a report to admins
  84.  
  85.     - Store sync log to db
  86.  
  87. b) For not empty building - update upload
  88.     - Get existing building units
  89.     - Get existing building residents
  90.     - For each row in file:
  91.         * Rename rows and check errors
  92.           If error: store row to report
  93.  
  94.         * Search for existing unit
  95.           If not exists: store for update
  96.  
  97.         * Search for person:
  98.           If not exists: store for update
  99.  
  100.         * Search for person role:
  101.           If not exists: store for update
  102.  
  103.         * Find existing person roles, that should be moved out
  104.  
  105.         * Find existing units that are not in file
  106.  
  107.     - Check the condition for update: if updating (new/move out) more than 15% and more than 4 rows - decline
  108.         Return *success flag*=False AND report data: {
  109.             'update': 5 - number of rows stored for update,
  110.             'moveout': 3 - number of residents to move out,
  111.             'total_count': 15 - total number of resident,
  112.             'missing_units': 2 - number of existing units not in file,
  113.             'skip_rows': [{}, {}, ...] - rows that contain errors
  114.         }
  115.  
  116.     - If condition passed - apply updated (create missing units, persons, person_roles and return report)
  117.         report: {
  118.             'units': 2, - number of created units
  119.             'persons': 2, - number of created persons
  120.             'associations': 2 - number of created associations
  121.         }
  122.  
  123.     - Apply move out
  124.  
  125.     - Return the success flag and report
  126.         success flag is true is there are no rows with errors (len(skip_rows) == 0)
  127.         report: {
  128.             ...update_report,
  129.             'moveout': 3 - number of residents to move out,
  130.             'missing_units': 2 - number of existing units not in file,
  131.             'skip_rows': [{}, {}, ...] - rows that contain errors
  132.         }
  133.  
  134.     - If success flag is false - send report to admins
  135.     - Store log to db
  136.  
  137.  
  138. 2. New residents upload
  139.  
  140. Flow:
  141.     - If building already has residents - return error
  142.     - For each row in file:
  143.         * Rename fields
  144.  
  145.         * Search for existing unit
  146.           If not exists: create
  147.  
  148.         * Search for existing person
  149.           If not exists: create person + assign role
  150.           ELSE: store person to report
  151.  
  152.     - Return the persons that were already present in the system prior to upload
  153.  
  154.     - Return report to uploader
  155.       report: [{
  156.             'unit': 'unit2',
  157.             'first_name': 'Magnus',
  158.             'last_name': 'Karlsen',
  159.             'email': 'random@check.com',
  160.             'phone_number': null
  161.           }, {...}, ...
  162.       ]
  163.  
  164.  
  165. Summary:
  166. All these endpoints/flows have much in common and use a lot of common utility functions. However all of them contain their own specific logic (like checking the updates amount in Yardi update or returning already existing persons in New residents upload), so I'm not sure if we can generalize everything to one flow that can cover all our needs.
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
 
Top