[autodeploy@spark4 fpspark]$ make run_api_1a
docker-compose -f ./docker-compose.yml run focal_point  bash -c "cd /FocalPointAttribution && sh ./api1a_data_curation_script_master.sh"
Starting postgresql_fp_milestone03 ... done
#################################################################################
Environment: Payer2
Data Source: MarketScan
Using Python at: /usr/local/bin/python
#################################################################################
INFO: Start: Pull and check data from database, derive additional features, aggregate claim lines, save as DataFrames
INFO: Parameter file is src/focalpoint/params/Payer2/MarketScan/data_curation.params
INFO: Metadata file is src/focalpoint/params/Payer2/MarketScan/MarketScan_Comm_Metadata.csv
INFO: Time period for pulling claim & enrollment data: 2012-01 to 2016-12
INFO: Using 3 processes
INFO: Checking database connection and verifying schema against metadata file
/usr/local/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
INFO: Directory data/ already exists
INFO: Start: Pull, process, and save claims data
/usr/local/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
/usr/local/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
/usr/local/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
INFO: Start: Execute claims SQL query for 2012-06
INFO: Start: Execute claims SQL query for 2012-01
INFO: Start: Execute claims SQL query for 2012-11
/usr/local/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
Traceback (most recent call last):
  File "src/focalpoint/nonspark/data_curation/pull_claim_enroll_monthly.py", line 501, in <module>
    sys.exit(main(sys.argv[1:]))
  File "src/focalpoint/nonspark/data_curation/pull_claim_enroll_monthly.py", line 456, in main
    pool.map(pull_aggreg_claims_one_month_star, argList)
  File "/usr/local/lib/python2.7/multiprocessing/pool.py", line 253, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/usr/local/lib/python2.7/multiprocessing/pool.py", line 572, in get
    raise self._value
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not resize shared memory segment "/PostgreSQL.45056658" to 4194304 bytes: No space left on device
 [SQL: 'SELECT sample_commercial.focalpoint_claims_1m.member_id, sample_commercial.focalpoint_claims_1m.claim_id, sample_commercial.focalpoint_claims_1m.svc_date, sample_commercial.focalpoint_claims_1m.paid_date, sample_commercial.focalpoint_claims_1m.allowed_amt, sample_commercial.focalpoint_claims_1m.coinsurance, sample_commercial.focalpoint_claims_1m.copay, sample_commercial.focalpoint_claims_1m.deductible, sample_commercial.focalpoint_membership_1m.gender, sample_commercial.focalpoint_membership_1m.state, sample_commercial.focalpoint_membership_1m.msa, sample_commercial.focalpoint_membership_1m.age, sample_commercial.focalpoint_membership_1m.age_category, sample_commercial.focalpoint_membership_1m.prod_type, sample_commercial.focalpoint_claims_1m.claim_type, sample_commercial.focalpoint_claims_1m.place_of_svc, sample_commercial.focalpoint_claims_1m.ndc, sample_commercial.focalpoint_claims_1m.proc_code, sample_commercial.focalpoint_claims_1m.proc_code_type, sample_commercial.focalpoint_claims_1m.svcscat, sample_commercial.focalpoint_claims_1m.dx1, sample_commercial.focalpoint_claims_1m.dx2, sample_commercial.focalpoint_claims_1m.dx3, sample_commercial.focalpoint_claims_1m.dx4, sample_commercial.focalpoint_claims_1m.icd10_yes, sample_commercial.focalpoint_claims_1m.svc_prov_specialty, sample_commercial.focalpoint_claims_1m.network_provider_yes, sample_commercial.focalpoint_claims_1m.table_type \nFROM sample_commercial.focalpoint_claims_1m LEFT OUTER JOIN sample_commercial.focalpoint_membership_1m ON sample_commercial.focalpoint_claims_1m.member_id = sample_commercial.focalpoint_membership_1m.member_id AND EXTRACT(year FROM sample_commercial.focalpoint_claims_1m.svc_date) = EXTRACT(year FROM sample_commercial.focalpoint_membership_1m.elig_date) AND EXTRACT(month FROM sample_commercial.focalpoint_claims_1m.svc_date) = EXTRACT(month FROM sample_commercial.focalpoint_membership_1m.elig_date) \nWHERE EXTRACT(year FROM sample_commercial.focalpoint_claims_1m.paid_date) = %(param_1)s AND EXTRACT(month FROM sample_commercial.focalpoint_claims_1m.paid_date) = %(param_2)s AND (%(param_3)s * (EXTRACT(year FROM sample_commercial.focalpoint_claims_1m.paid_date) - EXTRACT(year FROM sample_commercial.focalpoint_claims_1m.svc_date)) + EXTRACT(month FROM sample_commercial.focalpoint_claims_1m.paid_date)) - EXTRACT(month FROM sample_commercial.focalpoint_claims_1m.svc_date) <= %(param_4)s AND sample_commercial.focalpoint_membership_1m.pharmben_yes = %(pharmben_yes_1)s AND sample_commercial.focalpoint_membership_1m.prod_type NOT IN (%(prod_type_1)s, %(prod_type_2)s, %(prod_type_3)s) AND sample_commercial.focalpoint_membership_1m.hlthplan_yes = %(hlthplan_yes_1)s'] [parameters: {'prod_type_1': 'HMO', 'prod_type_2': 'POS with capitation', 'prod_type_3': 'POS with Capitation', 'param_4': 11, 'param_1': 2012, 'param_3': 12, 'param_2': 11, 'hlthplan_yes_1': 0, 'pharmben_yes_1': 1}] (Background on this error at: http://sqlalche.me/e/e3q8)
INFO: Start: Execute claims SQL query for 2013-04
INFO: Start: Aggregate monthly data into time periods
INFO: Data curation parameter file is src/focalpoint/params/Payer2/MarketScan/data_curation.params
INFO: Experiment parameter file is src/focalpoint/params/Payer2/MarketScan/experiment.params
INFO: Period 4: Incurred 2016-01 to 2016-12, paid 2016-01 to 2016-12
INFO: Period 3: Incurred 2015-01 to 2015-12, paid 2015-01 to 2015-12
INFO: Period 2: Incurred 2014-01 to 2014-12, paid 2014-01 to 2014-12
INFO: Period 1: Incurred 2013-01 to 2013-12, paid 2013-01 to 2013-12
INFO: Period 0: Incurred 2012-01 to 2012-12, paid 2012-01 to 2012-12
INFO: Directory expts/ already exists
INFO: Start: Process claims data for period 0
INFO: Processing claims paid in 2012-01
Traceback (most recent call last):
  File "src/focalpoint/nonspark/data_curation/create_claim_enroll_blocks.py", line 127, in <module>
    sys.exit(main(sys.argv[1:]))
  File "src/focalpoint/nonspark/data_curation/create_claim_enroll_blocks.py", line 84, in main
    dfClaims = aggreg_monthly_claims(dfClaims, idx, params)
  File "/FocalPointAttribution/src/focalpoint/nonspark/helper/create_claim_enroll_blocks_helper.py", line 21, in aggreg_monthly_claims
    dfClaimsNew = cPickle.load(open(os.path.join(params['dir']['data'], 'dfClaims'+str(month)+'.cpkl'), 'rb'))
IOError: [Errno 2] No such file or directory: u'data/dfClaims2012-01.cpkl'
make: *** [run_api_1a] Error 1