Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- Hi, i am currently trying to run a python script that navigate on the web with selenium and get the html of the product page on a web site and then do scraping to collect info. The info are stored in a database mariadb.
- I created a bash script that should run the python script, the bash script is this one:
- #!/bin/bash
- sleep 5
- source /home/realtalker/data_workers/scraping_best/.venv/bin/activate
- sleep 5
- /home/realtalker/data_workers/scraping_best/.venv/bin/python /home/realtalker/data_workers/scraping_best/get_info_best.py
- Then i did chmod +x to the bash script and to the python script. So it is executable.
- I tried first to run on the terminal the python script with python3 get_info_best.py and it worked. Then i tried to run the bash script on the terminal with at and it worked. The script run for like 10 minutes and after 7 minutes i tried atq and i could see that the script was running and checking the database i could see that the script worked. After checking this two things i
- added a job on cron that should execute the bash script at 4 am:
- 0 4 * * * /bin/bash -x /home/realtalker/data_workers/scraping_best/get_info_best.sh >> /home/realtalker/data_workers/scraping_best/cron.log 2>&1
- But the next morning when i checked the cron log i found this message:
- + sleep 5
- + source /home/realtalker/data_workers/scraping_best/.venv/bin/activate
- ++ deactivate nondestructive
- ++ '[' -n '' ']'
- ++ '[' -n '' ']'
- ++ hash -r
- ++ '[' -n '' ']'
- ++ unset VIRTUAL_ENV
- ++ unset VIRTUAL_ENV_PROMPT
- ++ '[' '!' nondestructive = nondestructive ']'
- ++ '[' linux-gnu = cygwin ']'
- ++ '[' linux-gnu = msys ']'
- ++ export VIRTUAL_ENV=/home/realtalker/data_workers/scraping_best/.venv
- ++ VIRTUAL_ENV=/home/realtalker/data_workers/scraping_best/.venv
- ++ _OLD_VIRTUAL_PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
- ++ PATH=/home/realtalker/data_workers/scraping_best/.venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
- ++ export PATH
- ++ '[' -n '' ']'
- ++ '[' -z '' ']'
- ++ _OLD_VIRTUAL_PS1=
- ++ PS1='(.venv) '
- ++ export PS1
- ++ VIRTUAL_ENV_PROMPT='(.venv) '
- ++ export VIRTUAL_ENV_PROMPT
- ++ hash -r
- + sleep 5
- + /home/realtalker/data_workers/scraping_best/.venv/bin/python /home/realtalker/data_workers/scraping_best/get_info_best.py
- Critic error in the process: Message: session not created: probably user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
- Stacktrace:
- #0 0xaaaabff0c898 <unknown>
- #1 0xaaaabfa6cbb4 <unknown>
- #2 0xaaaabfa9f7e4 <unknown>
- #3 0xaaaabfa9bc5c <unknown>
- #4 0xaaaabfadeedc <unknown>
- #5 0xaaaabfade79c <unknown>
- #6 0xaaaabfaa7a1c <unknown>
- #7 0xaaaabfaa87bc <unknown>
- #8 0xaaaabfed9b1c <unknown>
- #9 0xaaaabfeddda4 <unknown>
- #10 0xaaaabfec902c <unknown>
- #11 0xaaaabfede638 <unknown>
- #12 0xaaaabfeb04a4 <unknown>
- #13 0xaaaabfefb220 <unknown>
- #14 0xaaaabfefb3f4 <unknown>
- #15 0xaaaabff0b54c <unknown>
- #16 0xffffbd275958 <unknown>
- #17 0xffffbd2dba48 <unknown>
- I think that the problem at this point is with cron or chromium, i am on a Rasberry pi 5 on Ubuntu, the version of Chromium is Chromium 132.0.6834.159 snap and the version of the chromedriver is ChromeDriver 132.0.6834.83 (36d3e67544e087c21630e290d4f024723cd55338-refs/heads/main@{#1389833})
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement