#Common Commands

### System Utility
* Upgrade all packages with pip
```shell
sudo pip freeze --local | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 sudo pip install -U
```
* Split and Concat files in Shell
```shell
split -l 10000 new_data_lower.txt
cat *.txt > merged-file
```
* Change Mode to Executable
```bash
sudo chmod +x countdown.sh
sudo chmod +x new_thisweek.sh
```
* Kill a process by PID
```bash
kill -9 pid
```
* Print out first/last few lines
```bash
head -n 1 file  # change 1 to any number
tail -n 1 file
# if only print out first line
sed -n 1p file
head -n 1 file
awk 'NR==1' file
```
* Count total number of lines in a file
```bash
sed -n '$=' filename
wc -l filename
```
* Apply a change in a file (e.g., .bash_profile)
```bash
source .bash_profile
```
* VPN PrivateInternetAccess Usernam
```
p6441849
```
* remove a directory
```
rm -r directory
```
* Change default shell
```
chsh -s $(which zsh)
chsh -s $(which bash)
```

### Stata
* Plotting coefficient estimates
```stata
parmby "xtreg ..., fe r", norestore
eclplot estimate min95 max95 parmseq
```
```stata
//identify nonnumeric strings and drop
gen byte notnumeric = real(twitter_count)==.
/*makes indicator for obs w/o numeric values*/
tab notnumeric
/*==1 where nonnumeric characters*/
list  twitter_count  if notnumeric==1
/*will show which have nonnumeric*/
drop if notnumeric==1
```


### Python
* To get a value of a key in a dictionary
```python
dictionary.get('key', 'return_otherwise')
```
* String formatting
```python
a = 'John'
b = 'Doe'
'Hi there %s %s. How is it going' % (a,b)  #c-like placeholder
'Hi there {0} {1}. How is it going'.format(a,b) #python method
```

### System
* Get rid of the field codes with Endnote
```cmd
CMD + Shift + F9 + fn
```

### Git
```bash
cd /Users/Hong/Google Drive/website
git add.
git commit -m "message"
git push origin master
```

### Update personal website
```bash
cd "/Users/Hong/Google Drive/website"
scp * ykhong1@general.asu.edu:www  # push all files to the www folder under the asu public address
scp -r files/ ykhong1@general.asu.edu:www # push the files folder to the files folder under www folder in asu address
scp files/social.png ykhong1@general.asu.edu:www/files
ssh ykhong1@general.asu.edu
```

### Update the Hack Font
```
brew cask install caskroom/fonts/font-hack
```

### Access AWS Server
```bash
sudo ssh -i hongkey1.pem ubuntu@52.32.158.42

scp -i hongkey1.pem  /Users/Hong/Google\ Drive/Research/Freelancer\ Crawler/Final\ Data\ and\ Scripts/Latest/Database/FreelancerDump20160716.sql  ubuntu@52.43.46.23:~/data/

## upload a directory
sudo scp -r -i hongkey1.pem  /Users/Hong/Google\ Drive/Research/Freelancer\ Crawler/Final\ Data\ and\ Scripts/Latest/Scripts/  ubuntu@52.43.46.23:~/scripts/

##install mysqldb (mysql-python) on linux
sudo apt-get install libmysqlclient-dev python-dev
sudo ~/anaconda2/bin/pip install mysql-python
```
### MySQL
```mysql
#import:
mysql > set global max_allowed_packet=100000000

in command line: mysql -u root -p --max_allowed_packet=100m freelancer<e:dump.sql;

#dump:

mysqldump --user=root --password=****** --database gaf > d:dump.sql

#Grant only select function
GRANT SELECT ON *.* TO 'chen'@'%' IDENTIFIED BY 'isr';
```