/cng/ Chainlink Node General - Neet node setup tutorial

This tutorial was made to help make life a little easier for anons that want to setup a neet node. I will be using Google Cloud as a vm providor and this tutorial will be written to cater to that, along with running Ubuntu 18.04 LTS on the server, and ubuntu 18.04 lts on your home desktop. This is not easy, we were told this from the beginning. This tutorial will have bad grammar and spelling because it was written by a brainlet. Thats ok, if you dont like it then write your own. I am writing this as I learn stuff. Things will be changed and edited. My experience is shitposting and running linux for about 4 years fulltime on my desktop. It is written assuming you have about the same experience. If you dont know basic linux terminal commands, go shitpost on /g/ and then come back. if this helped you, i am going to shill my eth/link donation address below this. Donate or dont. If anything is wrong or could be done better, feel free to call me a faggot and what to change. Now that that is out of the way lets begin.

Eth/Link donation address: 0x871fDd795ee64ca6077fe70578d65291F365e627

Table of Contents
1. FAQ
2. Choosing a cloud provider
3. A journey into neetdom. Setting up the VM's
4. Installing the chainlink node

1. FAQ

>can i start making linkies right now with my neet node?
as of writing this no.
https://docs.chain.link/docs/becoming-a-chainlink-reviewed-node
you must first operate a testnet node for at least 1 week before a review. your node must accept jobs every 10 minutes with no interuptions with a cron job.

>what is a cron job? (answered by >>14262983)
In the context of Chainlink nodes. Cron is one of the available types of initiators.
Same as in Linux, Cron is a way to schedule jobs preemptively. Node jobs use JSON formatting and you use the same syntax as Linux on the schedule line.

{
"initiators": [
{
"type": "cron",
"params": {
"schedule": "* * * * * *"
}
}
],

Setting up the cron job can be confusing. You have to reference a working transaction and copy the hex code into the job while making sure you don't duplicate the transaction header. This is after you replace the original initiator with a cron initiator.

>will this cost money to run
yes

>how much?

GCP
two g1-small instances ($15 mo. each)
one postgreSQL database with high availability ($??)
fiews EaaS ($5 mo.)

AWS (>>14261881)
you're looking at about ~$700usd per year which includes:
-two t2.micro EC2 instances (one extra on standby for failover)
-t2.micro postgresql database with multi availability zones
-fiews EaaS (or you could use infura)

another anon (>>14274515 >>14274722) says it will cost less than $700
2 micro instances running all month will only cost $13.
postgresql should only cost 21$ a month (https://aws.amazon.com/rds/postgresql/pricing/
)

all and all i cant find 1 straight answer. i'd imagine the answer is somewhere in the middle.


>can i just buy a decent tower and run my node on there?
maybe
can you guarantee 100% uptime at your house? do you have multiple isp's in case one goes down? do you have the home network hardware capable of rolling over to a secondary isp if before scenario happens? do you have a battery backup to power your home network and the tower in case of power outage? is that battery backup capable of holding that load of possible hours? do you have a generator that can power that network in case of the previous problems? if you said yes to all those then it might be worth it to do it then. saves money on cloud hosting.

>what does that acronym mean?
EAAS - Ethereum As A Service
AWS - Amazon Web Services
GCP - Google Cloud Platform
VM - Virtual Machine (vm's being the plural)


2. Choosing a cloud provider

Its up to you to decide what you can afford and what will suit you best. There are the following options. I will include eaas here as well. In this tutorial we will be using GCP to run 2 nodes in different regions and a PostgreSQL database. We are taking advantage of that $300 credit here.

>cloud providers
https://azure.microsoft.com/en-us/
https://aws.amazon.com/ (free tiers)
https://www.digitalocean.com
https://www.cloud.google.com (free for 1y for new users (($300 credit)))
https://cloud.oracle.com/home (500 free credits with email signup)

>eaas
fiews.io
https://docs.linkpool.io/docs/node_rpc.html


3.A journey into neetdom. Setting up the VM's

So you signed up for GCP and you have your free $300 credit and you are ready to go. Great! You just completed the first step into neetdom. congrats.

First thing is first. We need to setup those vms. from the GCP console we are going to click on the hamburger menu, click on compute engine (you might want to pin it) and go to VM instances. Click Create Instance. We are going to startup our first node vm. Give it a name in the name field. next, choose a region and zone that makes sense for your location. For machine type we are going to chose g1-small. it will cost roughly $14 to run a month. Remember that $300 credit? Click on boot disk and we are going to select Ubuntu 18.04 LTS Minimal. Under firewall you will see Management, security, disks, networking, sole tenancy. Click that to expand and under management make sure automatic restart is on. Once you are done with that, go ahead and click create. Congrats. You just made your first link node vm. Now do it again with a different region. We need 2 nodes for redundancy. I'll wait while you do that.

Cool. You have both vms setup. Now we need to give them a static ip. Back to the hamburger menu and find VPC network (pin). We are going to select external ip addresses. Next, click on reserve static address. give it a name like node1ip. i selected standard for network service tier. Select your first link node on the drop down for attached to. Click on reserve and wait. Great job! now do it again for the second vm. Ill wait.

Righteous! both vms have a static ip now. Now lets make that PostgreSQL database instance. Hamburger menu, SQL, create instance. Click PostgreSQL, and give it a name and a password and then click create. before we continue however, go back to your vms and make note of the external ip. we are going to need that. after that go back to sql and click on your database  to go to the details page. we need to setup users and whitelisted ips (remember the static ip addresses we made?) Click on connections and tick public ip. hit add network. give the network a name like node1 and node2 or whatever you want. under network take one of the ips you took note of and put it in there with a /32 at the end. so like xx.xxx.xxx.xxx/32. hit done and do it again for your second nodes ip address. and thirdly, you might want to whitelist your home ip address so you can access the datatbase with SQL workbench from your desktop. after you are done click save. Next we are going to add a username. Click on users and select create user account. make a username and password. we are going to need that later. then click create. Next, click Databases and select create database. name it something. you will need that later. One last thing. In the overview tab look for the box that says configuration. at the bottom of it you will see edit configuration. click that. go to enable auto backups and high availability. Change single zone to high availability. it will increase costs, but since we are only having one database instance we want to be 100% sure that it never goes down. click save and wait for it to update.

we need to do one more thing before we can start installing the link node software. we need to generate a ssh key for each node. on your *nix desktop (im sure there is a way on windows. look it up if you insist on using windows on your home desktop) open up your handy dandy terminal. type in:

ssh-keygen -t rsa -f ~/.ssh/gc_rsa1 -C "username"

replace username with a username you will remember. it will then ask for a password. this will generate a ssh key and puts in in the .ssh folder in the home directory with a file called gc_rsa1. do it again and replace 1 with 2 for your second node. there will be a gc_rsa1.pub and gc_rsa2.pub also created in ~/.ssh. open those files up with a text editor of your choice. go into vm instances in GCP, click node 1, and click on edit. scroll down to SSH keys and expand it. click add item and paste the entire contents of gc_rsa1.pub into the field and click save. do it again for node 2 and gc_rsa2.pub. Now we need to secure those files. we do this with the following command

chmod 400 /home/user/.ssh/gc_rsa1
chmod 400 /home/user/.ssh/gc_rsa2

and while we are at it. make sure to backup those files. dont lose them.

lets test out our new SSH keys. open terminal on your home desktop. type in

ssh -i ~/.ssh/gc_rsa1 $username@$node

where $username is your gcp username and $node is your static ip address of node 1.

type in the password that you generated the keys with and type yes when it asks for adding a key. if successful, you should be greeted with a cursor in terminal on your vm running on GCP. Great job! you can now test out node2 and make sure you can log into it with ssh.

Did you make it this far? Awesome. We are done setting up the VM's (as far as i can tell). Pat yourself on the back. your journey to neetdom has finished a chapter.


4. Installing the chainlink node

So we have our new shiny vm's and database setup and ready to go. next step is to follow Thomas's instructions found here:

https://docs.chain.link/docs/running-a-chainlink-node

So ssh into your first node and lets begin! First we are going to update our node. Run

sudo apt update
sudo apt upgrade

once it is completed you now have a fully up to date version of ubuntu 18.04 lts. now to the fun stuff. we are going to now download docker via thomas's instructions.

curl -sSL https://get.docker.com/ | sh
sudo usermod -aG docker $USER
exit
# log in again

easy right? of course it is all you are doing is copying and pasting. The next step is up to you. You can either run a local geth/parity node, or you can use a eaas like fiews or linkpool. this guide we are going to use fiews. its easy, cheap, and saves you money on running another GCP VM. Fiews lowest tier is $5 a month. But you get a free week with signup. lets take advantage of that signup. Go to fiews.io and create an account. you will be given a ropsten link in your dashboard that looks like wss://cl-ropsten.fiews.io/v1/xxxxxxxxxxxxxxxxxxxx. make note of that url. we are going to need it. If you want to run your own eth node, thomas's instructions lay it out quite clearly. next we are going to do some more copying and pasting. we are going to make a directory in our home folder

mkdir ~/.chainlink-ropsten

and then create an enviroment file. this command will create a file called .env in ~/.chainlink-ropsten and populate it with LOG_LEVEL etc. just go ahead and copy all lines below and paste it in terminal as one paste.

echo "ROOT=/chainlink
LOG_LEVEL=debug
ETH_CHAIN_ID=3
MIN_OUTGOING_CONFIRMATIONS=2
LINK_CONTRACT_ADDRESS=0x20fe562d797a42dcb3399062ae9546cd06f63280
CHAINLINK_TLS_PORT=0
SECURE_COOKIES=false
ALLOW_ORIGINS=*" > ~/.chainlink-ropsten/.env

now we are going to edit it. but first we need to install nano, a text editor.

sudo apt install nano

then run

nano ~/.chainlink-ropsten/.env

in here you will see the above lines of text that we created when running the above command. time do add our own lines. remember that fiews url i told you to take note of? we are going to paste it in. Under ALLOW_ORIGINS add

ETH_URL=wss://cl-ropsten.fiews.io/v1/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

the x's being whatever yours is.

now its time to add our portgreSQL location. add the following line under ETH_URL

DATABASE_URL=postgresql://$USERNAME:$PASSWORD@$SERVER:$PORT/$DATABASE

remember when we set the username on portgreSQL? yup, thats the $USERNAME. remember when we set the password on portgreSQL? yup, thats the $PASSWORD. you will find $SERVER on your SQL Instance details where it says Public IP Address. $PORT is the default 5432 port. and finally $DATABASE is the name of your sql database that we set under databases earlier in this guide. feel free to first type this all into a notepad and the copy it over. it should looks something like this

DATABASE_URL=postgresql://anon:password@12.345.678.90:5432/database

lastly we are going to add another line under DATABASE_URL

DATABASE_TIMEOUT=0

this will make the node wait indefinitely for a connection to the database. this will become evident why later. for now we are done in here. go ahead and ctrl+o enter to save and ctrl+x to exit. lets spin up that node now. run the following command

cd ~/.chainlink-ropsten && docker run --name chainlink -p 6688:6688 -v ~/.chainlink-ropsten:/chainlink -it --env-file=.env smartcontract/chainlink local n

this will pull the latest build of the link node to you vm and launch. it will ask for a password for the node, then an email to login to the dashboard with, and then another password for dashboard login. if you are successful with everything. you chainlink node is now working. you should see readouts of blockHash and blockHeight. congrats. there is still way more work to be done but congrats. you have it running.

We are going to start referring to now

https://docs.chain.link/docs/miscellaneous

but before we continue i want you to try something. you have your chainlink node running in your terminal there. hit ctrl+c to terminate the node. now run

docker start -i chainlink

notice how it asks for a password again? we cant have that. if the event the vm restarts itself we dont want it to where you have to manually input the password everytime. So go ahead and hit ctrl+c and go to that link i posted above and lets move to where it says use password and api files on startup. copy and paste

echo "user@example.com" > ~/.chainlink-ropsten/.api

where "user@example.com" is the email you used for the chainlink dashboard when you created the node earlier. next

echo "password" >> ~/.chainlink-ropsten/.api

replace "password" with the dashboard password you set. and finally

echo "my_wallet_password" > ~/.chainlink-ropsten/.password

where "my_wallet_password" is the wallet password of your node. these commands will write user@example.com and "password" to a file called .api in your ~/.chainlink-ropsten folder along with "my_wallet_password" to a file called .password in the same folder. if you screwed up the entries somehow, you can always nano edit those files to fill in the appropriate information. lastly we are going to run

docker rm chainlink

this will remove the original docker container for chainlink. dont fret. all the information is on your sql database. we need to point the launch to those .api and .password files. our new launch command will also add a restart always function. this is for if the chainlink instance fails or the vm restarts it will automatically start the chainlink docker container. our new command is

cd ~/.chainlink-ropsten && docker run --restart always --name chainlink -p 6688:6688 -v ~/.chainlink-ropsten:/chainlink -it --env-file=.env smartcontract/chainlink local n -p /chainlink/.password -a /chainlink/.api

notice the --restart always flag along with our .password and .api files are being used in that command. go ahead and hit enter and your new chainlink node will automatically pull the info needed from the dot files and begin working. mess around with it for a few minutes. ctrl+p ctrl+q will detach the docker container from the screen and allow it to still run in the background. you can check this with

docker container ls

it should show a single docker container running. now go ahead and do a sudo reboot and make sure it starts up automatically with the vm. Great! we have our first node running! Now redo everything that was laid out in this chapter again for your rollover node. ill grab a cup of coffee (the link standard).

alright. you've come back here confused because when you run

cd ~/.chainlink-ropsten && docker run --name chainlink -p 6688:6688 -v ~/.chainlink-ropsten:/chainlink -it --env-file=.env smartcontract/chainlink local n

you get an output that looks something like waiting for postgreSQL lock or something along those lines (i forget). Remember how we set that variable in .env to say DATABASE_TIMEOUT=0? Thats that variable doing its job. right now your node 1 has that database on lock. and this node, node 2, is waiting on it to lose its lock. go ahead and shutdown node 1 and watch as node 2 shortly picks up the slack. Now go back at it.

So youre done with node 2. its now communicating with the sql database while node 1 is shutdown. go ahead and bring up node 1 now. if you do

docker attach chainlink

on node 1 you'll notice how nothing appears. its now waiting on the database. go ahead and shut down node 2 and watch node 1 pick up the slack. Congrats. both nodes are working and rollover is working as intended. you neet node is now operational. What you can do now is simulate outages on both nodes some more (shutting one down) and test this rollover.

there is one more thing we are going to cover in this chapter and thats accessing your chainlink dashboard. right now we have our ssh commands setup as

ssh -i ~/.ssh/gc_rsa1 $username@$node
ssh -i ~/.ssh/gc_rsa2 $username@$node

we are going to make a slight adjustment to both of them. if you look at the bottom of the running a chainlink node

https://docs.chain.link/docs/running-a-chainlink-node

youll see where thomas outlines how to access it. we are going to match his documentation

ssh -i ~/.ssh/gc_rsa1 $username@$node -L 6688:localhost:6688 -N
ssh -i ~/.ssh/gc_rsa2 $username@$node -L 6688:localhost:6688 -N

at this point, you might want to make an alias for these in your .bashrc file on your desktop (or zshrc or whatever you use). not required but it sure beats having to keep a notepad with these in it to copy and paste. you can google how to do that part.

go ahead and ssh into whatever node is active right now with your new revised command. when thats completed, open an internet browser of your choosing, and go to

http://localhost:6688

you will be greeted with your chainlink operator dashboard login page. go ahead and put the email and password in that you entered when creating the node. congrats you have logged into your dashboard.

to be continued.....