daily pastebin goal
21%
SHARE
TWEET

Untitled

a guest Aug 3rd, 2012 48 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. # Saved only the last 20 columns of the file first. No need to grab the rest.
  2.  
  3. # Calculate percent agreement
  4. install.packages("irr")
  5. agree(matrixdata, tolerance=0)
  6.    Percentage agreement (Tolerance=0)
  7.    Subjects = 7000
  8.      Raters = 20
  9.    %-agree = 0.0143
  10.  
  11. agree(matrixdata, 1)
  12.    Percentage agreement (Tolerance=1)
  13.    Subjects = 7000
  14.      Raters = 20
  15.    %-agree = 11.8
  16.  
  17. # Calculate krip's alpha. NOTE: This took hours on a MacBook Pro. If you've a
  18. # spare $10 just rent a cluster compute instance on EC2 for a few hours.
  19. install.packages("concord")
  20. dataset = read.delim("/path/to/results", header=T)
  21. matrixdata = data.matrix(dataset)
  22. krip.alpha(matrixdata, method = "nominal")
  23.     Krippendorffs alpha (data level - nominal) = 0.1529467
RAW Paste Data
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. OK, I Understand
 
Top