Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- <!DOCTYPE html><html><head><title>HIT</title><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/><script type='text/javascript' src='https://s3.amazonaws.com/mturk-public/externalHIT_v1.js'></script></head><body><form name="mturk_form" method="post" id="mturk_form" action="https://www.mturk.com/mturk/externalSubmit"><input type="hidden" value="" name="assignmentId" id="assignmentId" /><script type="text/javascript" language="javascript" src="https://code.jquery.com/jquery-1.12.3.min.js"></script><script type="text/javascript" class="init">
- $(document).ready(function() {
- $(".instButton").click(function () {
- //alert(wordsSeen.innerHTML); //(function(i, val) { return val*1+1 })
- $('#instructionSeen').val( parseInt($('#instructionSeen').val())+1);
- //alert($('#instructionSeen').val());
- if ($('.panel-body').css('display') == "none")
- {
- $('.panel-body').css('display','inline-block');
- }
- else{
- $('.panel-body').css('display','none');
- }
- });
- });
- </script>
- <p><input hidden="" id="instructionSeen" name="clickedInstructions" value="0" /></p>
- <link href="https://s3.amazonaws.com/mturk-public/bs30/css/bootstrap.min.css" rel="stylesheet" />
- <section class="container" id="Other" style="margin-bottom: 15px; padding: 10px; font-family: Verdana, Geneva, sans-serif; color: rgb(51, 51, 51);">
- <div class="row col-xs-12 col-md-12"><!-- Instructions -->
- <div class="panel panel-primary">
- <div class="panel-heading" style="font-size: 0.9em;"><strong>Instructions:</strong></div>
- <p style="color: rgb(51, 51, 51); font-size: 0.9em;"> </p>
- <div align="center" id="buttons" style="color: rgb(51, 51, 51);"><button class="instButton" id="showall" style="margin-right: 1em;" type="button">Expand/Contract Instructions</button></div>
-
- <div class="panel-body" hidden="">
- <p dir="ltr" style="margin-bottom: 15px; color: rgb(0, 0, 0); font-family: Lato, Helvetica, Arial, sans-serif; font-size: 15px;"><span style="color:#008000;"><strong>Congratulations!!</strong></span> You have been selected as a qualified judge. </p>
- <p dir="ltr" style="margin-bottom: 15px; color: rgb(0, 0, 0); font-family: Lato, Helvetica, Arial, sans-serif; font-size: 15px;"><span style="color: rgb(255, 0, 0);"><strong>Important:</strong> </span><span style="color: rgb(255, 0, 0); font-family: Lato, Helvetica, Arial, sans-serif; font-size: 15px;">We know the actual funniness scores of some of the edited headlines below which we will use to monitor your performance</span><span style="color: rgb(255, 0, 0);"> as you submit HITs, and we may disqualify you if we discover that your answers are not close to these scores.</span></p>
- <p dir="ltr" style="margin-bottom: 15px; color: rgb(0, 0, 0); font-family: Lato, Helvetica, Arial, sans-serif; font-size: 15px;">In this task, you are presented with the <u><strong>original</strong></u><strong> </strong>headlines of 10 online news articles (from 2017-2018) and their <u><strong>edited</strong></u><strong> </strong>versions. Sometimes the edit makes the headline funny, other times it does not. In particular, some of the edited headlines are intentionally not funny because we need data that includes both funny and not funny headline edits. Your task is to <strong>grade how funny each edited headline is</strong>. You are to call an edited headline funny if any of the following holds:</p>
- <ul style="font-family: Verdana, Geneva, sans-serif;">
- <li style="font-size: 0.9em;"><span style="font-size: 15px; line-height: 30.8px;">It is funny by itself regardless of the original headline.</span></li>
- <li style="font-size: 0.9em;"><span style="font-size: 15px; line-height: 30.8px;">It is only funny when you consider how the original headline was changed.</span></li>
- </ul>
- <p dir="ltr" style="margin-bottom: 15px; color: rgb(0, 0, 0); font-family: Lato, Helvetica, Arial, sans-serif; font-size: 15px;"><span style="color: rgb(0, 0, 0);">You will choose the funniness of the edited headlines on a range from 0 to 3 with 0 being "not funny" and 1,2,3 being <u><strong>uniformly increasingly</strong></u> funny. It is OK to give the same funniness score to two or more different edited versions of the same original headline. They might be the same funniness, but in different ways.</span></p>
- <p style="font-family: Verdana, Geneva, sans-serif; font-size: 0.9em;"><span style="font-family: Verdana, Geneva, sans-serif; font-size: 15px;"><u>Important Suggestions</u>:</span></p>
- <ul>
- <li style="font-family: Verdana, Geneva, sans-serif; font-size: 0.9em;"><span style="font-family: Verdana, Geneva, sans-serif; font-size: 15px;">Please judge the edited headline's funniness as it is, and do not consider other edits that could have been possible and made the headline funnier.</span></li>
- <li style="font-family: Verdana, Geneva, sans-serif; font-size: 0.9em;"><span style="font-family: Verdana, Geneva, sans-serif; font-size: 15px;">If the edited headline seems like it might be funny to other people, but you do not get the joke (e.g. because you may not know a person or a thing mentioned in the headline), then mark it as NOT FUNNY.</span></li>
- <li style="font-family: Verdana, Geneva, sans-serif; font-size: 0.9em;"><span style="font-family: Verdana, Geneva, sans-serif; font-size: 15px;">If you see two identical edits of the same headline, please assign both the same funniness score. </span></li>
- </ul>
- <p style="font-family: Verdana, Geneva, sans-serif; font-size: 0.9em;"><span style="font-size: 15px;">Our <u><strong>rules</strong></u> do not allow the following edits. If an edit includes one or more of the following, please mark the checkbox labeled "Rule Violation", but also assign a funniness grade (as you would assign to other headlines that do not violate these rules):</span></p>
- <ul style="font-family: Verdana, Geneva, sans-serif;">
- <li style="font-size: 0.9em;"><span style="font-family: Verdana, Geneva, sans-serif; font-size: 15px;"><span style="font-size: 15px; line-height: 20.8px;">An edit should not contain cheap and easy humor generation techniques. These include <u>profanity</u> (e.g. cunt, pussy), <u>slang words</u> (e.g. jackass, idiot), <u>bathroom/potty humor</u> (e.g. farting), or <u>sexual references</u>.</span></span></li>
- <li style="font-size: 0.9em;"><span style="font-size: 15px;">Editors are not allowed to add more than 1 word for an edit or </span><u style="font-size: 15px;">squeeze multiple words into one</u><span style="font-size: 15px;"> (e.g.</span><span style="font-size: 15px;"> "housecat", "VladimirPutin"</span><span style="font-size: 15px;">).</span></li>
- <li style="font-size: 0.9em;"><span style="font-family: Verdana, Geneva, sans-serif; font-size: 15px;"><u>Grammatical errors</u> (e.g. subject-verb agreement, plurality errors), <u>spelling mistakes</u>, </span><u style="font-family: Verdana, Geneva, sans-serif; font-size: 15px;">informal language</u><span style="font-family: Verdana, Geneva, sans-serif; font-size: 15px;"> (e.g. ain't) are not allowed, <span style="color:#0000CD;">however minor errors such as capitalization and "a/an" errors are ok. Also do not mark compound word errors like "Trump to build wall" (grammatically correct form: "Trump <em>wants</em> to build a wall") as Rule Violation because news headlines are often structured that way.</span></span></li>
- </ul>
- <p dir=" ltr"><span style="font-family: Verdana, Geneva, sans-serif; color: rgb(51, 51, 51); font-size: 15px; line-height: 20.8px;">We expect you to be an <u>objective judge</u> who is:</span></p>
- <ul style="font-family: Verdana, Geneva, sans-serif;">
- <li style="font-size: 0.9em;"><span style="font-size: 15px; line-height: 25.8px;">Great</span><span style="font-size: 15px; line-height: 20.8px;"> at recognizing and appreciating the </span><span style="font-size: 15px; line-height: 20.8px;">humor content in a headline regardless of your own opinions.</span></li>
- <li style="font-size: 0.9em;"><span style="font-size: 15px; line-height: 25.8px;">Objective about recognizing jokes that may not reflect your opinions but would still be funny to a large audience.</span></li>
- </ul>
- <p style="font-family: Verdana, Geneva, sans-serif; font-size: 0.9em;"><span style="font-size: 15px; line-height: 20.8px;">To be as objective as possible, you can take the following suggestion: when reading an edited headline, always think how a crowd would react to it, and then try to judge as if you represent the crowd. </span></p>
- <p style="font-family: Verdana, Geneva, sans-serif; font-size: 1em;"><strong style="color: rgb(0, 0, 0); font-family: Lato, Helvetica, Arial, sans-serif; font-size: 15px;">Note:</strong><span style="color: rgb(0, 0, 0); font-family: Lato, Helvetica, Arial, sans-serif; font-size: 15px;"> Our pay rate assumes that you will spend about 10s per headline funniness judgement.</span></p>
- </div>
- </div>
- <p dir="ltr" style="margin-bottom: 15px; color: rgb(0, 0, 0); font-family: Lato, Helvetica, Arial, sans-serif; font-size: 15px;"><span style="color: rgb(255, 0, 0);">Please do not attempt more than <b><u>15</u></b> HITs to allow sufficient judges to grade the edited headlines.</span></p>
- <table border="1">
- <tbody>
- <tr>
- <th>NOTE: Tick "Rule Violation" if the edit violates the rules described in instructions.</th>
- </tr>
- <tr>
- <td><strong>Orig:</strong> Turkey Gears Up for ‘ Risky ’ <span class="orig"><strong>Syria</strong></span> Mission With Russia and Iran <br />
- <strong>Edit:</strong> Turkey Gears Up for ‘ Risky ’ <span class="edit"><strong>Pizza</strong></span> Mission With Russia and Iran <br />
- <input name="violation0" type="checkbox" value="1" /> <span style="color:#3e73c9;font-weight:bold">Rule Violation</span><br />
- <br />
- <label><input name="h0Answer" required="" type="radio" value="0" /> 0 (Not Funny)</label>      <label><input name="h0Answer" type="radio" value="1" /> 1 (Slightly Funny)</label>      <label><input name="h0Answer" type="radio" value="2" /> 2 (Moderately Funny)</label>      <label><input name="h0Answer" type="radio" value="3" /> 3 (Funny)</label></td>
- </tr>
- <tr>
- <td><strong>Orig:</strong> Erick Erickson: Trump's <span class="orig"><strong>Russia</strong></span> Leak Is ' Far Worse ' Than Reported <br />
- <strong>Edit:</strong> Erick Erickson: Trump's <span class="edit"><strong>Tanning</strong></span> Leak Is ' Far Worse ' Than Reported <br />
- <input name="violation1" type="checkbox" value="1" /> <span style="color:#3e73c9;font-weight:bold">Rule Violation</span><br />
- <br />
- <label><input name="h1Answer" required="" type="radio" value="0" /> 0 (Not Funny)</label>      <label><input name="h1Answer" type="radio" value="1" /> 1 (Slightly Funny)</label>      <label><input name="h1Answer" type="radio" value="2" /> 2 (Moderately Funny)</label>      <label><input name="h1Answer" type="radio" value="3" /> 3 (Funny)</label></td>
- </tr>
- <tr>
- <td><strong>Orig:</strong> <span class="orig"><strong>Donald Trump Jr.</strong></span> will meet with House Intelligence Committee next week <br />
- <strong>Edit:</strong> <span class="edit"><strong>puppet</strong></span> will meet with House Intelligence Committee next week <br />
- <input name="violation2" type="checkbox" value="1" /> <span style="color:#3e73c9;font-weight:bold">Rule Violation</span><br />
- <br />
- <label><input name="h2Answer" required="" type="radio" value="0" /> 0 (Not Funny)</label>      <label><input name="h2Answer" type="radio" value="1" /> 1 (Slightly Funny)</label>      <label><input name="h2Answer" type="radio" value="2" /> 2 (Moderately Funny)</label>      <label><input name="h2Answer" type="radio" value="3" /> 3 (Funny)</label></td>
- </tr>
- <tr>
- <td><strong>Orig:</strong> <span class="orig"><strong>Survivor</strong></span>: We will only be heard if we scream @CNN <br />
- <strong>Edit:</strong> <span class="edit"><strong>Birds</strong></span>: We will only be heard if we scream @CNN <br />
- <input name="violation3" type="checkbox" value="1" /> <span style="color:#3e73c9;font-weight:bold">Rule Violation</span><br />
- <br />
- <label><input name="h3Answer" required="" type="radio" value="0" /> 0 (Not Funny)</label>      <label><input name="h3Answer" type="radio" value="1" /> 1 (Slightly Funny)</label>      <label><input name="h3Answer" type="radio" value="2" /> 2 (Moderately Funny)</label>      <label><input name="h3Answer" type="radio" value="3" /> 3 (Funny)</label></td>
- </tr>
- <tr>
- <td><strong>Orig:</strong> ' They basically have to let me <span class="orig"><strong>win</strong></span> ': Trump believes the media will help him get reelected <br />
- <strong>Edit:</strong> ' They basically have to let me <span class="edit"><strong>play</strong></span> ': Trump believes the media will help him get reelected <br />
- <input name="violation4" type="checkbox" value="1" /> <span style="color:#3e73c9;font-weight:bold">Rule Violation</span><br />
- <br />
- <label><input name="h4Answer" required="" type="radio" value="0" /> 0 (Not Funny)</label>      <label><input name="h4Answer" type="radio" value="1" /> 1 (Slightly Funny)</label>      <label><input name="h4Answer" type="radio" value="2" /> 2 (Moderately Funny)</label>      <label><input name="h4Answer" type="radio" value="3" /> 3 (Funny)</label></td>
- </tr>
- <tr>
- <td><strong>Orig:</strong> Key senator urges any wavering U.S. high court <span class="orig"><strong>justice</strong></span> to retire now <br />
- <strong>Edit:</strong> Key senator urges any wavering U.S. high court <span class="edit"><strong>jester</strong></span> to retire now <br />
- <input name="violation5" type="checkbox" value="1" /> <span style="color:#3e73c9;font-weight:bold">Rule Violation</span><br />
- <br />
- <label><input name="h5Answer" required="" type="radio" value="0" /> 0 (Not Funny)</label>      <label><input name="h5Answer" type="radio" value="1" /> 1 (Slightly Funny)</label>      <label><input name="h5Answer" type="radio" value="2" /> 2 (Moderately Funny)</label>      <label><input name="h5Answer" type="radio" value="3" /> 3 (Funny)</label></td>
- </tr>
- <tr>
- <td><strong>Orig:</strong> CNN ’s Don Lemon <span class="orig"><strong>Unloads</strong></span> On Trump Over Racist Insults In Epic New Takedown <br />
- <strong>Edit:</strong> CNN ’s Don Lemon <span class="edit"><strong>Squeezes</strong></span> On Trump Over Racist Insults In Epic New Takedown <br />
- <input name="violation6" type="checkbox" value="1" /> <span style="color:#3e73c9;font-weight:bold">Rule Violation</span><br />
- <br />
- <label><input name="h6Answer" required="" type="radio" value="0" /> 0 (Not Funny)</label>      <label><input name="h6Answer" type="radio" value="1" /> 1 (Slightly Funny)</label>      <label><input name="h6Answer" type="radio" value="2" /> 2 (Moderately Funny)</label>      <label><input name="h6Answer" type="radio" value="3" /> 3 (Funny)</label></td>
- </tr>
- <tr>
- <td><strong>Orig:</strong> Face facts, America, Donald Trump is a success. Let's count the <span class="orig"><strong>ways</strong></span>. <br />
- <strong>Edit:</strong> Face facts, America, Donald Trump is a success. Let's count the <span class="edit"><strong>lies</strong></span>. <br />
- <input name="violation7" type="checkbox" value="1" /> <span style="color:#3e73c9;font-weight:bold">Rule Violation</span><br />
- <br />
- <label><input name="h7Answer" required="" type="radio" value="0" /> 0 (Not Funny)</label>      <label><input name="h7Answer" type="radio" value="1" /> 1 (Slightly Funny)</label>      <label><input name="h7Answer" type="radio" value="2" /> 2 (Moderately Funny)</label>      <label><input name="h7Answer" type="radio" value="3" /> 3 (Funny)</label></td>
- </tr>
- <tr>
- <td><strong>Orig:</strong> Ben Carson ’s Department of Housing and Urban Development no longer cares about <span class="orig"><strong>discrimination</strong></span> <br />
- <strong>Edit:</strong> Ben Carson ’s Department of Housing and Urban Development no longer cares about <span class="edit"><strong>renters</strong></span> <br />
- <input name="violation8" type="checkbox" value="1" /> <span style="color:#3e73c9;font-weight:bold">Rule Violation</span><br />
- <br />
- <label><input name="h8Answer" required="" type="radio" value="0" /> 0 (Not Funny)</label>      <label><input name="h8Answer" type="radio" value="1" /> 1 (Slightly Funny)</label>      <label><input name="h8Answer" type="radio" value="2" /> 2 (Moderately Funny)</label>      <label><input name="h8Answer" type="radio" value="3" /> 3 (Funny)</label></td>
- </tr>
- <tr>
- <td><strong>Orig:</strong> After GOP Health <span class="orig"><strong>Failure</strong></span>, Next Battle Could Shut Down Government <br />
- <strong>Edit:</strong> After GOP Health <span class="edit"><strong>checkup</strong></span>, Next Battle Could Shut Down Government <br />
- <input name="violation9" type="checkbox" value="1" /> <span style="color:#3e73c9;font-weight:bold">Rule Violation</span><br />
- <br />
- <label><input name="h9Answer" required="" type="radio" value="0" /> 0 (Not Funny)</label>      <label><input name="h9Answer" type="radio" value="1" /> 1 (Slightly Funny)</label>      <label><input name="h9Answer" type="radio" value="2" /> 2 (Moderately Funny)</label>      <label><input name="h9Answer" type="radio" value="3" /> 3 (Funny)</label></td>
- </tr>
- </tbody>
- </table>
- <p> </p>
- <strong>Feedback </strong>(Optional). Please let us know if you have any confusion with the instructions; cases where you had difficulty deciding a score and why, and what we can do to clarify such cases, etc. Note that we won't provide any example grades of edited headlines.<textarea class="form-control" cols="120" name="feedback" rows="5"></textarea>
- <style type="text/css">/* show/hide button */
- #showall {
- background-color: #699DB6;
- border-color: rgba(0,0,0,0.3);
- text-shadow: 0 1px 0 rgba(0,0,0,0.5);
- color: #FFF;
- }
- #showall:hover {
- background-color: #4F87A2;
- border-color: rgba(0,0,0,0.5);
- }
- #showall:active {
- background-color: #3C677B;
- border-color: rgba(0,0,0,0.9);
- }
- table {
- font-family: arial, sans-serif;
- border-collapse: collapse;
- width: 100%;
- border: 1px solid black;
- }
- td{
- border: 2px solid black;
- text-align: left;
- padding: 8px;
- }
- th {
- border: 2px solid black;
- background-color: #95D6F9;
- text-align: left;
- padding: 8px;
- }
- tr:nth-child(even) {
- background-color: #dddddd;
- }
- .orig {
- color: #ff5500;
- background-color: yellow;
- font-size: 18px;
- }
- .edit {
- color: #009900;
- background-color: yellow;
- font-size: 18px;
- }
- </style>
- </div>
- </section>
- <p class="text-center"><input type="submit" id="submitButton" class="btn btn-primary" value="Submit" /></p></form><script language="Javascript">turkSetAssignmentID();</script></body></html>
Advertisement
Add Comment
Please, Sign In to add comment