Norming Words

Once eDom has loaded participants can use the software to norm words.  A detailed set of instructions that have been used in a previous norming experiment are included at the bottom of this page.  In brief, the instructions are as follows:

When the software loads completely, a blank norming screen will be presented. Participants can press 'begin Experiment' at the top left to begin norming words.

Participants will then be presented with a word and the definitions associated with each of its meanings (if any).  

On this screen, participants will be presented with each meaning of the word in a separate box.  The orders of the meanings is randomized in the display so that the dominant meaning is not always presented first.  Depending on how eDom has been set up, one or more of the empty definition windows will also be shaded in yellow.  These boxes are editable so that a participant can enter their own definitions for a word.  A progress meter appears in the top right which indicates the percentage of words in the experiment that have been completed (rounded down to the nearest decile).

After reading through the definitions (and optionally adding new definitions), participants must then rate the percentage of times that the word refers to each of the meanings when it is encountered.  These percentages are typed under each definition.  Percentages must sum to 100 across all definitions and only whole numbers may be entered.  An warning messagebox will be generated if participants add a new definition but do not assign a non-zero percentage to that definition, or if a non-zero percentage is entered in an empty box without adding a definition.  

Once participants are done rating the word, they can press 'Done Rating' to proceed to the next word, or to a break if enough words have been rated.

Alternatively, they can press 'Don't know word' if they do not know any of the meanings of the word and that word will be skipped.  No new definitions or non-zero percentages can be entered in this case.

Once all of the words have been rated, a message box will appear indicating that all of the data has been saved successfully.  eDom will exit once that message box is closed.

Sample Norming Instructions 

The following instructions have been used to solicit useful dominance ratings in some of our experiments:

Please review the following instructions carefully.  The entire experiment takes less than an hour, so there is no need to rush.  You may ask the experimenter for clarifications if you wish. 

Words can be classified based on a number of characteristics, one of which is the number of separate meanings that are associated with a word.  Broadly speaking, words can be classified as having a single meaning (sometimes called unambiguous words) or having multiple unrelated meanings (sometimes called ambiguous words).   For instance, the word BANK is associated with two separate meanings, one of which relates to a ‘financial institution’ interpretation, and the other of which relates to an ‘edge of a body of water’ interpretation.  Similarly, the word SECOND can refer to a unit of time, or to ranking which is one below ‘first’. 

The present study is concerned with assessing the relative frequency with which each meaning of an ambiguous word is intended when that word is used.  To do so, you will be asked to estimate, as a percentage, how often you think a particular meaning of a word is implied when you have encountered a given word.  For example, when you encounter the word BANK, you would have to estimate what percentage of encounters that word is being used in the ‘financial’ interpretation of the word versus the ‘water’ interpretation of the word.  In this case, you might decide that the financial interpretation of the word is more frequently encountered, and thus give a high percentage score to the financial interpretation and a low percentage score to the river interpretation.  In contrast, for the word SECOND, you might decide that the two meanings are fairly well balanced and thus give fairly similar percentage scores to each meaning of the word. 

To help remind you of the meanings associated with a word, the dictionary definitions associated with that word have been provided.  You should read over these definitions to help cue you as to which meaning of the word is intended and then base your decisions on your impression of how frequent that meaning has been in your own personal experience.  Note that the number of definitions and the length of these definitions should not directly impact your judgement, as some less frequent interpretations of a word might sometimes have rather detailed definitions, even though you don’t use that particular meaning of a word very frequently.   The order in which the definitions appear has also been randomized so which definition appears first does not provide any information about the frequency with which a particular meaning is encountered. 

For some of the words, it is also possible that you are familiar with another meaning associated with that word that doesn’t appear in any of the definitions that you are presented with.  For example, the word DOUGH might only have a definition which relates to the ‘bakery’ sense of the word, but you may know that this word sometimes relates to a definition linked to ‘money’.   If this occurs, you can list this definition in one of the two empty text boxes provided for this purpose (color coded in yellow) and provide a rating of how frequently that meaning of the word is intended when you encounter the word.  Please list each extra meaning you know for a given word in a separate yellow text box.  If you know more than two such meanings, list only the two most frequent additional meanings that you thought of in the two yellow boxes provided for this purpose. 

You should enter your frequency ratings for the different meanings of the ambiguous word as percentages ranging from 0-100.  Please only use whole numbers, not decimal places.  These numbers must sum to 100 across all of the definitions.  If you provided any extra definitions of a word, you should also list the percentage of times that you encounter these meanings as part of the total 100% that you are distributing across the different meanings. 

There are no right answers in this experiment – we’re interested in your personal impression of the relative frequency with which you encounter a given word.  Please try to be accurate in your ratings, without spending too much time thinking about how to rate a particular word. 

If you do not know a given word at all, you can press the ‘Don’t know word’ button. 

Periodically, you will be offered the chance to take a break, after which you can resume the experiment by pressing the ‘resume’ button. 

To summarize the instructions:

• your task is to rate, as a percentage between 0-100, the frequency with which you encounter a particular meaning of an ambiguous word.   The total percentage must sum to 100 across all of the definitions.  The dictionary definitions of each word are provided to help cue you as to which meaning is denoted in each instance.
• Two yellow boxes are provided where you can list any extra definitions of a word that you are familiar with that are not included in the definitions that we have provided.  If you know of an extra definition for a given word, please write a brief definition in one of these boxes (one definition per box).  You can then proceed with rating the frequencies of each meaning of the word as usual, including the meaning denoted in that box.  If you know more than two such definitions, only list the two most frequent definitions in the yellow boxes. 

This concludes your briefing.  If you have any questions, please ask the experimenter now.  Otherwise, you may begin the experiment.