Off topic: Help with academic study on question answering Thread poster: Sandra & Kenneth Grossman
|
|---|
As part of his PhD, my son-in-law is trying to enable machines to better answer questions in a way that is easy for people to understand.
The intention is to get as many responses as possible to understand how people from diverse backgrounds understand the English language. It's OK to do the survey even if you're not a native English speaker. Also, the survey is completely anonymous.
Help us teach computers to do a better job answering questions, and hopefully, you... See more As part of his PhD, my son-in-law is trying to enable machines to better answer questions in a way that is easy for people to understand.
The intention is to get as many responses as possible to understand how people from diverse backgrounds understand the English language. It's OK to do the survey even if you're not a native English speaker. Also, the survey is completely anonymous.
Help us teach computers to do a better job answering questions, and hopefully, you may enjoy the results in your search engine in the future!
Here is the link to the survey:
http://irsrv2.cs.biu.ac.il:9992/
Your participation is much appreciated!
Sandra ▲ Collapse
| | | | 564354352 (X) 
Denmark
Local time: 15:25
Danish to English
+ ...
| Not so anonymous | Sep 2, 2015 |
The first thing you are asked to do is to enter your name... Put me off straight away... sorry...
| | | | | This is just an optional field | Sep 2, 2015 |
... which is pre-filled.
You can use a pseudonym or leave it as is. Most people just stay with the default name.
| | | | Tom in London
United Kingdom
Local time: 14:25
Member (2008)
Italian to English
Sandra& Kenneth wrote:
As part of his PhD, my son-in-law is trying to enable machines to better answer questions in a way that is easy for people to understand.
Why?
| | |
|
|
|
Phil Hand
China
Local time: 21:25
Chinese to English
I tried, but there are too many questions, and the questionnaire is confusing.
For example, it offers both singular and plural verb forms. But some nouns are inherently plural, so I can't mark one or the other as grammatically incorrect. Also, there are several items on which I could go either way.
I would suggest that the questionnaire is redesigned, with real words instead of the dummy variables in angle brackets, and perhaps ask people to select the best answer out o... See more I tried, but there are too many questions, and the questionnaire is confusing.
For example, it offers both singular and plural verb forms. But some nouns are inherently plural, so I can't mark one or the other as grammatically incorrect. Also, there are several items on which I could go either way.
I would suggest that the questionnaire is redesigned, with real words instead of the dummy variables in angle brackets, and perhaps ask people to select the best answer out of 3 or 4, rather than this brain-melting list. ▲ Collapse
| | | |
I wanted to help out by responding, but I found the survey too confusing and finally gave up. I agree with the suggestions offered by Phil.
| | | | | filled it, but | Sep 2, 2015 |
Sandra& Kenneth wrote:
As part of his PhD, my son-in-law is trying to enable machines to better answer questions in a way that is easy for people to understand.
It might have to start with creating questions that people are able to understand, before worrying about machines being able to answer them. These questions were seriously messed up.
| | | |
Hi everyone, thank you so much for helping with my research!
Phil, thanks for the valuable feedback; I will incorporate some of your suggestions in the next round of annotations.
Perhaps I should give some broader context. The questionnaire isolates a specific problem in automatic question answering - answer verification. In this subtask, the machine is given a well-formed question and an automatically-retrieved text fragment containing a candidate answer. The machine m... See more Hi everyone, thank you so much for helping with my research!
Phil, thanks for the valuable feedback; I will incorporate some of your suggestions in the next round of annotations.
Perhaps I should give some broader context. The questionnaire isolates a specific problem in automatic question answering - answer verification. In this subtask, the machine is given a well-formed question and an automatically-retrieved text fragment containing a candidate answer. The machine must then determine whether the text fragment indeed contains an answer to the question. This task is extremely difficult for machines, since many of these decisions are based on context. For example, in "Which drug cures infections?", the fragment "<DRUG> eradicates infections" will probably contain a relevant answer, because "eradicates" implies "cures" when the object is "infections". However, this is most certainly not the case in "Which person cures sick patients?".
The questionnaire is challenging in two aspects. First, because roughly 30% of the candidate answer fragments are ungrammatical or nonsensical. This is an artifact of the automatic fragment retrieval process, which relies on other error-prone automatic processes such as syntactic parsing. The other reason is that the answer-verification task itself is semantically gray; different people tend to interpret language in different manners, resulting in a mix of different annotations for each question. This is exactly why it is so important for me to get a diverse sample of annotators, and why your participation is so helpful to my cause.
Finally, I'd like to address Phil's point on <PLACEHOLDERS>. Placeholders are a necessary evil, because in the first round of annotations, where we had the actual answer in view, annotators used their own world knowledge to decide whether the fragment contains an answer. While populating the machine with world knowledge is a valid approach for automatic question answering, this research project is focused on teaching it whether the context is indicative of a correct answer, regardless of the answer itself.
Again, thank you so much for participating in my questionnaire and helping my research!
[Edited at 2015-09-03 05:46 GMT] ▲ Collapse
| | |
|
|
|
| Thanks everyone for commenting and participating! | Sep 3, 2015 |
Your comments and participation are much appreciated.
The young scientist's reply is still being vetted by moderators, and hopefully will be released soon.
Thanks!
Sandra
| | | | | To change the form... | Sep 4, 2015 |
If you don't feel comfortable with the questions, you can regenerate the form any number of times to a different set by refreshing the page!
Thanks everyone!
Sandra
| | | | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » Help with academic study on question answering | CafeTran Espresso | You've never met a CAT tool this clever!
Translate faster & easier, using a sophisticated CAT tool built by a translator / developer.
Accept jobs from clients who use Trados, MemoQ, Wordfast & major CAT tools.
Download and start using CafeTran Espresso -- for free
Buy now! » |
| | TM-Town | Manage your TMs and Terms ... and boost your translation business
Are you ready for something fresh in the industry? TM-Town is a unique new site for you -- the freelance translator -- to store, manage and share translation memories (TMs) and glossaries...and potentially meet new clients on the basis of your prior work.
More info » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
