Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong stemming : A rule does not obey the spelling system #99

Open
bgnkim opened this issue Jun 30, 2016 · 3 comments
Open

Wrong stemming : A rule does not obey the spelling system #99

bgnkim opened this issue Jun 30, 2016 · 3 comments

Comments

@bgnkim
Copy link

bgnkim commented Jun 30, 2016

Thanks for your convenient and fast Korean parsing system.
However, I found there exists a wrong rule on stemming system, which does not follow the Korean spelling system.

For example,
"조사된" has root form of "조사(하다) + 되 + ㄴ", so correct stemming form should be one of "조사+되다" or "조사되다", while the stemming system understands that word as "조사돼다" (which means "조사하다+되+어+다", with redundant ending token "어").
There is another example like:
"대상이 되다" --> (Correct form) "대상 + 이 + 되다", so correct form should be "대상+이+되다" (System says) "대상 + 이 + 돼다"
"판정됐다" -> (Correct form) "판정 + 되 + 었 + 다", so correct form should be "판정되다" or "판정하다" (System says) "판정돼다"

Stemming system only misgiving this type of spelling error, for "되/돼" variations. Because "돼 = 되+어", usually "되" is the basic form ("어" is a ending morpheme to make the end of sentence or word).
Although it's very common error in Korean spelling system and doesn't related with the developing issue, I think you should provide it correctly.
FYI, here(korean) is an semi-official article from the National Institute of the Korean Language.

Thanks again with the convenient system.


다른 개발진에 외국인이 있을지 몰라서 영어로 적었습니다.
정리하자면 Stemming system이 "돼/되"를 반대로 구분하고 있습니다.

"조사된"의 기본형은 "조사되다"임에도 "조사돼다"로 출력되고,
"대상이 되다" 역시 "대상이 돼다"로 출력되며,
"판정됐다" 역시 "판정돼다"로 출력됩니다.

데이터 분석작업이나 개발에는 큰 문제가 없겠지만, 맞춤법을 따르도록 고쳐주시는 것이 어떤가 합니다.

감사합니다.

@hohyon-ryu
Copy link
Contributor

좋은 지적 감사드립니다. 정확한 학문적 분석보다는 실무에 사용하기 위해 만들다 보니 부족한 점이 많습니다. 참여해 주셔서 감사드립니다. 시간 되는대로 검토하여 반영하도록 하겠습니다.

혹시 직접 코드를 바꾸는 것에도 관심이 있으시면 알려주세요. 제가 함께 가이드 해 드리겠습니다.

감사합니다.

@tglstory
Copy link

tglstory commented Jul 7, 2016

I will take this.

@hohyon-ryu
Copy link
Contributor

@tglstory Thank you so much!! Please let me know if there's anything I can help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants