Hugging Face API (2)

Question Answering

 

NLP의 또 다른 일반적인 작업은 참조 텍스트에서 질문에 답하는 것이다. 다음 코드를 사용하여 이러한 모델을 로드한다.

 

reader = pipeline("question-answering")
question = "What now shall fade?"

 

이 예에서는 소네트 18번의 '허깅 페이스'에 "무엇이 사라질 것인가"라는 문제를 출제한다. "영원한 여름"이라는 정답이 보인다.

 

outputs = reader(question=question, context=text)
pd.DataFrame([outputs])

 

Language Translation

 

언어 번역은 NLP와 허깅 페이스의 또 다른 공통 작업이다.

 

!pip install sentencepiece

translator = pipeline("translation_en_to_de")

 

다음 코드는 소네트 18을 영어에서 독일어로 번역한다.

 

outputs = translator(text, clean_up_tokenization_spaces=True, min_length =100)
print(outputs[0]['translation_text'])

Sonnet 18 Originaltext William Shakespeare Du bist schöner und gemäßigter: Rough winds shaken the darling buds of May, And summer's lease hat all too short a date: Sometime too hot the eye of heaven shines, And often is his gold complexion dimm'd; And every fair from fair sometime declines, By chance or nature's changing course untrimm'd; But thy eternal summer shall not fade Nor lose possession of that fair thou owest; Nor shall death bra

 

Summarization

 

요약은 긴 텍스트를 단 몇 개의 문장으로 요약하는 NLP 작업이다.

 

text2 = """
An apple is an edible fruit produced by an apple tree (Malus domestica).
Apple trees are cultivated worldwide and are the most widely grown species
in the genus Malus. The tree originated in Central Asia, where its wild
ancestor, Malus sieversii, is still found today. Apples have been grown
for thousands of years in Asia and Europe and were brought to North America
by European colonists. Apples have religious and my thological significance
in many cultures, including Norse, Greek, and European Christian tradition.
"""

summarizer = pipeline("summarization", model="t5-base")

 

다음 코드는 "사과"에 대한 항목을 요약한 것이다.

 

summarizer = pipeline("summarization", model="t5-base")

apple trees are cultivated worldwide and are the most widely grown species in the genus Malus. the tree originated in Central Asia, where its wild ancestor, Malus sieversii, is

 

 

Text Generation

 

마지막으로 텍스트 생성은 입력 텍스트를 받아 미리 학습된 신경망에 해당 텍스트를 계속하도록 요청하는 기능이다.

 

from urllib.request import urlopen

generator = pipeline("text-generation")

 

다음은 소네트 18 다음에 추가 텍스트를 생성하는 예제이다.

 

URL = "https://data.heatonresearch.com/data/t81-558/datasets/sonnet_18.txt"

f = urlopen(URL)
text = f.read().decode("utf−8")

outputs = generator(text, max_length=400)
print(outputs[0]['generated_text'])

Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
Sonnet 18 original text
William Shakespeare

Shall I compare thee to a summer's day?
Thou art more lovely and more temperate:
Rough winds do shake the darling buds of May,
And summer's lease hath all too short a date:
Sometime too hot the eye of heaven shines,
And often is his gold complexion dimm'd;
And every fair from fair sometime declines,
By chance or nature's changing course untrimm'd;
But thy eternal summer shall not fade
Nor lose possession of that fair thou owest;
Nor shall Death brag thou wander'st in his shade,
When in eternal lines to time thou growest:
So long as men can breathe or eyes can see,
So long lives this and this gives life to thee. Notwithstanding the love of these, Notwithstanding the love of these thy own lives, Notwithstanding the love of these thy own lives, Notwithstanding the love of these thy own lives, Notwithstanding the love of these thy own lives, Notwithstanding the love of these thy own lives, Notwithstanding the love of these thy own lives, Notwithstanding the love of all thy life's lives, notwithstanding the love of all thy life's lives, Notwithstanding the love of all thou have heard, notwithstanding all thy life's days thee hear, thy love has passed with thee thy day hath gone thy day has passed, thy love hath been renewed thy love has been renewed thy love has been renewed thy love Has he not passed his life has passed his life hath not passed his life hath not passed his life the life of others thou doest live thou doest live thy life, thy life of thy life thou mayest live thy life, thy life of thy life Thou sayest, thy dear thou sayest, thy dear thou sayest, thy dear thee sayest, thy Dear, thy Dear, thou sayth thine prayers